PDA

View Full Version : indexing problem??


Chris2
02-20-2004, 05:53 PM
it's widely discussed around here already... the "just recently indexed problem", now i've followed all the threads, understood only half of it and finally just did two things:

hit "delete site" and "update form" on the admin/index.php page serveral times... (that surely wasn't it)

and:

rewrote my robots.txt:

problem: there was just a single space in fron of "User-Agent:" in the first line...

and see, jump, enjoy:

it's running again...

so to anyone who is getting crazy as well: check the robots.txt syntax, some spider engines may accept minor faults, others don't!

Charter
02-20-2004, 06:05 PM
Hi. Sounds like it may be a regex issue. What was the original robots.txt file, and what is the new robots.txt index file?

Chris2
02-21-2004, 07:23 AM
Hi Charter,

in the old robots.txt I had just a space at the beginning of the first line, now I've changed that and it is working again... so I don't have a prob. anymore but I just reported that so that other people might look into their robtos.txt before they get crazy trying to find out what a "NAT" is... (that's what happened to me when I first encounterted a prob. and read around here)

Keep up your good work, phpdig rocks.

Thanks

Chris2