|
08-31-2004, 12:34 PM | #1 |
Green Mole
Join Date: Aug 2004
Location: Chile
Posts: 13
|
404 error via shell... no pages indexed
When spidering via web I had no problems, the spider found links and index the pages, but when I do it via ssh:
-------------------------------------------------------------------------- HTTP/1.1 404 Not Found - http://www.chile-empresas.cl/robots.txt<br> See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html for explanation.<br> 404s are either dead links or something looked like a link to PhpDig so PhpDig tried to crawl it.<br> <br> -------------------------------------------------------------------------- This is a part of my spider.log... I have a cronlist.txt file containing a list of urls, but in all sites appear this message and 0 pages indexed. My spidering settings are: Search Depth:5 Link per:10 Days After:7 Reindex depth:5
__________________
http://claudiomet.phpnet.us Last edited by claudiomet; 08-31-2004 at 01:09 PM. |
08-31-2004, 02:50 PM | #2 |
Purple Mole
Join Date: Jan 2004
Posts: 694
|
That is just a warning message, and nothing to be concerned about. Check out this thread.
|
09-01-2004, 06:07 AM | #3 |
Green Mole
Join Date: Aug 2004
Location: Chile
Posts: 13
|
Oh! Thanks...
but viewing the shell console appear this messages: PHP Warning: eregi() REG_EMPTY in /home/.../robot_functions.php on line 1252 what means this? ... I have a lot of this messages.
__________________
http://claudiomet.phpnet.us |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
pages indexed | jmitchell | The Mole Hole | 8 | 02-15-2005 12:23 PM |
404 error although page exists | hendrix | Troubleshooting | 13 | 02-15-2005 05:58 AM |
Shell Spidering Quits After Indexing A Few Pages | vinyl-junkie | Troubleshooting | 11 | 10-09-2004 06:08 PM |
Error 404 Continuous on re-index | jacobwoods | Troubleshooting | 5 | 06-04-2004 03:19 PM |
404 Errors & only 1 page indexed | steve_true | Troubleshooting | 4 | 05-20-2004 05:10 AM |