View Single Post
Old 08-31-2004, 12:34 PM   #1
Green Mole
claudiomet's Avatar
Join Date: Aug 2004
Location: Chile
Posts: 13
404 error via shell... no pages indexed

When spidering via web I had no problems, the spider found links and index the pages, but when I do it via ssh:
HTTP/1.1 404 Not Found
See for explanation.<br>
404s are either dead links or something looked like a link to PhpDig so PhpDig tried to crawl it.<br>
This is a part of my spider.log... I have a cronlist.txt file containing a list of urls, but in all sites appear this message and 0 pages indexed. My spidering settings are:

Search Depth:5
Link per:10
Days After:7
Reindex depth:5

Last edited by claudiomet; 08-31-2004 at 01:09 PM.
claudiomet is offline   Reply With Quote