PDA

View Full Version : Problem with spidering with url tex list :-(


claudiomet
09-01-2004, 02:02 PM
When I start spidering via shell like:

php -f /home/.../spider.php http://www.123.cl

the site is enterely indexed... works perfectly.

But when this url is in a text file with other urls, the site is not indexed... any site is not indexed... (the content is not indexed ...but the urls are registered in the temporaly table with 0 links, My settings are Links per=0, Depth=0)

What's wrong?, the url text file must be in a specific format?

Charter
09-07-2004, 03:32 PM
Hi. The text file should have one URL per line.