Will not follow links
 

I feel really stupid that I can install this but I can not seem to make it work. I have just installed 1.8.7. Connect to DB fine. When I enter a URL to spider, it only crawls one page. I have tried a couple of URLs and get the same results. When entering the URL I set depth to 20 and links to 20 as I want it to crawl as deep as possible. I also have tried it from the command line and get the same results:

[prompt]# php -f ./spider.php http://www.fake_domain.com/destinations/
6485: old priority 0, new priority 18
Spidering in progress...
-----------------------------
SITE : http://www.fake_domain.com/
Exclude paths :
- @NONE@
XDuplicate of an existing document
1:http://www.fake_domain.com/destinations/
(time : 00:00:09)

No link in temporary table
links found : 1
Optimizing tables...
Indexing complete !

Can someone please tell me what I am doing wrong.

Thanks

T

Try here

http://www.phpdig.net/forum/showthre...ht=links+found

I've been running phpdig for a while on on our site, and despite some command-line issues, it's been working just fine in spidering our site. But when I try to get it to spider a second site, it comes up with nothing (as described above). Any idea why the spider would work for one site and not another?