PDA

View Full Version : Will not follow links


td234
01-17-2005, 02:44 PM
I feel really stupid that I can install this but I can not seem to make it work. I have just installed 1.8.7. Connect to DB fine. When I enter a URL to spider, it only crawls one page. I have tried a couple of URLs and get the same results. When entering the URL I set depth to 20 and links to 20 as I want it to crawl as deep as possible. I also have tried it from the command line and get the same results:

[prompt]# php -f ./spider.php http://www.fake_domain.com/destinations/
6485: old priority 0, new priority 18
Spidering in progress...
-----------------------------
SITE : http://www.fake_domain.com/
Exclude paths :
- @NONE@
XDuplicate of an existing document
1:http://www.fake_domain.com/destinations/
(time : 00:00:09)

No link in temporary table
links found : 1
Optimizing tables...
Indexing complete !

Can someone please tell me what I am doing wrong.

Thanks

T

Warmbells
01-17-2005, 02:57 PM
Try here

http://www.phpdig.net/forum/showthread.php?t=1577&highlight=links+found

drywall
01-27-2005, 05:37 AM
I've been running phpdig for a while on on our site, and despite some command-line issues, it's been working just fine in spidering our site. But when I try to get it to spider a second site, it comes up with nothing (as described above). Any idea why the spider would work for one site and not another?