I feel really stupid that I can install this but I can not seem to make it work. I have just installed 1.8.7. Connect to DB fine. When I enter a URL to spider, it only crawls one page. I have tried a couple of URLs and get the same results. When entering the URL I set depth to 20 and links to 20 as I want it to crawl as deep as possible. I also have tried it from the command line and get the same results:
[prompt]# php -f ./spider.php
http://www.fake_domain.com/destinations/
6485: old priority 0, new priority 18
Spidering in progress...
-----------------------------
SITE :
http://www.fake_domain.com/
Exclude paths :
- @NONE@
XDuplicate of an existing document
1:
http://www.fake_domain.com/destinations/
(time : 00:00:09)
No link in temporary table
links found : 1
Optimizing tables...
Indexing complete !
Can someone please tell me what I am doing wrong.
Thanks
T