PhpDig.net - View Single Post

X-Warlock · 05-04-2005, 07:39 AM

I've got a fresh install of phpdig on my server, the site's links are all properly linked and should be totally accessible, and I've tried numerous combinations on the search depth and links per, even as far as modifying the config to allow me a search and links setting of 100 each. But so far the absolute best I have been able to manage to index is less than 40 pages, on a site that should be returning several hundred.

What's causing it to just stop, and is there any way I can fix this or get around it and have it index the site properly?

05-04-2005, 07:39 AM	#1
X-Warlock Green Mole Join Date: May 2005 Posts: 3	Spider Crawl stopping too early I've got a fresh install of phpdig on my server, the site's links are all properly linked and should be totally accessible, and I've tried numerous combinations on the search depth and links per, even as far as modifying the config to allow me a search and links setting of 100 each. But so far the absolute best I have been able to manage to index is less than 40 pages, on a site that should be returning several hundred. What's causing it to just stop, and is there any way I can fix this or get around it and have it index the site properly?