![]() |
not spidering all pages (too many links on page?)
I have trouble getting all my pages spidered.
I don't think it has to do with levels, since it does find some pages at deeper levels. However it doesn't spider all links on pages. I was thinking it might have to do with the LINKS_MAX_LIMIT being 20. In config.php I've tried changing LINKS_MAX_LIMIT to 100, but that doesn't seem to make any difference. Any idea? |
mirdin, i think that if you leave the box "links per:" at 0 this mean for spider
to follow all the links in the page. |
That's what I thought also, I did leave it at 0 a couple of times and that didn't work.
After trying a few times with limit=10 en perlinks=100 it worked, so no more problems here :) |
All times are GMT -8. The time now is 03:44 AM. |
Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.