|
- and also an addition to the question from tryangle - how can I limit the spider to only spider the pages that I list in a text file fed to the spider?
I've already tried changing the default level of spider recursion to be 0. But, still, the spider is very eager to index everything it finds!
|