|
bloddjelly - at the moment, i have the respider limit and the default limit both set to be 0. but yet - when i set the spider off again, i find that it continues to visit the old pages which have already been spidered.
i think that it just passes over them quickly - but it's taking 1 or 2 seconds just for looking over these pages, that i don't want to index.
the other setting that i've customized is the default reindex period - which i've set to be 500 days. basically, once something has been indexed, i don't want to return to this page again.
|