In spider.php there's a line that says sleep(5). This makes the spider pause for five seconds in between pages, and while it's good to leave in when spidering other people's sites, if you're spidering your own you could just take it out completely.
If I remember correctly, there's another sleep(2) call somewhere in there, too.
|