sleep(5) in spider.php?
Anybody know why the sleep(5) is in the big loop in spider.php? I took it out and surprisingly saved 5 seconds per file indexed. Besides the obvious consequences of not giving the CPU a rest, is there any good reason to have a sleep?
|