View Single Post
Old 07-12-2004, 09:02 AM   #4
bloodjelly
Purple Mole
 
Join Date: Dec 2003
Posts: 106
Hi Niall -

There is a bit of "sleep" code in spider.php that prevents phpDig from requesting pages too quickly from web hosts. You can find this line and change the sleep time of 5 seconds between links to your choosing:
PHP Code:
 // Spidering ...
while($level <= $limit) {
     
sleep(5); 
This will help make spidering faster, as long as the site your spidering doesn't mind the increased load.
bloodjelly is offline   Reply With Quote