Go Back > PhpDig Forums > How-to Forum

Thread Tools
Old 02-18-2004, 06:47 PM   #1
Green Mole
Join Date: Feb 2004
Posts: 3
sleep(5) in spider.php?

Anybody know why the sleep(5) is in the big loop in spider.php? I took it out and surprisingly saved 5 seconds per file indexed. Besides the obvious consequences of not giving the CPU a rest, is there any good reason to have a sleep?
bsw114 is offline   Reply With Quote
Old 02-19-2004, 06:40 AM   #2
Head Mole
Charter's Avatar
Join Date: May 2003
Posts: 2,539
Hi. If you crawl sites that are not yours, setting a delay is the courteous thing to do so that you do not place too high of a load on the other person's machine.
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 03-12-2004, 02:31 PM   #3
Green Mole
Join Date: Mar 2004
Location: Russia
Posts: 21
Why don't make a settings for that? for example "force indexing!" And when it's on, indexing is speeded up. Or force every 5 urls and sleep after that for a 5 secs, then again... and so on...
Konstantine is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
autorefresh spider.php zaartix Mod Requests 0 12-21-2005 10:11 PM
Change sleep noel How-to Forum 1 10-27-2005 10:58 AM
Sleep in the code davids211082 How-to Forum 2 05-19-2005 12:22 AM
I have one bug with spider.php Booboo Troubleshooting 1 03-01-2005 02:46 AM
spider.php via bash tomas Troubleshooting 16 02-07-2004 05:23 PM

All times are GMT -8. The time now is 05:34 PM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.