PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > How-to Forum

Reply
 
Thread Tools
Old 02-18-2004, 05:47 PM   #1
bsw114
Green Mole
 
Join Date: Feb 2004
Posts: 3
sleep(5) in spider.php?

Anybody know why the sleep(5) is in the big loop in spider.php? I took it out and surprisingly saved 5 seconds per file indexed. Besides the obvious consequences of not giving the CPU a rest, is there any good reason to have a sleep?
bsw114 is offline   Reply With Quote
Old 02-19-2004, 05:40 AM   #2
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. If you crawl sites that are not yours, setting a delay is the courteous thing to do so that you do not place too high of a load on the other person's machine.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 03-12-2004, 01:31 PM   #3
Konstantine
Green Mole
 
Join Date: Mar 2004
Location: Russia
Posts: 21
Why don't make a settings for that? for example "force indexing!" And when it's on, indexing is speeded up. Or force every 5 urls and sleep after that for a 5 secs, then again... and so on...
Konstantine is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
autorefresh spider.php zaartix Mod Requests 0 12-21-2005 09:11 PM
Change sleep noel How-to Forum 1 10-27-2005 09:58 AM
Sleep in the code davids211082 How-to Forum 2 05-18-2005 11:22 PM
I have one bug with spider.php Booboo Troubleshooting 1 03-01-2005 01:46 AM
spider.php via bash tomas Troubleshooting 16 02-07-2004 04:23 PM


All times are GMT -8. The time now is 11:10 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.