PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > How-to Forum

Reply
 
Thread Tools
Old 12-16-2004, 12:57 PM   #1
jmitchell
Orange Mole
 
Join Date: Dec 2004
Location: Tennessee
Posts: 60
Question Multiple Spiders

Hi there,

I have just installed phpdig 2 days and I'm very excited about using it! I have been spidering for almost 2 days, and I would like to know, is there any way to run multiple spiders?

I have seen several threads asking the same question, and some of them had answers, but can I get some tested code that is being used currently for multiple spiders?

thanks

J Mitchell
jmitchell is offline   Reply With Quote
Old 12-16-2004, 03:15 PM   #2
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Maybe you can get an idea from this thread or try using multiple browser windows.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 12-16-2004, 04:52 PM   #3
jmitchell
Orange Mole
 
Join Date: Dec 2004
Location: Tennessee
Posts: 60
so, I can have multiple browsers on multiple pcs running the spider at the same time and it will work?
jmitchell is offline   Reply With Quote
Old 12-16-2004, 05:43 PM   #4
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
If you are going to use multiple browsers, then you might want to comment out the following line in the spider.php file:
PHP Code:
array_unshift($list_sites,$site_datas); 
If you are going to use shell, then you might want to send in only one site per process.

Either way, you should test it out on some of your demo sites and see if it works, whether you need to do other modifications to the code, etcetera.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
wrapper/multiple spiders... CentaurAtlas Mod Submissions 5 11-19-2006 01:07 PM
Locked one of my spiders? megabaz Troubleshooting 4 10-23-2004 05:37 PM
1.8.3 spiders slow, 1.6.3 spiders same site fast Wayne McBryde Troubleshooting 0 09-21-2004 08:10 PM
Multiple spiders tryangle How-to Forum 3 04-24-2004 03:43 AM
2 or more Spiders firestarter Mod Requests 2 03-29-2004 12:59 PM


All times are GMT -8. The time now is 02:23 PM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.