PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > How-to Forum

Reply
 
Thread Tools
Old 12-14-2003, 09:33 AM   #1
fr :: anonymus
Green Mole
 
fr :: anonymus's Avatar
 
Join Date: Dec 2003
Location: Lyon, France
Posts: 17
Number of ' spider ' simultaneous

Hi,

i would like to know how many spider can we launch simultaneously. So : Is it possible, and no damageous to have two windows like that ..../admin/spider.php?site_id=........ ?

Thanks.
fr :: anonymus is offline   Reply With Quote
Old 12-14-2003, 05:31 PM   #2
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. Assuming you mean to use the same database tables, I'm not sure that PhpDig will 'mutlithread' appropriately, especially if two spiders hit the same site. If you are looking to crawl somewhat faster, I would suggest crawling from the command line to circumvent the browser interface.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 01-24-2004, 11:06 AM   #3
fzxdude
Green Mole
 
Join Date: Sep 2003
Posts: 4
I have about 450 domains spidered with it so far ... run up to about 12 here with no troubles ... things do slow down (which isn't so bad cuz I'm not hammering the other sites so hard)
fzxdude is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Number of pages indexed claudiomet How-to Forum 0 08-30-2004 03:26 PM
Set time limit on spider.php or number of pages paullind Troubleshooting 1 05-01-2004 08:25 AM
pages number limited indexing J Gius│ Mod Requests 4 01-13-2004 12:17 PM
Number of links found bloodjelly How-to Forum 2 01-09-2004 10:50 AM
Reduce number of connections druesome Troubleshooting 1 10-14-2003 08:42 AM


All times are GMT -8. The time now is 03:27 PM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.