Go Back > PhpDig Forums > How-to Forum

Thread Tools
Old 12-14-2003, 08:33 AM   #1
fr :: anonymus
Green Mole
fr :: anonymus's Avatar
Join Date: Dec 2003
Location: Lyon, France
Posts: 17
Number of ' spider ' simultaneous


i would like to know how many spider can we launch simultaneously. So : Is it possible, and no damageous to have two windows like that ..../admin/spider.php?site_id=........ ?

fr :: anonymus is offline   Reply With Quote
Old 12-14-2003, 04:31 PM   #2
Head Mole
Charter's Avatar
Join Date: May 2003
Posts: 2,539
Hi. Assuming you mean to use the same database tables, I'm not sure that PhpDig will 'mutlithread' appropriately, especially if two spiders hit the same site. If you are looking to crawl somewhat faster, I would suggest crawling from the command line to circumvent the browser interface.
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 01-24-2004, 10:06 AM   #3
Green Mole
Join Date: Sep 2003
Posts: 4
I have about 450 domains spidered with it so far ... run up to about 12 here with no troubles ... things do slow down (which isn't so bad cuz I'm not hammering the other sites so hard)
fzxdude is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Number of pages indexed claudiomet How-to Forum 0 08-30-2004 02:26 PM
Set time limit on spider.php or number of pages paullind Troubleshooting 1 05-01-2004 07:25 AM
pages number limited indexing JÿGius³ Mod Requests 4 01-13-2004 11:17 AM
Number of links found bloodjelly How-to Forum 2 01-09-2004 09:50 AM
Reduce number of connections druesome Troubleshooting 1 10-14-2003 07:42 AM

All times are GMT -8. The time now is 08:56 PM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.