Infos about multiple spider change

noel · 11-11-2005, 02:18 PM

Hi CHARTER,

When using multiple spider with browser, why comment this line :

array_unshift($list_sites,$site_datas);

Thank You

Noel

Charter · 11-11-2005, 03:19 PM

The thought was that, if you are trying to spider using multiple browsers, you don't want one browser adding domains to the list that the other browser is already indexing, or you might find yourself in a quasi-loop.

noel · 11-11-2005, 04:16 PM

OK I will do the necessary in order to not be in a "quasi-loop" ;-)

11-11-2005, 02:18 PM	#1
noel Orange Mole Join Date: Aug 2005 Posts: 44	Infos about multiple spider change Hi CHARTER, When using multiple spider with browser, why comment this line : array_unshift($list_sites,$site_datas); Thank You Noel

11-11-2005, 03:19 PM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	The thought was that, if you are trying to spider using multiple browsers, you don't want one browser adding domains to the list that the other browser is already indexing, or you might find yourself in a quasi-loop. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Change phpdigExclude to not spider Links as well as text	gkdeveloper	How-to Forum	0	09-12-2007 03:54 AM
Spider indexing/request speed/rate - How to change?	JAB Creations	How-to Forum	1	09-07-2005 05:46 AM
Change Identity of Spider	Kishin	How-to Forum	1	07-18-2005 04:03 PM
multiple crawlers	searchboy	How-to Forum	1	09-11-2004 06:10 AM
QUESTION: How-to Spider Multiple URL's, not just one at a time.	2wheelin	How-to Forum	4	06-13-2004 10:42 PM

11-11-2005, 04:16 PM	#3
noel Orange Mole Join Date: Aug 2005 Posts: 44	OK I will do the necessary in order to not be in a "quasi-loop" ;-)