PhpDig.net - View Single Post

Wayne McBryde · 09-24-2004, 01:23 PM

If you post the URL of the site, I will try spidering it with PHPDig versions 1.6.3 & 1.8.3 and post back the time per page to spider. Of course I don't plan to spider the several thousand pages you got but I will spider 10 to 20 pages to get an average.

I currently spider over 1000 sites for my database and some sites do spider slow like the page per minute you talked about. Others will spider faster in the range of 3 to 4 seconds per page. I always guessed the slow sites were slow because of the server they were running on.

09-24-2004, 01:23 PM	#2
Wayne McBryde Orange Mole Join Date: Oct 2003 Location: NC, USA Posts: 34	If you post the URL of the site, I will try spidering it with PHPDig versions 1.6.3 & 1.8.3 and post back the time per page to spider. Of course I don't plan to spider the several thousand pages you got but I will spider 10 to 20 pages to get an average. I currently spider over 1000 sites for my database and some sites do spider slow like the page per minute you talked about. Others will spider faster in the range of 3 to 4 seconds per page. I always guessed the slow sites were slow because of the server they were running on. __________________ Wayne Mcbryde http://LakeNormansWeb.com We search all of Lake Norman!