How can I make phpdig spider faster

jakeres · 11-02-2004, 01:36 AM

Hi there....

I've installed phpdig and tested with my local site....and it takes almost 5 minutes to scan all 10 pages from site.... WHY?

I've looked over the code ...and see how it does a lot of tests before getting the actual html content....Does fsockopen take so long ?!?

Can you tell me how to speep up things a little bit?!

Thanks.

Kim Steinhaug · 11-29-2004, 11:05 AM

I am wondering on the exact same thing. I havnt had time to study the code to much, since Im kinda "depending" on someone else which handles the project to look into this.... umm..

Well, there might be reasons why the digging is so annoyingly slow - running the dig on a brand new computer with loads of CPU power doesnt really seem to speed it up either...

Any feedback on this would be wonderful, some explernations would also make me feel better since I would then be happy with those, hehe.

Well, okidoki.

11-02-2004, 01:36 AM	#1
jakeres Green Mole Join Date: Nov 2004 Posts: 2	How can I make phpdig spider faster Hi there.... I've installed phpdig and tested with my local site....and it takes almost 5 minutes to scan all 10 pages from site.... WHY? I've looked over the code ...and see how it does a lot of tests before getting the actual html content....Does fsockopen take so long ?!? Can you tell me how to speep up things a little bit?! Thanks.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Can I make the spider stop and start on a dime?	dewed	How-to Forum	0	11-25-2005 11:54 AM
How to make phpdig index certain content, located in certain html tags?!	r3m	How-to Forum	1	11-18-2004 05:27 PM
Make yourself as an avatar	Charter	The Mole Hole	0	07-24-2004 07:34 PM
Optimized search (around 60% faster)	synnalagma	Mod Submissions	6	06-10-2004 03:47 AM
how to make phpdig support GB2313?	slackware	How-to Forum	1	09-20-2003 10:52 AM

11-29-2004, 11:05 AM	#2
Kim Steinhaug Green Mole Join Date: Nov 2004 Posts: 1	I am wondering on the exact same thing. I havnt had time to study the code to much, since Im kinda "depending" on someone else which handles the project to look into this.... umm.. Well, there might be reasons why the digging is so annoyingly slow - running the dig on a brand new computer with loads of CPU power doesnt really seem to speed it up either... Any feedback on this would be wonderful, some explernations would also make me feel better since I would then be happy with those, hehe. Well, okidoki.