PhpDig.net - View Single Post - QUESTION: How-to Spider Multiple URL's, not just one at a time.

2wheelin · 05-22-2004, 06:00 PM

Is there any way to spider Multiple URL's instead of all the pages in a single URL?

I need to index subject specific web sites, the top or index page of each site only. If phpDig will do this, please tell me how to do it or better yet, direct me to an existing thread with this info.

THANKS!

05-22-2004, 06:00 PM	#1
2wheelin Green Mole Join Date: Apr 2004 Posts: 3	QUESTION: How-to Spider Multiple URL's, not just one at a time. Is there any way to spider Multiple URL's instead of all the pages in a single URL? I need to index subject specific web sites, the top or index page of each site only. If phpDig will do this, please tell me how to do it or better yet, direct me to an existing thread with this info. THANKS!