PhpDig.net - View Single Post

sf44 · 01-28-2005, 02:56 AM

I have prepared all files I need to index in special directory.
Is there any possibility to setup spider to index only this special directory files?
I am possible to add all file names to database - tempspider table.

Why to do this? Because spidering many special selected files using external urls but also http://localhost is too slow - it is about 15s for one page. I have changed everything to boost spidering proces, but all I have done is decreasing time from 20s to 15s for one page - it's still too much time.

01-28-2005, 02:56 AM	#1
sf44 Green Mole Join Date: Jul 2004 Posts: 9	How to index only local directory files? I have prepared all files I need to index in special directory. Is there any possibility to setup spider to index only this special directory files? I am possible to add all file names to database - tempspider table. Why to do this? Because spidering many special selected files using external urls but also http://localhost is too slow - it is about 15s for one page. I have changed everything to boost spidering proces, but all I have done is decreasing time from 20s to 15s for one page - it's still too much time.