View Single Post
Old 03-09-2004, 01:48 PM   #1
airplay
Green Mole
 
Join Date: Mar 2004
Posts: 7
Very Slow Indexing

Hi, this is my first attempt at using php dig.
I have a site that has around 32,000 pages.
When I first ran the spider it ran overnight and had around 10,000 pages indexed when I woke up.
I notced there was a long time inbetween pages (around 6 seconds). I found the "sleep" in your code so I went ahead and changed it to 2 seconds.

I then cleared out all of the tables and started over.
This sped up the process at first, but now it has been 24 hours and the darn thing is still running.
Is this normal for it to take this long to index 32,000 pages?

Any ideas?

Thanks!
Airplay....
airplay is offline   Reply With Quote