PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 05-04-2005, 06:31 AM   #1
ossi01
Green Mole
 
Join Date: May 2005
Location: Munic
Posts: 3
Question Indexing are very slow, how can i speed up?

The indexing of my htm-files is very slow and i have over 14500 of it.

(6 seconds per page = 1450 minutes = over 24 hours for one run,
the spider.php runs local on the same hard disk, where the htm-files,
the CPU utilization is below 1%, from HD are reading below 129kb/s)

But, i must reindex my htm-files every 3 hours.

How can i speed up the indexing? Please help me.
ossi01 is offline   Reply With Quote
Old 05-04-2005, 07:16 AM   #2
bloodjelly
Purple Mole
 
Join Date: Dec 2003
Posts: 106
In spider.php there's a line that says sleep(5). This makes the spider pause for five seconds in between pages, and while it's good to leave in when spidering other people's sites, if you're spidering your own you could just take it out completely.

If I remember correctly, there's another sleep(2) call somewhere in there, too.
bloodjelly is offline   Reply With Quote
Old 05-06-2005, 03:26 AM   #3
ossi01
Green Mole
 
Join Date: May 2005
Location: Munic
Posts: 3
Yep, that is it.
Now i need less than 1 second per page...
(my new (maximum) reindex-time for 15037 links: 5h 12min)

Thanx
ossi01 is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Slow indexing Alex-FBTech Troubleshooting 0 02-27-2008 05:30 AM
How to increase indexing speed sf44 How-to Forum 0 01-28-2005 03:57 AM
Indexing slow.... no, _really_ slow bluntman Troubleshooting 1 09-24-2004 01:23 PM
Very Slow Indexing airplay Troubleshooting 2 03-09-2004 02:20 PM
Again, indexing speed Richie Troubleshooting 1 01-18-2004 11:21 PM


All times are GMT -8. The time now is 03:11 PM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.