PhpDig.net

Go Back   PhpDig.net > General Forums > The Mole Hole

Reply
 
Thread Tools
Old 02-10-2005, 10:57 AM   #1
Dave A
Purple Mole
 
Dave A's Avatar
 
Join Date: Aug 2004
Location: North Island New Zealand
Posts: 170
Search times and speed

Heaps of people are presently chatting about search times.
If your version of PHPDIG is running on a normal web host then it is sharing resources with other users include processor time and db access.
I am presently changing over to a dedicted server and the early tests show that search times are a whole heap quicker plus it gives me more scope for creating a larger index without speed issues.
The trouble can be that web hosting companies, get titchy when you start to consume a heap of bandwidth and start to hog the processor usage.
Dave A is offline   Reply With Quote
Old 02-11-2005, 07:31 AM   #2
jmitchell
Orange Mole
 
Join Date: Dec 2004
Location: Tennessee
Posts: 60
I'm running on a dedicated server myself (and love the control that I have) but I'm still getting slow response times.

It could be how many pages I have indexed (60,000), but if so, I'm not sure what I can do to fix that.

any ideas anyone?

JMitchell
__________________
60,000 pages indexed!!!!! http://www.sharemylink.com
jmitchell is offline   Reply With Quote
Old 02-11-2005, 01:24 PM   #3
Dave A
Purple Mole
 
Dave A's Avatar
 
Join Date: Aug 2004
Location: North Island New Zealand
Posts: 170
Spped questions

Yes I also have a heap of pages indexed, when it was on a hosted server and not a standalone, a search for say " Camp sites Auckland " could take up to 44 seconds, in some cases due to the processor usage and further pushed by Multiple searches at the same time.
Since then I found the best way around it was to set up the mysql database in the ram in the host (Cached) and I have set up a small Php code addition that stores previous searches for five days. So that searches are checked via the cache then run across the database.
If you try two searches that are identicle within twenty seconds the old search should remain in your hosts memory and so the time will be a heap shorter for the second search.
Since I am not the brightest bulb in the light house, it may well be worth someone having a look into storing the search results in other ways to see if we can all work out the problem of slow search speeds.
It could be a way of turning phpdig into something really special.
Everyone who uses Phpdig should be looking at ways of combining our knowledge and experiences to help each other, Charter and Vinyl Junky are brilliant and the support they offer, is really neat and we must bear in mind that support isn't guaranteed it's offered freely and must take a heap of time for them.
One thing that I had considered that might be useful would be a simple drop down chart/menu made of boxes with problems that answer yes or no too leading to the possible cure. It may save people time in searching the forums but it's just an idea.

I hope this helps you out, I have popped an email over to your email address found on your web site..

All the very best
Dave Andrews
Dave A is offline   Reply With Quote
Old 02-23-2005, 10:54 PM   #4
Andypale
Green Mole
 
Join Date: Nov 2004
Posts: 2
Wow! 60,000 pages in a website? Do you run amazon.com?
Andypale is offline   Reply With Quote
Old 03-20-2005, 08:25 AM   #5
jmitchell
Orange Mole
 
Join Date: Dec 2004
Location: Tennessee
Posts: 60
no, just trying to get a small search engine going.

Charter, have you thought of programming a search engine that can take several million results? it would probably have to be in something like C or C++.

JMitchell
__________________
60,000 pages indexed!!!!! http://www.sharemylink.com
jmitchell is offline   Reply With Quote
Old 03-20-2005, 09:59 AM   #6
Dave A
Purple Mole
 
Dave A's Avatar
 
Join Date: Aug 2004
Location: North Island New Zealand
Posts: 170
Well I have just upgraded my host to twinned 360gb drives (Mirrored) a gig of RAM and it's running twin 3.2 Gig processors and on a multiple word search (Five words searched) it manages to whiz through an index of seven million in less than a second which is qiute speed increase so it may be something to do with the make and access times of the drives and the memory which is in the host.
The system runs red hat so I am not sure if the operating system may have some influence on it.
But if we can compare notes we may be able to find out what makes some systems slow down.
All the best
Dave Andrews
Dave A is offline   Reply With Quote
Old 03-20-2005, 09:59 AM   #7
bloodjelly
Purple Mole
 
Join Date: Dec 2003
Posts: 106
Have you guys seen this thread: http://www.phpdig.net/forum/showthread.php?t=978 ? It speeds up search queries quite a bit. With 1.8.7 though it doesn't seem to recognize the different types of queries, except for start and exact. Basically it gives the searching job to the MySQL DB, which can perform the functions faster than PHP. I'd be curious how much of a difference it would make for you, JM, since you have so many pages indexed
bloodjelly is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
speed of search and filter out double results marb How-to Forum 1 03-29-2004 11:38 AM
Speed up phpdig? Konstantine Mod Requests 0 03-13-2004 10:31 PM
Again, indexing speed Richie Troubleshooting 1 01-18-2004 11:21 PM
Typical run times... rayvd Troubleshooting 3 11-19-2003 01:20 PM
2 Linux-related articles from today's NY Times maggiemel The Mole Hole 0 08-05-2003 04:14 AM


All times are GMT -8. The time now is 09:58 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.