Go Back > PhpDig Forums > How-to Forum

Thread Tools
Old 03-25-2004, 11:23 PM   #1
Green Mole
Join Date: Mar 2004
Posts: 19
speed of search and filter out double results

I'm running 3 versions of PHPDIG 180 in different dirs.
All versions use the same DB, work good!
I can spider more than one URL at the same time, work good also.
If I do a search on the same time(with spidering) the speed of search is sometimes ferry slow,
and my server go slow. Depend of the results I get of the query.
Results 1-10, 1475 total, on "vermeer" (25.41 seconds)
Sometimes I have also a lot of double results of the same url.

I'm running regular "clean index" I'm a bit careful to run "clean directory".
I dont no of I lose results that I want to keep.

How can I speed up this a bit?
And how can I filter of double results?

marb is offline   Reply With Quote
Old 03-29-2004, 11:38 AM   #2
Head Mole
Charter's Avatar
Join Date: May 2003
Posts: 2,539
Hi. Searching while spidering can be slow, as spidering, especially with three running, can be intense. Perhaps this thread might help. Also, for double results, PhpDig doesn't specifically account for multithreading issues. Perhaps try adding a column to the tempspider table indictaing whether a link has been grabbed and modify the code accordingly.
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Language filter Dayman Feedback & News 1 07-03-2008 01:26 AM
Blank results & Speed Dave A Troubleshooting 0 06-30-2007 02:05 AM
How to filter your searches yveslebeau How-to Forum 1 05-22-2006 11:01 PM
Search times and speed Dave A The Mole Hole 6 03-20-2005 09:59 AM
double install marb How-to Forum 1 03-10-2004 03:40 PM

All times are GMT -8. The time now is 01:21 AM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.