Duplicate/Similar search results?

ChadK · 08-20-2004, 04:52 AM

When you search on my site for something like "Macaw" you will get a bunch of results. The problem is, many of these results are the same thing with slightly different URLs. You may see:

URL/index.php?showthread=1
this is the snippet with macaw in it

URL/index.php?markthread=1
this is the snippet with macaw in it

URL/index.php?reply=1
this is the snippet with macaw in it

URL/index.php?notification=1
this is the snippet with macaw in it

and so on... they're all essentially the same page it's just showing the links to actions that can be taken on those pages because those links also show the original post in the action. So if you click REPLY, you see the original post... but it's a unique URL so it's added twice.

Any ideas on how to eliminate this problem?
Maybe we could add something that forces a duplicate flag to mark it as a duplicated page?
MessageIDTags=="showtopic,t";
then as the spider works it's magic, it checks for "showtopic" and "t" in the url and if it's found, it checks the value of "showtopic" or "t" and uses it as a "unique index"? So two urls like this:
URL/index.php?showtopic=12?reply=true
and
URL/index.php?showtopic=12?reply=false
would be marked as "duplicate" and only one would be saved.

Charter · 08-20-2004, 05:59 AM

Check for PHPDIG_SESSID_VAR in the config file.

ChadK · 08-20-2004, 06:02 AM

Yes but that doesn't address things like TOPIC Id's does it? Isn't that for .. well, session ids?

Charter · 08-20-2004, 06:07 AM

sid and/or variable...

PHP Code:


			
define('PHPDIG_SESSID_VAR','PHPSESSID,s');  

// name of SID or variable to remove

// can be 's' or comma delimited 's,id,var,foo,etc'

08-20-2004, 04:52 AM	#1
ChadK Green Mole Join Date: May 2004 Posts: 23	Duplicate/Similar search results? When you search on my site for something like "Macaw" you will get a bunch of results. The problem is, many of these results are the same thing with slightly different URLs. You may see: URL/index.php?showthread=1 this is the snippet with macaw in it URL/index.php?markthread=1 this is the snippet with macaw in it URL/index.php?reply=1 this is the snippet with macaw in it URL/index.php?notification=1 this is the snippet with macaw in it and so on... they're all essentially the same page it's just showing the links to actions that can be taken on those pages because those links also show the original post in the action. So if you click REPLY, you see the original post... but it's a unique URL so it's added twice. Any ideas on how to eliminate this problem? Maybe we could add something that forces a duplicate flag to mark it as a duplicated page? MessageIDTags=="showtopic,t"; then as the spider works it's magic, it checks for "showtopic" and "t" in the url and if it's found, it checks the value of "showtopic" or "t" and uses it as a "unique index"? So two urls like this: URL/index.php?showtopic=12?reply=true and URL/index.php?showtopic=12?reply=false would be marked as "duplicate" and only one would be saved.

08-20-2004, 05:59 AM	#2
Charter Head Mole Join Date: May 2003 Posts: 2,539	Check for PHPDIG_SESSID_VAR in the config file. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

08-20-2004, 06:07 AM	#4
Charter Head Mole Join Date: May 2003 Posts: 2,539	sid and/or variable... PHP Code: `define('PHPDIG_SESSID_VAR','PHPSESSID,s'); // name of SID or variable to remove // can be 's' or comma delimited 's,id,var,foo,etc'` __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Restricting search results by URL at the search form level	innerfire	How-to Forum	3	08-01-2005 08:36 AM
No most searched terms, biggest results, most 0 results, last search queries, etc.	jongag1	How-to Forum	6	04-22-2005 10:43 AM
CVS or similar file system for the code	blueyed	The Mole Hole	0	12-13-2004 04:45 AM
Indexing duplicate descriptions and keywords causing false search results	jerrywin5	Mod Requests	3	05-04-2004 08:27 AM
'Duplicate' Search Results	siliconkibou	Troubleshooting	1	01-13-2004 08:00 AM

08-20-2004, 06:02 AM	#3
ChadK Green Mole Join Date: May 2004 Posts: 23	Yes but that doesn't address things like TOPIC Id's does it? Isn't that for .. well, session ids?