obottek
08-21-2006, 08:17 AM
If you have an XHTML conform URL like
www.mydomain.com/?id=1&page=2
the URL part behind the & seams to be ignored when spidering. Using the normal & works, but the XHTML conform version not. I have a site where this is used widely and actually no way to remove it there.
Here's a little test site with links (language switching) using these links:
http://test.mg02.serverarea.com/
http://test.mg02.serverarea.com/?&L=2 - not spidered
http://test.mg02.serverarea.com/?&L=2 - spidered
In this example the spider ignores the L=2 which make the pages equal to L=0 (the default) which makes him saying "already indexed".
I thought that this bug was existing in an earlier version already, but can not find the item back. It's a least exisiting in 1.8.9 RC1, 1.8.4 and 1.8.2.
Any ideas?
Greetings,
Olaf
www.mydomain.com/?id=1&page=2
the URL part behind the & seams to be ignored when spidering. Using the normal & works, but the XHTML conform version not. I have a site where this is used widely and actually no way to remove it there.
Here's a little test site with links (language switching) using these links:
http://test.mg02.serverarea.com/
http://test.mg02.serverarea.com/?&L=2 - not spidered
http://test.mg02.serverarea.com/?&L=2 - spidered
In this example the spider ignores the L=2 which make the pages equal to L=0 (the default) which makes him saying "already indexed".
I thought that this bug was existing in an earlier version already, but can not find the item back. It's a least exisiting in 1.8.9 RC1, 1.8.4 and 1.8.2.
Any ideas?
Greetings,
Olaf