View Full Version : Spidering problem please help
I am trying to spider certian urls like http://cgi.ebay.com/Sweetie-Pies-Baby-Pie-Diaper-Cake_W0QQitemZ5641413114QQcategoryZ102376QQssPageNameZWDVWQQrdZ1QQcmdZViewI tem
and when it tryes to crawl it it adds a / to the end of the url which makes the url a 404 how would i fix this issue.
thanks in advance
KaZ
This isnt advertising that is not my link it is just an example i am creating a auction search website and need to crawl auction listings from all auction sites only site im having an issue with so far is Ebay the auction listing URl's are weird and if you add a trailing slash to them it makes the link a 404 but it seems to only do it on these urls other urls like http://auctionsitetest.com/auctiondetails.php?id=222 get listed fine with no trailing slash but for some reason only the ebay links that i have encountered so far do this any suggestions please?
vBulletin® v3.7.3, Copyright ©2000-2025, Jelsoft Enterprises Ltd.