PhpDig.net - View Single Post

drywall · 05-17-2004, 10:20 AM

I'd like to expand on this problem a little bit, in case anyone feels like tackling it. I'm indexing a reasonably complicated site and I've noticed that in some cases it's managing to index dynamic pages with different numbers in their GET string, but not others.

I'm not sure about this, but it appears to only be able to grab one per page. For example, on http://www.freepress.net/news/releases.php, it will only spider the first release on the list (ID 17). However, it appear to be spidering several news article pages (which have urls of the form news/article.php?id=XXXX), because it's finding them via separate pages, rather than on a single page as with the press releases.

Or maybe it's dying simply because it stops looking at the releases once it hits the word doc? Not sure... but it's fishy, and frustrating.

05-17-2004, 10:20 AM	#5
drywall Green Mole Join Date: May 2004 Posts: 25	I'd like to expand on this problem a little bit, in case anyone feels like tackling it. I'm indexing a reasonably complicated site and I've noticed that in some cases it's managing to index dynamic pages with different numbers in their GET string, but not others. I'm not sure about this, but it appears to only be able to grab one per page. For example, on http://www.freepress.net/news/releases.php, it will only spider the first release on the list (ID 17). However, it appear to be spidering several news article pages (which have urls of the form news/article.php?id=XXXX), because it's finding them via separate pages, rather than on a single page as with the press releases. Or maybe it's dying simply because it stops looking at the releases once it hits the word doc? Not sure... but it's fishy, and frustrating.