PhpDig.net - View Single Post - indexing for the 1st time but getting "duplicate of existing doc" msg with some files

Morphea · 12-30-2004, 01:48 PM

At first I edited my last post, but seeing you replied in the meantime I thought I'd post it in a new message.

So I increased the CHUNK_SIZE (doubled it to 2048), but it still wouldn't work.
My pages are simple php files, with no variables called in the URL like in the exemple in the link provided. They're like entries of a dictionary or an encyclopedia, so each page is quite different from any other (maybe not filesize-wise though).
After changing the CHUNK_SIZE, and trying to index a specific page with its full URL, with depth to 0, the spider didn't even seem to try indexing this one page, but instead began trying to index files in another directory... (and not even the root directory)

I'm sure there HAS to be some way to fix this... But right now I'm quite discouraged...

12-30-2004, 01:48 PM	#9
Morphea Green Mole Join Date: Dec 2004 Posts: 6	At first I edited my last post, but seeing you replied in the meantime I thought I'd post it in a new message. So I increased the CHUNK_SIZE (doubled it to 2048), but it still wouldn't work. My pages are simple php files, with no variables called in the URL like in the exemple in the link provided. They're like entries of a dictionary or an encyclopedia, so each page is quite different from any other (maybe not filesize-wise though). After changing the CHUNK_SIZE, and trying to index a specific page with its full URL, with depth to 0, the spider didn't even seem to try indexing this one page, but instead began trying to index files in another directory... (and not even the root directory) I'm sure there HAS to be some way to fix this... But right now I'm quite discouraged...