PhpDig.net - View Single Post - indexing for the 1st time but getting "duplicate of existing doc" msg with some files

rAdoN · 12-30-2004, 03:03 PM

remove mods - do virgin 1.8.6 install - use CHUNK_SIZE 2048 or 4096 - use dynamic title - use LIMIT_TO_DIRECORY false - use SPIDER_MAX_LIMIT 100 - use LINKS_MAX_LIMIT 100 - use search depth 100 - use links per 0 - links in iframe or heavy javascript not followed - index after setting config - expect duplicate of an existing document at high depth - may be real duplicate - same link is duplicate - see

http://www.phpdig.net/forum/showthread.php?t=1139

relax - know not what more

12-30-2004, 03:03 PM	#10
rAdoN Green Mole Join Date: Oct 2004 Posts: 27	remove mods - do virgin 1.8.6 install - use CHUNK_SIZE 2048 or 4096 - use dynamic title - use LIMIT_TO_DIRECORY false - use SPIDER_MAX_LIMIT 100 - use LINKS_MAX_LIMIT 100 - use search depth 100 - use links per 0 - links in iframe or heavy javascript not followed - index after setting config - expect duplicate of an existing document at high depth - may be real duplicate - same link is duplicate - see http://www.phpdig.net/forum/showthread.php?t=1139 relax - know not what more __________________ rAdoN was here