PhpDig.net - View Single Post

Charter · 03-10-2004, 10:32 AM

Hi. A robots.txt file with the following should exclude the directories from PhpDig prior to index:

Code:

User-agent: PhpDig
Disallow: /developers/
Disallow: /developers/community/forums/
Disallow: /lps/
Disallow: /lps-2.0/docs/lzx-developers-guide/

If a directory that you don't want indexed has already been indexed, just click the red circle to delete and exclude it, making sure that the tempspider table is empty prior to reindex.

03-10-2004, 10:32 AM	#4
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. A robots.txt file with the following should exclude the directories from PhpDig prior to index: Code: User-agent: PhpDig Disallow: /developers/ Disallow: /developers/community/forums/ Disallow: /lps/ Disallow: /lps-2.0/docs/lzx-developers-guide/ If a directory that you don't want indexed has already been indexed, just click the red circle to delete and exclude it, making sure that the tempspider table is empty prior to reindex. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.