View Single Post
Old 12-23-2003, 03:36 PM   #4
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Hi. If you've already crawled directories that start with an underscore, you can delete them from the admin panel by clicking the site, the update button, and then the appropriate red X symbol.

To exclude all folders that start with an underscore, the robots.txt file can be like so (wildcards are nonstandard):

User-agent: PhpDig
Disallow: /_
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote