PhpDig.net - View Single Post

Charter · 12-24-2003, 05:32 PM

Hi. Add the following to the top of the robots.txt file and then make the code change listed in this thread.

Code:

User-agent: PhpDig
Disallow:
# whatever else below this

This should let PhpDig follow all the links it finds, and then you can go and delete/exclude certain links/directories from the admin panel.

12-24-2003, 05:32 PM	#17
Charter Head Mole Join Date: May 2003 Posts: 2,539	Hi. Add the following to the top of the robots.txt file and then make the code change listed in this thread. Code: User-agent: PhpDig Disallow: # whatever else below this This should let PhpDig follow all the links it finds, and then you can go and delete/exclude certain links/directories from the admin panel. __________________ Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.