View Single Post
Old 01-10-2005, 10:53 PM   #1
djavet
Orange Mole
 
Join Date: Jan 2005
Posts: 31
robots.txt and URL

Hello,

I've a robot.txt file wich work perfectly with PhpDig.
Notice the forum folder into:
Code:
User-agent: *
Disallow: /cgi-bin/
Disallow: /flash/
Disallow: /forum/
Into my forum (PhpBB), i've a FAQ section wich I'd index with PhpDig.
It is possible to force a few URLs (in fact 19 entries) when the forum is exclude form indexing?

Thx for your help and time.
Regards; Dominique
djavet is offline   Reply With Quote