Hello,
I've a robot.txt file wich work perfectly with PhpDig.
Notice the forum folder into:
Code:
User-agent: *
Disallow: /cgi-bin/
Disallow: /flash/
Disallow: /forum/
Into my forum (PhpBB), i've a FAQ section wich I'd index with PhpDig.
It is possible to force a few URLs (in fact 19 entries) when the forum is exclude form indexing?
Thx for your help and time.
Regards; Dominique