Samar
07-31-2006, 11:43 AM
I have a website in english and french and I want to have 2 phpdig installation, one to search the french pages and one for the english pages.
I want to block access to the english pages to the french phpdig and vice versa in my robots.txt file.
For example my robots.txt would look like :
#phpdig for english pages
User-agent: PhpDig
Disallow: /fr/
#phpdig for french pages
User-agent: PhpDigFr
Disallow: /en/
I have read some post on how to change the user-agent name and I've changed the robot_functions.php and spider.php files without success.
Any help would be much appreciated.
I want to block access to the english pages to the french phpdig and vice versa in my robots.txt file.
For example my robots.txt would look like :
#phpdig for english pages
User-agent: PhpDig
Disallow: /fr/
#phpdig for french pages
User-agent: PhpDigFr
Disallow: /en/
I have read some post on how to change the user-agent name and I've changed the robot_functions.php and spider.php files without success.
Any help would be much appreciated.