View Single Post
Old 08-15-2005, 07:37 AM   #1
influxx
Green Mole
 
Join Date: Jul 2005
Posts: 1
/~username robots.txt issue - With Fix

PhpDig v.1.8.7

Issue:
If a robots.txt file Disallows searching a user directory (/~username) PhpDig interprets this as Disallow: / (everything)

Solution:
In the robot_functions.php in function phpdigReadRobotsTxt on Line 917 change:
Code:
if (eregi('[[:blank:]]*disallow:[[:blank:]]*([/]?([a-z0-9_/*+%.-]*))',$line,$regs))
To:
Code:
if (eregi('[[:blank:]]*disallow:[[:blank:]]*([/]?([a-z0-9_/*+%.-~]*))',$line,$regs))
Note the addition of a '~' in the regular expression.
influxx is offline   Reply With Quote