PhpDig v.1.8.7
Issue:
If a robots.txt file Disallows searching a user directory (/~username) PhpDig interprets this as Disallow: / (everything)
Solution:
In the robot_functions.php in function phpdigReadRobotsTxt on Line 917 change:
Code:
if (eregi('[[:blank:]]*disallow:[[:blank:]]*([/]?([a-z0-9_/*+%.-]*))',$line,$regs))
To:
Code:
if (eregi('[[:blank:]]*disallow:[[:blank:]]*([/]?([a-z0-9_/*+%.-~]*))',$line,$regs))
Note the addition of a '~' in the regular expression.