[boomboom100]

Hi

Thanks for the reply, Charter.

I tried replacing the line but am getting parse errors. Here is the function from the original robots_functions.php:

PHP Code:

//=================================================
//search robots.txt in a site
function phpdigReadRobotsTxt($site)  //don't forget the end backslash
{
if (phpdigTestUrl($site.'robots.txt') == 'PLAINTEXT')
     {
     $robots = file($site.'robots.txt');
     while (list($id,$line) = each($robots))
            {
            if (ereg('^user-agent:[ ]*([a-z0-9*]+)',strtolower($line),$regs))
                {
                     $user_agent = $regs[1];
                }
            if (eregi('[[:blank:]]*disallow:[[:blank:]]*(/([a-z0-9_/*+%.-]*))',$line,$regs))
                {
                if ($regs[1] == '/')
                     {
                     $exclude[$user_agent]['@ALL@'] = 1;
                     }
                else
                     {
                     $exclude[$user_agent][str_replace('*','.*',str_replace('+','\+',str_replace('.','\.',$regs[2])))] = 1;
                     }
                }
            elseif (($user_agent == 'phpdig') && (eregi('[[:blank:]]*disallow:[[:blank:]]*',$line,$regs)))
                {
                $exclude['@NONE@'] = 1;
                return $exclude;
                }
            }
     if (isset($exclude['phpdig']) && is_array($exclude['phpdig']))
         return $exclude['phpdig'];
     elseif (isset($exclude['*']) && is_array($exclude['*']))
         return $exclude['*'];
     }
$exclude['@NONE@'] = 1;
return $exclude;
}

//================================================= 


How should I edit this block?

Thanks again.