Hi. Please run the following. It may help me determine the problem.
PHP Code:
<?php
$site = 'http://user:pass@www.vdoh.de/';
$robots = file($site.'robots.txt');
for ($i=0; $i<count($robots); $i++) {
echo $robots[$i].'<br>';
}
?>
What do you get? I get the following when viewing the HTML source:
Code:
User-agent: *
<br>Disallow:<br>
Also, do the username and password that you are using in the URL match those that are in the sites table for this site?