Don't want to spid some site
 

Hello,
Why PHPDIG don't spid this sites, he gives an error message , and google can spid these sites, there are in his index in totality ?

HTTP/1.1 403 Forbidden - http://www.asacorsica.com/robots.txt
See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html for explanation.

HTTP/1.1 403 Forbidden - http://www.centralhotel.fr/robots.txt
See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html for explanation.

HTTP/1.1 403 Forbidden - http://www.corsicamoto.com/robots.txt
See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html for explanation.

HTTP/1.1 403 Forbidden - http://www.corsica-taxi.com/robots.txt
See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html for explanation.

Have I to change a parameter to make it possible ?


Thank you

Noël

There is nothing to change in PhpDig to make it possible. The 403 Forbidden message is a response from the server telling PhpDig that it cannot access the page.