hmm, i've tried to put the cookies for
http://umvf.cochin.univ-paris5.fr/ always permitted, but it seems not to change anything in the crawl :
And here is the output of the "access_combined_log" from apache
Quote:
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "HEAD /css/style_uol.css HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "HEAD /css/habillage.css HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "HEAD /css/impression.css HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "HEAD /css/lien.css HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "HEAD /article.php3?id_article=177 HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "GET /article.php3?id_article=177 HTTP/1.0" 302 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "GET /desole.html HTTP/1.0" 200 1036 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "HEAD /menu.css HTTP/1.1" 404 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "HEAD /css/spip_style.css HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "HEAD /rubrique.php3?id_rubrique=99 HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "HEAD /menu.css HTTP/1.1" 404 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "HEAD /css/spip_style.css HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "HEAD /avare3.html HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:37 +0100] "GET /avare3.html HTTP/1.0" 200 61 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:38 +0100] "HEAD /avare2.pdf HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:38 +0100] "HEAD /avare2.pdf HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:38 +0100] "HEAD /ecrire/ HTTP/1.1" 302 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:38 +0100] "HEAD //../spip_login.php3 HTTP/1.1" 200 0 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
umvf.cochin.univ-paris5.fr - - [09/Jan/2004:11:36:38 +0100] "GET /spip_login.php3 HTTP/1.0" 200 2222 "-" "PhpDig/1.6.4 (+http://www.phpdig.net/robot.php)"
nestor.lurt-cochin.prd.fr - admin [09/Jan/2004:11:36:38 +0100] "POST /recherche/admin/spider.php HTTP/1.1" 200 8691 "http://umvf.cochin.univ-paris5.fr/recherche/admin/index.php" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
|