can index
 

hi
I have installed phpdig, and run for the first time indexation but I got this error message

HTTP/1.1 403 Forbidden - http://www.ps-yerres.org/robots.txt
See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html for explanation.

could you help me

regards

NO reply at this time. I have look at the forum and did not an answer.
I have saw that could be a problem if phpdig is on the same machine than the http.
i can explain my configuration:
the http machine is not dedicated to us. It is a sharer machine.
Phpdig is on a directory under web
the mysql database is on an other machine

need help

best regards

Hi, it´s not normal.

the robots.txt file at "http://www.ps-yerres.org/robots.txt", is protected.

normally it can be read, like- http://www.google.pt/robots.txt or http://www.jjoel.com/robots.txt

Inté

jjoel

This file is created by me and it is empty. i had the same error message before without this file.

could you tell me why this file is necessary , what this file should containt ?

regards

Hi, you must read this.

http://www.searchengineworld.com/rob...s_tutorial.htm

Inté
jjoel

Now only index.html
 

Hi

thank's for your anser. I have put a robots.txt and i can index the site but only the first page (index.htlm) phpdig does'nt go through the entire site ?

have you an idea ?

best regards

a precision
 

And I have already do that

Try setting search depth to a large number, links per to zero, LIMIT_TO_DIRECTORY to false, PHPDIG_IN_DOMAIN to true, and choose the no option.

regards

SOLVED
 

Hello
I have finally understand that i should put in ly local /etc/hosts file the ip adress of my host because it is a "rack machine" with multiple cpu and raid hard disk

regards