classical... Links found 0
 

Hi all,
first of all, be sure i've carefully read all the threads about this neverending problem.

So I've installed and set up phpdig 1.6.4 on my ISP site.
I actually don't know what php/apache/linux version is running :-/

Anyway, I can correctly crawl some sites (i.e. www.expertilog.com) very straight forward.

But when I try do crawl any site hosted at my isp, including my site, I get the famous :

SITE : http://www.netstudio.fr/
Exclude paths :
-
- @NONE@
No link in temporary table

--------------------------------------------------------------------------------

links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !


I did activate error trace in spider.php ( I'm a newbie to php).
I get several lines of :

Notice: Undefined variable: user_agent in /mnt/pro/client/939/742/searchengine/admin/robot_functions.php on line 564


But the fact is that i actually do not have any robots.txt file.
I tried both (with and without) and the result is the same.

The site i'd like to crawl is www.netstudio.fr .....

any comment or suggestion will be very appreciated !
:-)
cheers!

note : I'm aware about the \n\r issue, I had previously 1.6.2 with the same problem, and tried to fix it by editing the correct file, and then installed 1.6.4 which is supposed to fix it.

more info
 

Hi again,

I got more info about the system running my phpdig instance.

version PHP : 4.2.3
Apache : 1.3.26


I hope it helps !

it works
 

Hi,


I finally discoverd why my site didn't work.

There was a file named robots.txt in the root dir of my target site !!! A file that has nothing to do with my phpdig install.

That drove phpdig crazy.

Thank you all.
:-)


docphilz

same problem, but I haven't robots.txt
this morning it worked fine, but now I have this problem
I reinstalled the sql but the problem remains :(