classical... Links found 0

[docphilz]

Hi all,
first of all, be sure i've carefully read all the threads about this neverending problem.

So I've installed and set up phpdig 1.6.4 on my ISP site.
I actually don't know what php/apache/linux version is running :-/

Anyway, I can correctly crawl some sites (i.e. www.expertilog.com) very straight forward.

But when I try do crawl any site hosted at my isp, including my site, I get the famous :

SITE : http://www.netstudio.fr/
Exclude paths :
-
- @NONE@
No link in temporary table

--------------------------------------------------------------------------------

links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !


I did activate error trace in spider.php ( I'm a newbie to php).
I get several lines of :

Notice: Undefined variable: user_agent in /mnt/pro/client/939/742/searchengine/admin/robot_functions.php on line 564


But the fact is that i actually do not have any robots.txt file.
I tried both (with and without) and the result is the same.

The site i'd like to crawl is www.netstudio.fr .....

any comment or suggestion will be very appreciated !
:-)
cheers!

note : I'm aware about the \n\r issue, I had previously 1.6.2 with the same problem, and tried to fix it by editing the correct file, and then installed 1.6.4 which is supposed to fix it.

[docphilz]

Hi again,

I got more info about the system running my phpdig instance.

version PHP : 4.2.3
Apache : 1.3.26


I hope it helps !

[docphilz]

Hi,


I finally discoverd why my site didn't work.

There was a file named robots.txt in the root dir of my target site !!! A file that has nothing to do with my phpdig install.

That drove phpdig crazy.

Thank you all.
:-)


docphilz

[Freddie]

same problem, but I haven't robots.txt
this morning it worked fine, but now I have this problem
I reinstalled the sql but the problem remains :(