docphilz
12-09-2003, 03:09 PM
Hi all,
first of all, be sure i've carefully read all the threads about this neverending problem.
So I've installed and set up phpdig 1.6.4 on my ISP site.
I actually don't know what php/apache/linux version is running :-/
Anyway, I can correctly crawl some sites (i.e. www.expertilog.com) very straight forward.
But when I try do crawl any site hosted at my isp, including my site, I get the famous :
SITE : http://www.netstudio.fr/
Exclude paths :
-
- @NONE@
No link in temporary table
--------------------------------------------------------------------------------
links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !
I did activate error trace in spider.php ( I'm a newbie to php).
I get several lines of :
Notice: Undefined variable: user_agent in /mnt/pro/client/939/742/searchengine/admin/robot_functions.php on line 564
But the fact is that i actually do not have any robots.txt file.
I tried both (with and without) and the result is the same.
The site i'd like to crawl is www.netstudio.fr .....
any comment or suggestion will be very appreciated !
:-)
cheers!
note : I'm aware about the \n\r issue, I had previously 1.6.2 with the same problem, and tried to fix it by editing the correct file, and then installed 1.6.4 which is supposed to fix it.
first of all, be sure i've carefully read all the threads about this neverending problem.
So I've installed and set up phpdig 1.6.4 on my ISP site.
I actually don't know what php/apache/linux version is running :-/
Anyway, I can correctly crawl some sites (i.e. www.expertilog.com) very straight forward.
But when I try do crawl any site hosted at my isp, including my site, I get the famous :
SITE : http://www.netstudio.fr/
Exclude paths :
-
- @NONE@
No link in temporary table
--------------------------------------------------------------------------------
links found : 0
...Was recently indexed
Optimizing tables...
Indexing complete !
I did activate error trace in spider.php ( I'm a newbie to php).
I get several lines of :
Notice: Undefined variable: user_agent in /mnt/pro/client/939/742/searchengine/admin/robot_functions.php on line 564
But the fact is that i actually do not have any robots.txt file.
I tried both (with and without) and the result is the same.
The site i'd like to crawl is www.netstudio.fr .....
any comment or suggestion will be very appreciated !
:-)
cheers!
note : I'm aware about the \n\r issue, I had previously 1.6.2 with the same problem, and tried to fix it by editing the correct file, and then installed 1.6.4 which is supposed to fix it.