View Single Post
Old 09-30-2006, 05:30 PM   #3
Green Mole
Join Date: Sep 2006
Posts: 6
I just contacted the server admin, and he switched the allow_url_fopen value to 1 (ON). But then, same thing still happened. I put the website name, click 'Dig This!', and spider.php was still blank and no indexing activity.

Btw the server that I use is Linux Apache Version 1.3.27 and PHP Version 4.2.3.

I tried to copy the EXACT same phpdig folder to another website on a different server system (this time windows server), and voila, it works. Even when I put the website address of the one in the linux server, it could crawl and index that website.

Then I thought, it might be a path issue (because all the pages are in /usr/home/....../public_html/ and this search folder is in /usr/home/....../public_html/phpdig), so I tried to changed the 'ABSOLUTE_SCRIPT_PATH' to '/usr/home/../../phpdig', still it wouldn't work.

What else should I do?

Btw I noticed the following paragraph in the documentation:

"Note that if your OS/setup is for example a CGI loadbalanced cluster of servers, it may not possible to index sites on the cluster as there cannot be a connection back to the loadbalanced address. Also note that PhpDig is a web spider and search engine, meaning that you may have to edit you hosts file with something like "" in order to get PhpDig to crawl on localhost."

What does this mean?
digdug is offline   Reply With Quote