Cannot spider some pages and ABSOLUTE_SCRIPT_PATH and /usr/local/bin/
 

Hi. I recently re-installed php dig on another machine:
PhpDig v.1.8.8/apache_1.3.34/Php 4.4.2/MySQL 4.1.18/winXP

I have had limited success with spidering and searching for the spiderd page. Some pages have worked perfectly for spidering and then searching for them.

Most sites I attempt to spider give these errors after I hit the Dig button:
1] pop up error 'PHP Script Interpreter Error'...needs to close..Send error Mesage button and 2] 500 Internal Server Error from spider.php and then 3] the site appears in my list but cannot be searched for, no pages found when clicking update button for the site.

I am not certain if this value is correct or has anything to do with the error:
define('ABSOLUTE_SCRIPT_PATH','/phpdig');
The 'phpdig' folder is in my root webserver folder (C:/Web/)
(I also see many references to 'usr/bin...' where is that located?)

Here's a couple sample pages that do not work for me:
http://www.allgoods.ca
http://www.hotlistrookies.com
http://arizona.diamondbacks.mlb.com

here's a page that worked
http://www.cdncc.com

Your assistance is appreciated,

Paul L

how to determine ABSOLUTE_SCRIPT_PATH ?
 

I see another post on troubleshooting how to set this variable, it uses this code:

<?php
echo $_SERVER['DOCUMENT_ROOT']."<br>";
echo $_SERVER['SCRIPT_FILENAME']."<br>";
echo $_SERVER['PATH_TRANSLATED']."<br>";
?>

My output is:
c:/web
c:/php/php.exe
c:\web\phpdig\includes\test.php

Can someone tell me if one of these 3 values are what my ABSOLUTE_SCRIPT_PATH should be?

thx

Cannot spider redirects
 

My spidering problem seems to largely involve redirect/re-direct pages. Like http://hawkeyesports.cstv.com/ , http://arizona.diamondbacks.mlb.com

Here is some info I found from the forums to troubleshoot this from Charter:
Charter sample forum entry:
$url = "http://somewhere.com/path1/path1/file1.php?someid,1,1,1";
print_r(parse_url($url));
Charters output:
Array ( [scheme] => http [host] => somewhere.com [path] => /path1/path1/file1.php [query] => someid,1,1,1 )

My attempt:
$url = "http://houston.astros.mlb.com";
print_r(parse_url($url));
My output:
Array ( [scheme] => http [host] => houston.astros.mlb.com )
another sample:
Array ( [scheme] => hhttp [host] => hawkeyesports.cstv.com )

The path and query are not reading the pages that it should be re-directed to in mine. Any suggestions for failed redirects?

thx