PDA

View Full Version : Error while spidering


jimigisme
10-01-2003, 01:50 PM
I've got phpdig running from the command line as follows:
C:\php>php.exe -f C:\phpdig\admin\spider.php sites.txt
THe spider runs for a bit:
Spidering in progress...
Then:
Notice: Undefined index: path in C:\phpdig\admin\robot_functions.php on line 347
Any ideas would be helpful,
Thank you!

Charter
10-01-2003, 07:26 PM
What's in sites.txt, i.e., is there a full URL, one per line?

jimigisme
10-02-2003, 09:27 AM
Example of sites.txt
http://www.sitename.com
http://www.sitename.com
http://www.sitename.com
http://www.sitename.com

Charter
10-05-2003, 10:37 AM
Where is sites.txt located? Line 347 in robot_functions.php is:

$path = $newurl["path"];

Could you change that to:

$path = $newurl["path"];
echo $path . " <--- Path<br><br>";

to see what $path is?

jimigisme
10-06-2003, 09:03 AM
Thank you for your help, here is what I got:

SITE : http://www.welcometoourworldofgifts.com/
Exclude paths :
- @NONE@
/welcome/default.asp <--- Path<br><br>+1:http://www.welcom
(time : 00:00:28)
/welcome/default.asp <--- Path<br><br>/welcome/default.asp

Charter
10-06-2003, 03:51 PM
Hmm... How about try the following:

In the config file, set the following:

define('PHPDIG_SESSID_REMOVE',true); // remove SIDS from indexed URLS
define('PHPDIG_SESSID_VAR','mscssid'); // name of the SID variable

Anyone out there ever have problems indexing *.asp files?