Error while spidering

[jimigisme]

I've got phpdig running from the command line as follows:
C:\php>php.exe -f C:\phpdig\admin\spider.php sites.txt
THe spider runs for a bit:
Spidering in progress...
Then:
Notice: Undefined index: path in C:\phpdig\admin\robot_functions.php on line 347
Any ideas would be helpful,
Thank you!

[Charter]

What's in sites.txt, i.e., is there a full URL, one per line?

[jimigisme]

Example of sites.txt
http://www.sitename.com
http://www.sitename.com
http://www.sitename.com
http://www.sitename.com

[Charter]

Where is sites.txt located? Line 347 in robot_functions.php is:

$path = $newurl["path"];

Could you change that to:

$path = $newurl["path"];
echo $path . " <--- Path<br><br>";

to see what $path is?

[jimigisme]

Thank you for your help, here is what I got:

SITE : http://www.welcometoourworldofgifts.com/
Exclude paths :
- @NONE@
/welcome/default.asp <--- Path<br><br>+1:http://www.welcom
(time : 00:00:28)
/welcome/default.asp <--- Path<br><br>/welcome/default.asp

[Charter]

Hmm... How about try the following:

In the config file, set the following:

define('PHPDIG_SESSID_REMOVE',true); // remove SIDS from indexed URLS
define('PHPDIG_SESSID_VAR','mscssid'); // name of the SID variable

Anyone out there ever have problems indexing *.asp files?