Ok.. Here is some food for thought. I took that little script from the link that you wanted me to check out. and created a page like so:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<title>Untitled Document</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body>
<?php
$fd = fopen( "http://www.mydomain.com/index.php/", "r" );
if( !$fd )
{
echo "Cannot open URL";
} else {
while ( !feof( $fd ) ) {
$buffer = fgets( $fd, 4096 );
echo $buffer;
}
fclose ( $fd );
}
?>
</body>
</html>
I called it testing.php. Then I went to a web browser and called testing.php, and it printed the desired page fully intact into the browser window. Totally awesome. Then I went and to PhpDig and typed in that very same URL for testing.php, and guess what. It spidered it no problem. So.... what do you think it could be? If I try spidering the URL/page with it's actual URL it doesn't work, but when I spider the testing.php which essentially opens and reads the page that I want to spider and echos it back then the spidering works. It only works on that one level, because other links that are generated on that particular page also use that dis=1,0,0,....... Now I am really lost here. Could it be that the code needs to be changed or adjusted so that the dis variable with the string of number with the commas won't throw a fit?
Please help
Mr. L