PhpDig.net - links not found (well, most of them)

PhpDig.net (http://www.phpdig.net/forum/index.php)

- Troubleshooting (http://www.phpdig.net/forum/forumdisplay.php?f=22)

- - links not found (well, most of them) (http://www.phpdig.net/forum/showthread.php?t=1084)

b2l_grefix

07-14-2004 11:58 PM

links not found (well, most of them)

Version 1.8.3

I've just installed version 1.8.3 And tried to spider http://grefix.be with the standard settings. The site is actually only one index.php wich is dynamically changed through the vars in the request uri

PHPDig only finds 2 links in the main page, while there are at least 15. :(

I've tried the solution suggested by nathanc (see this)

Alltough this works in a browser environment, it doesn't seem to work when spidering from the shell...

Any suggestions?

Charter

07-15-2004 08:00 AM

Hi. Perhaps it's a location issue...

PHP Code:


		
			
// turn off magic_quotes_runtime for escaping purposes

@ini_set("magic_quotes_runtime","0");

// turn off magic_quotes_sybase for escaping purposes

@ini_set("magic_quotes_sybase","0");

Those are set in config.php but if you stick them at the top of spider.php does it help?

Charter

07-15-2004 09:32 PM

Hi. So I don't think it is a location issue but maybe it's a value issue...

In the config.php file replace:

PHP Code:


		
			
// turn off magic_quotes_runtime for escaping purposes

@ini_set("magic_quotes_runtime","0");

// turn off magic_quotes_sybase for escaping purposes

@ini_set("magic_quotes_sybase","0");

with the following:

PHP Code:


		
			
// turn off magic_quotes_runtime for escaping purposes

@ini_set("magic_quotes_runtime",false);

// turn off magic_quotes_sybase for escaping purposes

@ini_set("magic_quotes_sybase",false);

Does that have any effect?

b2l_grefix

07-16-2004 03:19 AM

Thanksfor the response :)

Anyway I've tried both now, but it doesn't seem to have much effect.

Results remain the same...

I was wondering if you (or somebody else ) gets other results then me? If so, then we can be certain that it has something to do with the setup of php.ini.

Just gonna post the result here of what I get:

Quote:

1209: old priority 0, new priority 18
Spidering in progress...
-----------------------------
SITE : http://grefix.be/
Exclude paths :
- @NONE@
+1:http://grefix.be/
(time : 00:00:07)
+ +
level 1...
+2:http://grefix.be/index.php?st=10
(time : 00:00:18)
+
+3:http://grefix.be/index.php?st=5
(time : 00:00:24)

level 2...
+4:http://grefix.be/index.php?st=0
(time : 00:00:36)

No link in temporary table
links found : 4
Optimizing tables...
Indexing complete !

Charter

07-16-2004 10:47 AM

Hi. I found the problem and it's not your setup. Download the attachment in this post and replace your files with the ones in the ZIP file.

b2l_grefix

07-16-2004 11:57 AM

Whatever you did, it worked :) And as a plus, it's even faster :)

Thanks:D

All times are GMT -8. The time now is 10:14 PM.