nevsie
02-28-2005, 12:55 PM
Hi,
I am using PHPDIG and like the programme... However, i have a problem with the spidering stopping after a set amount of pages. One site it might be 7 pages another it might be 10, but never more. The site then stays in the state of being locked.
I have set:
Depth to '10'
Links per '0'
'LIMIT_TO_DIRECTORY',false
'PHPDIG_IN_DOMAIN',true
Other comments have been that you should use a browser such as FireFox to stop timeout issues... I am using Firefox, although i never see the second page where the details of page spidering arise... Does this mean the browser is timing out... If so can i change the timeout period, as i cannot access shell to run PHP DIG.
Alternatively could my host be ending the spidering. Whether it be at the mysql end or at the spidering end? I have contacted them and asked, but as of yet no reply.
Any help is truely appreciated.
Thanks N
I am using PHPDIG and like the programme... However, i have a problem with the spidering stopping after a set amount of pages. One site it might be 7 pages another it might be 10, but never more. The site then stays in the state of being locked.
I have set:
Depth to '10'
Links per '0'
'LIMIT_TO_DIRECTORY',false
'PHPDIG_IN_DOMAIN',true
Other comments have been that you should use a browser such as FireFox to stop timeout issues... I am using Firefox, although i never see the second page where the details of page spidering arise... Does this mean the browser is timing out... If so can i change the timeout period, as i cannot access shell to run PHP DIG.
Alternatively could my host be ending the spidering. Whether it be at the mysql end or at the spidering end? I have contacted them and asked, but as of yet no reply.
Any help is truely appreciated.
Thanks N