Niall Fernie
07-12-2004, 04:24 AM
Hi, first post here as this is the first problem I've come across. Been using 1.4x until now and decided to go for the new version (shiney new features were to much to resist)
Now, though, I find that my site is impossible to spider. I takes for ever, or more like 4.5 seconds per page. I have no access to shell to see if this would be any faster, but 1.4x used to spider the site in about 10 minutes (13-1400 pages) but at 4.5 secs per page, I cannot get it to finish.
Now I have the time to waste, and on several occasions I've left the spider spidering for around 2 hours but for one reason or another my browser ends up saying "done" long before the spider process in finished and the spider page doesn't show the list of pages it found.
Is there something new thats causing things to take soooooo long. I will have to check with my hosts for any server info you might need to help and would like to offer my thinks in advance.
(if this has already been covered in another thread, please delete this and pm me the link)
Now, though, I find that my site is impossible to spider. I takes for ever, or more like 4.5 seconds per page. I have no access to shell to see if this would be any faster, but 1.4x used to spider the site in about 10 minutes (13-1400 pages) but at 4.5 secs per page, I cannot get it to finish.
Now I have the time to waste, and on several occasions I've left the spider spidering for around 2 hours but for one reason or another my browser ends up saying "done" long before the spider process in finished and the spider page doesn't show the list of pages it found.
Is there something new thats causing things to take soooooo long. I will have to check with my hosts for any server info you might need to help and would like to offer my thinks in advance.
(if this has already been covered in another thread, please delete this and pm me the link)