PhpDig.net - View Single Post - Spider stops before all pages are indexed

halide · 07-09-2005, 09:21 AM

Hi,

I am trying to spider 3 sections of my site via a sitemap which I have put together for the purpose. The problem is that sometimes the spider will index all the pages but often it will stop before the end. There is no error message or completion message, just nothing. If I go back to the admin page it says *Locked* - This behaviour appears to be completely r****m because each time it stops at a different point.

The site is http://www.lightingdirectory.com and the site map is http://www.lightingdirectory.com/site-map (This is the URL I'm using for indexing).

Observation - All of the pages appear in the tempspider table every time. So it appears that the pages are being indexed but not spidered.

Any help would be appreciated. Thanks.

07-09-2005, 09:21 AM	#1
halide Green Mole Join Date: Jun 2005 Posts: 2	Spider stops before all pages are indexed Hi, I am trying to spider 3 sections of my site via a sitemap which I have put together for the purpose. The problem is that sometimes the spider will index all the pages but often it will stop before the end. There is no error message or completion message, just nothing. If I go back to the admin page it says Locked - This behaviour appears to be completely r****m because each time it stops at a different point. The site is http://www.lightingdirectory.com and the site map is http://www.lightingdirectory.com/site-map (This is the URL I'm using for indexing). Observation - All of the pages appear in the tempspider table every time. So it appears that the pages are being indexed but not spidered. Any help would be appreciated. Thanks.