PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Troubleshooting

Reply
 
Thread Tools
Old 07-09-2005, 09:21 AM   #1
halide
Green Mole
 
Join Date: Jun 2005
Posts: 2
Spider stops before all pages are indexed

Hi,

I am trying to spider 3 sections of my site via a sitemap which I have put together for the purpose. The problem is that sometimes the spider will index all the pages but often it will stop before the end. There is no error message or completion message, just nothing. If I go back to the admin page it says *Locked* - This behaviour appears to be completely r****m because each time it stops at a different point.

The site is http://www.lightingdirectory.com and the site map is http://www.lightingdirectory.com/site-map (This is the URL I'm using for indexing).

Observation - All of the pages appear in the tempspider table every time. So it appears that the pages are being indexed but not spidered.

Any help would be appreciated. Thanks.
halide is offline   Reply With Quote
Old 07-15-2005, 04:59 AM   #2
llamastephen
Green Mole
 
Join Date: Jul 2005
Posts: 3
Did you manage to solve this? I am having very similar problems.
llamastephen is offline   Reply With Quote
Old 07-18-2005, 04:38 PM   #3
Charter
Head Mole
 
Charter's Avatar
 
Join Date: May 2003
Posts: 2,539
Once spidering is complete, the tempspider table should be empty. Check whether a server timeout issue is happening or whether the host is killing the spidering process.
__________________
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 07-19-2005, 12:26 AM   #4
halide
Green Mole
 
Join Date: Jun 2005
Posts: 2
I haven't solved it yet. But when I set up the next crawl I'll also increase the time out value of my apache server and see whether that helps.
halide is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Indexing stops after a few pages Sibona Troubleshooting 1 05-03-2005 10:27 AM
Spidering Stops, Site is Locked after 7 Pages nevsie Troubleshooting 3 03-02-2005 03:52 PM
Pages not re-indexed wx3 Troubleshooting 0 09-16-2004 05:53 PM
Spidering stops after 6 level 1 pages Steve Troubleshooting 3 05-20-2004 04:56 PM
Spider stops indexing (Browser: Done) rainbow_six Troubleshooting 2 04-24-2004 08:16 AM


All times are GMT -8. The time now is 02:38 PM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.