How to change?

[a3719]

I have a large site about 800 pages but everytime i index it stops about 70 pages in. I noticed while it was doing the index that it is using

admin/spider.php?site_id=2&mode=small

Is there a way to change the mode from small to something else so that i can index the whole site?

[Charter]

Hi. Do you mean that when it stops it isn't completely done with the index, or do you mean that it only indexes 70 of the 800 pages?

[a3719]

yea when it stops it not done with the index i have to go back in a restart it and tell it to index again and it will get about 40 more pages then stop again and i have to keep redoing this

[Charter]

Hi. This sounds like an issue possibly related to timeout or memory. What do you get in your error logs?

[a3719]

the error logs that i have access to do not show timeouts or memory issues they are very basic logs and i can not access the other logs cause my host does not have them setup

[a3719]

when i go back and tell it to respider the root it will go through all the previous pages saying date unchanged which is fine and then it will start to add about 40 or so more pages after it checks all the ones it did before. I got it to about 400 pages so far and it runs for about 40 minutes so i dont think that its a timeout error.

[Charter]

Hi. Do you notice whether or not it only stops when it encounters a large file? If so, there is some code in this (http://www.phpdig.net/showthread.php?threadid=534) thread that might help.

[a3719]

all of my files are small they are basic web pages with text most are about 300k or less. i will try that code tho just to make sure.

[Charter]

Hi. Could it be that your host has some sort of auto shutdown for runaway or resource intensive scripts?

[a3719]

it may be that cause thats what i am starting to think. so i guess i will just index the pages about 50 at a time. Once i get all the pages indexed it reindexes them all fine. Like right now i have gone though and reindexed the root enough times to get about 500 pages now and it goes though all the previuos pages without a problem so i guess once i get all the pages indexed everything should be ok when it does the automatic reindex every week.