PDA

View Full Version : How to change?


a3719
03-31-2004, 01:18 PM
I have a large site about 800 pages but everytime i index it stops about 70 pages in. I noticed while it was doing the index that it is using

admin/spider.php?site_id=2&mode=small

Is there a way to change the mode from small to something else so that i can index the whole site?

Charter
04-01-2004, 01:03 AM
Hi. Do you mean that when it stops it isn't completely done with the index, or do you mean that it only indexes 70 of the 800 pages?

a3719
04-01-2004, 05:04 AM
yea when it stops it not done with the index i have to go back in a restart it and tell it to index again and it will get about 40 more pages then stop again and i have to keep redoing this

Charter
04-01-2004, 05:19 AM
Hi. This sounds like an issue possibly related to timeout or memory. What do you get in your error logs?

a3719
04-01-2004, 05:31 AM
the error logs that i have access to do not show timeouts or memory issues they are very basic logs and i can not access the other logs cause my host does not have them setup

a3719
04-01-2004, 05:34 AM
when i go back and tell it to respider the root it will go through all the previous pages saying date unchanged which is fine and then it will start to add about 40 or so more pages after it checks all the ones it did before. I got it to about 400 pages so far and it runs for about 40 minutes so i dont think that its a timeout error.

Charter
04-01-2004, 06:06 AM
Hi. Do you notice whether or not it only stops when it encounters a large file? If so, there is some code in this (http://www.phpdig.net/showthread.php?threadid=534) thread that might help.

a3719
04-01-2004, 06:08 AM
all of my files are small they are basic web pages with text most are about 300k or less. i will try that code tho just to make sure.

Charter
04-01-2004, 06:19 AM
Hi. Could it be that your host has some sort of auto shutdown for runaway or resource intensive scripts?

a3719
04-01-2004, 06:26 AM
it may be that cause thats what i am starting to think. so i guess i will just index the pages about 50 at a time. Once i get all the pages indexed it reindexes them all fine. Like right now i have gone though and reindexed the root enough times to get about 500 pages now and it goes though all the previuos pages without a problem so i guess once i get all the pages indexed everything should be ok when it does the automatic reindex every week.