Quote:
|
Originally Posted by Charter
>> I have no idea what kind of server load there might be, don't know how one measures that.
From the shell prompt, type top or uptime and hit return. You should see the load average with three numbers showing the average load over the last 1, 5, and 15 minutes.
|
Thanks. I'll try that next time and let you know the results.
Quote:
>> It says indexing is complete, then stops.
>> I still had the spidering process just hang...
Sometimes it hangs and sometimes it completes? Does anything unusual show in your raw access or error logs?
|
When spidering completes, it doesn't index very many pages at all through shell. However, I've spidered now a couple of times through the secure web page since moving my site, and it functioned just as I would have expected.
This whole thing really bugs me, because I would eventually like to have this run as a cron job and dispense with the other two methods. However, I don't have a lot of confidence at this point that a cron job would do any different than shell. This is all so strange.
Nothing unusual at all in the server log. My provider is at a loss to explain why it would just hang, too.
I've been chewing up lots of bandwidth messing with this. Still have plenty to play with for now, but need to watch it.