Thanks Charter - I added the code with the AND LOCKED=0 clause but it didn't seem to work. In fact now I'm having problems of runaway spidering when running any of the methods, even after I set the spider code back to normal and the tempspider table is empty to start. I have a feeling it's my other tweaks that are the cause of the problem. Really, the only 3 things I want to do are:
1) limit the number of links found per site (done this)
2) invert the common_words application so that only words in that file are spidered (done this as well)
3) perform updates using multiple spiders to index the whole lot faster, with links limit in place for updates as well (buggy - text file or not)
I think maybe I'll start over, then make changes slowly, one at a time, and see when the problem starts. I appreciate the help, though - awesome customer service you can't even get when you pay people for it!
