CHUNK_SIZE in the 1.8.1 config file I have is 1024 vs. 2048 in my original config file.
When I re-spidered last night after copying the 1.8.1 config file to the server, I got 2 more pages indexed this time. Still far short of what I should have.
Charter, when you get a chance, could you put together a complete 1.8.1 zip file with all the latest stuff? I want to re-download again to make absolute certain I'm using all files from that. Then I'll re-spider again and let you know what happens.
Nice to have plenty of bandwidth to do that.