I've got a bunch of pages with a navigation menu on it, as well as some text I don't want indexed.
I want to exclude everything but a <div> tag that contains all the text I want to index.. However...
it seems that if I do that, all of the links in my navigation menu aren't spidered. That is not helpful.
How can I get phpDig to spider my entire site, but only index one section of every page?