Indexing file and down
 

I read the thread titled "Subdirectories but not higher directories" found here.

I need to take this a little further. I want to index pages on a remote server in a directory but only the page that I specify and the pages it links to in the same directory and in sub directories. Here are two examples:

http://www.elks.org/lodges/home.cfm?LodgeNumber=835
http://members.tripod.com/~Scott_Mic...stook-War.html

Thank you!

That thread should get you what you want, unless I'm misunderstanding.

If I index the first link, I will index all elks lodges rather than just the lodge I intend to index. I need the spider to crawl that page and any links found on that page. I don't want it to crawl the entire directory that the page is in.

If I index the second link, I will index all pages in the ~Scott_Michaud directory. Instead, I want to index the page Aroostook_War.html and all pages that it links to in that directory.

Unfortunately, I have many situations like this.