Indexing file and down

[jerrywin5]

I read the thread titled "Subdirectories but not higher directories" found here. (http://www.phpdig.net/forum/showthread.php?t=1749)

I need to take this a little further. I want to index pages on a remote server in a directory but only the page that I specify and the pages it links to in the same directory and in sub directories. Here are two examples:

http://www.elks.org/lodges/home.cfm?LodgeNumber=835
http://members.tripod.com/~Scott_Michaud/Aroostook-War.html

Thank you!

[Charter]

That thread should get you what you want, unless I'm misunderstanding.

[jerrywin5]

If I index the first link, I will index all elks lodges rather than just the lodge I intend to index. I need the spider to crawl that page and any links found on that page. I don't want it to crawl the entire directory that the page is in.

If I index the second link, I will index all pages in the ~Scott_Michaud directory. Instead, I want to index the page Aroostook_War.html and all pages that it links to in that directory.

Unfortunately, I have many situations like this.

[Charter]

http://www.phpdig.net/forum/showthread.php?t=1852