View Single Post
Old 03-10-2004, 12:42 AM   #1
bigals
Orange Mole
 
Join Date: Nov 2003
Posts: 41
Remove Index pages

Hi I have php dig set up on a web site and I'm having trouble getting round a problem.

PHP DIG indexes pages when spidering, these pages are roots of folders, they appear in the spider directory as '-' entries, these are then listed when a user searches, when they are clicked they just open folder pages on my web site such as:

www.domain.com/html/one/

thats a bit confusing so try this description:

if you have a page called:

www.domain.com/html/one/one.html

the spider indexes this link but also indexes the root folder of that html page, so you get both of these results:

www.domain.com/html/one/one.html
www.domain.com/html/one/

I don't want these folders indexed, i just want the html pages indexed,

SO, does anyone know how to get round this, your help will be much appreciated.
bigals is offline   Reply With Quote