|
i dont think there are links from filenames.html
i dont think there are links from filenames.html to dir/
couldn't i add '-' to the forbidden extentions list or will that just mess it all up?
the html files are named by regions and towns in england, i.e. 'norwich.html', they are not called 'index.html' if thats what you were thinking perhaps.
do you get these directory indexs in you spider results?
alex
|