Not really a bug
 

Hi charter,
i've indexed a site with a rm file in it
and the spider accepted the content of the file.
I know this is due to the server on which the site resides,
since it returns a mimetype text/html also for rm files.

I've added rm extension to the list of yours
FORBIDDEN_EXTENSIONS

Maybe adding the extension real media by default to the list of forbidden could be...

Regards,
Simone Capra

capra__nospam__@erweb.it
E.R.WEB - s.r.l.
http://www.erweb.it