View Single Post
Old 12-19-2004, 02:19 AM   #3
Slider
Orange Mole
 
Join Date: Jan 2004
Posts: 30
Thanks vinyl-junkie,

You explained that very well.
I'm am wanting to ban links like "links" as in a links page or links/index.html or forum directories. Will i have to make a new line and a brand new define and then simply try to imitate what BANNED is doing?

Line 1264 in robot_functions.php is the only reference I found to BANNED
Code:
if ($regs[5] && $regs[5] != $localdomain && !eregi(BANNED,$regs[5]) && ereg('[a-z]+',$regs[5])) {
So I could make a BANNED2 just under BANNED in config
then would Line 1264 in robot_functions.php be written this way?
Code:
if ($regs[5] && $regs[5] != $localdomain && !eregi(BANNED,$regs[5]) && !eregi(BANNED2,$regs[5]) && ereg('[a-z]+',$regs[5])) {
I'm crawling a lot of websites and a universal setting like this would really help the huge mysql database I have now. I have produced way too many uninformative links in the database.

A little info:
800 sites @ level 1 depth has me at a 20 mb database size. Time to downsize and then decide to get more database space if needed.

Thanks again foryour reply

Last edited by Slider; 12-19-2004 at 02:29 AM.
Slider is offline   Reply With Quote