PhpDig.net - robots.txt comments

PhpDig.net (http://www.phpdig.net/forum/index.php)

- Mod Submissions (http://www.phpdig.net/forum/forumdisplay.php?f=24)

- - robots.txt comments (http://www.phpdig.net/forum/showthread.php?t=595)

edkay

03-02-2004 04:54 AM

robots.txt comments

At present (1.8.0), the phpdigReadRobotsTxt function doesn't understand comments.

Robots.txt comments begin with a #. At present, these are ignored and so commented out lines are still interpretted (not good!!)

As a simple fix, I added the following additional check:

PHP Code:


		
			
while (list($id,$line) = each($robots))

            {

            // Ignore comments

            if (strpos(trim($line),"#") === 0)

                continue;

            if (ereg('^user-agent....

I would recommend that this (or similar) is included in future releases.

marb	03-11-2004 09:14 AM

In which file and where can I place this code?
I think it is robot_functions.php but I'm not sure.

Marten :)

Konstantine

03-12-2004 12:41 PM

Yes, it's robot_functions.php. Hope, it will be in new version of phpdig :)

All times are GMT -8. The time now is 10:22 PM.