PhpDig.net

Go Back   PhpDig.net > PhpDig Forums > Mod Submissions

Reply
 
Thread Tools
Old 03-02-2004, 05:54 AM   #1
edkay
Green Mole
 
Join Date: Feb 2004
Posts: 1
robots.txt comments

At present (1.8.0), the phpdigReadRobotsTxt function doesn't understand comments.

Robots.txt comments begin with a #. At present, these are ignored and so commented out lines are still interpretted (not good!!)

As a simple fix, I added the following additional check:

PHP Code:
while (list($id,$line) = each($robots))
            {
            
// Ignore comments
            
if (strpos(trim($line),"#") === 0)
                continue;
            if (
ereg('^user-agent.... 
I would recommend that this (or similar) is included in future releases.
edkay is offline   Reply With Quote
Old 03-11-2004, 10:14 AM   #2
marb
Green Mole
 
Join Date: Mar 2004
Posts: 19
In which file and where can I place this code?
I think it is robot_functions.php but I'm not sure.


Marten
marb is offline   Reply With Quote
Old 03-12-2004, 01:41 PM   #3
Konstantine
Green Mole
 
Join Date: Mar 2004
Location: Russia
Posts: 21
Yes, it's robot_functions.php. Hope, it will be in new version of phpdig
Konstantine is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
robots.txt seems to be ignored :? galacticvoyager Bug Tracker 1 11-12-2005 01:52 PM
robots.txt and URL djavet How-to Forum 4 01-11-2005 04:19 AM
robots.txt versus robotsxx.txt Charter IPs, SEs, & UAs 0 03-11-2004 07:00 PM
robots.txt ignored roy Troubleshooting 3 02-20-2004 09:02 PM
robots.txt renehaentjens Troubleshooting 3 12-05-2003 03:40 PM


All times are GMT -8. The time now is 02:11 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.