|
03-02-2004, 04:54 AM | #1 |
Green Mole
Join Date: Feb 2004
Posts: 1
|
robots.txt comments
At present (1.8.0), the phpdigReadRobotsTxt function doesn't understand comments.
Robots.txt comments begin with a #. At present, these are ignored and so commented out lines are still interpretted (not good!!) As a simple fix, I added the following additional check: PHP Code:
|
03-11-2004, 09:14 AM | #2 |
Green Mole
Join Date: Mar 2004
Posts: 19
|
In which file and where can I place this code?
I think it is robot_functions.php but I'm not sure. Marten |
03-12-2004, 12:41 PM | #3 |
Green Mole
Join Date: Mar 2004
Location: Russia
Posts: 21
|
Yes, it's robot_functions.php. Hope, it will be in new version of phpdig
|
Thread Tools | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
robots.txt seems to be ignored :? | galacticvoyager | Bug Tracker | 1 | 11-12-2005 12:52 PM |
robots.txt and URL | djavet | How-to Forum | 4 | 01-11-2005 03:19 AM |
robots.txt versus robotsxx.txt | Charter | IPs, SEs, & UAs | 0 | 03-11-2004 06:00 PM |
robots.txt ignored | roy | Troubleshooting | 3 | 02-20-2004 08:02 PM |
robots.txt | renehaentjens | Troubleshooting | 3 | 12-05-2003 02:40 PM |