A follow up to my earlier post, to edit the spider user agent edit the file admin/robot_functions.php, not the spider.php file I mentioned earlier.
Another good idea would be to have a test in your site's authentication mechanism to check that the IP address of the spider is what you expect it to be, just in case.
- Ben
|