View Single Post
Old 11-30-2004, 10:57 AM   #7
thematrixbe
Green Mole
 
Join Date: Nov 2004
Posts: 2
Dave_A: your totaly wrong sorry to have to say. The user agent is the way you uniquely identify your crawler to the websites.

Imho its nearly mandatory to identify to the sites your crawling, so the sites know how to contact you in case of problems or if they wish to talk about excluding or including certain things.

You can change the user agent in the robot_functions.php file.

Just change the lines starting with:

Code:
."User-Agent:
thematrixbe is offline   Reply With Quote