Go Back > General Forums > IPs, SEs, & UAs

Thread Tools
Old 09-07-2004, 02:28 PM   #1
Head Mole
Charter's Avatar
Join Date: May 2003
Posts: 2,539
Rogue Bot Rant

So here is the first request: - - [07/Sep/2004:04:09:26 -0700] "GET / HTTP/1.1" 200 31891 "-" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"

And after 4 hours, 11 minutes, 13 seconds and 15447 pages, 16717 hits, and 374.30 MB later...

Here is the last request: - - [07/Sep/2004:08:20:39 -0700] "GET /forum/showthread.php?t=942&page=3&pp=15 HTTP/1.1" 200 7583 "-" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"

There is absolutely no reason that the user of this rogue bot had to suck that much content and bandwidth, not to mention that there was no request for a robots.txt file and no bot name was identified. This mofo, yes mofo, and others that do the same, really tick me off!

With that said, PhpDig is scripted to follow a robots.txt file and allow for a five-second delay when indexing. If you are indexing a site that is not your own, please do not bypass these features, as you are using another person's resources and extra bandwidth can cost people money!

Another thing while I'm at it, some people do not know about PhpDig so there may not be a block in their robots.txt file. If you do index a site that is not your own, please be kind and do not index every single page on their site. Such action is simply not necessary to create a decent index.

Don't run a rogue bot. Take the high road, and use PhpDig kindly and wisely.
Responses are offered on a voluntary if/as time is available basis, no guarantees. Double posting or bumping threads will not get your question answered any faster. No support via PM or email, responses not guaranteed. Thank you for your comprehension.
Charter is offline   Reply With Quote
Old 09-13-2004, 07:47 PM   #2
WebDiva 2.0
Former Member
Join Date: Aug 2004
Posts: 18
That IP address come back to this, Charter:

Don't know if that helps you though, as this would be a proxy IP.
WebDiva 2.0 is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
URL Bot RaGe Mod Requests 0 04-29-2004 05:59 AM
The Dot Bot Charter IPs, SEs, & UAs 0 04-14-2004 03:56 PM
Bot from IP Charter IPs, SEs, & UAs 0 03-10-2004 09:56 AM

All times are GMT -8. The time now is 10:06 AM.

Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright © 2001 - 2005, ThinkDing LLC. All Rights Reserved.