OK... ever since i upgraded to 2.0, i have been getting nonstop hits from bots...and its not that they are just hitting my site, they are creating more false hits by trying to go to pages and files that aren't even and never have been on my site...
whats going on? i used to get random hits from them from time to time, but its going non stop now...
Install Bad Behavior and enjoy your freedom from bad bots.
ok, because i'm new to all this...
that won't affect my site in any way right? not the way things are posted, comments, etc...nothing?
all i do is upload it and it will keep all those bots from creating false hits?
thank you
That's the idea. ^_-
thank you!!!!
oh...
i was just reading on the webpage...
it isn't very user friendly... to view the log you have go in an view the database... and edit some of the files for things like whitelisting...
i'm just looking for a quick and easy solution for this...sorry
It's mostly a set-it-and-forget-it plugin. You will only have to check the logs and use the whitelist if you notice any odd behavior or if some visitors (usually behind outdated proxy filters) complain of any inability to access your site.
does it take up a lot of database space?
either way... I am sure that i am not the only one haveing this problem and i think maybe its something that should be looked at, cause like i mentioned above...whereas before it wasn't often and was random that bots like googlebot and the others would give my site false hits, now its nonstop. not only do they go to every single page and post, but they try to go to pages that don't exist... i mean, what is it about 2.0 that is doing this where 1.5 wasn't?
Thank you
Does it take up a lot of database space?
No, not really. In the past week, Bad Behavior has blocked a total of 1,354 attempted hits on my blog from "bad bots", and that only accounts for 636.2kb of my database.
It wasn't often and was random that bots like googlebot and the others would give my site false hits, now its nonstop. Not only do they go to every single page and post, but they try to go to pages that don't exist.
Do you have the user agent of the bot that is spidering those pages? Is it the Googlebot or some other well-know bot? These "bot invasions" are typically done at random by "bad bots" attempting to steal content, post spam, harvest email addresses, or look for security exploits. Bad Behavior blocks all of those "bad bots".
yeah, its googlebot and the other common ones and now its others i've never heard of before...
but...i still find it odd that it started with the non stop stuff as soon as i upgraded to 2.0...
Well, due to the change in your blog's generator, the Googlebot and other major search bots may be re-indexing your site and comparing their existing index with your current site to remove any "dead" pages from their indexes.
If you're concerned, just install Bad Behavior. It has a set of functions which compares the major search bots against their known IPs and known behavior and blocks any bots which are in violation. So, if this is a "bad bot" masquerading as the Googlebot, it will be stopped.
maybe you could explain it better than the webpage did...
if i find one that is missed by the plugin, how do i add the IP or referrer to a blacklist or to one of those lists?
**EDIT**
I'm still getting googlebot... i thought it would prevent the bots from giving me false hits...
I'm not trying to preven spam, i'm using captcha for that and its working wonderfully... i'm talkin about my stats where i find out info about hits on my site... all these bots are making it seem as though people are going nuts over my blog...lol...which would be great to some i guess
BTW, I'm using the plugin counterize for my stats...
Well, if Bad Behavior is not blocking it, then that means that it is the one and only genuine Googlebot. And, you certainly don't want to block that. It's probably just re-indexing your site. That does happen from time to time.
ok... i'm confused by all this...
just from what you've said, i gathered that this plugin would stop the false hits... from reading the webpage the plugin is from, it makes it seem like its supposed to stop comment spam...
It stops hits from known "bad bots" including those which falsely pretend to be major search bots, attempt to steal content, post spam, harvest email addresses, or hunt for security exploits.
thank you macmanx!!!
**EDIT**
ok, honestly... i'm not seeing any difference... they just keep comin...
besides that they are still coming... the common ones still won't stop and they are still trying to got o pages and files that just aren't there and creating hits that way... they've never done that before....just started as soon as i upgraded
As I have already stated:
"Well, due to the change in your blog's generator, the Googlebot and other major search bots may be re-indexing your site and comparing their existing index with your current site to remove any "dead" pages from their indexes."
Bad Behavior will not stop legitimate bots. If you want to disappear from the listings and results of all major search engines, feel free to use a .htaccess file to block those bots by their user-agents. Otherwise, just get some sleep and let them run their course.
oh...sorry... i just get worried when i look and see them trying to hit files and pages that aren't there and sometimes the link they are looking for is just a bunch of random letters and numbers...
Well, like I said, "If you want to disappear from the listings and results of all major search engines, feel free to use a .htaccess file to block those bots by their user-agents." If it really is a major concern to you, then I suggest that you contact the administrators of the "offending" search engines.
Additionally, this article may interest you:
http://ajaydsouza.wordpress.com/2005/10/02/protecting-your-blog-from-spam/
Here is the list of the "bots" that are doing this... I've had maybe 10 legitimate hits since the counter reset at midnight and its 8 AM here now and it says that I am pushing nearly 200 hits. I am wondering if others are having this happen as well. One imaginary page they keep trying to go to is robots.txt, which simply isn't there and never has been. Also, it these "legitimate" bots are just updating their listings, why do they keep coming back and just repeating the hits over and over... I mean, I can understand what macmanx is saying that there are times when all the bots do things like this, but its been almost non stop for two days, ever since the minute I upgraded to wordpress 2.0...
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
PHP/4.2.2
msnbot/1.0 (+http://search.msn.com/msnbot.htm)
Googlebot/2.1 (+http://www.google.com/bot.html)
GivingYouFreeLinks/0.1 (+http://referer.org/)
Baiduspider+(+http://www.baidu.com/search/spider.htm)
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Larbin/2.6.3 (larbin@unspecified.mail)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
Gigabot/2.0
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312465)
Mozilla/4.0 (compatible;)
PEAR HTTP_Request class ( http://pear.php.net/ )
Java/1.4.1_04
WordPress/2.0
I've been having the same problems for the whole mobth of December and I am still on 1.5.2. I've had to increase my bandwidth several times throughout the month. I'm now up to 1GB. It seems that half are the "search bots". I am concerned, too.
exactly jwurster... my bandwidth is going way up!!! and after looking over the log, you're right, it has been december, prior to my upgrade to 2.0...
i know that most people wouldn't complain about their hit counter running up cause it makes it look as though you site is getting all these visitors, but when they run up your bandwidth, it becomes a problem...
and like i said, i've gotten these bots in the past, no big deal, but its just been nonstop lately to the same pages over and over and to pages/files that don't exist...
i'm glad its not just me...
OK... Now its a problem cause my site is down...and i get this messege...
Precondition Failed
We're sorry, but we could not fulfill your request for /wp-admin/link-manager.php on this server.
We have established rules for access to this server, and any person or robot that violates these rules will be unable to access this site.
To resolve this problem, please try the following steps:
* Ensure that your computer is free of viruses, Trojan horses, spyware or any other sort of malicious software.
* If you are using any sort of personal firewall or browser privacy software, check to ensure that its settings do not cause your web browser to inadvertently violate any of the rules listed below.
* If you are behind a Web proxy or corporate firewall, the proxy must conform to the HTTP specification with respect to proxy servers. Contact your network administrator if the trouble persists, or bypass the proxy and connect directly if possible.
* Disable any download accelerators you may be using. They don't speed up your downloads anyway; in most cases, they actually run slower!
* If all else fails, try using a different Web browser, such as Firefox.
If you still need assistance, please contact admin at mykahil.com.
More Information
For your reference, the conditions for access to this server are:
Robots:
* MUST read and obey robots.txt.
* MUST identify themselves properly; for example MUST NOT identify as Mozilla.
* MUST NOT pretend to be a human.
Humans:
* MUST NOT pretend to be a robot.
* MUST NOT use a computer infected with viruses, Trojan horses or other malicious software.
Both:
* MUST NOT harvest email addresses.
* MUST NOT attempt to send spam.
* MUST NOT attempt to compromise server security.
* MUST NOT use excessive amounts of bandwidth or other server resources.
The precondition on the request for the URL /wp-admin/link-manager.php evaluated to false.
ahhh... figured it out...
it was the plugin "bad behavior" that someone suggested i use for the above bot problem... bad idea i guess...
Kahil, I'm sorry, I guess that I've given you all the wrong advice, or that you just don't want to listen to it. I will re-post the important points, incase you are in a mood to listen.
* Bad Behavior stops hits from known "bad bots" including those which falsely pretend to be major search bots, attempt to steal content, post spam, harvest email addresses, or hunt for security exploits. If you are suspicious of these bots, activate the plugin, and let it decide on whether these bots are legitimate or malicious.
* If you get blocked by Bad Behavior, read this: http://error.wordpress.com/2005/09/30/what-to-do-when-bad-behavior-blocks-you-or-your-friends/
* Due to the change in your blog's generator, the Googlebot and other major search bots may be re-indexing your site and comparing their existing index with your current site to remove any "dead" pages from their indexes.
* If Bad Behavior is not blocking it, then that means that it is the one and only genuine Googlebot. The same can be said for the Yahoo and MSN bots. And, you certainly don't want to block those. They're probably just re-indexing your site. That does happen from time to time.
* If you want to disappear from the listings and results of all major search engines, feel free to use a .htaccess file to block those bots by their user-agents. Otherwise, just get some sleep and let them run their course.
* If it really is a major concern to you, then I suggest that you contact the administrators of the "offending" search engines.
easy macmanx...
summerdonna...
i'm just saying that there was no change in bot activity and the plugin blocked me all of a sudden... i was in my admin section one moment, then all of a sudden i get blocked...
and since someone else has come forward with the same problem, there has to be something going on. it has become a problem cause its eating up all the bandwidth.
i'm not saying you aren't right with your explanation, just that there may be another and we shouldn't just toss it aside.
Have you contacted the administrators of the "offending" search engines yet?
no... cause other than google, yahoo and msn, there are still several others doing this... like just a few mins ago, java something put almost 50 hits on my site in a matter of minutes...
i have a large bandwidth, and haven't had the issue this has caused to jwurster above, so i'm not gonna go that far yet. but since others have that problem, i think we should find out why. its not gonna do us no good if we ask them to stop and not know why all the bots are doing this...
This topic has been closed to new replies.
