Stop Web Crawlers


Stop Web Crawlers has been developed to block referrer spam attacks on your WordPress website.

Stop Web Crawlers will reduce the likelihood that your site will be crawled by the Referrer spammers like;

> ,, buttons-for-website,,,,

In total Stop Web Crawlers protects your site from 1439 known referrer spammers.

If these Crawlers attempt to crawl your website they sent 403 access prohibited error response

The additional functionality planned for this plugin for the future is also to remove Ghost Referrer spammers from accessing your Google Analytics.

There is no need to edit your .htaccess or mess around with robots.txt, all is required is too install the plugin and activate the plugin.

The plugin is currently designed to run unattended and automatically.

There is a settings section, but this is currently only a placeholder for future functionality, which will be activated in future releases of the plugin.

Other bots will be added to the block list over time via updates to this plugin.



  1. Upload the stop-web-crawlers folder to the /wp-content/plugins/ directory
  2. Activate the Stop Web Crawlers plugin through the ‘Plugins’ menu in WordPress
  3. Configure the plugin by going to the Stop Web Crawlers menu that appears in your admin menu



Our new website, without even a load of new content was flooded with loads of websites “linking” traffic to us. but it all seemed suspicious . After installing your plugin over night it seemed they stopped! Thanks!


Perfect, recommend it to everyone … got rid of semalt[.]com and break even killed over 1000 bad spiders … thank you for developing and maintaining compatible with the current version of wordpress.

Read all 3 reviews

Contributors & Developers

“Stop Web Crawlers” is open source software. The following people have contributed to this plugin.


Translate “Stop Web Crawlers” into your language.

Interested in development?

Browse the code, check out the SVN repository, or subscribe to the development log by RSS.



Database schema improvements
Database upgrade feature
Dashboard UI enhancements
Crawler Logging feature
Enhanced Activation
Improved Uninstall
Improved crawler list updates


Added submit a feature email
Responsive Image in Sidebar
Algorithm improvements to Bot detection
Updated product description url
added version number checker
added access to settings on plugin settings


Removed duplicate menu label in admin menu
Fixed error to relative paths in views
Added Twitter follow icon on dashboard


Improved Dashboard page
Added Admin CSS styles
Tidied up List Table columns


Code clean up removing deprecated classes


Major refactor of the code base 


Fixed Urgent bug caused by deprecated call


Typo fixed in readme file
Fixed error in which we were inadvertently linking to a spmmer in readme
updated dashboard 


Updated the database to include over 1100 known referer spam bots
Improve blocking
Updated Dashboard Text
Tested against WordPress beta 4.7


Added Logo to Admin Dashboard Menu
Add New Web Crawler functionality implemented
Dashboard View Revamp
Additional Refactoring to improve long term support


Updated license header
Improved refactoring
Added minimum WordPress version check


Added functionality for filter not run for logged in users or admin users


Updated readme text 


Fix upgrade bug and patched HTML formatting


Bug fix from a source control merge issue


Cosmetic changes


Initial release