Title: Latchbot &#8211; AI Crawler Control
Author: maksymkotenko
Published: <strong>November 14, 2026</strong>
Last modified: November 14, 2026

---

Search plugins

![](https://ps.w.org/latchbot/assets/banner-772x250.png?rev=3801798)

![](https://ps.w.org/latchbot/assets/icon.svg?rev=3801798)

# Latchbot – AI Crawler Control

 By [maksymkotenko](https://profiles.wordpress.org/maksymkotenko/)

[Download](https://downloads.wordpress.org/plugin/latchbot.0.1.0.zip)

 * [Details](https://wordpress.org/plugins/latchbot/#description)
 * [Reviews](https://wordpress.org/plugins/latchbot/#reviews)
 *  [Installation](https://wordpress.org/plugins/latchbot/#installation)
 * [Development](https://wordpress.org/plugins/latchbot/#developers)

 [Support](https://wordpress.org/support/plugin/latchbot/)

## Description

AI companies send crawlers for three different reasons: to collect training data,
to build an AI search index, and to fetch a page because a user asked a chatbot 
about it. Latchbot lets you decide per crawler, from one screen under **Settings
> AI Crawlers**.

#### Features

 * **One rule per crawler** – allow or block GPTBot, ClaudeBot, Google-Extended,
   PerplexityBot, CCBot, Applebot-Extended, Bytespider, Meta-ExternalAgent and about
   20 more, each labelled with its operator and purpose (AI training, AI search 
   or fetches for a user).
 * **Presets** – “Block AI training, allow AI search”, “Block all AI crawlers” or“
   Allow all” in one click, then fine-tune.
 * **robots.txt rules** – added to the robots.txt WordPress generates, one group
   per blocked crawler, with a preview on the settings screen.
 * **Refuse crawlers that ignore robots.txt** – optionally answer 403 to blocked
   crawlers; robots.txt itself always stays readable.
 * **TDMRep** – reserve text and data mining rights under the W3C TDMRep protocol:`/.
   well-known/tdmrep.json`, the `tdm-reservation` header and meta tag, with an optional
   licensing policy URL.
 * **Content Signals** – add a `Content-Signal: search=yes, ai-input=yes, ai-train
   =no` line to robots.txt, with each value your choice.
 * **noai and noimageai** – in the robots meta tag and the `X-Robots-Tag` header.
 * **Visit log** – how often each AI crawler came in the last 30 days, when it was
   last seen, and which kinds of pages it read (posts, pages, products, feeds and
   so on).
 * **New crawlers** – updates add newly announced crawlers; a setting blocks the
   new training crawlers automatically.

On activation Latchbot blocks the AI training crawlers and allows AI search and 
user fetches. Nothing else changes until you turn it on.

#### Privacy

The visit log stores the crawler name, the day, the kind of page and two counters.
It never stores IP addresses, page addresses, cookies or any other visitor data,
and it keeps 90 days. Latchbot makes no requests to other sites and loads nothing
from them.

#### Good to know

 * robots.txt is a request, not a lock: well-behaved crawlers follow it. Turn on“
   Refuse blocked crawlers” for those that do not.
 * If your site root has a physical robots.txt file, the web server sends that file
   and WordPress rules do not apply. Latchbot warns about it and shows the lines
   to copy.
 * Page caches and CDNs answer many requests without running WordPress. Those visits
   are not counted and not refused by the plugin; block crawlers at the CDN too 
   if it offers that.
 * Google-Extended and Applebot-Extended are robots.txt tokens only: Google and 
   Apple crawl with their normal search bots and read the token to decide whether
   pages may be used for AI. They never show up in the visit log.

#### For developers

 * `latchbot_bots` filter – add, change or remove crawlers; the list is built once
   per request, so add the filter from a plugin or your theme’s functions.php. Each
   entry has `token`, `name`, `operator`, `purpose` (`training`, `search` or `user`)
   and `ua` (the User-Agent substring; empty for robots.txt-only tokens).
 * `latchbot_refuse` filter – decide whether a request from a known crawler gets
   a 403. Arguments: the decision, the crawler id and the kind of page.
 * `latchbot_visit` action – runs after a crawler visit is counted, with the crawler
   id, the kind of page and whether it was refused.

## Screenshots

[⌊Every AI crawler with its operator, purpose, rule and visits.⌉⌊Every AI crawler
with its operator, purpose, rule and visits.⌉[

Every AI crawler with its operator, purpose, rule and visits.

[⌊Signals for all crawlers: noai, TDMRep and Content Signals.⌉⌊Signals for all crawlers:
noai, TDMRep and Content Signals.⌉[

Signals for all crawlers: noai, TDMRep and Content Signals.

[⌊The robots.txt additions and what AI crawlers read.⌉⌊The robots.txt additions 
and what AI crawlers read.⌉[

The robots.txt additions and what AI crawlers read.

## Installation

 1. Install and activate Latchbot from Plugins > Add New.
 2. Open Settings > AI Crawlers, pick a preset or set each crawler, and save.
 3. Open your robots.txt from the link on the settings screen to check the result.

## FAQ

### Does blocking GPTBot or Google-Extended hurt my Google ranking?

No. Google-Extended only controls whether Google may use your pages for its AI models;
Google Search keeps crawling with Googlebot, which Latchbot never touches. The same
goes for Applebot-Extended and Applebot.

### What is the difference between training, search and user fetches?

Training crawlers collect pages to train AI models. AI search crawlers build the
index that AI answers cite and link to. User fetches happen when someone asks an
assistant about your page. Many sites block training and allow the other two, which
is the first preset.

### Does it work with a caching plugin?

The robots.txt rules, the TDMRep file and the meta tags are part of the page, so
cached copies keep them. The X-Robots-Tag and tdm-reservation headers, the visit
counts and the 403 answers need WordPress to run, so they cover only requests your
cache passes through; some caches store headers too.

### My site is in a subdirectory.

Crawlers read robots.txt and /.well-known/tdmrep.json only at the root of the domain.
Latchbot shows a warning; copy the rules from the preview into the files at the 
root.

## Reviews

There are no reviews for this plugin.

## Contributors & Developers

“Latchbot – AI Crawler Control” is open source software. The following people have
contributed to this plugin.

Contributors

 *   [ maksymkotenko ](https://profiles.wordpress.org/maksymkotenko/)

[Translate “Latchbot – AI Crawler Control” into your language.](https://translate.wordpress.org/projects/wp-plugins/latchbot)

### Interested in development?

[Browse the code](https://plugins.trac.wordpress.org/browser/latchbot/), check out
the [SVN repository](https://plugins.svn.wordpress.org/latchbot/), or subscribe 
to the [development log](https://plugins.trac.wordpress.org/log/latchbot/) by [RSS](https://plugins.trac.wordpress.org/log/latchbot/?limit=100&mode=stop_on_copy&format=rss).

## Changelog

#### 0.1.0

 * First release: per-crawler rules for 28 AI crawlers, presets, robots.txt rules,
   optional 403, TDMRep, Content Signals, noai, and a visit log without IP addresses.

## Meta

 *  Version **0.1.0**
 *  Last updated **4 days ago**
 *  Active installations **Fewer than 10**
 *  WordPress version ** 6.5 or higher **
 *  Tested up to **7.1.4**
 *  PHP version ** 7.4 or higher **
 * Tags
 * [ai bots](https://wordpress.org/plugins/tags/ai-bots/)[AI Crawlers](https://wordpress.org/plugins/tags/ai-crawlers/)
   [block ai](https://wordpress.org/plugins/tags/block-ai/)[robots.txt](https://wordpress.org/plugins/tags/robots-txt/)
 *  [Advanced View](https://wordpress.org/plugins/latchbot/advanced/)

## Ratings

No reviews have been submitted yet.

[Your review](https://wordpress.org/support/plugin/latchbot/reviews/#new-post)

[See all reviews](https://wordpress.org/support/plugin/latchbot/reviews/)

## Contributors

 *   [ maksymkotenko ](https://profiles.wordpress.org/maksymkotenko/)

## Support

Got something to say? Need help?

 [View support forum](https://wordpress.org/support/plugin/latchbot/)