GlobalDev AI Traffic Control

Description

GlobalDev AI Traffic Control is primarily an AI crawler traffic monitoring and access-control plugin. It is not an llms.txt generator with a few extra settings: its core job is to identify known AI crawler requests, record what they request, evaluate those requests against your policy, and optionally enforce that policy.

Monitor identifiable AI crawler traffic. The dashboard records requests from supported AI crawler user agents, including which crawler requested which URL, when it happened, and whether the request matched or violated your configured policy. Ordinary visitors are not logged. Raw IP addresses are not stored; when an identifier is needed, the plugin stores a one-way hash.

Create a policy hierarchy. Set a site-wide default, override it for a specific crawler, and override that again for an individual post or page. This lets you block model-training crawlers while still allowing AI search crawlers or live assistant fetchers, or exclude only selected content.

Choose the enforcement level. You can record policy violations only, send X-Robots-Tag: noai, noimageai, or return HTTP 403 for recognized crawler requests that are not allowed.

Publish matching robots.txt rules. The plugin adds AI crawler directives to WordPress’ virtual robots.txt and can include URL-specific exclusions. A preview screen shows the rules that would be published.

Keep an audit trail. Crawler requests and policy decisions are stored locally in a dedicated WordPress database table, with configurable retention and automatic cleanup.

Optionally publish llms.txt. llms.txt support is an additional feature and is disabled by default. If enabled, the plugin can publish a Markdown summary at /llms.txt, omit content you have excluded, and include your own description and usage terms.

What makes this different

The plugin combines four functions in one policy workflow:

• request-level analytics for identifiable AI crawlers;
• site-wide, per-crawler, and per-content access decisions;
• active enforcement through headers or HTTP 403 responses; and
• robots.txt output derived from the same policy.

The optional llms.txt output is generated from that policy rather than being the main purpose of the plugin.

What this plugin does not claim

User-Agent identification is not a firewall. A scraper that disguises itself as an ordinary browser cannot be reliably identified by this plugin. robots.txt is also advisory and depends on crawler cooperation.

The plugin therefore manages crawlers that identify themselves through supported User-Agent tokens and provides enforcement for requests that WordPress actually receives.

External services

This plugin does not connect to, depend on, or send data to any third-party service. AI crawler identification is performed locally using User-Agent strings included with incoming requests. The plugin does not call crawler-provider APIs, load remote scripts, or send analytics to GlobalDev.