BotTrail – AI Crawler Tracker, robots.txt Fix & llms.txt

Description

AI assistants like ChatGPT, Gemini and Perplexity now answer the questions your customers used to search for. This plugin shows how AI sees your WordPress site.

Free, works straight away:

  • AI readiness score. Open the plugin and get a score out of 100 straight away — no account, no waiting for data: whether AI crawlers can read the site, “Discourage search engines”, llms.txt, XML sitemap, structured data, HTTPS, and how well your pages are written for AI to quote. Each check links to its fix.
  • Which pages are ready for AI. Every published post, page and product checked for what helps AI assistants quote it, weakest first, with how often AI crawlers read each one and how many visits it gets from AI answers.
  • AI share of your traffic. Counts page visits without cookies and shows what share came from ChatGPT, Perplexity, Gemini, Copilot and Claude answers, which assistant sent them, and where they land. Works with page caches. Stores only the page, the day and the AI source — no IP address, nothing sent anywhere. Can be turned off.
  • Revenue from AI (WooCommerce). Orders and revenue from people who came from AI answers, by assistant, and what they bought — from WooCommerce’s own order attribution.
  • AI visitors. Counts requests from AI crawlers (GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, ChatGPT-User, Google-CloudVertexBot, Applebot and more — list version 2026-10-06) and visits that arrive from AI answers, by crawler, by page and by month, for 12 months — kept in your own WordPress database.
  • AI crawler access. Reads your robots.txt and shows which AI crawlers are blocked, and the exact line that blocks them. One click allows the ones you choose (WordPress admin stays off-limits; undo any time). Warns when “Discourage search engines” is on.
  • Blocked-crawler alerts. Checks your robots.txt daily and warns you — in WordPress, and by email if you want — when a security or SEO plugin or a site change starts blocking an AI crawler.
  • AI check in the editor. A panel in the block editor checks each post for what helps AI assistants quote it (a direct answer up top, question headings, lists, FAQ, sources, image descriptions) and shows how often AI crawlers read it.
  • llms.txt. Builds an llms.txt from your pages and posts, lets you edit it, and can serve it at /llms.txt.
  • Dashboard widget with this week’s AI crawler visits, your AI readiness, and revenue from AI.

Optionally, add your email to keep your AIORank details for the site and get tips and offers (an unticked box). Every feature works without it.

With a free AIORank account (optional):

  • Your AI visibility score, which AI engines name you, and which competitor they recommend instead — right in WordPress.
  • Paid AIORank plans add more questions, daily checks, a to-do list of fixes, and verified AI crawler history (fake bots filtered out using each AI company’s published IP list).

External services

This plugin connects to AIORank (https://www.aiorank.io), a service that measures how AI assistants talk about websites. It sends data only in these cases:

  • When you enter your email — to https://www.aiorank.io/api/wp/lead: the email, your name if given, the site’s address and name, its timezone and language setting, the plugin and WordPress versions, and whether you ticked the box for tips and offers.
  • When you click Connect — your browser opens https://www.aiorank.io/connect/wordpress (with the site’s address and name); after you sign in there, the plugin sends the one-time code it receives back to https://www.aiorank.io/api/wp/exchange to get its connection key.
  • While connected — the plugin reads your results from https://www.aiorank.io/api/wp/summary (at most every few hours, and when you open the BotTrail page).
  • While connected on an AIORank plan with AI crawler tracking — requests from AI crawlers (user agent, page path, status code and the crawler’s IP address, to verify it) and visits that came from AI answers (page path and the AI site that referred them — never the visitor’s IP address) are sent to https://www.aiorank.io/api/agent-logs/ingest. No other traffic is sent.

The robots.txt check reads your own site’s robots.txt. AIORank Terms of Service: https://www.aiorank.io/terms — Privacy Policy: https://www.aiorank.io/privacy-policy

Screenshots

Installation

  1. Plugins Add New search “BotTrail”, or upload the zip. Activate.
  2. Open BotTrail in the admin menu. The free tools work straight away.
  3. Optional: click Check with a free AIORank account to connect.

FAQ

Does it slow my site down?

No. It checks the user agent after the page has been generated. When connected on a plan with crawler tracking, events are sent in the background in small batches; if AIORank can’t be reached, your site is unaffected.

What data is sent to AIORank?

Only what is listed under External services. Your AI visitor counts stay in your WordPress database.

Why don’t I see AI crawler visits to cached pages?

Full-page caches serve pages without running WordPress or this plugin, and AI crawlers don’t run JavaScript. Visits from people (including those from AI answers) are counted on cached pages too. If your site is on Cloudflare, the AIORank Cloudflare Worker sees every crawler request.

Does visit counting use cookies or need a consent banner?

No cookies, and no personal data: for each visit it stores the page address, the day and — if the visitor came from an AI answer — which assistant. No IP address is stored and nothing leaves your site. A suggested privacy policy paragraph is added under Settings Privacy. Turn counting off under Advanced if you don’t want it.

Where does “Revenue from AI” come from?

WooCommerce’s order attribution (WooCommerce 8.5 or later, on by default): an order counts when the visitor came from an AI assistant’s website or a link tagged with its utm_source (ChatGPT adds utm_source=chatgpt.com). Completed, processing and on-hold orders, refunds taken off.

Reviews

There are no reviews for this plugin.

Contributors & Developers

“BotTrail – AI Crawler Tracker, robots.txt Fix & llms.txt” is open source software. The following people have contributed to this plugin.

Contributors

Changelog

2.5.1

  • Code clean-up for the WordPress.org review (Plugin Check): output escaping and database table names. No change in features.

2.5.0

  • AI readiness score on first open: robots.txt, “Discourage search engines”, llms.txt, sitemap, schema, HTTPS and page content, each with its fix.
  • Pages list: every published post, page and product against the AI checks, weakest first, with AI crawler reads and AI visits.
  • AI share of traffic: cookieless visit counting (works with page caches) and which AI assistant sent each visit. Visits from AI answers on cached pages are now counted.
  • Revenue from AI for WooCommerce stores, with what AI shoppers buy.
  • Fix: two Connect buttons on one screen no longer make the first one expire.

2.4.0

  • Plugin name: BotTrail (it connects to the AIORank service).
  • Section bar at the top of the plugin screen (Overview, Buyer questions, AI visitors, Crawler access, llms.txt, Settings) with status at a glance.
  • Clearer score trend: labelled axis, a dot per check with its score on hover.
  • Buyer questions list the misses first, with a named/missed bar.
  • AI visitors: monthly chart with 7-day, 30-day and AI-answer totals; crawler table on its own row; page bars.
  • Dashboard widget shows the score trend and which AI assistants name you.

2.3.0

  • Redesigned plugin screens: score, trend, buyer questions, share of AI mentions, top fixes, AI visitors by month and crawler, in AIORank’s look.
  • Redesigned dashboard widget.
  • Any AIORank connection key works (including one pasted from Bot Monitor), and a refused key now says why.

2.2.0

  • One-click fix for AI crawlers blocked by robots.txt (when WordPress serves robots.txt), with undo.
  • Daily check that warns when an AI crawler becomes blocked; optional email to the site admin.
  • “AI check” panel in the block editor.
  • robots.txt check works when the site can’t fetch its own URL.
  • Fix: saving Advanced settings no longer clears the email registration.

2.1.0

  • AI visitor history kept 12 months, with a month-by-month view.
  • Optional email sign-up; tips and offers only if you tick the box.

2.0.0

  • Free tools: AI visitor counts, robots.txt AI crawler check, llms.txt builder, dashboard widget.
  • One-click connect to a free AIORank account.

1.0.0

  • First release.