Title: ReadyStack Robots Txt AI Crawler Audit Author: readystack Published: October 13, 2026 Last modified: October 13, 2026 --- Search plugins ![](https://s.w.org/plugins/geopattern-icon/readystack-robots-txt-ai-crawler-audit. svg) # ReadyStack Robots Txt AI Crawler Audit By [readystack](https://profiles.wordpress.org/readystack/) [Download](https://downloads.wordpress.org/plugin/readystack-robots-txt-ai-crawler-audit.1.0.0.zip) * [Details](https://wordpress.org/plugins/readystack-robots-txt-ai-crawler-audit/#description) * [Reviews](https://wordpress.org/plugins/readystack-robots-txt-ai-crawler-audit/#reviews) * [Installation](https://wordpress.org/plugins/readystack-robots-txt-ai-crawler-audit/#installation) * [Development](https://wordpress.org/plugins/readystack-robots-txt-ai-crawler-audit/#developers) [Support](https://wordpress.org/support/plugin/readystack-robots-txt-ai-crawler-audit/) ## Description Checks robots.txt line by line and says what each AI crawler token actually controls– training, AI-search citation, or user fetch – and which lines. #### What it does It reads a robots.txt line by line and reports what each line really controls. It flags tokens that no crawler answers to, such as GPT-Bot or the retired Claude-Web, and separates training crawlers like GPTBot from citation crawlers like OAI-SearchBot, so you can see which lines cost you visibility in AI answers. #### How to use it 1. Install and activate the plugin. 2. Edit the page or post where you want the tool and add the shortcode `[readystack_robots_txt_ai_crawler_audit]`. In the block editor, use a Shortcode block. 3. Open the page, paste the file you want to check into the box, and read the findings listed underneath. Nothing is uploaded. #### What you can do with the results Each finding names the rule it breaks and the fix. Correct the flagged lines, paste the file again, and keep the clean result as a dated record of the check for your checklist, ticket or client report. #### Privacy The tool runs in the browser of whoever opens the page. The plugin makes no external requests, sets no cookies, adds no tracking and stores nothing in your database. It has no settings screen. #### Source code The JavaScript and CSS files in the plugin’s assets folder are the complete, human- readable source. There is no build step: nothing is compiled, bundled or minified, so what you read is what runs. #### Free and paid The free tool on this page is complete for one pasted file at a time: every finding is listed with the rule it breaks and the fix. Nothing in this plugin is locked or time-limited. A separate paid version for teams is sold on our website, not inside this plugin. Full version: https://getreadystack.com/tools/robots-txt-ai-crawler- audit?ref=wporg ## Installation 1. Upload the plugin folder to `/wp-content/plugins/`, or install it from the Plugins screen. 2. Activate it through the Plugins screen in WordPress. 3. Add the shortcode `[readystack_robots_txt_ai_crawler_audit]` to the page where you want the tool. ## FAQ ### Why not just ask a chatbot to write my robots.txt? Assistants confidently produce tokens that do nothing, because the crawler names changed after their training data was collected. Claude-Web and anthropic-ai were retired in 2024, and GPT-Bot with a hyphen is not a real token. robots.txt ignores unknown names silently, so a wrong file looks correct and blocks nothing. ### Does the plugin send data anywhere? No. Everything is worked out in the browser and nothing leaves the page. ### Do I need an account or an API key? No. The free tool is self-contained. ## Reviews There are no reviews for this plugin. ## Contributors & Developers “ReadyStack Robots Txt AI Crawler Audit” is open source software. The following people have contributed to this plugin. Contributors * [ readystack ](https://profiles.wordpress.org/readystack/) [Translate “ReadyStack Robots Txt AI Crawler Audit” into your language.](https://translate.wordpress.org/projects/wp-plugins/readystack-robots-txt-ai-crawler-audit) ### Interested in development? [Browse the code](https://plugins.trac.wordpress.org/browser/readystack-robots-txt-ai-crawler-audit/), check out the [SVN repository](https://plugins.svn.wordpress.org/readystack-robots-txt-ai-crawler-audit/), or subscribe to the [development log](https://plugins.trac.wordpress.org/log/readystack-robots-txt-ai-crawler-audit/) by [RSS](https://plugins.trac.wordpress.org/log/readystack-robots-txt-ai-crawler-audit/?limit=100&mode=stop_on_copy&format=rss). ## Changelog #### 1.0.0 * First release. ## Meta * Version **1.0.0** * Last updated **1 day ago** * Active installations **Fewer than 10** * WordPress version ** 6.0 or higher ** * Tested up to **7.1.3** * PHP version ** 7.4 or higher ** * Tags * [AI](https://wordpress.org/plugins/tags/ai/)[audit](https://wordpress.org/plugins/tags/audit/) [compliance](https://wordpress.org/plugins/tags/compliance/)[crawler](https://wordpress.org/plugins/tags/crawler/) [robots](https://wordpress.org/plugins/tags/robots/) * [Advanced View](https://wordpress.org/plugins/readystack-robots-txt-ai-crawler-audit/advanced/) ## Ratings No reviews have been submitted yet. [Your review](https://wordpress.org/support/plugin/readystack-robots-txt-ai-crawler-audit/reviews/#new-post) [See all reviews](https://wordpress.org/support/plugin/readystack-robots-txt-ai-crawler-audit/reviews/) ## Contributors * [ readystack ](https://profiles.wordpress.org/readystack/) ## Support Got something to say? Need help? [View support forum](https://wordpress.org/support/plugin/readystack-robots-txt-ai-crawler-audit/)