Title: Plugins categorized as crawlers | WordPress.org --- Search * [All](https://wordpress.org/plugins/) * [Community](https://wordpress.org/plugins/tags/crawlers/?plugin_business_model=community) * [Commercial](https://wordpress.org/plugins/tags/crawlers/?plugin_business_model=commercial) # Plugin Tag: crawlers * ![](https://ps.w.org/block-ai-crawlers/assets/icon-256x256.png?rev=3074623) * ### 󠀁[Block AI Crawlers](https://wordpress.org/plugins/block-ai-crawlers/)󠁿 * (8 total ratings) * Adds robots.txt rules blocking AI crawlers and AI bots. * lastsplash (a11n) 1,000+ active installations Tested with 7.0.4 * ![](https://ps.w.org/robotstxt-rewrite/assets/icon-128x128.png?rev=1409259) * ### 󠀁[Robots.txt rewrite](https://wordpress.org/plugins/robotstxt-rewrite/)󠁿 * (2 total ratings) * Provide the easy managment of your robots.txt from admin side. It propose you the advanced then standard robots.txt content too. * Eugen Bobrowski 1,000+ active installations Tested with 4.7.35 * ![](https://ps.w.org/ai-content-signals/assets/icon-256x256.gif?rev=3397536) * ### 󠀁[AI Content Signals](https://wordpress.org/plugins/ai-content-signals/)󠁿 * (3 total ratings) * Control how AI uses your content — search, AI answers or training — via robots. txt Content Signals, HTTP headers, meta tags and TDMRep. * Fernando Tellado 600+ active installations Tested with 7.1 * ![](https://ps.w.org/coding-bunny-llms-generator/assets/icon-256x256.png?rev= 3386275) * ### 󠀁[CodingBunny LLMs.txt Generator](https://wordpress.org/plugins/coding-bunny-llms-generator/)󠁿 * (1 total ratings) * Advanced llms.txt generator with AI metadata, Schema.org types, canonical URLs, and differentiated licenses for optimal LLM indexing. * CodingBunny 100+ active installations Tested with 6.9.7 * ![](https://s.w.org/plugins/geopattern-icon/meta-robots-by-seo-sign.svg) * ### 󠀁[MetaRobots by SEO-Sign](https://wordpress.org/plugins/meta-robots-by-seo-sign/)󠁿 * (1 total ratings) * The easiest way to manage meta robots tag. * Artem Pilipets 100+ active installations Tested with 4.1.42 * ![](https://s.w.org/plugins/geopattern-icon/robotswhiz.svg) * ### 󠀁[Robots Meta Whiz](https://wordpress.org/plugins/robotswhiz/)󠁿 * (0 total ratings) * Easy way to discourage search engines from indexing only specific pages / posts with custom meta tags. * Marat Nepomnyashy 100+ active installations Tested with 4.7.35 * ![](https://ps.w.org/serve-md/assets/icon-256x256.png?rev=3483080) * ### 󠀁[Serve Markdown](https://wordpress.org/plugins/serve-md/)󠁿 * (0 total ratings) * Serve Markdown versions of your posts and pages to AI agents and crawlers. Content negotiation, .md URLs, auto-discovery, and crawler logging. * Azey 70+ active installations Tested with 6.9.7 * ![](https://ps.w.org/parseless/assets/icon-256x256.png?rev=3562481) * ### 󠀁[ParseLess](https://wordpress.org/plugins/parseless/)󠁿 * (0 total ratings) * Serve your posts as Markdown to AI crawlers and CLI tools — typically 95%+ less bandwidth, ~60–80% fewer DB queries, no theme bloat. * phalkmin 50+ active installations Tested with 7.0.4 * ![](https://ps.w.org/ceewp-robots-txt-manager/assets/icon-256x256.png?rev=3344900) * ### 󠀁[CeeWP Robots.txt Manager](https://wordpress.org/plugins/ceewp-robots-txt-manager/)󠁿 * (0 total ratings) * Easily manage your WordPress site's virtual robots.txt file with custom rules. No physical file needed! * CeeWP 40+ active installations Tested with 6.8.8 * ![](https://ps.w.org/seo-crawlytics-data-export/assets/icon-128x128.png?rev=982868) * ### 󠀁[Seo Crawlytics Data Export & Delete](https://wordpress.org/plugins/seo-crawlytics-data-export/)󠁿 * (0 total ratings) * Exports all entries of Seo Crawlytics and gives you the option to delete the entries that are older than 1 month. * apasionados 20+ active installations Tested with 6.3.10 * ![](https://ps.w.org/temso-ai/assets/icon.svg?rev=3567519) * ### 󠀁[Temso AI](https://wordpress.org/plugins/temso-ai/)󠁿 * (0 total ratings) * Stream front-end request logs from your WordPress origin to Temso so AI-crawler and bot traffic shows up in your dashboard. * Temso AI 20+ active installations Tested with 7.0.4 * ![](https://ps.w.org/loghero/assets/icon-256x256.png?rev=1892779) * ### 󠀁[Log-Hero.com Plugin](https://wordpress.org/plugins/loghero/)󠁿 * (1 total ratings) * See which search engines, AI crawlers and other bots visit your site, and what they find. The official plugin for log-hero.com. * Log Hero 20+ active installations Tested with 7.1 * ![](https://s.w.org/plugins/geopattern-icon/ljpl-armored-robotstxt.svg) * ### 󠀁[LJPL Armored robots.txt](https://wordpress.org/plugins/ljpl-armored-robotstxt/)󠁿 * (0 total ratings) * Add some directives to your robots.txt file to keep your site safer * Lukasz Jasinski 10+ active installations Tested with 3.4.2 * ![](https://ps.w.org/ai-rights-protection-by-simplefeed/assets/icon-128x128.png? rev=3164318) * ### 󠀁[AI Rights Protection by SimpleFeed](https://wordpress.org/plugins/ai-rights-protection-by-simplefeed/)󠁿 * (1 total ratings) * Protect your rights by reviewing and blocking AI bots from using your content without compensation. * Alex Khomichenko Fewer than 10 active installations Tested with 6.7.7 * ![](https://ps.w.org/rawi-md/assets/icon-256x256.png?rev=3655277) * ### 󠀁[Rawi MD — Markdown for AI Crawlers](https://wordpress.org/plugins/rawi-md/)󠁿 * (0 total ratings) * Give ChatGPT, Claude and other AI crawlers a clean Markdown copy of your posts, pages and products. Nothing changes for your visitors. * Ahmed Abdelhamed Fewer than 10 active installations Tested with 7.0.4 * ![](https://ps.w.org/ai-mention-visibility/assets/icon-256x256.png?rev=3649526) * ### 󠀁[AI Mention Visibility](https://wordpress.org/plugins/ai-mention-visibility/)󠁿 * (0 total ratings) * See which AI crawlers (GPTBot, ClaudeBot, Perplexity and more) reach your site, and the visitors AI sends you. Verified, and free. * aimention Fewer than 10 active installations Tested with 7.0.4 * ![](https://ps.w.org/botkibble/assets/icon.svg?rev=3470658) * ### 󠀁[Botkibble](https://wordpress.org/plugins/botkibble/)󠁿 * (0 total ratings) * Serves every published post and page as Markdown for AI agents and crawlers. No configuration, no API keys. Activate and it works. * gregrandall Fewer than 10 active installations Tested with 6.9.7 * ![](https://ps.w.org/supertab-connect/assets/icon.svg?rev=3508698) * ### 󠀁[Supertab Connect](https://wordpress.org/plugins/supertab-connect/)󠁿 * (1 total ratings) * Connect your WordPress site to the Supertab Connect platform for RSL license serving and enabling crawler authentication protocol. * getsupertab Fewer than 10 active installations Tested with 7.0.4 * ![](https://ps.w.org/puffersights-ai-crawler-insights/assets/icon.svg?rev=3574171) * ### 󠀁[PufferSights – AI Crawler Insights](https://wordpress.org/plugins/puffersights-ai-crawler-insights/)󠁿 * (0 total ratings) * Monitor 100+ known AI crawler and AI agent user agents with local analytics, AI referrals, llms.txt, and opt-in robots.txt policy tools. * senols Fewer than 10 active installations Tested with 7.0.4 * ![](https://ps.w.org/frenemy/assets/icon-256x256.png?rev=3612092) * ### 󠀁[Frenemy](https://wordpress.org/plugins/frenemy/)󠁿 * (0 total ratings) * See every bot that reaches WordPress — search crawlers, AI agents, scrapers, impostors — classified and identity-verified, in one dashboard. * frenemydev Fewer than 10 active installations Tested with 7.0.4