{"id":357449,"date":"2026-08-27T06:13:48","date_gmt":"2026-08-27T06:13:48","guid":{"rendered":"https:\/\/wordpress.org\/plugins\/adg-ai-markdown-llms-txt\/"},"modified":"2026-09-03T10:14:59","modified_gmt":"2026-09-03T10:14:59","slug":"adg-markdown-for-ai-crawlers","status":"publish","type":"plugin","link":"https:\/\/wordpress.org\/plugins\/adg-markdown-for-ai-crawlers\/","author":15667861,"comment_status":"closed","ping_status":"closed","template":"","meta":{"version":"1.1.0","stable_tag":"1.1.0","tested":"7.1","requires":"6.0","requires_php":"7.4","requires_plugins":null,"header_name":"ADG Markdown for AI Crawlers","header_author":"A-DevGroup","header_description":"Serves AI crawlers clean Markdown of the main content on the same URL (content negotiation) and auto-generates a virtual llms.txt \/ llms-full.txt. No .htaccess edits, no external services.","assets_banners_color":"384047","last_updated":"2026-09-03 10:14:59","external_support_url":"","external_repository_url":"","donate_link":"","header_plugin_uri":"https:\/\/www.a-devgroup.com\/adg-ai-markdown\/","header_author_uri":"https:\/\/a-devgroup.com","rating":0,"author_block_rating":0,"active_installs":0,"downloads":92,"num_ratings":0,"support_threads":0,"support_threads_resolved":0,"author_block_count":0,"sections":["description","installation","faq","changelog"],"tags":{"1.0.0":{"tag":"1.0.0","author":"andyodua","date":"2026-08-27 06:13:39","revision":3668173},"1.1.0":{"tag":"1.1.0","author":"andyodua","date":"2026-09-03 10:14:59","revision":3679537}},"upgrade_notice":{"1.1.0":"<p>YAML front matter, custom fields\/ACF, taxonomy archives, WooCommerce support, term-based exclusions, cache compatibility, and a Markdown preview link.<\/p>","1.0.0":"<p>Initial release.<\/p>"},"ratings":[],"assets_icons":{"icon-256x256.png":{"filename":"icon-256x256.png","revision":3668173,"resolution":"256x256","location":"assets","locale":"","width":256,"height":256},"icon.svg":{"filename":"icon.svg","revision":3668173,"resolution":false,"location":"assets","locale":false}},"assets_banners":{"banner-772x250.png":{"filename":"banner-772x250.png","revision":3668173,"resolution":"772x250","location":"assets","locale":"","width":772,"height":250}},"assets_blueprints":{},"all_blocks":[],"tagged_versions":["1.0.0","1.1.0"],"block_files":[],"assets_screenshots":{"screenshot-1.png":{"filename":"screenshot-1.png","revision":3679615,"resolution":"1","location":"assets","locale":"","width":2000,"height":4720},"screenshot-2.png":{"filename":"screenshot-2.png","revision":3679615,"resolution":"2","location":"assets","locale":"","width":800,"height":500}},"screenshots":{"1":"The settings screen: content negotiation, editable AI-bot list, and llms.txt options.","2":"The \"Hide from AI bots\" checkbox on the post editor."}},"plugin_section":[],"plugin_tags":[2353,9837,247966,244604,4608],"plugin_category":[],"plugin_contributors":[277767],"plugin_business_model":[],"class_list":["post-357449","plugin","type-plugin","status-publish","hentry","plugin_tags-ai","plugin_tags-crawler","plugin_tags-gptbot","plugin_tags-llms-txt","plugin_tags-markdown","plugin_contributors-andyodua","plugin_committers-andyodua"],"banners":{"banner":"https:\/\/ps.w.org\/adg-markdown-for-ai-crawlers\/assets\/banner-772x250.png?rev=3668173","banner_2x":false,"banner_rtl":false,"banner_2x_rtl":false},"icons":{"svg":"https:\/\/ps.w.org\/adg-markdown-for-ai-crawlers\/assets\/icon.svg?rev=3668173","icon":"https:\/\/ps.w.org\/adg-markdown-for-ai-crawlers\/assets\/icon.svg?rev=3668173","icon_2x":false,"generated":false},"screenshots":[{"src":"https:\/\/ps.w.org\/adg-markdown-for-ai-crawlers\/assets\/screenshot-1.png?rev=3679615","caption":"The settings screen: content negotiation, editable AI-bot list, and llms.txt options."},{"src":"https:\/\/ps.w.org\/adg-markdown-for-ai-crawlers\/assets\/screenshot-2.png?rev=3679615","caption":"The \"Hide from AI bots\" checkbox on the post editor."}],"raw_content":"<!--section=description-->\n<p>When a known AI crawler (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and others) requests one of your posts or pages, this plugin returns clean <strong>Markdown<\/strong> of the main content on the <strong>same URL<\/strong> instead of the full theme HTML \u2014 no header, footer, sidebar or scripts. Regular visitors keep seeing your normal site.<\/p>\n\n<p>It also generates a virtual <strong>llms.txt<\/strong> and <strong>llms-full.txt<\/strong> \u2014 a sitemap-style index of your content for AI tools \u2014 served at <code>\/llms.txt<\/code> and <code>\/llms-full.txt<\/code> without creating any physical files.<\/p>\n\n<p><strong>What it does<\/strong><\/p>\n\n<ul>\n<li>Detects AI crawlers by User-Agent (the list is editable in the settings).<\/li>\n<li>Serves clean Markdown on the same URL with <code>Content-Type: text\/markdown<\/code>, <code>Vary: User-Agent<\/code> and <code>X-Robots-Tag: noindex<\/code>.<\/li>\n<li>Works with any public post type \u2014 including WooCommerce products \u2014 plus a Markdown index for category, tag and custom-taxonomy archives.<\/li>\n<li>Configurable <strong>YAML front matter<\/strong> (title, url, date, author, tags, canonical) with per-field toggles.<\/li>\n<li>Include your own <strong>custom fields \/ ACF<\/strong> values in the front matter.<\/li>\n<li>Adds machine-readable Schema.org JSON-LD (title, dates, author, cover image, publisher).<\/li>\n<li>Generates virtual <code>\/llms.txt<\/code> and <code>\/llms-full.txt<\/code>.<\/li>\n<li>Caches the converted Markdown in post meta for fast responses; the cache is cleared automatically when a post is edited.<\/li>\n<li>Plays nice with page caches: sets <code>DONOTCACHEPAGE<\/code> (WP Rocket, W3 Total Cache, WP Super Cache, LiteSpeed) so bots are never served a stale HTML cache.<\/li>\n<li>Exclude content with a per-post \"Hide from AI bots\" checkbox or by category\/tag slug.<\/li>\n<li>\"View Markdown\" link in the editor and post list \u2014 preview exactly what crawlers receive.<\/li>\n<\/ul>\n\n<p><strong>Privacy and external services<\/strong><\/p>\n\n<p>This plugin does <strong>not<\/strong> connect to any external service, send any data off your site, or load any remote scripts. All processing happens on your own server.<\/p>\n\n<p><strong>Good to know<\/strong><\/p>\n\n<ul>\n<li>No .htaccess edits and no Composer \u2014 a built-in, zero-dependency HTML\u2192Markdown converter is included.<\/li>\n<li>Does not interfere with SEO plugins, caches, or WooCommerce shop\/cart\/checkout pages.<\/li>\n<li>Private, password-protected and unpublished content is never served to bots \u2014 access rights are not bypassed.<\/li>\n<\/ul>\n\n<p><strong>Pro version<\/strong><\/p>\n\n<p>A Pro version is available with bot IP verification (anti-spoofing), bot-visit analytics with CSV export, a one-click self-test, and priority support. See the plugin's settings page for details. The free version is fully functional on its own and is not time-limited.<\/p>\n\n<!--section=installation-->\n<ol>\n<li>Upload the <code>adg-markdown-for-ai-crawlers<\/code> folder to <code>\/wp-content\/plugins\/<\/code>, or install the plugin through the WordPress \"Plugins\" screen.<\/li>\n<li>Activate the plugin.<\/li>\n<li>Go to <strong>Settings \u2192 AI Bots &amp; Markdown<\/strong> to configure it.<\/li>\n<li>On Nginx, if <code>\/llms.txt<\/code> returns a 404, make sure your config falls through to <code>index.php<\/code> (the standard <code>try_files $uri $uri\/ \/index.php?$args;<\/code> block covers this). On Apache it works out of the box.<\/li>\n<\/ol>\n\n<!--section=faq-->\n<dl>\n<dt id=\"will%20private%20or%20draft%20pages%20be%20exposed%20to%20bots%3F\"><h3>Will private or draft pages be exposed to bots?<\/h3><\/dt>\n<dd><p>No. Markdown is served only for published, non-password-protected, non-private posts of the allowed types. Access rights are never bypassed.<\/p><\/dd>\n<dt id=\"does%20it%20send%20my%20content%20to%20any%20external%20server%3F\"><h3>Does it send my content to any external server?<\/h3><\/dt>\n<dd><p>No. There are no outbound requests at all \u2014 detection is by User-Agent, and all conversion happens locally.<\/p><\/dd>\n<dt id=\"does%20it%20work%20with%20page%20builders%20like%20elementor%3F\"><h3>Does it work with page builders like Elementor?<\/h3><\/dt>\n<dd><p>Yes. Content is taken through <code>the_content<\/code>, so shortcodes and builder output are rendered before conversion. Testing on your specific setup is recommended.<\/p><\/dd>\n<dt id=\"a%20bot%20still%20receives%20html%20instead%20of%20markdown.%20why%3F\"><h3>A bot still receives HTML instead of Markdown. Why?<\/h3><\/dt>\n<dd><p>Most often the request is blocked at the server\/CDN level (HTTP 403) or served from a page cache before the plugin runs. Make sure your cache honors <code>Vary: User-Agent<\/code> or excludes bot user-agents.<\/p><\/dd>\n\n<\/dl>\n\n<!--section=changelog-->\n<h4>1.1.0<\/h4>\n\n<ul>\n<li>New: configurable YAML front matter (title, url, date, author, tags, canonical) with per-field toggles.<\/li>\n<li>New: include custom fields \/ ACF values in the Markdown front matter.<\/li>\n<li>New: Markdown index for category, tag and custom-taxonomy archives.<\/li>\n<li>New: WooCommerce products (and any public post type) can be served as Markdown and listed in llms.txt.<\/li>\n<li>New: exclude content by category\/tag slug, in addition to the per-post checkbox.<\/li>\n<li>New: \"View Markdown\" preview link in the post editor and post list.<\/li>\n<li>Improved: page-cache compatibility \u2014 sets DONOTCACHEPAGE for WP Rocket, W3 Total Cache, WP Super Cache and LiteSpeed.<\/li>\n<li>Fixed: list item text (simple &lt;li&gt; entries) was dropped from the Markdown output.<\/li>\n<\/ul>\n\n<h4>1.0.0<\/h4>\n\n<ul>\n<li>Initial release: content negotiation (Markdown for AI bots on the same URL), Schema.org\/JSON-LD front matter, post-level Markdown cache, virtual llms.txt \/ llms-full.txt, and a \"Hide from AI bots\" metabox.<\/li>\n<\/ul>","raw_excerpt":"Serve AI crawlers clean Markdown of your content on the same URL, and auto-generate a virtual llms.txt \/ llms-full.txt. No external services.","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin\/357449","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin"}],"about":[{"href":"https:\/\/wordpress.org\/plugins\/wp-json\/wp\/v2\/types\/plugin"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.org\/plugins\/wp-json\/wp\/v2\/comments?post=357449"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.org\/plugins\/wp-json\/wporg\/v1\/users\/andyodua"}],"wp:attachment":[{"href":"https:\/\/wordpress.org\/plugins\/wp-json\/wp\/v2\/media?parent=357449"}],"wp:term":[{"taxonomy":"plugin_section","embeddable":true,"href":"https:\/\/wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_section?post=357449"},{"taxonomy":"plugin_tags","embeddable":true,"href":"https:\/\/wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_tags?post=357449"},{"taxonomy":"plugin_category","embeddable":true,"href":"https:\/\/wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_category?post=357449"},{"taxonomy":"plugin_contributors","embeddable":true,"href":"https:\/\/wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_contributors?post=357449"},{"taxonomy":"plugin_business_model","embeddable":true,"href":"https:\/\/wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_business_model?post=357449"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}