Description
Gonzales AI Agent Markdown gives AI agents a clean Markdown version of your public WordPress content and helps you see when they request it. Your visitors continue to see your usual website.
It is for site owners, publishers and businesses that want to make their content easier for automated tools to read, while keeping control over what is available. It works alongside your existing website and SEO setup.
Main features
- Serve Markdown when a client explicitly requests it, or through a page’s /index.md address.
- Choose public content types, exclude individual paths or groups of pages, and remove unwanted page elements.
- Keep useful headings, links, tables, images and metadata in the Markdown output.
- Respect private, password-protected and noindex content, your WordPress privacy page and protected WooCommerce pages.
- See request history, claimed crawler names, channels and estimated tokens; filter results and export CSV.
- Recognize about 70 crawlers and AI agents by purpose (AI training, AI search, AI assistants acting for a user, search engines, SEO tools, link previews), list the user agents that are not recognized, and prepare robots.txt lines for the bots you choose to block. The plugin never edits robots.txt.
- Optionally check whether a crawler really is who it claims to be, using the official IP lists and host names published by its operator (off by default).
- Review practical diagnostics and run a local self-test in your WordPress dashboard.
- Use an optional personal dashboard widget. The interface is translation-ready through translate.wordpress.org.
Markdown makes content available in another useful format. It does not guarantee that an AI service will discover, cite or use your website, or that your search rankings will improve. Crawler names in the log come from their user agents. Unless you enable the optional bot identity check, they are not verified identities.
Hosting and compatibility
The plugin supports Apache-compatible routing and provides an Nginx configuration snippet in Diagnostics. Some hosting setups need help from the hosting provider before Markdown can bypass an existing page cache. Diagnostics explains the checks and supplies commands for external testing.
Pages that another plugin builds from one shared WordPress page through its own address rules (for example a template page used for many locations) are converted at their own address. Content can come from the rendered page (the default), with an automatic fallback to filtered article content if the internal request fails. SEO integrations support Yoast SEO, Rank Math, All in One SEO, SEOPress and The SEO Framework. Site-specific themes, cache configurations and multilingual setups should be checked on your own site.
Markdown is converted when it is first requested and kept in a protected per-page cache, outside the plugin directory, together with the server-rules backups. Page Markdown supports conditional requests and a configurable HTTP cache lifetime.
On Apache and LiteSpeed the plugin adds a small marked block to .htaccess that sends Markdown requests to WordPress’s own index.php with an internal parameter, so a page cache does not answer them with HTML. For these Markdown requests only, the block also adds a server-side cookie that page caches such as W3 Total Cache treat as uncacheable; it is never sent to visitors. The block is removed on deactivation. The X-AI-Agent-Markdown-Cache debug header is emitted only when WP_DEBUG is enabled.
Data and privacy
The request log stays in your own WordPress database. It records the requested public URL without query parameters, time, channel, sanitized user agent, claimed bot, response details, estimated tokens and an anonymized IP address. IPv4 addresses lose their last octet and IPv6 addresses their last 80 bits. Cookies and raw IP addresses are not stored by the plugin.
Logging is enabled by default with a 90-day retention period. You can change retention or disable new logging in Log & statistics. Existing records remain available until their retention expires. URLs and user agents may still contain personal information, so review the suggested text in WordPress’s privacy tools for your site. The plugin does not send logs or content to an external service; its content requests stay on your site.
When the optional bot identity check is enabled, the complete IP address of a request from a crawler that can be verified is used in memory during that request only, to compare it with the official lists or to look up its host name with your server’s DNS resolver. The stored address stays anonymized; only the result (verified, unconfirmed or not verifiable) is added to the log.
Normal HTML visits and authenticated internal generation are not logged. Requests served entirely by another cache cannot be observed by WordPress. Token counts are estimates, not billing measurements; HEAD and 304 responses send no body tokens.
External services
The plugin contacts no external service by default. Only if you enable “Check bot identity” in Log & statistics, your server downloads, once a day, the official IP address lists that these operators publish for their crawlers. Each download is a plain HTTPS request from your server: no visitor data, content, logs or site details are sent, beyond what any HTTPS request carries (your server’s IP address and the WordPress user agent). Disabling the option stops the downloads.
- OpenAI (GPTBot, OAI-SearchBot, ChatGPT-User): https://openai.com/gptbot.json, https://openai.com/searchbot.json, https://openai.com/chatgpt-user.json. Documentation: https://platform.openai.com/docs/bots. Terms: https://openai.com/policies/terms-of-use/. Privacy: https://openai.com/policies/privacy-policy/.
- Anthropic (ClaudeBot, Claude-User, Claude-SearchBot): https://claude.com/crawling/bots.json. Terms: https://www.anthropic.com/legal/consumer-terms. Privacy: https://www.anthropic.com/legal/privacy.
- Perplexity (PerplexityBot, Perplexity-User): https://www.perplexity.com/perplexitybot.json, https://www.perplexity.com/perplexity-user.json. Documentation: https://docs.perplexity.ai/guides/bots. Terms: https://www.perplexity.ai/hub/legal/terms-of-service. Privacy: https://www.perplexity.ai/hub/legal/privacy-policy.
- Google (Googlebot, GoogleOther, Google-CloudVertexBot): https://developers.google.com/static/search/apis/ipranges/googlebot.json, https://developers.google.com/static/search/apis/ipranges/special-crawlers.json. Documentation: https://developers.google.com/search/docs/crawling-indexing/verifying-googlebot. Terms: https://policies.google.com/terms. Privacy: https://policies.google.com/privacy.
- Microsoft (Bingbot): https://www.bing.com/toolbox/bingbot.json. Documentation: https://www.bing.com/webmasters/help/verify-bingbot-2195837f. Terms: https://www.microsoft.com/servicesagreement. Privacy: https://privacy.microsoft.com/privacystatement.
- Apple (Applebot): https://search.developer.apple.com/applebot.json. Documentation: https://support.apple.com/en-us/119829. Privacy: https://www.apple.com/legal/privacy/.
For Googlebot, GoogleOther, Bingbot and Applebot requests whose address is not in the downloaded list, the plugin also performs a forward-confirmed reverse DNS lookup through your server’s DNS resolver, as each operator documents. The result is cached for one day per address.
Bundled libraries
- league/html-to-markdown (MIT license), used to convert HTML to Markdown.
- symfony/css-selector (MIT license), used for the element selectors in the Content settings.
Both are included unminified in the vendor folder with their license files, under a plugin-specific namespace.
Screenshots






Installation
- Install and activate Gonzales AI Agent Markdown on your WordPress site.
- Open Settings > AI Agent Markdown and review the General and Content tabs.
- Add any paths you do not want to publish, then save your changes.
- Open Diagnostics and run a test. If server configuration is needed, share the supplied instructions with your hosting provider.
- Clear your website’s HTML cache after changing content exclusions. Check Log & statistics for incoming requests.
Network activation is not supported. WordPress 6.1 or later and PHP 8.1 or later are required.
FAQ
-
Will visitors see a different website?
-
Visitors still receive your usual HTML pages. Markdown is returned only for an explicit Markdown request or a supported /index.md address.
-
Does this make AI services use or cite my content?
-
No. It provides another way to read your public content and records requests that reach WordPress. Discovery, citations and use are decided by each service.
-
Will it work with my cache plugin?
-
It includes routing and cache integrations, but hosting configurations differ. Run Diagnostics and test both HTML and Markdown after clearing the cache. Your host may need to apply the Nginx snippet or adjust server rules.
-
Can I keep some pages out?
-
Yes. In Content, add one path per line, such as /contact/, or a path ending in * to exclude a group. Private, password-protected and noindex content is excluded automatically, along with the configured WordPress privacy page and protected WooCommerce pages.
-
What data is collected, and where does it go?
-
The optional request log stays on your site, uses anonymized IP addresses and has configurable retention. You can disable new logging at any time. No log data is sent to an external service. See Data and privacy above for the recorded fields.
-
Can the plugin tell a real crawler from one using a false name?
-
Yes, for the crawlers whose operators publish a verification method, if you enable “Check bot identity” in Log & statistics. Each request is then marked as verified, unconfirmed or not verifiable. Unconfirmed does not always mean fake: some operator lists are not kept up to date. See External services for the lists the plugin downloads.
-
What happens if I deactivate or delete the plugin?
-
Deactivation stops publication and removes managed routing and scheduled jobs while keeping your settings and request history. Retired cache files are cleaned in small batches while the plugin is active. Deleting the plugin also removes its settings, request table and other owned data. Unrelated site files are preserved.
-
Can I use the plugin on a multilingual site?
-
Yes. Each page keeps its original language and its own address. With WPML, the self-test page search covers every active language and shows the language of each result. With other multilingual plugins, check the expected language versions on your own site.
Reviews
There are no reviews for this plugin.
Contributors & Developers
“Gonzales AI Agent Markdown” is open source software. The following people have contributed to this plugin.
ContributorsTranslate “Gonzales AI Agent Markdown” into your language.
Interested in development?
Browse the code, check out the SVN repository, or subscribe to the development log by RSS.
Changelog
1.2.1
- First public release.
