Title: AI Crawler and SEO Manager by akbansa Author: Anshul Published: October 27, 2025 Last modified: October 27, 2025 --- Search plugins This plugin **hasn’t been tested with the latest 3 major releases of WordPress**. It may no longer be maintained or supported and may have compatibility issues when used with more recent versions of WordPress. ![](https://s.w.org/plugins/geopattern-icon/akbansa-ai-seo-manager.svg) # AI Crawler and SEO Manager by akbansa By [Anshul](https://profiles.wordpress.org/akbansa/) [Download](https://downloads.wordpress.org/plugin/akbansa-ai-seo-manager.1.0.4.zip) * [Details](https://wordpress.org/plugins/akbansa-ai-seo-manager/#description) * [Reviews](https://wordpress.org/plugins/akbansa-ai-seo-manager/#reviews) * [Installation](https://wordpress.org/plugins/akbansa-ai-seo-manager/#installation) * [Development](https://wordpress.org/plugins/akbansa-ai-seo-manager/#developers) [Support](https://wordpress.org/support/plugin/akbansa-ai-seo-manager/) ## Description The AI Crawler and SEO Manager by akbansa plugin generates an `llms.txt` file for crawler discovery and logs visits by AI-based crawlers such as GPTBot, ClaudeBot, and others. It also provides an analytics page in the WordPress admin dashboard to view crawler activity by path, URL, and timestamp. ### Key Features: * Generates `llms.txt` file for crawler discovery. * Removes `llms.txt` on plugin uninstall * Schedules automatic generation of `llms.txt` daily at midnight (frequency configurable: daily, twice daily, hourly) * Uses atomic file writes for `llms.txt` and `sitemap.xml` for safety * Integrates with Yoast SEO, Rank Math, or All in One SEO for metadata (falls back to DB if not present) * Admin panel to select/exclude categories and content types for `llms.txt` * Logs every AI crawler request from the frontend (user agent, crawl info, page URL, IP, referrer) * Filters crawlers using `config.json` and stores all details in a dedicated DB table * Analytics dashboard in admin to view recent crawler activity * Robust AJAX security and error handling * Detects and logs visits from AI crawlers based on their User-Agent. * Stores crawler logs in a database for analysis. * Supports popular AI-based crawlers (e.g., OpenAI, Anthropic, GPTBot, etc.). * Option to customize the list of crawlers via a `config.json` file. ### Short Description Advanced AI-powered SEO and analytics for WordPress. Generates llms.txt for AI crawler discovery, logs AI bot visits, and provides comprehensive analytics dashboard. ### Usage * After activation, the plugin will automatically start generating the `llms.txt` file and log visits from AI-based crawlers. * The crawler logs are stored in the database for analysis. * You can view analytics by navigating to **LLMS Crawlers** in the WordPress admin menu. ### Admin Analytics The admin panel will display a list of the most recent crawler visits, showing the following information: – Path – URL – Crawler Name – Timestamp ### Support If you encounter any issues, please feel free to reach out via the plugin support page or create an issue on the GitHub repository. ### License This plugin is licensed under the GPLv2 or later. ### New Features 1. **Admin Settings Filter**: You can now filter active pages, products, categories, and posts for generating the `llms.txt`. 2. **Sitemap Handling**: If a `sitemap.xml` exists, it will be used to get the list of pages. If not, the database will be used as a fallback. 3. **SEO Plugin Integration**: The plugin integrates with Yoast SEO, All in One SEO, and Rank Math to fetch metadata for the `llms.txt` file. 4. **Frequency Setting**: You can choose the frequency for regenerating the `llms. txt` file (daily, weekly, monthly, hourly). 5. **Traffic Capture and AI Agent Logging**: A front-end script captures traffic, sends it to the backend, and logs AI agents based on a `config.json` file. 6. **Sitemap Update**: The `sitemap.xml` is automatically updated with the absolute path to the `llms.txt` and the updated timestamp. ## Installation 1. Upload the `akbansa-ai-seo-manager` plugin folder to your WordPress installation’s` wp-content/plugins/` directory. 2. Activate the plugin through the ‘Plugins’ menu in WordPress. 3. Ensure the `config.json` file is in the plugin directory. You can update this file to include or exclude specific AI crawlers as needed. ## FAQ ### How can I add more AI crawlers? You can add more AI crawlers to the `config.json` file located in the plugin directory. Simply add the User-Agent strings of the crawlers you want to track. ### Can I exclude certain crawlers from being logged? Yes, you can remove any crawlers from the `config.json` file to prevent them from being detected and logged. ### How do I view the crawler logs? You can view the logs by navigating to the **LLMS Crawlers** section in the WordPress admin menu. This will display the most recent AI crawler visits. ## Reviews There are no reviews for this plugin. ## Contributors & Developers “AI Crawler and SEO Manager by akbansa” is open source software. The following people have contributed to this plugin. Contributors * [ Anshul ](https://profiles.wordpress.org/akbansa/) [Translate “AI Crawler and SEO Manager by akbansa” into your language.](https://translate.wordpress.org/projects/wp-plugins/akbansa-ai-seo-manager) ### Interested in development? [Browse the code](https://plugins.trac.wordpress.org/browser/akbansa-ai-seo-manager/), check out the [SVN repository](https://plugins.svn.wordpress.org/akbansa-ai-seo-manager/), or subscribe to the [development log](https://plugins.trac.wordpress.org/log/akbansa-ai-seo-manager/) by [RSS](https://plugins.trac.wordpress.org/log/akbansa-ai-seo-manager/?limit=100&mode=stop_on_copy&format=rss). ## Changelog #### 1.3 * Added support for detecting and logging visits from a wider range of AI-based crawlers (e.g., GPTBot, ClaudeBot). * Introduced `config.json` file for easy crawler list updates. * Improved logging of crawler visits to a dedicated database table. * Added admin analytics page for viewing crawler logs. #### 1.2 * Initial release. ## Meta * Version **1.0.4** * Last updated **11 months ago** * Active installations **Fewer than 10** * WordPress version ** 5.0 or higher ** * Tested up to **6.4.11** * PHP version ** 7.4 or higher ** * Tags * [AI Crawlers](https://wordpress.org/plugins/tags/ai-crawlers/)[analytics](https://wordpress.org/plugins/tags/analytics/) [llms.txt](https://wordpress.org/plugins/tags/llms-txt/)[seo](https://wordpress.org/plugins/tags/seo/) * [Advanced View](https://wordpress.org/plugins/akbansa-ai-seo-manager/advanced/) ## Ratings No reviews have been submitted yet. [Your review](https://wordpress.org/support/plugin/akbansa-ai-seo-manager/reviews/#new-post) [See all reviews](https://wordpress.org/support/plugin/akbansa-ai-seo-manager/reviews/) ## Contributors * [ Anshul ](https://profiles.wordpress.org/akbansa/) ## Support Got something to say? Need help? [View support forum](https://wordpress.org/support/plugin/akbansa-ai-seo-manager/)