WordPress.org
Get WordPress
WordPress.org

Plugin Directory

Search results for: “website crawler”

351 plugins

Search results for: “website crawler”

  • IntentDeep Virtual Files – AI-Ready: Robots.txt, Security.txt, Ads.txt, LLMS.txt

    (1 total ratings)

    Create robots.txt, ads.txt, security.txt, llms.txt & JSON files with AI-ready content generation (ChatGPT, Claude, Gemini) at any path. No FTP needed.

    IntentDeep Fewer than 10 active installations Tested with 6.9.4

  • Forerunner® AI Live Chat

    (1 total ratings)

    Modern AI-powered live chat for WordPress with webhook integration, Azure Functions support, and customizable appearance.

    designforonline 10+ active installations Tested with 7.0

  • Serve Static – Automatic WordPress Static Page generator

    (9 total ratings)

    Serve Static is a static HTML page generator WordPress plugin to create and serve static copies of your existing web pages to avoid PHP/DB load.

    Rajin Sharwar 100+ active installations Tested with 6.7.5

  • QMSpace Business Facts for AI Assistants

    (0 total ratings)

    Make your business visible to AI assistants like ChatGPT, Claude & AI Search. When customers ask about your business, give them the right answer.

    qmspace Fewer than 10 active installations Tested with 6.9.4

  • Block Chat GPT via robots.txt

    (1 total ratings)

    Blocks the ChatGPT-User bot that is used by plugins in ChatGPT to crawl websites through the WordPress virtual robots.txt file

    apasionados 100+ active installations Tested with 6.3.8

  • BreznGEO

    (1 total ratings)

    AI meta descriptions, GEO blocks, internal link suggestions, Schema.org structured data, and llms.txt for WordPress. No subscription.

    mifupa Fewer than 10 active installations Tested with 6.9.4

  • AgencyBoost Lite

    (1 total ratings)

    Complete SEO suite for agencies. On-page SEO meta box, XML sitemap, robots.txt manager, lead capture and analytics.

    shashank23jan Fewer than 10 active installations Tested with 6.9.4

  • Static Cache Wrangler – Coverage Assistant

    (0 total ratings)

    Monitor cache coverage and identify uncached content for Static Cache Wrangler.

    derickschaefer Fewer than 10 active installations Tested with 7.0

  • Super Duper SEO Title and Description

    (0 total ratings)

    Lightweight SEO: titles, descriptions, Open Graph, XML sitemap, 301 redirects, WooCommerce-friendly indexing and AI crawler control.

    Super Duper Plugins Fewer than 10 active installations Tested with 6.9.4

  • Issues Tracker

    (3 total ratings)

    Issues Tracker allows you view and search WordPress logs, receive security advice, track 404 errors, and view your server settings.

    Oleksandr Lysyi 50+ active installations Tested with 6.7.5

  • Fstyle Honeypot Bot Blocker

    (0 total ratings)

    Protect your site from malicious bots and scanners using a smart honeypot trap.

    fstylegear09543 Fewer than 10 active installations Tested with 6.9.4

  • SFR Analytics

    (0 total ratings)

    Lightweight, privacy-focused WordPress analytics. Track pageviews, visitors, campaigns and more — all data stored locally in your own database.

    SupportFromRichard 20+ active installations Tested with 6.9.4

  • Llmify – AI Readable

    (0 total ratings)

    Make your WordPress site readable by AI. Install and activate — no configuration needed.

    Kazuto Nakazawa Fewer than 10 active installations Tested with 6.9.4

  • GoBookMart Robots Editor

    (1 total ratings)

    A lightweight plugin to edit or create robots.txt directly from your WordPress dashboard. Developed by GoBookMart.

    Gobookmart Fewer than 10 active installations Tested with 6.9.4

  • Recognyze.AI

    (0 total ratings)

    Protect your content and earn from AI usage. Manage posts, track AI crawlers, and verify authenticity with Recognyze.AI.

    recognyze Fewer than 10 active installations Tested with 6.9.4

  • GEOmatic AI

    (0 total ratings)

    AI-first SEO: minimal JSON-LD, an /ai-sitemap.xml, and AI-friendly HTML attributes. No tracking.

    GEOmatic AI Fewer than 10 active installations Tested with 6.8.5

  • DataStackLogic Schema Manager

    (0 total ratings)

    Structured data plugin for WordPress. Outputs Organization, LocalBusiness, and FAQPage schema, and manages AI crawler directives in robots.txt.

    DataStackLogic Fewer than 10 active installations Tested with 6.9.4

  • Sitemap Generator Professional

    (23 total ratings)

    An easy to use XML sitemap generator with support for image and video sitemaps for WordPress.

    Marco Beierer 4,000+ active installations Tested with 6.0.11

  • Link Checker Professional

    (30 total ratings)

    An easy to use link checker for WordPress to detect broken links and images on your website.

    Marco Beierer 4,000+ active installations Tested with 6.0.11

  • Block old browser versions and suspicious browsers

    (0 total ratings)

    With the help of WhatIsMyBrowser the plugin detects old and bad browsers and denies them access. A special robots.txt denies crawling by bad bots.

    hupe13 Fewer than 10 active installations Tested with 7.0