{"id":19009634,"date":"2026-09-02T13:12:45","date_gmt":"2026-09-02T13:12:45","guid":{"rendered":"https:\/\/wordpress.org\/support\/topic\/disable-filter-links-for-crawlers\/"},"modified":"2026-09-02T13:12:45","modified_gmt":"2026-09-02T13:12:45","slug":"disable-filter-links-for-crawlers","status":"publish","type":"topic","link":"https:\/\/wordpress.org\/support\/topic\/disable-filter-links-for-crawlers\/","title":{"rendered":"Disable filter links for crawlers"},"content":{"rendered":"\n<ol start=\"3\"><\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Detailed Review: &#8220;Disable filter links for crawlers&#8221; &#8211; A Game-Changer for Crawl Budget &amp; SEO Efficiency<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I wanted to leave a detailed breakdown of the new &#8220;Disable filter links for crawlers&#8221; feature, as it addresses one of the most critical structural problems for large WooCommerce stores and high-facet catalogs.<\/p>\n\n\n\n<ol>\n<li>The Problem It Solves<br \/>In typical WooCommerce stores using faceted navigation, every filter combination generates a distinct URL. Even if these pages have dynamic noindex tags, or lack active SEO rules, standard crawlers (Googlebot, Bingbot, and modern AI bots\/crawlers) still see physical links on every page.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">They follow every AND \/ OR combination endlessly. In my case, this created an inflation of 700,000+ garbage URLs queued for crawling. While these pages offered zero SEO value, crawlers spent massive amounts of resources scanning them, draining the site&#8217;s overall crawl budget and putting unnecessary stress on the database.<\/p>\n\n\n\n<ol start=\"2\">\n<li>How This Feature Works<br \/>When enabled, the plugin intelligently alters how filter elements are presented to search bots and AI crawlers versus real users:<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">For real visitors: Seamless UX remains identical. Navigation, AJAX loading, and filtering work without any compromise.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For search crawlers:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Only URLs with active, custom SEO rules (index, follow) remain as crawlable standard HTML links ().<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Deep junction links (AND\/OR logic) and index-restricted pages without SEO rules no longer contain crawlable href targets for bots.<\/p>\n\n\n\n<ol start=\"3\">\n<li>Key Benefits &amp; Real-World Impact<br \/>Crawl Budget Optimization: Googlebot stops wasting request quotas on low-value\/no-index filter pages.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Faster Indexation of Key Pages: Search engines divert 100% of their scanning capacity toward high-value landing pages, active SEO filter pages, and main product categories.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Database &amp; Server Load Reduction: Prevents continuous background execution of heavy SQL queries triggered whenever bots hit complex non-indexed filter URLs.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Clean Search Engine Diagnostics: Prevents Google Search Console from bloating with hundreds of thousands of URLs under the &#8220;Excluded by \u2018noindex\u2019 tag&#8221; report, keeping your indexing reports clear, actionable, and manageable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Summary<br \/>This feature turns Filter Everything from a basic AJAX filtering tool into an enterprise-grade SEO platform for WooCommerce. If you manage a site with hundreds or thousands of products and attributes, enabling &#8220;Disable filter links for crawlers&#8221; is an absolute must. Highly recommended!<\/p>\n","protected":false},"template":"","class_list":["post-19009634","topic","type-topic","status-publish","hentry"],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/wordpress.org\/support\/wp-json\/wp\/v2\/topic\/19009634","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.org\/support\/wp-json\/wp\/v2\/topic"}],"about":[{"href":"https:\/\/wordpress.org\/support\/wp-json\/wp\/v2\/types\/topic"}],"version-history":[{"count":0,"href":"https:\/\/wordpress.org\/support\/wp-json\/wp\/v2\/topic\/19009634\/revisions"}],"wp:attachment":[{"href":"https:\/\/wordpress.org\/support\/wp-json\/wp\/v2\/media?parent=19009634"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}