Excessive Crawling / CPU Load

Hi,

I’ve been using the free version of this plugin for around 18 months.

Most of that time the plugin worked without any issues. About 3 months ago my site started getting very slow to load pages, the hosting CPU usage was exceeding its limits and I was getting repeated 508 errors. In other words, the website had become unusable.

I enabled Cloudflare Under Attack Mode and it helped temporarily, but every time I turned it off the same problem came back.

After a lot of intermittent back and forth with my hosting company, we eventually found last night that the load was being caused by an excessive number of bots crawling filter URLs created by the plugin. I have now set Cloudflare rules to block those requests and the website load times has noticeably improved.

When I checked the plugin settings, I noticed that the Pro version now has an option called “Disable filter links for crawlers”. The description says this replaces <a> tags with <span> tags to stop crawlers following the filter links and overloading the website, so this appears to be a known issue.

I have a few questions.

• Has the free version always used crawlable <A> links for filters, or has this changed at some point over the last 3–4 months?
• If it has changed, when was it changed and why?
• If it has always worked this way, why wasn’t protection against excessive crawling included by default? This issue has now caused me around 3 months of poor website performance and most likely lost sales plus the time I have put into finding the issue. As the plugin had worked perfectly for around 15 months beforehand, I had no reason to suspect it was the cause.
• Why is the solution to a problem that can make a website unusable only available in the paid version, when the free version directly creates the problem in the first place?

From my point of view, the free version creates an almost unlimited number of crawlable filter URLs. Bots then follow those URLs, which causes excessive server load, high CPU usage and 508 errors. The Pro version then offers the option to stop crawlers following those same links when its the plugin that caused it to happen.

I’d appreciate it if you could explain if this behaviour has changed over time and whether there is any supported way to prevent this excessive crawling while continuing to use the free version, as I don’t really require any of the other Pro features. I would have no issue buying the PRO version as a one off, but a yearly payment just to stop something directly caused by the free version seems a bit excessive.

Finally, could you also explain whether replacing <a> tags with <span> tags provides any other benefits to website performance, SEO or general site functionality beyond reducing crawler activity?

Thank you.