Allow custom User-Agent for WordPress REST API
-
We need our SEO tooling to call the WordPress REST API (
/wp-json/*) with Application Passwords on two sites protected by CleanTalk (Anti-Crawler / SpamFireWall):What works elsewhere
On https://torchsa.com/ we allowlisted User-Agent wildcard*WeblabSEOBot*at the edge (Cloudflare). Requests using:WeblabSEOBot/1.0 (+https://theweblab.co.za; SEO agent)then get 200 on the homepage,
/wp-json/, and authenticated/wp-json/wp/v2/users/me.What fails on CleanTalk
The same User-Agent still receives CleanTalk’s “Blocked: Security by CleanTalk” 403 on/wp-json/(and often/robots.txton osight.africa). Homepages can load; REST does not.We tried Personal List CSV IP allow (
140.248.50.53,allow), but our bot egress IPs rotate, so a single IP whitelist is unreliable. Your docs also say custom User-Agents cannot be added to the Anti-Crawler trusted list ourselves — only the built-in bot list.Request
Please either:- Add
WeblabSEOBot(match substring / wildcard*WeblabSEOBot*) to the trusted / allowed User-Agents for these two websites, or - Tell us the supported way to allow this custom User-Agent for SpamFireWall / Anti-Crawler so REST API traffic is not challenged or blocked.
We only need read/write via official WordPress Application Passwords; not form spam bypass.
Thank you.
- Add
You must be logged in to reply to this topic.