Sorry about that! My accusation was unfounded, and I should’ve been more considerate.
After rereading all the data you gave, I believe that comment pagination is the culprit combined with caching. Even though you might have comment pagination disabled, someone or something can still activate a query for it.
You can disable the protective feature in TSF by implementing this snippet:
add_filter( 'the_seo_framework_enable_noindex_comment_pagination', '__return_false' );
However, I don’t think the filter will resolve the issue entirely. Moreover, you may get away without using a filter, but by switching an option instead — more on that in the following paragraphs.
I’ve also used memcached for Object Cache and Cloudlare proxy. Could they affect on noindex? How I could check late 404-response myself on demo install?
Yes. That’s the final piece to the puzzle!
I’m going to remove the extended object caching support from TSF in the next update because it’s causing rare issues with some of our PHP APIs. Reassessing the object key generation of TSF, I see it doesn’t consider comment pagination nor advanced query protection. Thank you for allowing me to find this deeply nested bug, making the feature removal even more justified!
The object caching would also explain why bypassing the FastCGI cache still allowed it to show the noindex value. Cloudflare’s proxy shouldn’t affect any of this, provided that they treat queries as unique requests. But, again, given your data, neither FastCGI nor Cloudflare plays an obvious role.
So, if you haven’t yet, you should uncheck “Enable object cache?” at “SEO Settings > General > Performance > Object Cache Settings.” This should take care of the issue.
If you have already had that feature disabled, I’m afraid I need another round to look at the issue.
You can test the issue by performing these two steps:
- Go to
<post-url>?cpage=2. TSF should output “noindex” (unless the filter above is implemented). TSF should output a URL in the og:url meta tag that points to the comment section.
- Go to
<post-url>. TSF shouldn’t output “noindex”.
I hope this helps!
