I’ve just noticed that WP Super Cache adds an HTML comment <!-- --> to my robots.txt file that I generate using the KB Robots.txt plugin. This is not (AFAIK) part of the file format for robots.txt. I’m sure most browsers and spiders are smart enough to ignore this, but:
If you are generating your robots (eg with KB Robots.txt) then WP Super Cache will add this comment, which can be removed by adding
/robots.txt
in the rejected URIs section of the WP Super cache config page.
The topic ‘[Plugin: WP Super Cache] robots.txt can be cached and made non-format compliant’ is closed to new replies.
(@malcolmm)
15 years, 6 months ago
I’ve just noticed that WP Super Cache adds an HTML comment
<!-- -->to my robots.txt file that I generate using the KB Robots.txt plugin. This is not (AFAIK) part of the file format for robots.txt. I’m sure most browsers and spiders are smart enough to ignore this, but:If you are generating your robots (eg with KB Robots.txt) then WP Super Cache will add this comment, which can be removed by adding
/robots.txt
in the rejected URIs section of the WP Super cache config page.