[resolved] [Plugin: WP Super Cache] robots.txt can be cached and made non-format compliant (1 post)

  1. malcolmm
    Posted 7 years ago #

    I've just noticed that WP Super Cache adds an HTML comment <!-- --> to my robots.txt file that I generate using the KB Robots.txt plugin. This is not (AFAIK) part of the file format for robots.txt. I'm sure most browsers and spiders are smart enough to ignore this, but:

    If you are generating your robots (eg with KB Robots.txt) then WP Super Cache will add this comment, which can be removed by adding


    in the rejected URIs section of the WP Super cache config page.

Topic Closed

This topic has been closed to new replies.

About this Topic