robots.txt

  • Resolved jdelarbre

    (@jdelarbre)


    4 years ago

    Hi, thank you for the plugin. It works well.

    Just a remark, it would be cleaner to always add robots.txt (and eventually sitemap.xml) to the white list. Since they are virtually generated by wordpress, they are not accessible by bots.

    Ok for a password protected site, robots.txt would be quite short, but Google and others will officially know: nothing to see their.

    I think it would be better to mention it in the FAQ.

Viewing 1 replies (of 1 total)
  • Plugin Author Kevin Vess

    (@kevinvess)

    3 years, 12 months ago

    Hi– thanks for using Force Login!

    Not all sites have virtually generated robots.txt and sitemap.xml files.

    If those files actually exist on the server, they’re publicly accessible because no WordPress / PHP script runs prior to loading those files. The server should just load the file.

    Since, each site handles these files differently, I can’t say for sure whether they need to be whitelisted or not in the FAQ.

    However, if you find that Force Login is blocking certain URLs that you want publicly accessible, the FAQ does detail how you can add exceptions for those URLs.

    Thanks for the suggestion though!

Viewing 1 replies (of 1 total)
  • The topic ‘robots.txt’ is closed to new replies.