Support » Fixing WordPress » Why is my robots.txt keep being removed

  • The robots.txt file on websites keep being removed every time my page updates.

    Can someone help me figure out why is this happening and how to resolve this issue

    Thanks

Viewing 7 replies - 1 through 7 (of 7 total)
  • Jason King

    (@jasoncharlesstuartking)

    There is no robots.txt file that you will be able to see using FTP.

    Instead, WordPress creates it on the fly.

    So there probably isn’t a problem but if you want to be sure, visit http://yourdomain.com/robots.txt – is it displayed?

    I did that and the file is in my WordPress site but when I go to my http://yourdomain.com/robots.txt it’s not showing up.

    Jason King

    (@jasoncharlesstuartking)

    I can see it. It’s there.

    It’s there now but the next day it goes away, and I’m trying to figure out why is this happening.

    the only thing that is showing is:

    User-Agent: *
    Disallow:

    Is there a way to correct this from happening so i won’t have to continuing re-applying robots.txt files to my WordPress website?

    Moderator Steven Stern (sterndata)

    (@sterndata)

    Forum Moderator & Support Team Volunteer

    WordPress does not delete a robots.txt file if it’s there. If you actually place a file of that name in your site’s root and it gets deleted, it’s not by WP.

    That’s not to say that a plugin isn’t doing it, but there’s no core functionality that removes the file.

    I re-uploaded my robots.txt file to my website server and http://yourdomain.com/robots.txt it shows up with all the items I want blocked.

    If I got and update anything on my website and go to the http://yourdomain.com/robots.txt again all the items on my robots.txt is gone.

    I’m trying to figure out why is this occurring…. could it be a cache issue or plugin issue?

    Moderator Steven Stern (sterndata)

    (@sterndata)

    Forum Moderator & Support Team Volunteer

    Disable all plugins and try it again.

Viewing 7 replies - 1 through 7 (of 7 total)
  • The topic ‘Why is my robots.txt keep being removed’ is closed to new replies.