WordPress.org

Support

Support » Multisite » Network sites 404 Error on robots.txt

Network sites 404 Error on robots.txt

Viewing 15 replies - 1 through 15 (of 20 total)
  • That’s a mapped domain, right?

    What happens if you go to the non-mapped URL?

    networksite.mainsite.com/robots.txt

    duncmorley
    Member

    @duncmorley

    It is yes. The network sites are installed in sub directories.

    The mainsite.com/networksite automatically redirects to networksite.com.

    So mainsite.com/networksite/robots.txt redirects to networksite.com/robots.txt which goes to a 404 error page.

    Do you have a non-mapped site to check with?

    Also which plugin are you using? Can you link to it?

    duncmorley
    Member

    @duncmorley

    I am using Multisite Robots.txt Manager. Please see link below:

    http://wordpress.org/extend/plugins/multisite-robotstxt-manager/

    I don’t have a non mapped site to check unfortunately.

    So make a test one 🙂

    duncmorley
    Member

    @duncmorley

    In fact i did have one domain that wasn’t mapped yet. mainsite.com/networksite/robots.txt still 404’s.

    Okay then, it’s not mapped domains 🙂 It doesn’t like subfolders!

    Annnnd yeah, looks like we ran into this a year ago.

    http://wordpress.org/extend/plugins/pc-robotstxt/ works.

    I’m having the same issue…I just installed it and it doesn’t work – still got a 404 error for the subdirectory’s file.

    KGarrett – If it’s the same plugin, it’s apparently a known issue. If pc-robotstxt doesn’t work, make a new topic for yourself 🙂

    duncmorley
    Member

    @duncmorley

    pc-robotstxt does also not work. Both the mapped domain and unmapped give me 404 error!

    tribalNerd
    Participant

    @tribalnerd

    Hello everyone…

    Currently the MS Robots.txt Manager Plugin doesn’t render the robots.txt file within directories. However, the next release of the plugin renders the robots.txt file within a directory. It should be ready in the next day or two.

    Now…. the robots.txt file shouldn’t be in a directory at all. Spiders only read robots.txt files from the root of the Website. This is why WordPress itself, doesn’t render the robots.txt file within directories.

    Hope that helps clear up some confusion…

    ~Chris

    That makes a ton of sense, tribalNerd 🙂 Thanks!

    duncmorley
    Member

    @duncmorley

    tribalNerb, you’re a ledge! Thanks for the info.

    pepe
    Participant

    @pputzer

    @tribalnerd: Any roadmap for the update? I have the same problem and I’d really like the plugin to work. I’ve got a mapped domain (based on a subfolder network installation) and the robots.txt doesn’t render correctly. If I change the URL in the network admin sitemanager, it does render, but that breaks the WordPress admin interface.

    JustinRS83
    Member

    @justinrs83

    I’m running a multisite instance with 6 sites, 5 of them are working correctly but one is sending me to a 404 page. Any idea what could cause this?

    And yes they are mapped and when I remove mapping so it’s in the subfolder that one site still breaks.

Viewing 15 replies - 1 through 15 (of 20 total)
  • The topic ‘Network sites 404 Error on robots.txt’ is closed to new replies.