• Resolved jtroyer

    (@jtroyer)


    I’m using multisite with subdirectories, ie blog.example.com/foo and blog.example.com/bar. So I need a robots.txt file at the root blog.example.com that disallows not only Disallow: /wp- but that also Disallows /foo/wp- /bar/wp- etc.

    Do you have any suggestions or examples for this kind of setup? I’m going with the following for now:

    # robots.txt
    User-agent: *
    Disallow: /*?
    Disallow: /wp-
    Disallow: /feed/
    Disallow: */feed/
    Disallow: /cgi-bin/
    Disallow: /comments/
    Disallow: */comments/
    Disallow: /trackback/
    Disallow: */trackback/
    Disallow: /wp-admin/
    Disallow: /wp-content/
    Disallow: /wp-includes/
    Disallow: /wp-login.php
    Disallow: /wp-content/cache/
    Disallow: /wp-content/themes/
    Disallow: /wp-content/plugins/
    Disallow: /*/wp-admin/
    Disallow: /*/wp-content/
    Disallow: /*/wp-includes/
    Disallow: /*/wp-login.php
    Disallow: /*/wp-content/cache/
    Disallow: /*/wp-content/themes/
    Disallow: /*/wp-content/plugins/
    Allow: /

    Also, why do you include both the ‘wp-‘ line as well as the full directories?

    http://wordpress.org/extend/plugins/multisite-robotstxt-manager/

Viewing 2 replies - 1 through 2 (of 2 total)
  • Thread Starter jtroyer

    (@jtroyer)

    Also a nice feature would be an option to insert sitemap fields for each blog in my system when using subdirectories. (I have about 40.)

    Plugin Author tribalNerd

    (@tribalnerd)

    Hello jtroyer,
    I’ve updated the plugin to version 0.3.1, it will allow you to insert a unique sitemap url for each Website. I don’t have any examples on what you’re trying to do, however what you posted does look correct.

    I include both wp- and the full dir, because not all search engines read the robots.txt file the same.

    Please let me know if you have any other questions or find any issues.

    ~Chris

Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘[Plugin: Multisite Robots.txt Manager] multisite using directories’ is closed to new replies.