[Plugin: Multisite Robots.txt Manager] multisite using directories
-
I’m using multisite with subdirectories, ie
blog.example.com/foo
andblog.example.com/bar
. So I need a robots.txt file at the root blog.example.com that disallows not onlyDisallow: /wp-
but that also Disallows/foo/wp-
/bar/wp-
etc.Do you have any suggestions or examples for this kind of setup? I’m going with the following for now:
# robots.txt User-agent: * Disallow: /*? Disallow: /wp- Disallow: /feed/ Disallow: */feed/ Disallow: /cgi-bin/ Disallow: /comments/ Disallow: */comments/ Disallow: /trackback/ Disallow: */trackback/ Disallow: /wp-admin/ Disallow: /wp-content/ Disallow: /wp-includes/ Disallow: /wp-login.php Disallow: /wp-content/cache/ Disallow: /wp-content/themes/ Disallow: /wp-content/plugins/ Disallow: /*/wp-admin/ Disallow: /*/wp-content/ Disallow: /*/wp-includes/ Disallow: /*/wp-login.php Disallow: /*/wp-content/cache/ Disallow: /*/wp-content/themes/ Disallow: /*/wp-content/plugins/ Allow: /
Also, why do you include both the ‘wp-‘ line as well as the full directories?
http://wordpress.org/extend/plugins/multisite-robotstxt-manager/
Viewing 2 replies - 1 through 2 (of 2 total)
Viewing 2 replies - 1 through 2 (of 2 total)
- The topic ‘[Plugin: Multisite Robots.txt Manager] multisite using directories’ is closed to new replies.