Support » Plugin: All in One SEO Pack » Robots.txt file is not updating correctly in multisite

Viewing 4 replies - 1 through 4 (of 4 total)
  • Plugin Author Steve Mortiboy

    (@wpsmort)

    Support Manager at Semper Fi

    It all depends on whether you set the multisite up as sub domain or sub directory. If it’s sub directory then each site will have it’s own directory and you can place the robots.txt file in that directory.

    AH I see, ok we had to add a custom robots.txt file to stop bots from hitting the site as much (using crawl-delay) and that is being readin instead of the virtual robots.txt file

    Is there anyway you could add that option to the plugin so we could use site specific robot.txt files?

    Plugin Author Steve Mortiboy

    (@wpsmort)

    Support Manager at Semper Fi

    Do you mean adding crawl-delay as an option in our Robots.txt Generator module?

    Yes indeed, that would be great. I’ve had sites taken down by bots and setting the crawl delay helped alot.

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘Robots.txt file is not updating correctly in multisite’ is closed to new replies.