WordPress.org

Forums

WordPress big site and Google Bot (4 posts)

  1. LapanWebsite
    Member
    Posted 1 year ago #

    5 000 posts
    100 000 tags

    Google Bot alone taking me 700MB bandwidth daily.
    I need to stop it somehow.

    Right now, I installed new "One page" WordPress on old domain, and put my old WordPress with content on subdomain and restrict access for bots.

    I would return it to old domain, I mean on this old content which is now moved to subdomain, if there is a way to somehow "calm down" a Google Bot.

    Even in Webmaster tools, 0.003 visits per second is lowest, but still too high for website with so much content.

  2. JumboClicks
    Member
    Posted 1 year ago #

    actually i have a few sites with 25k posts or more

    I would shy away from webmaster tools throttling
    I tried that with a website a few summers ago
    It resulted in me losing about 200k in quantcast rank and losing all most all my ad income for well over a year.

    The bot just gets mad and you can never make it happy again... heh

    it would probably be better to throttle it with an ip logger

  3. Tara
    Member
    Posted 1 year ago #

  4. LapanWebsite
    Member
    Posted 1 year ago #

    Hey, Tara. Thanks.
    That could be usefull if I decide to leave my content on subdomain with restricted access for bots.

    If I don't find a way (maybe plugin) to restrict some of my categories, tags etc. I will definitelly have to leave content hidden from bots.

Topic Closed

This topic has been closed to new replies.

About this Topic