WordPress.org

Forums

WordPress big site and Google Bot (4 posts)

  1. LapanWebsite
    Member
    Posted 1 year ago #

    5 000 posts
    100 000 tags

    Google Bot alone taking me 700MB bandwidth daily.
    I need to stop it somehow.

    Right now, I installed new "One page" WordPress on old domain, and put my old WordPress with content on subdomain and restrict access for bots.

    I would return it to old domain, I mean on this old content which is now moved to subdomain, if there is a way to somehow "calm down" a Google Bot.

    Even in Webmaster tools, 0.003 visits per second is lowest, but still too high for website with so much content.

  2. JumboClicks
    Member
    Posted 1 year ago #

    actually i have a few sites with 25k posts or more

    I would shy away from webmaster tools throttling
    I tried that with a website a few summers ago
    It resulted in me losing about 200k in quantcast rank and losing all most all my ad income for well over a year.

    The bot just gets mad and you can never make it happy again... heh

    it would probably be better to throttle it with an ip logger

  3. Tara
    Volunteer Moderator
    Posted 1 year ago #

  4. LapanWebsite
    Member
    Posted 1 year ago #

    Hey, Tara. Thanks.
    That could be usefull if I decide to leave my content on subdomain with restricted access for bots.

    If I don't find a way (maybe plugin) to restrict some of my categories, tags etc. I will definitelly have to leave content hidden from bots.

Topic Closed

This topic has been closed to new replies.

About this Topic