The Support Forums will be in read-only mode for a scheduled maintenance window on 01 September 2016 14:00 UTC - 20:00 UTC. More information.

Blog updates propogating across internet to Yahoo, etc. (2 posts)

  1. FatalException
    Posted 9 years ago #

    I've been using WordPress 2.0.4 on my server for a while now (hosted through a private web service provider). I recently modified the code for the index.php file to log all IP addresses and user-agents to a new table in a SQL database on my server.

    I was surprised to find that immediately after I make a post to my blog (within 2 minutes after I post), Yahoo's RSS feed reader visits my index page. Shortly after Yahoo's server hits my site, search engine "bots" from other companies also hit the site. My site only gets hit by these search aggregators and by Yahoo immediately after I post. If I don't post for 3 or 4 days, they don't visit.

    In my opinion, it seems like somehow my site is telling Yahoo that I've posted and it's time to come visit to get the latest post. I don't want this to happen - in fact, I don't want Yahoo or any other engine to visit my site at all! How do I turn off this "feature?" I've looked through the code, and from what little I know of PHP, I can't figure out what is making this happen. Any ideas about what is going on? I've already tried using a "disallow all" robots.txt file on my site and it hasn't kept these search engines from visiting me. Thanks for your help.

  2. Options->Writing. Scroll down to "Update Services". The default setting is for you to tell the Ping-o-matic that you have a new post. It will then tell dozens and dozens of other sites.

    If you don't want to ping an update service, then remove everything listed in the box under Update Services.

    As for robots.txt, if you make the file under the root of your site, and put this in it, it will prevent them from indexing you. It won't prevent them from visiting when they get an update ping, but over time it will cause them to remove you from their indexes.
    User-agent: *
    Disallow: /

Topic Closed

This topic has been closed to new replies.

About this Topic