WordPress.org

Forums

Google tool: robot.txt and network unreachable? (1 post)

  1. jeffqc
    Member
    Posted 1 year ago #

    for 2 day suddnly withous any chnage on my site the network is unrecahble and all page/robot and siemap are unreacable.

    i contact my host to white list all ip's and all is ok form their side

    my robot is:

    Sitemap: http://www.tvqc.com/sitemap.xml
    Sitemap: http://www.tvqc.com/sitemap-news.xml
    Sitemap: http://www.tvqc.com/sitemap-posttype-post.2014.xml
    Sitemap: http://www.tvqc.com/sitemap-posttype-post.2012.xml
    Sitemap: http://www.tvqc.com/sitemap-posttype-post.2011.xml
    Sitemap: http://www.tvqc.com/sitemap-posttype-post.2013.xml
    Sitemap: http://www.tvqc.com/sitemap-posttype-post.2010.xml
    Sitemap: http://www.tvqc.com/sitemap-posttype-post.2009.xml
    Sitemap: http://www.tvqc.com/sitemap-posttype-page.xml

    User-agent: *
    Crawl-delay: 2
    Disallow: /cgi-bin/
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /comments/feed/
    Disallow: /trackback/
    Disallow: /xmlrpc.php
    Allow: /wp-content/uploads
    User-agent: NinjaBot
    Allow: /

    User-agent: Googlebot-Image
    Allow: /wp-content/uploads/

    User-agent: Adsbot-Google
    Allow: /

    User-agent: Googlebot-Mobile
    Allow: /

    User-agent: Mediapartners-Google*
    Disallow:
    Allow: /*

    # Internet Archiver Wayback Machine
    User-agent: ia_archiver
    Disallow: /

    # digg mirror
    User-agent: duggmirror
    Disallow: /

Topic Closed

This topic has been closed to new replies.

About this Topic