Support » Networking WordPress » Google doesn't crawl or index all my posts

  • On a multisite i realise one of my sites Google doesn’t crawl or index all my posts or make a lot of days to index news posts.

    Example: from 27th-30th july i have make 11 posts and only 5 have indexing. Why?!
    my site is active 3 years.

    i use the ultimate seo & google xml sitemap 4.0b11

    my robots.txt is

    sitemap: http://sub1.com/sitemap.xml
    
    # Google Image
    User-agent: Googlebot-Image
    Disallow: /files
    Disallow: /wp-content/uploads
    Disallow: /wp-content/blogs.dir
    
    User-agent: Googlebot-Video
    Allow: /
    
    # Google AdSense
    User-agent: Mediapartners-Google*
    Allow: /
    
    User-agent: Googlebot-News
    Allow: /
    
    User-agent: Googlebot-Mobile
    Allow: /
    
    User-agent: AdsBot-Google
    Allow: /
    
    User-agent: ia_archiver
    Disallow: 
    
    User-agent: bingbot
    Allow: /
    
    User-agent: msnbot
    Allow: /
    
    User-agent: msnbot-news
    Allow: /
    
    User-agent: msnbot-media
    Allow: /
    
    User-agent: msnbot-products
    Allow: /
    
    User-Agent: Slurp
    Allow: /
    
    # digg mirror
    User-agent: duggmirror
    Disallow: /
    
    # global
    User-agent:  *
    Disallow: /cgi-bin/
    Disallow: /wp-admin/*
    Disallow: /wp-content/*
    Disallow: /wp-includes/*
    Disallow: /wp-content/plugins/
    Disallow: /wp-content/cache/
    Disallow: /wp-content/themes/
    Disallow: /banners/
    Disallow: /wp-*
    Disallow: /comments/feed/
    Disallow: /feed/
    Disallow: */feed/
    Disallow: /*/feed/rss/$
    Disallow: /trackback/
    Disallow: */trackback/
    Disallow: /index.php
    Disallow: /xmlrpc.php
    Disallow: ?comments=*
    Disallow: /comment-page-
    Disallow: /*?s=
    Disallow: /search?
    Disallow: /?p=*
    Disallow: /*.pdf$
    Disallow: /*.php$
    Disallow: /*.js$
    Disallow: /*.cgi$
    Disallow: /*.xhtml$
    Disallow: /*.php*
    Disallow: /*.inc$
    Disallow: /*.css$
    Disallow: /*.txt$
    Disallow: /*?*
    Disallow: /*.gz$
    Disallow: /*.wmv$
    Disallow: /*.zip$
    Disallow: ?theme=
    Disallow: /category/*
    Disallow: /category/*/*
    Disallow: /transfer/
    Disallow: /tweets/
    Disallow: /mint/
    Disallow: /readme.html
    Disallow: /license.txt
    Disallow: /wp-content/downloads/
    Disallow: /downloads/
    Disallow: /recommends/
    Disallow: /go/
    Disallow: /category/
    Disallow: /tag/
    Disallow: /tag/*
    Disallow: /archives/
    Disallow: /author/
    Disallow: /search?
    Disallow: /recommended/
    Disallow: *?wptheme
    Disallow: ?comments=*
    Disallow: /?p=*
    Disallow: /images/
    Disallow: /embed.js?pname=wordpress&pver=*
    Disallow: /*?replytocom=*
    Disallow: /stats/
    Disallow: /general/
    Disallow: /date/
    Disallow: /?attachment_id*
    Disallow: /search/?*
    Disallow: */trackback/*
    Disallow: /?cat*
    Disallow: /?m*
    Disallow: /*?utm_source*
    Disallow: /index.php/*
    Disallow: /custom-search/
Viewing 2 replies - 1 through 2 (of 2 total)
Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘Google doesn't crawl or index all my posts’ is closed to new replies.