Google doesn't crawl or index all my posts

  • Be Dark

    (@b_dark)


    10 years, 8 months ago

    On a multisite i realise one of my sites Google doesn’t crawl or index all my posts or make a lot of days to index news posts.

    Example: from 27th-30th july i have make 11 posts and only 5 have indexing. Why?!
    my site is active 3 years.

    i use the ultimate seo & google xml sitemap 4.0b11

    my robots.txt is

    sitemap: http://sub1.com/sitemap.xml

# Google Image
User-agent: Googlebot-Image
Disallow: /files
Disallow: /wp-content/uploads
Disallow: /wp-content/blogs.dir

User-agent: Googlebot-Video
Allow: /

# Google AdSense
User-agent: Mediapartners-Google*
Allow: /

User-agent: Googlebot-News
Allow: /

User-agent: Googlebot-Mobile
Allow: /

User-agent: AdsBot-Google
Allow: /

User-agent: ia_archiver
Disallow: 

User-agent: bingbot
Allow: /

User-agent: msnbot
Allow: /

User-agent: msnbot-news
Allow: /

User-agent: msnbot-media
Allow: /

User-agent: msnbot-products
Allow: /

User-Agent: Slurp
Allow: /

# digg mirror
User-agent: duggmirror
Disallow: /

# global
User-agent:  *
Disallow: /cgi-bin/
Disallow: /wp-admin/*
Disallow: /wp-content/*
Disallow: /wp-includes/*
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-content/themes/
Disallow: /banners/
Disallow: /wp-*
Disallow: /comments/feed/
Disallow: /feed/
Disallow: */feed/
Disallow: /*/feed/rss/$
Disallow: /trackback/
Disallow: */trackback/
Disallow: /index.php
Disallow: /xmlrpc.php
Disallow: ?comments=*
Disallow: /comment-page-
Disallow: /*?s=
Disallow: /search?
Disallow: /?p=*
Disallow: /*.pdf$
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.cgi$
Disallow: /*.xhtml$
Disallow: /*.php*
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.txt$
Disallow: /*?*
Disallow: /*.gz$
Disallow: /*.wmv$
Disallow: /*.zip$
Disallow: ?theme=
Disallow: /category/*
Disallow: /category/*/*
Disallow: /transfer/
Disallow: /tweets/
Disallow: /mint/
Disallow: /readme.html
Disallow: /license.txt
Disallow: /wp-content/downloads/
Disallow: /downloads/
Disallow: /recommends/
Disallow: /go/
Disallow: /category/
Disallow: /tag/
Disallow: /tag/*
Disallow: /archives/
Disallow: /author/
Disallow: /search?
Disallow: /recommended/
Disallow: *?wptheme
Disallow: ?comments=*
Disallow: /?p=*
Disallow: /images/
Disallow: /embed.js?pname=wordpress&pver=*
Disallow: /*?replytocom=*
Disallow: /stats/
Disallow: /general/
Disallow: /date/
Disallow: /?attachment_id*
Disallow: /search/?*
Disallow: */trackback/*
Disallow: /?cat*
Disallow: /?m*
Disallow: /*?utm_source*
Disallow: /index.php/*
Disallow: /custom-search/
Viewing 2 replies - 1 through 2 (of 2 total)
  • Pat Victor

    (@instovo)

    10 years, 8 months ago

    What’s your domain? Did you submit your sitemap to Google Webmaster tools? Are you following all the best practices?

    See: https://support.google.com/webmasters/answer/35291?hl=en

    Thread Starter Be Dark

    (@b_dark)

    10 years, 8 months ago

    yes i have submit the sitemap on webmaster tool of google ,
    example today i publish a new post, after 1 hour of the publish my post is doesn’t indexing on google but the simultaneous repost(the same title with only the excerpt and link to original post) in a blogpost site(6 months active) is indexing on google :/ strange

Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘Google doesn't crawl or index all my posts’ is closed to new replies.