WordPress.org

Ready to get started?Download WordPress

Forums

URL restricted by robots.txt possible plugin issue (3 posts)

  1. Jimmi_Henderson
    Member
    Posted 5 years ago #

    Ok here it goes..
    google error = URL restricted by robots.txt
    I have read all posts on this subject... still no solution...

    privacy settings = allow Google bot
    plugins installed = XML-site map generator and meta robot

    I would really love to know the best way to set this frigging thing up so as to allow unique content disallow duplicate content and it would be great to be able to submit the frigging thing and get googlebot to crawl...
    here is the url Http://garagedoortoday.com
    here is the site map from the plugin http://www.garagedoortoday.com/sitemap.xml and of course /robots.txt is there too....
    I have know frigging clue how to programm php mabey I will learn but can some one please tell me why this thing wont work...

  2. iridiax
    Member
    Posted 5 years ago #

    Since you are having problems, I'd deactivate the site map and meta robot plugins and delete the robots.txt and sitemap. XML site maps are not necessary, and having one with errors will hurt your Google indexing. Try a seo plugin if you are worried about duplicate content issues and meta tags.

  3. Jimmi_Henderson
    Member
    Posted 5 years ago #

    according to google they are a good idea check it out:
    http://services.google.com/breeze/webmasters/googleforwebmasters/

    slide 5/30 not only recommends an xml sitemap but says it is especially usefull for sites with dynamic content... eg a blog

Topic Closed

This topic has been closed to new replies.

About this Topic