WordPress.org

Ready to get started?Download WordPress

Forums

robots.txt problems (5 posts)

  1. fifthhouse
    Member
    Posted 5 years ago #

    Hello. I installed a plugin for my site that creates a site map and a robots.txt file. The sitemap checks out well, but when I use my google webmaster tools to check the robots.txt file, in six out of the eight lines it says: Syntax not understood.

    I was wondering if anyone might be able to help me with my robots.txt file. I have tried using various ones offered in the codex documentation pages in the SEO section on robots.txt, but none of them check out well with google. Furthermore, I'm just shooting in the dark, as I have no idea what I'm doing to my site with these files.

    Any help would be much appreciated.

  2. whooami
    Member
    Posted 5 years ago #

    http://www.angerwise.com/robots.txt

    {\rtf1\ansi\ansicpg1252\cocoartf949\cocoasubrtf350
    {\fonttbl\f0\fswiss\fcharset0 Helvetica;}
    {\colortbl;\red255\green255\blue255;}
    \margl1440\margr1440\vieww9000\viewh8400\viewkind0
    \pard\tx566\tx1133\tx1700\tx2267\tx2834\tx3401\tx3968\tx4535\tx5102\tx5669\tx6236\tx6803\ql\qnatural\pardirnatural

    \f0\fs24 \cf0 User-agent: *\
    Disallow:}

    do NOT edit that file in MS Word. Do NOT save that file as an .rtf in Wordpad and then rename it. An .rtf (rich text format) file gets MS specific code added to it.

    Use notepad, or some other plain text editor.

  3. fifthhouse
    Member
    Posted 5 years ago #

    HI there and thank you for your help.

    I'm not sure what I'm to do with this. I see that this is my robots.txt file, but when I check it with google sitemaster tools, I get the message as described above.

    Do you know how I need to edit the file to eliminate the errors?

  4. Make a new Robots.txt file, in NOTEPAD (not word or wordpad or anything like that).

    No one can tell you what to set your robots.txt to since ... well we don't know what you want it as. What plugin did you install?

    As an example, here's a working robots:

    User-Agent: *
    Disallow: /cgi_bin/
    
    # BEGIN XML-SITEMAP-PLUGIN
    Sitemap: http://domain.com/blog/sitemap.xml.gz
    # END XML-SITEMAP-PLUGIN
  5. fifthhouse
    Member
    Posted 5 years ago #

    The plugin I installed is the DagonDesign site map generator.

    It creates a robots.txt file automatically.

    I don't know what I want in the file, but I have read numerous times that you need one in order to make sure google doesn't somehow mistakenly ignore your site.

    So I think I just want a basic robots.txt file, without specic things that it stipulates.

Topic Closed

This topic has been closed to new replies.

About this Topic

Tags

No tags yet.