WordPress.org

Ready to get started?Download WordPress

Forums

[closed] how to add robots.text file (4 posts)

  1. tradtips
    Member
    Posted 2 years ago #

    How to upload robots.txt file,and how to check if already uploaded in website.

  2. esmi
    Forum Moderator
    Posted 2 years ago #

    Try using Google. This isn't a WordPress specific issue.

  3. tradtips
    Member
    Posted 2 years ago #

    ok thanks.

  4. jtt89
    Member
    Posted 2 years ago #

    You need to make a robots.txt file. You can start with this:

    User-agent: *
    Disallow: /cgi-bin
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /trackback
    Disallow: /comments
    Disallow: /category/*/*
    Disallow: /tag/
    Disallow: */trackback
    Disallow: */comments

    ...and then upload it to root folder of your website (where wp-admin, wp-content, and wp-includes folders are). After you upload it you can enter http://your_domain_name.com/robots.txt to see it in a browser or go to something like: http://www.spidertest.com/ to see if it works or not etc (I did not check this one, but I know it is not able to spider my site as I am not letting anybody in right now)

Topic Closed

This topic has been closed to new replies.

About this Topic

Tags

No tags yet.