WordPress.org

Ready to get started?Download WordPress

Forums

Fetch as google, robots.txt, .htaccess? (7 posts)

  1. LapanWebsite
    Member
    Posted 12 months ago #

    I will paste code from robots.txt and htaccess on pastebin.

    Here is robots.txt: http://pastebin.com/S69qDrqM
    And .htaccess: http://pastebin.com/H4qCAmdD

    When I "fetch as google" urls like

    domain.com.ba/filmovi/filmovi-strani-filmovi/filmovi-strani-filmovi-animacije/potraga-za-nemom-finding-nemo-2003/
    or

    domain.com.ba/oznaka/online-serije-sa-prevodom

    Sometimes it work, but most time keeps refreshing and after hour or so it would fail.

    I changed permalinks from %postname% to %category%/%postname% last week, don't know should I wait more for google to update to my new permalinks, maybe the time is answer, or maybe robots.txt or .htaccess aren't configured on the right way.

  2. Mark Senff
    Member
    Posted 12 months ago #

    So what's your question?

  3. LapanWebsite
    Member
    Posted 12 months ago #

    So what's your question?

    Are my .htaccess and robots.txt correct, in order to allow
    domain.com.ba/filmovi/ << and subdirectories, sublinks.
    and to allow some of links like this one is
    domain.com.ba/oznaka/online-serije-sa-prevodom

    That's the question.
    I don't know do I have my side problem or Google side.

  4. kmessinger
    Moderator
    Posted 12 months ago #

    What theme are you using? Please post a url to your site. Are you verified by Google?

    Are my .htaccess and robots.txt correct

    This is not a WordPress problem. You may or may not get assistance

  5. LapanWebsite
    Member
    Posted 12 months ago #

    Hi, kmessinger.

    http://lapan.com.ba/ is my site.
    http://lapan.com.ba/robots.txt

    I was easily adding URLs to Google in Webmaster tools before I did changed my permalinks and blocked some content by robots.txt

    Now, whenever I add link, allowed by robots.txt, google page keeps refreshing, and after hour it would fail.
    Not all but 90% of added links.

    Yes, maybe this is not a WordPress problem, I just want to be sure are my robots.txt and .htaccess ok, as my site is built with WordPress.

    Theme that I'm using is a child of the Hueman theme.

  6. kmessinger
    Moderator
    Posted 12 months ago #

    Do you have a backup of your robots.txt before you changed things?

  7. LapanWebsite
    Member
    Posted 12 months ago #

    Yes, I do have.
    But backup of robots.txt are not for %category%/%postname% permalinks structure.

    I wanted to hide some whole categories from Google.
    That's why I changed to this permalinks structure.
    I can block all or none of posts with %postname% permalinks structure.
    With this new I can block all category posts, without blocking some another category posts.
    That's why I did this change.

    All is triggered by Google Bot that was eating my bandwidth way too much.
    I had to hide some parts of the site, and show just a key content.

Reply

You must log in to post.

About this Topic

Tags

No tags yet.