Fetch as google, robots.txt, .htaccess? (7 posts)

  1. LapanWebsite
    Posted 1 year ago #

    I will paste code from robots.txt and htaccess on pastebin.

    Here is robots.txt: http://pastebin.com/S69qDrqM
    And .htaccess: http://pastebin.com/H4qCAmdD

    When I "fetch as google" urls like



    Sometimes it work, but most time keeps refreshing and after hour or so it would fail.

    I changed permalinks from %postname% to %category%/%postname% last week, don't know should I wait more for google to update to my new permalinks, maybe the time is answer, or maybe robots.txt or .htaccess aren't configured on the right way.

  2. Mark Senff
    Posted 1 year ago #

    So what's your question?

  3. LapanWebsite
    Posted 1 year ago #

    So what's your question?

    Are my .htaccess and robots.txt correct, in order to allow
    domain.com.ba/filmovi/ << and subdirectories, sublinks.
    and to allow some of links like this one is

    That's the question.
    I don't know do I have my side problem or Google side.

  4. kmessinger
    Forum Moderator
    Posted 1 year ago #

    What theme are you using? Please post a url to your site. Are you verified by Google?

    Are my .htaccess and robots.txt correct

    This is not a WordPress problem. You may or may not get assistance

  5. LapanWebsite
    Posted 1 year ago #

    Hi, kmessinger.

    http://lapan.com.ba/ is my site.

    I was easily adding URLs to Google in Webmaster tools before I did changed my permalinks and blocked some content by robots.txt

    Now, whenever I add link, allowed by robots.txt, google page keeps refreshing, and after hour it would fail.
    Not all but 90% of added links.

    Yes, maybe this is not a WordPress problem, I just want to be sure are my robots.txt and .htaccess ok, as my site is built with WordPress.

    Theme that I'm using is a child of the Hueman theme.

  6. kmessinger
    Forum Moderator
    Posted 1 year ago #

    Do you have a backup of your robots.txt before you changed things?

  7. LapanWebsite
    Posted 1 year ago #

    Yes, I do have.
    But backup of robots.txt are not for %category%/%postname% permalinks structure.

    I wanted to hide some whole categories from Google.
    That's why I changed to this permalinks structure.
    I can block all or none of posts with %postname% permalinks structure.
    With this new I can block all category posts, without blocking some another category posts.
    That's why I did this change.

    All is triggered by Google Bot that was eating my bandwidth way too much.
    I had to hide some parts of the site, and show just a key content.

Topic Closed

This topic has been closed to new replies.

About this Topic


No tags yet.