Fetch as google, robots.txt, .htaccess?

  • pinokio

    (@lapanwebsite)


    10 years, 1 month ago

    I will paste code from robots.txt and htaccess on pastebin.

    Here is robots.txt: http://pastebin.com/S69qDrqM
    And .htaccess: http://pastebin.com/H4qCAmdD

    When I “fetch as google” urls like

    domain.com.ba/filmovi/filmovi-strani-filmovi/filmovi-strani-filmovi-animacije/potraga-za-nemom-finding-nemo-2003/
    or

    domain.com.ba/oznaka/online-serije-sa-prevodom

    Sometimes it work, but most time keeps refreshing and after hour or so it would fail.

    I changed permalinks from %postname% to %category%/%postname% last week, don’t know should I wait more for google to update to my new permalinks, maybe the time is answer, or maybe robots.txt or .htaccess aren’t configured on the right way.

Viewing 6 replies - 1 through 6 (of 6 total)
  • Senff – a11n

    (@senff)

    10 years, 1 month ago

    So what’s your question?

    Thread Starter pinokio

    (@lapanwebsite)

    10 years, 1 month ago

    So what’s your question?

    Are my .htaccess and robots.txt correct, in order to allow
    domain.com.ba/filmovi/ << and subdirectories, sublinks.
    and to allow some of links like this one is
    domain.com.ba/oznaka/online-serije-sa-prevodom

    That’s the question.
    I don’t know do I have my side problem or Google side.

    kmessinger

    (@kmessinger)

    10 years, 1 month ago

    What theme are you using? Please post a url to your site. Are you verified by Google?

    Are my .htaccess and robots.txt correct

    This is not a WordPress problem. You may or may not get assistance

    Thread Starter pinokio

    (@lapanwebsite)

    10 years, 1 month ago

    Hi, kmessinger.

    http://lapan.com.ba/ is my site.
    http://lapan.com.ba/robots.txt

    I was easily adding URLs to Google in Webmaster tools before I did changed my permalinks and blocked some content by robots.txt

    Now, whenever I add link, allowed by robots.txt, google page keeps refreshing, and after hour it would fail.
    Not all but 90% of added links.

    Yes, maybe this is not a WordPress problem, I just want to be sure are my robots.txt and .htaccess ok, as my site is built with WordPress.

    Theme that I’m using is a child of the Hueman theme.

    kmessinger

    (@kmessinger)

    10 years, 1 month ago

    Do you have a backup of your robots.txt before you changed things?

    Thread Starter pinokio

    (@lapanwebsite)

    10 years, 1 month ago

    Yes, I do have.
    But backup of robots.txt are not for %category%/%postname% permalinks structure.

    I wanted to hide some whole categories from Google.
    That’s why I changed to this permalinks structure.
    I can block all or none of posts with %postname% permalinks structure.
    With this new I can block all category posts, without blocking some another category posts.
    That’s why I did this change.

    All is triggered by Google Bot that was eating my bandwidth way too much.
    I had to hide some parts of the site, and show just a key content.

Viewing 6 replies - 1 through 6 (of 6 total)
  • The topic ‘Fetch as google, robots.txt, .htaccess?’ is closed to new replies.