stop robots.txt blocking one page

  • persist9

    (@persist9)


    1 day ago

    Google Search Console shows one page as

    “Indexed though (despite) being blocked by robots.txt

    [ redundant link deleted ]

    blocked by

    disallow: /*/feed

    I could change the name of my page, but I would prefer not to do that.

    I could delete that disallow instruction, but it blocks pages that I want blocked, so I would prefer not to do that.

    Can I make a change to my robots.txt file to stop the system from blocking this page, perhaps something like:

    Allow: /*/feedback*

    I think the blocking is stopping me from running the Facebook debugger on this web page

    https://developers.facebook.com/tools/debug/

    The page I need help with: [log in to see the link]

Viewing 3 replies - 1 through 3 (of 3 total)
  • Moderator threadi

    (@threadi)

    22 hours, 37 minutes ago

    It appears that the display of SEO information on your site is controlled by the JetPack plugin you’re using. I would recommend that you post your question on their support forum: https://wordpress.org/support/plugin/jetpack/

    Thread Starter persist9

    (@persist9)

    22 hours, 20 minutes ago

    I will put a query on the JetPack forum. Thanks.

    Could you tell me what is it in my question that points to JetPack. I have written my own robots.txt file assisted by AI, which may have misled me.

    Moderator threadi

    (@threadi)

    21 hours, 36 minutes ago

    The URL of the sitemap is usually listed in the robots.txt file. When you access it, you’ll see a Jetpack header above the list of your files 🙂 This strongly suggests that both the sitemap and the robots.txt file are managed through Jetpack – a practice common to many other SEO-focused plugins (which makes sense, since they go hand in hand).

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.