Support » Plugin: XML Sitemap & Google News » (Suggest) robots.txt

  • Resolved davide

    (@davidemetupit)


    Hi,
    Your plugin is fantastic! I use the robots.txt editor with great satisfaction! I understand the decision to trim newlines after saving the editor, the problem is that I’m using another plugin (JM Twitter Cards), which implies that there is a newline at the end, and appends its text in the format:

    User-agent: Twitterbot\n
    Disallow:\n

    Can you help me? Maybe adding automatically a newline in outputting robots.txt?

    Thanks a lot
    Davide

    https://wordpress.org/plugins/xml-sitemap-feed/

Viewing 8 replies - 1 through 8 (of 8 total)
  • Hi Davide,

    Can you share a link to your robots.txt? I’d like to see the exact effect of these two plugins together…

    Thread Starter davide

    (@davidemetupit)

    Unfortunately I had to remove option from other plugin to have a correct Robots.txt , previous robots.txt was (from http://velvetgossip.it/robots.txt ):

    # XML Sitemap & Google News Feeds version 4.3.2 - http://status301.net/wordpress-plugins/xml-sitemap-feed/
    Sitemap: http://velvetgossip.it/sitemap.xml
    Sitemap: http://velvetgossip.it/sitemap-news.xml
    
    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Disallow: */xmlrpc.php
    Disallow: */wp-*.php
    Disallow: */trackback/
    Disallow: *?wptheme=
    Disallow: *?comments=
    Disallow: *?replytocom
    Disallow: */comment-page-
    #Disallow: *?s=
    Allow: */wp-content/uploads/
    Disallow: */wp-content/
    
    User-agent: *
    Disallow: /43775887/
    Disallow: /ngg_tag/
    Disallow: /2012/*/*/*/feed/
    Disallow: /2013/*/*/*/feed/
    Disallow: /2014/*/*/*/feed/
    Disallow: /gallery/vip-upskirt-foto/
    Disallow: /gallery/foto-kelly-brook-in-topless/
    # CustomUser-agent: Twitterbot
    Disallow:
    Thread Starter davide

    (@davidemetupit)

    Unfortunately I had to remove option from other plugin [and add its rows manually] to have a correct Robots.txt

    So everything until and including that

    ...
    # Custom

    was in the robots.txt field on Settings > Reading while the part

    User-agent: Twitterbot
    Disallow:

    after that came from the Twitter Cards plugin… Do I understand it correctly?

    Or does the robots.txt field content on Settings > Reading end with the default

    ...
    Disallow: */wp-content/

    ?

    Thread Starter davide

    (@davidemetupit)

    the first you said

    OK, thanks for the report 🙂

    It should be solved in the new development version. If you like, you can download it from http://wordpress.org/plugins/xml-sitemap-feed/developers/

    Thread Starter davide

    (@davidemetupit)

    Thanks,
    I Can’t, I must wait stable, I’m in enterprise environment 🙂

Viewing 8 replies - 1 through 8 (of 8 total)
  • The topic ‘(Suggest) robots.txt’ is closed to new replies.