Support » Plugin: XML Sitemap & Google News feeds » (Suggest) robots.txt

Viewing 8 replies - 1 through 8 (of 8 total)
  • Plugin Author RavanH

    (@ravanh)

    Hi Davide,

    Can you share a link to your robots.txt? I’d like to see the exact effect of these two plugins together…

    davide

    (@davidemetupit)

    Unfortunately I had to remove option from other plugin to have a correct Robots.txt , previous robots.txt was (from http://velvetgossip.it/robots.txt ):

    # XML Sitemap & Google News Feeds version 4.3.2 - http://status301.net/wordpress-plugins/xml-sitemap-feed/
    Sitemap: http://velvetgossip.it/sitemap.xml
    Sitemap: http://velvetgossip.it/sitemap-news.xml
    
    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Disallow: */xmlrpc.php
    Disallow: */wp-*.php
    Disallow: */trackback/
    Disallow: *?wptheme=
    Disallow: *?comments=
    Disallow: *?replytocom
    Disallow: */comment-page-
    #Disallow: *?s=
    Allow: */wp-content/uploads/
    Disallow: */wp-content/
    
    User-agent: *
    Disallow: /43775887/
    Disallow: /ngg_tag/
    Disallow: /2012/*/*/*/feed/
    Disallow: /2013/*/*/*/feed/
    Disallow: /2014/*/*/*/feed/
    Disallow: /gallery/vip-upskirt-foto/
    Disallow: /gallery/foto-kelly-brook-in-topless/
    # CustomUser-agent: Twitterbot
    Disallow:
    davide

    (@davidemetupit)

    Unfortunately I had to remove option from other plugin [and add its rows manually] to have a correct Robots.txt

    Plugin Author RavanH

    (@ravanh)

    So everything until and including that

    ...
    # Custom

    was in the robots.txt field on Settings > Reading while the part

    User-agent: Twitterbot
    Disallow:

    after that came from the Twitter Cards plugin… Do I understand it correctly?

    Plugin Author RavanH

    (@ravanh)

    Or does the robots.txt field content on Settings > Reading end with the default

    ...
    Disallow: */wp-content/

    ?

    davide

    (@davidemetupit)

    the first you said

    Plugin Author RavanH

    (@ravanh)

    OK, thanks for the report 🙂

    It should be solved in the new development version. If you like, you can download it from http://wordpress.org/plugins/xml-sitemap-feed/developers/

    davide

    (@davidemetupit)

    Thanks,
    I Can’t, I must wait stable, I’m in enterprise environment 🙂

Viewing 8 replies - 1 through 8 (of 8 total)
  • The topic ‘(Suggest) robots.txt’ is closed to new replies.