• Resolved Max

    (@makspostal)


    Hi,

    How can I turn OFF Tools - File Editor feature in YOAST SEO?

    I’m using WordPress in multisite mode and plugin Robots.txt Editor in order to manage each site’s robots.txt.
    But this YOAST SEO feature adds code block:

    
    # START YOAST BLOCK
    # ---------------------------
    User-agent: *
    Disallow:
    
    Sitemap: http://en.wpwc.ru/sitemap_index.xml
    # ---------------------------
    # END YOAST BLOCK
    

    in my robots.txt files, so they become broken for robots.txt validator.

Viewing 15 replies - 1 through 15 (of 21 total)
  • Plugin Support Maybellyne

    (@maybellyne)

    Hello Max,

    Thanks for reaching out regarding your robots.txt file. If I understand correctly, you use another plugin to manage your robots.txt file and will like to disable to directives added by our plugin.

    Since you’re in a multisite environment, that block of text in the file can’t be removed. But there’s a filter to disable the subsites sitemap link from robots.txt. Further information is available here: https://github.com/Yoast/wordpress-seo/pull/18909.

    You may also consider customizing the robots.txt crawl rules using Yoast SEO’s hook instead of writing yours. But again, there’s no way to remove the existing crawl rules. Here’s the relevant hook to work with https://github.com/Yoast/wordpress-seo/blob/f43026341e91e40bae9950ff9f6f4dae616bf172/src/integrations/front-end/robots-txt-integration.php#L97

    Thread Starter Max

    (@makspostal)

    Hello Maybellyne,

    Thanks for reaching out regarding your robots.txt file. If I understand correctly, you use another plugin to manage your robots.txt file and will like to disable to directives added by our plugin.

    Yes, this is correct.

    Yoast\WP\SEO\register_robots_rules hook returns robots_txt_user_agents and robots_txt_sitemaps
    But I need to remove whole section

    
    # START YOAST BLOCK
    ...
    # END YOAST BLOCK
    

    Because it contains directive User-agent: * witch doublicate my own ‘User-agent: *’ directive. And this causes robots.txt validation error.

    So, I don’t need to exclude multisite xml sitemaps, I need to turn Yoast robots.txt support off at all and managed them by another plugin which works with multisite network more correctly.
    But also, I need to use Yoast XML sitemaps and don’t want to change them, because they work perfectly.

    Thank you in advance for your help.

    Plugin Support Maybellyne

    (@maybellyne)

    Hello @makspostal

    I’m sorry that what you currently want is not possible. Please submit a feature request for our product team on GitHub at https://github.com/Yoast/wordpress-seo/issues

    This thread was marked resolved due to a lack of activity, but you’re always welcome to re-open the topic. Please read this post before opening a new request.

    Thread Starter Max

    (@makspostal)

    Hello @maybellyne

    Thank you for your help.
    Could you tell me what version of the plugin doesn’t contain this feature?
    I mean add Yoast SEO block to robots.txt?

    # START YOAST BLOCK

    # END YOAST BLOCK

    I would like to downgrade the plugin.

    Thanks.

    Hi @makspostal,

    Just a quick fix from my side to disable the corresponding filter to attach to the WordPress hook robots_txt.

    @maybellyne: Thanks for your effort to making the Yoast plugin such as great tool. However, it appears to be a very invasive way to attach to robots_txt at priority “99999” without letting people knowing that the plugin starts manipulating robots.txt. Please consider to add a custom preference setting or a filter, which can be overwritten in function.php of a theme, to control the call of the function filter_robots.

    
    --- src/integrations/front-end/robots-txt-integration.org.php      2022-11-08 15:04:33.425277229 +0100
    +++ src/integrations/front-end/robots-txt-integration.php  2022-11-08 15:04:42.677050142 +0100
    @@ -64,7 +64,7 @@
             * @return void
             */
            public function register_hooks() {
    -               \add_filter( 'robots_txt', [ $this, 'filter_robots' ], 99999 );
    +       //      \add_filter( 'robots_txt', [ $this, 'filter_robots' ], 99999 );
            }
    
            /**
    

    Cheers,

    Matthieu

    Thread Starter Max

    (@makspostal)

    Hi @theschappy

    I totaly agree with you. Yoast SEO is the best SEO sulution for WordPress, but this featue is unacceptable in some cases. In my opinion, it has to be optional.
    Thanks for your solution.

    https://github.com/Yoast/wordpress-seo/issues/19161

    • This reply was modified 1 year, 6 months ago by Max. Reason: Add Github link

    I think they turned it off with the latest update, right?

    No, the plugin developers have not fixed this problem yet. It’s a pity, it’s a big inconvenience because of this.

    Hello all, I did this solution, hope it help for someone.

    add_action('wp_loaded', 'show_all_filters');
    /**
     * List of all filters with their function names.
     */
    function show_all_filters()
    {
        remove_filter_by_method_name('filter_robots');
    }
    
    /**
     * Remove needed filter from list
     *
     * @param string $method_name Name of the method called to the filter.
     * @param int $priority Set the priorite from that filter if we need to remove just from it.
     *
     * @return void
     */
    function remove_filter_by_method_name(string $method_name, int $priority = 0)
    {
        global $wp_filter;
    
        if (isset($wp_filter['robots_txt']->callbacks) && is_array($wp_filter['robots_txt']->callbacks)) {
            foreach ($wp_filter['robots_txt']->callbacks as $callback_priority => $callback) {
                foreach ($callback as $function_key => $function) {
                    if ('filter_robots' === $function['function'][1]) {
                        $priority = 0 === $priority ? $callback_priority : $priority;
                        unset($wp_filter['robots_txt']->callbacks[$priority][$function_key]);
                    }
                }
            }
        }
    }
    Thread Starter Max

    (@makspostal)

    Hi @nsukonny

    Thank you for the solution!

    I just simplified your snippet a bit for this specific case:

    /**
     * Fix Yoast SEO robots.txt changes.
     * https://wordpress.org/support/topic/disable-robots-txt-changing-by-yoast-seo/#post-16648736
     */
    function wpwc_fix_yoast_seo_robots_txt() {
    
    	global $wp_filter;
    
    	if ( isset( $wp_filter['robots_txt']->callbacks ) && is_array( $wp_filter['robots_txt']->callbacks ) ) {
    
    		foreach ( $wp_filter['robots_txt']->callbacks as $callback_priority => $callback ) {
    			foreach ( $callback as $function_key => $function ) {
    
    				if ( 'filter_robots' === $function['function'][1] ) {
    					unset( $wp_filter['robots_txt']->callbacks[ $callback_priority ][ $function_key ] );
    				}
    			}
    		}
    	}
    }
    
    add_action( 'wp_loaded', 'wpwc_fix_yoast_seo_robots_txt' );
    Sergio Urra

    (@sergiourra)

    @nsukonny and @makspostal

    Where this piece of code must be placed?

    /**
     * Fix Yoast SEO robots.txt changes.
     * https://wordpress.org/support/topic/disable-robots-txt-changing-by-yoast-seo/#post-16648736
     */
    function wpwc_fix_yoast_seo_robots_txt() {
    
    	global $wp_filter;
    
    	if ( isset( $wp_filter['robots_txt']->callbacks ) && is_array( $wp_filter['robots_txt']->callbacks ) ) {
    
    		foreach ( $wp_filter['robots_txt']->callbacks as $callback_priority => $callback ) {
    			foreach ( $callback as $function_key => $function ) {
    
    				if ( 'filter_robots' === $function['function'][1] ) {
    					unset( $wp_filter['robots_txt']->callbacks[ $callback_priority ][ $function_key ] );
    				}
    			}
    		}
    	}
    }
    
    add_action( 'wp_loaded', 'wpwc_fix_yoast_seo_robots_txt' );
    nsukonny

    (@nsukonny)

    @sergiourra Just put it to functions.php in your theme.

    Sergio Urra

    (@sergiourra)

    It works. Thank you @nsukonny.
    I hope Yoast fixes this issue.

    Linuxvit Santos

    (@linuxvit-santos)

    Testing wordpress 6.0 yoast. Thanks for the info!

Viewing 15 replies - 1 through 15 (of 21 total)
  • The topic ‘Disable robots.txt changing by YOAST SEO’ is closed to new replies.