Default behavior of Yoast SEO’s noindex status and indexing via Algolia

Yoast SEO is widely used (5+ million active installs according to WP.org as of writing), and so I’m left wondering why the setting of noindex by that plugin isn’t honored by this plugin by default.

The current solution is to add something to the effect of

/**
 * Don't have Algolia Search index pages where the robot index
 * option in the Yoast SEO plugin is set to noindex.
 * Via: https://github.com/algolia/algoliasearch-wordpress/blob/master/docs/src/indexing-flow.md
 *
 * @param bool    $should_index
 * @param WP_Post $post
 *
 * @return bool
 */
function filter_post_for_yoast_noindex_with_algolia( $should_index, WP_Post $post ){
    if ( false === $should_index ){
        return false;
    }
    return get_post_meta($post->ID, '_yoast_wpseo_meta-robots-noindex', true) == 1 ? false : true;
}
add_filter( 'algolia_should_index_searchable_post', 'filter_post_for_yoast_noindex_with_algolia', 10, 2 ); // Hook into Algolia to manipulate the post that should be indexed.

to the theme’s functions.php file.

However, why should this be necessary?

Wouldn’t it make more sense if this was the default behavior of this plugin (don’t serve up search results for things marked to not be indexed by search engines by the most used WordPress SEO plugin)?

At that point, sites could use a modified version of the code snippet above to include items in the Algolia search results that were set to be noindex for other search engines. Seems like this would better serve the majority of use cases while still fitting the needs of the less common use cases if/when needed (via a custom code snippet at that point.)

I’d love to see this officially adopted as part of this plugin so I’m not needing a custom code snippet on all of my sites in order to make sure something that isn’t set to be indexed isn’t then showing up on the site’s own search results (I’m also wondering how many sites are out there where this aspect has been overlooked when it definitely shouldn’t have been.)

Also, I’ve mentioned this at https://github.com/WebDevStudios/wp-search-with-algolia/issues/105 just in case there’s more technical discussion to be had on the GitHub repo rather than here.

Thanks,
Kurt