the following is the robots.txt of my website:
User-agent: * Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
User-agent: * Disallow: /wp-content/uploads/wpo-plugins-tables-list.json
Please help me remove ” User agent * ” in the last line.
-
This reply was modified 1 year, 1 month ago by
janardhan.
@janardhanjp It is not possible to remove single line. You can remove all the lines added by WP-Optimize by adding following code in your theme’s functions.php file
add_action('after_setup_theme', 'ab123_remove_robots_txt_entry');
function ab123_remove_robots_txt_entry() {
if (function_exists('WP_Optimize')) {
remove_filter('robots_txt', array(WP_Optimize(), 'robots_txt'), 99, 1);
}
}
-
This reply was modified 1 year, 1 month ago by
Venkat Raj.
-
This reply was modified 1 year, 1 month ago by Venkat Raj.
the following is th Robots.txt as on today:
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
User-agent: *
Disallow: /wp-content/uploads/wpo-plugins-tables-list.json
the robots text analysis is telling like this,
Results of robots.txt analysis
1 error
lines used
User-agent: * Multiple ‘User-agent: *’ rules found
So there are two User Agent:*
So pl try to remove the last but one line in robots text
that is remove User Agent:* before
Disallow: /wp-content/uploads/wpo-plugins-tables-list.json
which is repetitive
@janardhanjp At this moment there is no option to remove that single line. To implement this, could you please let us know which robots.txt analyzer throws error? Thanks
Yandex Site diagnostics says,
Robots.txt contains errors. This can lead to incorrect crawling and indexing of your site.
Check the robots.txt file.
User-agent: *
Disallow: /wp-content/uploads/wpo-plugins-tables-list.json
WP Optimize was added by me recently. After that I found this error message in Yandex.
Thanks Venkat Raj. I have added the code given by you in functions.php. And the lines added by WPoptimize are gone. Thank you very much.
