Title: Robots.txt error Last modified: September 5, 2021 --- # Robots.txt error * Resolved [dmitresku](https://wordpress.org/support/users/dmitresku/) * (@dmitresku) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/) * Hi. I received an error notification with robots.txt How can I exclude this? * # BEGIN W3TC ROBOTS User-agent: * Disallow: /wp-content/cache/ # END W3TC ROBOTS * At the moment, I have manually changed the file robots.txt, but when clearing the cache in the plugin, this entry appears again. * The page I need help with: _[[log in](https://login.wordpress.org/?redirect_to=https%3A%2F%2Fwordpress.org%2Fsupport%2Ftopic%2Frobots-txt-error%2F%3Foutput_format%3Dmd&locale=en_US) to see the link]_ Viewing 15 replies - 1 through 15 (of 17 total) 1 [2](https://wordpress.org/support/topic/robots-txt-error/page/2/?output_format=md) [→](https://wordpress.org/support/topic/robots-txt-error/page/2/?output_format=md) * Thread Starter [dmitresku](https://wordpress.org/support/users/dmitresku/) * (@dmitresku) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14840025) * [Screen error Yandex Webmaster](https://dropmefiles.com/H6Ucw) * [Galfom](https://wordpress.org/support/users/galfom/) * (@galfom) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14840893) * See [https://wordpress.org/support/topic/robots-txt-cache-directory/](https://wordpress.org/support/topic/robots-txt-cache-directory/) for a quick fix (basically `chmod 444 robots.txt`). * Plugin Contributor [Marko Vasiljevic](https://wordpress.org/support/users/vmarko/) * (@vmarko) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14840988) * Hello [@dmitresku](https://wordpress.org/support/users/dmitresku/) [@galfom](https://wordpress.org/support/users/galfom/) * Thank you for reaching out and I am sorry about the issue you are experiencing. This issue is reported numerous times and we already gave a [Github issue](https://github.com/W3EDGE/w3-total-cache/issues/420) reported for this. The temporary solution is either to revert back to the previous version of W3TC or to change the permission for the robots.txt file. Please refer to the existing topic for this issue where the Github issue is posted [https://wordpress.org/support/topic/w3tc-conflict-with-aioseo/](https://wordpress.org/support/topic/w3tc-conflict-with-aioseo/) * Thanks! * [abogachev](https://wordpress.org/support/users/abogachev/) * (@abogachev) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14843995) * Hi, i am using Yoast, but see error in robots: * # BEGIN W3TC ROBOTS User-agent: * Disallow: /wp-content/cache/ # END W3TC ROBOTS * Last update (2.1.7) W3 Total Cache have a bug. Version 2.1.6 works correctly! * [chaosfranklin](https://wordpress.org/support/users/chaosfranklin/) * (@chaosfranklin) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14844622) * How do we change permission for the robots.txt.file or revert the plugin to a previous version? I am not a coder. There’s needs to be a solution for this type of users as well. * Plugin Contributor [Marko Vasiljevic](https://wordpress.org/support/users/vmarko/) * (@vmarko) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14846001) * Hello [@chaosfranklin](https://wordpress.org/support/users/chaosfranklin/) * The best thing to do is to delete the robots.txt file, create your own and set the permissions to 444. Alternatively, you can install the WP rollback plugin and revert to the previous version of the plugin. The files that were changed are: Generic_Environment.php Util_Rule.php w3-total-cache-api.php So you can download the previous version from [here](https://downloads.wordpress.org/plugin/w3-total-cache.2.1.6.zip) and upload the files that were changed. Once again thank you for your patience and we are working on a fix for this * [chaosfranklin](https://wordpress.org/support/users/chaosfranklin/) * (@chaosfranklin) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14846132) * [@vmarko](https://wordpress.org/support/users/vmarko/) * I don’t know how to delete the robots.txt.file. Instructions would be more helpful. * Plugin Contributor [Marko Vasiljevic](https://wordpress.org/support/users/vmarko/) * (@vmarko) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14847024) * Hello [@chaosfranklin](https://wordpress.org/support/users/chaosfranklin/) * The robots.txt file should be generated at the root of your website public_html/ yourwebsite.com/ You can cancel or FTP to your server and delete the file manually. I hope this helps! * [chaosfranklin](https://wordpress.org/support/users/chaosfranklin/) * (@chaosfranklin) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14847139) * [@vmarko](https://wordpress.org/support/users/vmarko/) * I’ve tried the WP rollback plugin because I am not a coder. I’m a usual user of WordPress. It did not resolve the problem. I’m still getting errors. * Furthermore, my hosting package does provide me access to the root of my website. I have a package only for WordPress. * Plugin Contributor [Marko Vasiljevic](https://wordpress.org/support/users/vmarko/) * (@vmarko) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14847691) * Hello [@chaosfranklin](https://wordpress.org/support/users/chaosfranklin/) * Thank you for the info. Even on shared hosting, you should have the access to your server for accounts, files, installing WordPress, etc. * I am not sure where you hosted your website but you should have access to your website files. Please reach out to your hosting support for more details about how to access it. Thanks! * [chaosfranklin](https://wordpress.org/support/users/chaosfranklin/) * (@chaosfranklin) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14847797) * [@vmarko](https://wordpress.org/support/users/vmarko/) * This is WordPress hosting from a web host. WordPress is pre-installed. I do not have access to WordPress’s files. None of your advice in any conversation about this issue has been helpful. * Please provide a solution for people and casual users, who are not coders. This glitch is hurting my business. * [abogachev](https://wordpress.org/support/users/abogachev/) * (@abogachev) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14848172) * Игнорируете мой вопрос на английском, тогда напишу по-русски! Бажное правило “ Disallow: /wp-content/cache/”, которое автоматически добавляется в роботс блокирует доступ Гуглу к минифицированным данным кэша, после чего в Консоли Гугла пачками появляются ошибки по мобильной оптимизации. * Во всех ваших рекомендациях указано, что такая проблема появляется только у сайтов, использующих All in One Seo, но я пользуюсь Yoast! Откатил плагин к старой версии и всё работает. Ваше последнее обновление содержит баг, который и создаёт вышеуказанные проблемы. * Plugin Contributor [Marko Vasiljevic](https://wordpress.org/support/users/vmarko/) * (@vmarko) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14848277) * Hello [@abogachev](https://wordpress.org/support/users/abogachev/) * Thank you for the information. Yes, you are correct. The problem is in the robots. txt file and we are working on a fix [https://github.com/W3EDGE/w3-total-cache/issues/420](https://github.com/W3EDGE/w3-total-cache/issues/420) * [@chaosfranklin](https://wordpress.org/support/users/chaosfranklin/) I’ve never heard of a hosting provider that does not allow users to access their files. Please let me know the URL of your hosting provider or drop us a note with the details via the plugin in Performance>Support. * Thanks! * Plugin Contributor [Marko Vasiljevic](https://wordpress.org/support/users/vmarko/) * (@vmarko) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14850103) * Hello [@abogachev](https://wordpress.org/support/users/abogachev/) [@chaosfranklin](https://wordpress.org/support/users/chaosfranklin/) [@dmitresku](https://wordpress.org/support/users/dmitresku/) * We have released a patch in version 2.1.8. We do apologize for any inconvenience. Please update and do let us know if there are any issues. We will be happy to assist you. Thank you. * [chaosfranklin](https://wordpress.org/support/users/chaosfranklin/) * (@chaosfranklin) * [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/#post-14852707) * [@vmarko](https://wordpress.org/support/users/vmarko/) * The update has not resolved this issue on my end. Google Search Console continues to generate errors and failed validations. I’ve updated the plugin and cleaned the site’s cache. Viewing 15 replies - 1 through 15 (of 17 total) 1 [2](https://wordpress.org/support/topic/robots-txt-error/page/2/?output_format=md) [→](https://wordpress.org/support/topic/robots-txt-error/page/2/?output_format=md) The topic ‘Robots.txt error’ is closed to new replies. * ![](https://ps.w.org/w3-total-cache/assets/icon-256x256.png?rev=1041806) * [W3 Total Cache](https://wordpress.org/plugins/w3-total-cache/) * [Frequently Asked Questions](https://wordpress.org/plugins/w3-total-cache/#faq) * [Support Threads](https://wordpress.org/support/plugin/w3-total-cache/) * [Active Topics](https://wordpress.org/support/plugin/w3-total-cache/active/) * [Unresolved Topics](https://wordpress.org/support/plugin/w3-total-cache/unresolved/) * [Reviews](https://wordpress.org/support/plugin/w3-total-cache/reviews/) * 17 replies * 6 participants * Last reply from: [Endymion00](https://wordpress.org/support/users/endymion00/) * Last activity: [4 years, 8 months ago](https://wordpress.org/support/topic/robots-txt-error/page/2/#post-14852747) * Status: resolved