Cache you can stop if you wanted – it’s off be default on new installs but was on for previous installs.
To switch it off, put this is inside wp-config.php:
define('DISABLE_CACHE', true);
Following that you should be able to delete or change permissions.
Backup. If you do not use the plugin with WP
OR
your download your backups** then delete the folder or make it unwritable then ignore the warnings the plugin will make.
**because leaving backups of your data on a server in case the server crashes is ….. pointless really