My install was not working because of WP_PLUGIN_DIR being defined in the wp-cache-config.php:
if ( ! defined('WPCACHEHOME') )
define( 'WPCACHEHOME', WP_PLUGIN_DIR . '/wp-super-cache/' );
I had to man change to this
if ( ! defined('WPCACHEHOME') )
define( 'WPCACHEHOME', WP_CONTENT_DIR . '/plugins/wp-super-cache/' );
I thought this was defined in the WordPress defaults?