I think I may have figured out the problem. I turned debugging on in wp and slowly went through all the errors and fixed them one by one. I eventually came to this:
ob_end_flush(): failed to send buffer of zlib output compression (1)
Some research indicates that WordPress does not like zlib compression and has caused problems with updating plugins among other things.
http://core.trac.wordpress.org/ticket/18525
I disabled zlib compression in my /etc/php5/fpm/php.ini
; Transparent output compression using the zlib library
; Valid values for this option are 'off', 'on', or a specific buffer size
; to be used for compression (default is 4KB)
; Note: Resulting chunk size may vary due to nature of compression. PHP
; outputs chunks that are few hundreds bytes each as a result of
; compression. If you prefer a larger chunk size for better
; performance, enable output_buffering in addition.
; Note: You need to use zlib.output_handler instead of the standard
; output_handler, or otherwise the output will be corrupted.
; http://php.net/zlib.output-compression
zlib.output_compression = Off
Now all those issues seem to be fixed! Theme previews now load just fine.
wordpress theme preview blank
Correction. I can see themes from the super admin account, but I still cannot see them from certain sub sites. This is very odd.
