I had this almost working 5 minutes ago like this:
http://domain.com/ <<< main site (with blog)
http://admin.domain.com/ <<< wordpress site
The only downside: you'll no longer be logged in on main site when you log into WordPress.
Overview of what I did:
- Moved WP core files to a different directory
- Moved index.php to main directory (1 and 2 more or less as per here
- Created subdomain in Apache
- Restarted Apache
- Went to main site
And that was it, basically! I was a little 'cheaty' since my admin subdomain was pointing to the main site + /admin, but that should be OK as long as you change that directory (I imagine it would work with any, even above 'main' path) in index.php.
I haven't been investigating it any further yet (and I'm thinking if I will really use it). In fact, I arrived here looking for some info.
Apart from not seeying the 'edit' and other admin options in main site, everything was OK.