I'm in the same boat with several clients that want to use Google Cal to "feed" the calendar on their site.
In the past I have used "Stout Google Calendar" http://wordpress.org/plugins/stout-google-calendar/ which is old and no longer supported, but so far it does still work. I use it on this site to display a full agenda (you can set it up to display other views):
The issue I have now is getting a Google calendar feed into a widget. It can be done with Stout Google Calendar, but the styling is not to my liking. So I started hunting for other solutions. I too found and tested the Time.ly plugin which has great potential but I ended up coming to the same conclusion as you ...bloated and didn't function correctly.
Anyway, not sure if Stout Google Calendar will solve your problem but figured I would try and help when I read your post here.