Hi Travis:
I came here looking for a solution to the exact same issue. After finding nothing I goofed around in WP for a while and I hope your fix is as simple as mine was!
What solved it for me was going into WordPress General Settings (Settings -> General) and setting the site timezone to my local time rather than the default UTC. After that, I cleared the cache on my calendar(s) and that fixed the issue for me.