Performance and load times are a huge focus as we work to build v2.0
We're hoping this will be released mid in February (if testing goes well).
In most of the cases, the reason of this high memory usage is GOOGLE CRAWLING. Please read our article here on how to limit Google crawling: support.time.ly/limiting-excessive-google-crawls/
Please clear your cache periodically from Wp-content -> plugins-> All in one calendar -> Cache
Try installing a cache plugin such as WP Total Cache. It will help reduce calls to the server. If possible, install APC or some form of Opcode Caching.
Your patience is greatly appreciated.