Theme Mods & Memory issue

Hi,
I am receiving Fatal Error on customizer screen, its fine until I save any settings on customizer screen then it gives error.

I have dozens of theme customization options in customizer.

I do
`
remove_theme_mods();
`
and it works again, but until I save any setting.

I have memory hungry plugins install, woocommerce, events calendar pro and such.

What is the memory limit for theme mods etc???

How I can overcome this?

Reducing amount of customizer settings is not an option.