None of the cache plugins work

I have had issues with every wordpress cache plugin out there. Every time I enable any of them my server gets slammed and crashes very quickly. Heres some info about my site: 6,171 posts and 25,119 comments
I usually have anywhere from 5-15 posts per day, and comments per day usually are around 100-300 or so. I have a feeling that all of the content on my wordpress site is whats doing it, but I could be wrong. Maybe something isn’t configured correctly? Any thoughts on what I can do?