Template Error When Loading 3.4

I’m having some problems setting up a fresh and basic WordPress site. I’m loading 3.4 and I get the following error.

“Template Error: The template file must be given”

There are problems starting in the admin dashboard and then spills over into the public view of the site. I can only see the public view when I load a non-default theme. I have been talking with my webhost tech guy (who knows WordPress very well) and is having problems finding the answer on the server. Thinking it could be an update problem. Any thoughts?

P.S. I get the error without adding anything to the install.