You should be able to find a free theme in the Theme Directory that comes pretty close.
I wonder how much it would cost to get a customized theme
Rough guess: $200 - $1000 depending on the design complexity, source graphics needed, extra functionality etc.
how do you know if the quality is good enough.. security, stability, SEO
Unless the theme includes extra functionality beyond what WP normally offers, security shouldn't be an issue as that's handled by WP.
Stability can only be assessed by throwing different content into the site and then viewing it using varying screen resolutions and browsers. A good theme should use W3C compliant markup & CSS that will maximise its chances of performing effectively across as many browsers as possible.
In terms of SEO, have a look at:
http://quirm.net/2009/07/02/wordpress-search-engine-optimisation/
Includes some comments about choosing the right theme but you can also use to, essentially, work backwards and apply some of the ideas to a theme. For example, does it display a unique meta title based on the post title on every page? Do hard-coded page elements - such widgets, search results or comment displays use semantic markup with good use of headers?
If you're going to spend hard cash on a custom theme, don't use the guy down the road 'cos he knows a bit of web stuff. Use a company or individual with a web development track record who can supply references - that you can contact - on request.