I'll just explain that a little further.
Inside each theme is the html for how the page looks. There is an initial page, index.php (inside your theme dir as explained in the previous post), which is basically a page that calls up the entries from every category and makes a list. The index.php page will call header.php, sidebar.php, and footer.php. Header is all the html until just past the body tag, sidebar is the side navigation html, and footer closes off the page with the closing body and html tags.
There are other pages that have slightly different tasks, like single.php, archives.php etc. They all want to do different things on the page, like display a single post, or display all posts from a single category, but they will all generally call the header and footer into the page, and often the sidebar.
By having common header, footer, and sidebar files, each page will built pretty much the same. A lot of what you want to customize will be in the header, footer, and sidebar files. These files frame the page. To get specific in the middle of the page with the meat of the content, look to index.php, single.php, archive.php, archives.php, etc.
The is an over simplificaion, but I hope it gives you the general picture. Each theme has it's own unique header, footer, etc., and of course, a unique style.css file. Most of the colour stuff, and a LOT of the layout is actually in the style.css file. In short, if you want to change how it looks, edit the style.css file. If you want to change what appears, look to the php files, and look for the html within then.