Installing offline?

Well, I want to try this WordPress.org thing out since it will give me plenty more freedom than the .com version does. My problem, however, is the fact that web hosts are simply too expensive to just jump in without knowing a whole lot yet. What I want to do is be able to set up a website that is on my computer, visible only to me, so I can practice editing and whatnot. I know you can do this with html files, so php must be able to do this, too, right?