In your dashboard area, go to Appearance->Menus. You can add items to create a custom menu on this page.
Click on the “Screen Options” tab at the top of the screen and check the “CSS Classes” tab from the resulting panel. You will then have the option to add CSS classes for each individual <li> item via the main section of the menu page.
You can also add custom <a href=""> links to your menu via this page.