Won’t Log Out

I just installed WordPress on my server to test it as a possibility to use for a new site I’m starting, and I am encountering a problem whenever I try and log out of the admin account. When I return to the home-page as a visitor it automatically logs me back in and I can’t seem to stay logged out. Even if I was previously logged into a different account when I log out and return to the home-page it logs me in as admin. Any suggestions?