Different Sidebar for Single Post depending on the Category ID of the Post

Hi…
I would like to know how to change the sidebar for a single post view…depending on what category that single post view belongs to. I have 4 different categories on my blog and each has its own modified sidebar. It works great but when someone clicks on a post to view in a single post view…the sidebar changes to the default sidebar rather then staying with the corresponding category sidebar.

In single.php I am guessing <?php get_sidebar(); ?> needs to be modified to check if it belongs in category 1,2,3 or 4 and then show the corresponding sidebar1.php, sidebar2.php, sidebar3.php, or sidebar4.php

any help one how I could implement this? Thank you…