Perma link (back and forth?)

If I choose my permalink to be month and name and then I realized I want to change that, am I going to harm myself? (search engines losing my pages, users get error pages because they bookmarked a different URL..etc)