This is a horrible, kludgy solution to this. It will break the next time you upgrade WordPress, but hopefully, there will be a feature to choose default categories for “press this” soon.
Edit wp-admin/press-this.php
<div id="categorydiv" class="stuffbox">
<h2><?php _e('Categories') ?></h2>
<div class="inside">
<div id="categories-all" class="ui-tabs-panel">
<ul id="categorychecklist" class="list:category categorychecklist form-no-clear">
<!-- Edit this line to suit your setup - my default category number is "11" and its name is "random" -->
<li id='category-11'><label class="selectit"><input value="11" type="checkbox" checked name="post_category[]" id="in-category-11"/>random</label></li>
<?php wp_category_checklist($post->ID, false, false, $popular_ids) ?>
</ul>
</div>