ok if you have decided to hack the plugin file, I figured out how to make it add the menu to the right place, from my experience with dealing with custom post types and adding custom features to them.
1) like I said - change the post_type= part of LINE60 to your custom post type.
2) postMash adds it's menus at the end of the same file, in a function named "postMash_add_pages". In this case I added postMash menu to my custom post type named "staff" :
function postMash_add_pages(){
//add menu link
global $minlevel, $wp_version;
if($wp_version >= 2.7){
$page = add_submenu_page('edit.php?post_type=staff', 'postMash: Order Posts', 'Arrange', $minlevel, __FILE__, 'postMash_main');
}else{
$page = add_management_page('postMash: Order Posts', 'postMash', $minlevel, __FILE__, 'postMash_main');
}
add_action("admin_print_scripts-$page", 'postMash_head'); //add css styles and JS code to head
}