So I've seen this post on the forums before, but none of the solutions which don't involve editing core files actually work.
The only solution that does work is to edit /wp-includes/post-template.php on line 1170 (in WP 3.1):
return apply_filters( 'wp_get_attachment_link', "$link_text", $id, $size, $permalink, $icon, $text );
Can anyone explain how this could be put into the functions.php file instead of editing the core file?