I strongly disagree. It's easy to remove them manually. I would hate to see the .po and .mo files deleted automatically based only on the language choice of the install.
At present, it's easy to change the language of an install with a couple of minor changes BECAUSE the .po .mo files exist.
Please don't change that.
I wouldn't give any star, but since there is no way to give "negative stars", I'll enter a half-star to bring down the count.
The only thing that I would agree with would be an interface in the admin area where would could go to choose what .po .mo files they want to get rid of, instead of having to go through ftp or cpanel file manager.