Posting via e-mail is open to a lot of spam and bulk mail. When using insecure networks (e.g. public access wi-fi spots), the secret email ID can be recovered easily by a packet sniffer. One alternative would be to enable SSH on the host server and send out all emails using an SSH tunnel, but a better alternative would be to add GPG signature verification support for WordPress. The way it would work is the following:
1) WP carries a list of allowed public keys of all the people who are authorised to post.
2) While posting via e-mail, the outgoing mail should be digitally signed.
3) WP should /dev/null all e-mails which are not signed appropriately
optionally, a WP public key can also be set up to encrypt e-mails when posting to WP, but since everything would go on display, no sense in hiding the e-mail traffic.
I'm not a very proficient PHP coder, and hence I do not know how to integrate GPG support in WP, but any help required with handling GPG messages, I shall be glad to provide.