When used with a CAPTCHA plugin and/or custom registration forms fields, users are created on failed form submission. What this means is that when a user corrects their mistakes and submits a second time, their username is now already taken :(
Custom: require your custom fields in your code. When registering don't fill them in. Upon error, submit again: your username is now already taken.
BWP reCAPTCHA: Register w/o entering reCAPTCHA. Upon error, submit again: your username is now already taken.