Disable payment without signing in

I have disabled guest checkout.

I manually create an order for an existing customer/user. I send them the invoice email through the actions part of the order edit screen. In the email, they have a pay button. It appears that they are able to pay for that order without logging into the site.

I’m not even sure that this is an issue but seems like anyone that grabs a hold of this link can pay without knowing the credentials of the user.

Should I be worried about this? Is there a way to force the user to login before they are able to pay?

Thanks in advance for the help.

http://wordpress.org/plugins/woocommerce/