Gave Replies Only a test try as well, broken as well.
Also, there's no option to switch from "replies only" to ALL within normal subscription pages.
Also, when advanced subscription is enabled, the checkbox is missing, which I presume means anytime a person makes a comment, they will be subscribed by default.
Normal subscriptions appear to work as intended. Management pages for readers work as well otherwise, so it just seems to be the replies only option that's buggy.