migrating from wp.com to wp.org – but keeping my (hyperlocal) host

I have a wordpress.com website and blog, miriamseidel.com. I had the domain before setting up the wordpress.com site, and it is with a small local host.

I would like to transfer over to wordpress.org, and would rather stick with my host if possible. I believe I can do this if I switch over manually. BUT from reading other threads, it looks like there might be a problem moving my subscribers over. If I understand right, the ONLY way to move the subscribers over is by using wordpress.com’s paid Guided Transfer service, and they require me to switch to one of the same domain hosts that wordpress.org also recommends. Am I understanding this right?

If there is a way around this, are there any other hidden problems I might encounter in changing over to wordpress.org if I keep my current host?

Thanks.