Files Gone Public

I have a problem in that Google has listed for me pages it is not indexing but this reveals the fact that the documents are publically accessible despite having been loaded via this plugin and being in the “special” folder they can be accessed via a simple URL like “mydomain.com/mydocument.pdf”.

If I look at the file in the plugin it is set to Members Only but I can retrieve them without being logged in just using that simple URL. The URL showing there is the special one.

How does that work and how do I fix it?