Plugin does not protect site from crawlers
-
I intend to set up a private forum in which sensitive business issues can be discussed without leaking out.
After installing and configuring your plugin, I have installed a plugin showing the crawler activity on my site.
How can it be that there are still crawlers around?
What exactly does your plugin do to make a site “private”?
-
My guess is you mis-configured something. One of the more regular (if odd) complaints the plugin gets is once you block logins, you’re also blocking robots, and therefore private sites are terrible for SEO. So since we know that bots are blocked (because we’ve had users complain it pummels SEO), then I’m guessing you probably need to tweak something.
I would say this though, and this is as a security professional, not a WordPress plugin developer: if you have sensitive business information or you need to comply with regulations like HIPAA and FERPA, I wouldn’t rely on a free add-on to WordPress to secure my information. I’d invest in an entire end-to-end security management system (maybe something like SharePoint with Microsoft enterprise support).
Alternatively, you could put up a server behind a firewall and only allow those with local network access to see it (and those who can tunnel in via a secured VPN connection).
My Private Site can certainly keep out prying eyes (I use it for some of my experimental sites where I don’t want others to dig in to work I’m tinkering with), but I sure wouldn’t use this to keep medical patient information private, for example.
In security, it’s a matter of degree. What are you protecting and who do you need to keep out? How much of a risk is it if there is exposure? Are you keeping out random visitors or are you trying to protect against a directed assault from a nation state? That’s a very large range of difference.
–David
Hello David,
Thank you for your detailed explanation. I have not considered all of it. As the site is supposed to bring people from different companies together to discuss their startup projects as peers, I decided to go with a wordpress.com blog and set it private. It is https by default.
Obviously, one potential risk that sensitive information might leak is by the behavior of the forum users. If they make use of the RSS feeds on devices that are lying around etc.
So I guess it comes down to being a good host by giving a technically secure enough virtual room and ask for good behavior.
Re. the settings of your plugin – What might I have done wrong? I clicked the “Make this site private” and saved it. Apart from that, I did not set up anything in your plugin. On a wordpress settings level I disabled “anyone can register” and checked “discourage crawlers from crawling this site”.
Nicole
p.s. I’ll google HIPAA and FERPA
My own experience with the plugin is that crawlers will still have access to your site when it is protected by this plugin, but all they will see is the Login screen, just like anyone else who somehow discovered the URL of your site.
In the final analysis, there is no way to completely hide the existence of a domain name, since it can be found in a WHOIS listing. Which means that there will always be something unexpected in the weblogs.
As for the plugin protecting “everything”, the second paragraph of the Plugin Description in the Plugin Directory is worth a read:
This plugin does not control non-WordPress web pages, such as .html and .php files created by hand or by other software products. Or images and other media and text files directly accessed by their URL, or from a browser’s directory view, if available.
Thank you, jonradio, now I understand the crawler traffic.
Re. images – I am aware that they are not protected. I have the same problem with WooThemes Sensei. The course lessons are protected; the embedded images not.
This support threat also discusses ways of blocking RSS feeds:
https://wordpress.org/support/topic/not-blocking-site-rss-feed?replies=4
HIPAA and FERPA are among the many federal regulations for privacy. HIPAA is for medical records and FERPA is for student records.
Just keep in mind what level of security you’re going for and you should be okay. I generally recommend not posting ANYTHING on the Internet you wouldn’t want a future employer or a news reporter to see. But we do now need to use private messaging systems and expect them to be reasonably confidential. What you’re looking for the balance between those extremes.
–David
Thank you, David, for giving this further thought.
I’ll try the private feed key.
Nicole
The topic ‘Plugin does not protect site from crawlers’ is closed to new replies.