"This is what happens on the internet" isn't a rationale, it's a reality.
You can definitely go through the motions of notifying the offending party that they are in breach of copyright and that the content should be removed immediately or legal action will be taken. Similarly, depending on where the site is hosted, you can check their host's web site to see if they have an 'acceptable use' policy that covers misuse of copyrighted material, though even if they do you should prepare yourself that they may have little or no intention of enforcing it. You might want to contact a lawyer and get a generic 'cease and desist' letter drafted, using suitably threatening legal language, and send it off to anyone who has copied your content, as a matter of course.
Ultimately, it's up to you to determine exactly how valuable the content is to you, and what lengths to which you are willing or prepared to go to protect it. Yes, it's illegal to use copyrighted material in the way you're describing, but it's painful and usually expensive to enforce.
And, I speak from experience, in that up to a couple of years ago I ran a site that had content stolen and reproduced a number of times. I did what I could to disuade the offenders from maintaining my content on their sites, but had to accept that even the most aggressive cease-and-desist campaign would often fall into a black hole of indifference, and ultimately I wasn't making enough money from the content to make chasing even one offender financially feasible.
It's frustrating, yes, but it's that reality thing.
 Or, more accurately, responds to a right-click event by displaying a "This content is copyrighted" notice, etc, instead of the expected menu with the "Copy" item.