Fatal error: Allowed memory size of 134217728 bytes exhausted. > Why?

Hello,
I am using WordPress on several sites and on several hosts. It works great!

But sometimes I get an error like this:
“Fatal error: Allowed memory size of 134217728 bytes exhausted”+ some more text about the host and a filename.

For example, on one blog when I use a ‘delete duplicate posts’ plugin I get this following error:
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes) in /home/username/public_html/domain.tld/wp-includes/plugin.php on line 138
In this case, deleting the duplicate posts doesn’t work.

I sometimes got these errors also when doing other things. Mostly it are things that are related to ‘heavier’ work, like for example using a plugin that creates a backup of database and all files.

It happens ons several hosts. (multiple linux hosts and one IIS host) And it happens in all WordPress versions, from older to 2.9. Plugins are all up to date.

I am wondering what I can do against these ‘memory size exhausted’ errors?