In order to display Latvian characters properly, WP 2.1 should include two lines in the wp-settings.php file after the line no. 267
mysql_query("SET CHARACTER SET utf8");
mysql_query("SET NAMES utf8");
When these 2 lines are added all text is shown properly. I think it's something to do with the way how connection is established with MySQL database v.4.1 and later. I have MySQL v.5 and without these two lines all text is garbage...