I will look into this tonight. Do you have some links already defined in your WordPress installation with associated categories? Could you post a list of the settings that you have configured for the plugin under the admin panel?
Looking at the source of the web page, I think that the problem is that it is not finding anything based on the way that you configured the plugin and this is resulting in some tags not being opened or closed correctly.
After doing some testing, I was able to reproduce the problem that you are seeing. It only occurs when there are no link categories to display.
I will post a new version (1.1.7) later tonight to correct this.
Now, to get in this situation did you try to specify some category IDs to be included or excluded under the admin settings? Did you use the numeric IDs? I would first recommend remove all include and exclude conditions to see if the plugin works for you correctly. Typing names of categories in these fields will not work. Also, have you deleted all of the default links before trying to use this plugin?