Hi Chris
The plugin has a housekeeping option – if you have switch that on before deleting the plugin cleans up after itself and leaves nothing – but that is not your issue.
The ‘fragments’ you are seeing would be the gallery css that is cached either by your own browser ( mostly likely ) or your site caching.
Purge your sites CSS cache or do a full cache cache purge. After that if you are still seeing the issue – clear your browser cache.
Regards
Steve