It seems like you are saying the tracking script and not the plugin that inserts it is slowing down the site. The plugin only runs the queries it needs when it needs it to avoid extra processing. My test this morning show no impact on load time and the addition of only 1-3 queries to the 15 running by default.
More information would be very helpful (your website, wordpress version, other plugins, etc). Feel free to email us at email@example.com and I'll look into it further. Otherwise I'll pass this on to HubSpot to see if there's anything they can do on the tracking script side.