Moved site to new server, Categories are all screwed up

I moved my site from one server to another recently. The new site looks pretty good except the categories are all screwed up. I have 80 on the old site and when it migrated there over 500. What had happened is that it duplicated many of them 10 times over. So I deleted all the duplicates and I’m down to the 80 on my new site, but the numbers are all off.

What was category number 6 on the old site is category number 31 on the new site. This screws up a lot of my stuff where I’m excluding categories from the main page, etc.

So my questions deals with importing/exporting. Can I export just the categories tables from my old database and import them into the new database without it screwing with the rest of the database?

I’m new to the database. I know that it’s wp_terms, wp_terms_relationships, and wp_terms_taxonomy that I should be dealing with. Can I just export those three and import them?

Thank you.