That's a bit much... Relevanssi database should have a lot more rows than your wp_posts database, but it shouldn't be very large. For example, I have a site with about thousand posts: wp_posts is 9,900 rows and 70 MB, while wp_relevanssi is 250,000 rows and 25 MB.
How many posts you have? Your database sounds large to me, which may mean it's too large for Relevanssi. Previous experience says Relevanssi stops being usable when the database size reaches 100,000 posts or so. Relevanssi hasn't been optimized for large databases, it's a simple blog search engine that will handle thousands of posts just fine and will do with tens of thousands, but hundreds of thousands will be too much.
Make sure you enable "Limit searches" and adjust it to something fairly small, 100 or maybe even less. That should give a significant performance boost in many cases, though with a really large database it may not be enough.