Which WooCommerce Extension do I need?

This is possible with core WooCommerce plus a tiny bit of (technically optional) custom code.

First, you need a shipping zone configured like so:

https://d.pr/i/TufmbY/ACUTVzkTMr

In my example, my zone is for New York in the U.S., but you can use whatever region.

Inside the free shipping method, the configuration is like so:

https://d.pr/i/m54LHj/abd4Tu1XFr

Inside flat rate:

https://d.pr/i/Tq8ZPw/LDq9BO52W9

To get that to work, you’d need to add a Free Shipping class:

https://docs.woocommerce.com/document/product-shipping-classes/

… and then assign a cost of 0 to that class inside the flat rate shipping method.

As a result of all this, items in the Free Shipping class will always be 0 to ship, while everything else will cost 3.95 per item. The Free Shipping method will take over only when the order exceeds a total of 25.

The one issue here is that, by default, it will look like this after 25 is exceeded:

https://d.pr/i/dWUbot/opQAtFkMjh

This custom code will work around that by removing other shipping options when free shipping is available:

https://docs.woocommerce.com/document/hide-other-shipping-methods-when-free-shipping-is-available/

Let me know if that works for you. 🙂 Thanks!