Filter products

Hello, I am currently working on products filters for an online shop, on woocommerce and shop isle theme, and I am stuck.

This website is selling decoration products, such as carpets that comes in differents sizes. I have created size attributs that work fine on a product page, but where I have an issue is on the research products page (the shop main page), on the left I have my filters (size / colors), but I can always see them, even if I am not in the carpet category, so what I want to do is that the size attributs I created for the carpets shows only when I select the carpet category in the filters.