Core post privileges also enable custom post types

I have an issue with enabling ONLY core post privileges. I want a user to be able to create and publish posts. But when I enable edit_posts in Core>posts, it automatically allows all custom post types as well (which gives them access to
a restricted section).