User Roles

Hey
I run a directory site on wordpress where users can create a listing and others can search and book this listing.

my problem is that at the moment EVERY registered user can add photos to any listings created by any users.
I want to change the user role or whatever is needed to achieve that ONLY the listing owner can add/edit photos to his/her listing.

I tried to fix this problem with a plugin called USER ROLE EDITOR, where i can add or change capabilities to user roles, but this does not seem to help because every registered user does have the same user role.

Is it possible to hide the “add photos” button from everyone besides the listing owner or something like this ?

thanks for your help !
Niklas