Exclude posts from loop based on custom field

I have a custom post type that I am generating a query for. I am ordering the query using meta_value_num based on a date in a custom field (event_start_date). But I also want to filter out all posts from the loop where another custom field contains Yes (as opposed to No).

My current query:

query_posts($query_string . '
&post_type=events
&order=ASC
&orderby=meta_value_num&
meta_key=event_start_date
&posts_per_page=5');

So this is for an event listing, and I am displaying posts that only have a end date (end_date_e) that is + 86340 greater than the current time (which I think is about 2 months) and also if the event is not ongoing (ongoing_e != YES)

<?php
if (strtotime($end_date_e)+86340 > time() && ($ongoing_e != "Yes")) {
$numDisplayed++;
?>

This works to a point, but the ongoing events are still part of the loop query and thus are counted against my posts_per_page count.

My question is how would I exclude (ongoing_e == Yes) from the loop query rather than just hiding it in the output? I looked at using meta_compare, but it seems to only compare to whatever the current meta_key value is.