When we encode video we should adhere to specific standard heights (lines per frame, such as 480, 720, 1080, etc.). This is somewhat irrelevant to the height of the video viewer element a user sees in their browser. The element height is managed by element attributes and CSS and is unaffected by the video encoding height.
The answer to your question is do both. The encoding height relates to the overall video quality but does not directly impact the viewer element height. You must make a choice when encoding a video. You should also specify the desired viewer height. This could vary depending on the browser’s viewport size, so you would likely want to follow responsive design principles.
Note that embed codes obtained from video streaming services often include arbitrary viewer sizing. Our ability to alter this varies by the nature of embed codes.