Link comment gravatar to user website

I’m trying to link the comment gravatars to the user websites, however, I’m having trouble figuring out how to do it only for commentators who have websites.

This is my code, but obviously I get an error because of the !empty. What do I have to use instead?

<?php if (!empty(comment_author_url())) { ?>
      <a href="<?php comment_author_url(); ?>">
			<?php } ?>
				<?php echo get_avatar( $comment, 70 ); ?>
			<?php if (!empty(comment_author_url())) { ?>
      </a>
			<?php } ?>