If you are referring to NextGEN Smooth Gallery, then I have good news! It is possible.
But, it does require a little more 'hack' than before.
The first problem, is that NextGEN Smooth Gallery doesn't seem to care about the unique image id, but we need it. So, to solve that you need to change line 102 of nggSmoothSharedFunctions.php from the blank line it is now, to this:
$aux["pid"] = $picture->pid;
Now, we can get the custom field created with NextGEN Custom Fields to use as the link like this:
On line 182 of nggSmoothSharedFunctions.php (if you didn't create any new lines with the previous edit), you will see the
<a> link code. You need to replace that line with this:
$out .= " <a target=\"_blank\" href=\"" . nggcf_get_field($picture["pid"], "custom link") . "\" title=\"open image\" class=\"open\"></a>";
Remember to use the name you called your custom field in the place of "custom link", as in the example I showed awarner20.
That should be it, let me know how that works out for you :)
[edit: fixed funky markup that broke this post, x2]