In Next Gen Gallery 1.5.5, line 261 of core.php reads:
$custom_template = apply_filters( 'ngg_render_template', false, $template_name );
The apply_filters function only takes two arguments, not three; The hook $tag and the $value to be modified. Because $template_name is the third argument, it doesn't get passed to the filter. Instead the filter is always passed FALSE, which makes it impossible for plugins to detect what template is being requested through this filter.
Instead, the line should read:
$custom_template = apply_filters( 'ngg_render_template', $template_name );