I’ve not dug into 5.9 and FSE at all, but my guess is that this would be theme-dependent.
For instance, I just tested with the “Block-based Bosco” theme, and I see Frontpage, Archive, Search and 404; whereas another theme, Zelia, has only Frontpage.
I also did some other tests and came to the same conclusion. One possibility is to create your own template files in the “template” directory. Which, in my opinion, contradicts the idea of “full site editing”.
Hiya,
When you are in the editor, working on a normal page, the Settings sidebar has a new section called “Template”, you are here able to create a new custom template, not tied to any of the Template Hierarchy pages.
The interface at “Themes > Editor” is meant for template pages in general, and are based on HTML files in your theme, not specific page templates.