Soapbox for Blygger

Description

An independent implementation of the Blygger protocol; not affiliated with the Blygger project.

Soapbox for Blygger makes your WordPress site a publisher under the Blygger protocol, an open protocol for publishing on your own domain in a way other people’s software can subscribe to, archive completely, and quote exactly.

Once it is switched on, every post you publish also becomes an item of your blyg, at an address such as https://example.com/blyg/. For each item the plugin keeps:

• A permanent id, which never changes when you edit, rename or move the post.
• Versions. Each time the visible content changes, the item gets a new version. Saving without changing anything does not.
• Withdrawal, not deletion. Trash or delete a post and its item stays at its address, empty, marked withdrawn. Republish and it returns under the same id.
• Pins. Pin a version and exactly that version is served at its own address forever, so somebody can cite it and find the same words there later.

Readers get a manifest, an RSS feed that any feed reader can use, a complete archive index, and a document for every item, with the content as both Markdown and clean, self-contained HTML.

Before you publish

The protocol is a draft. This plugin implements Blygger 0.3, which is before 1.0. Its wire format may change, and the plugin will follow it. What will not change is what publishing means: every address your blyg serves is a promise to keep answering there, and a pin cannot be undone. Read “Can I undo this?” below first.

Nothing is published until you say so. Activating the plugin publishes nothing. You choose where the blyg lives and how it is laid out, save that, and then switch publishing on. Before you run the backfill, which is the step that puts your existing posts out, do these four things:

1. Check what a post looks like on the blyg if you use a membership, paywall or content-restriction plugin. The plugin renders each post as a logged-out visitor would see it, but not on the post’s own page, and a plugin that hides content only there may not hide it here. See “Does it work with membership or paywall plugins?” below. Test this first: what the backfill publishes cannot be unpublished.
2. Run the backfill’s dry run (Tools > Soapbox for Blygger). It renders every post and publishes nothing. Read some of what it shows.
3. Check Site Health for the entries about the blyg, the one about caching above all. A cache that holds the blyg’s documents would keep serving a post after you withdrew it.
4. Be sure of the mount path and the layout. Both are permanent from the first item published.

This is an independent implementation. Soapbox for Blygger is not written by, endorsed by or affiliated with the authors of the Blygger protocol. The protocol’s specification is at blygger.org.

What it does not do

• It does not change how your site looks, and it does not replace your existing RSS feed.
• It does not read or subscribe to other blygs. It publishes.
• It makes no requests to any other server, unless you switch on Webmentions (see below).

Works the WordPress way

• Any post type can publish: tick it in the settings, or add blyg support to your own post type.
• Settings are under Settings > Soapbox for Blygger; the backfill and integrity checks are under Tools > Soapbox for Blygger, and need no command line.
• A panel in the editor shows each post’s id, versions and pins.
• Site Health checks that the blyg is reachable and that no cache in front of your site is holding it.
• WP-CLI: wp blyg backfill, status, pin, verify and origins.
• Multisite: each site has its own blyg and its own switch.

External requests

With its default settings this plugin makes no requests to any other server, and sends no data anywhere.

If, and only if, you switch on “Accept Webmentions from other blygs”:

• When a Webmention arrives, your site later fetches at most three documents to verify it: the source address the sender gave; the item document that page points to; and, when that document is not at the protocol’s usual address for it, the manifest (blyg.json) of the blyg it says it belongs to, which is asked where that blyg keeps its items. Counting redirects, that is six HTTP GET requests at most, so three documents leave room for three redirects between them: a sender whose addresses each redirect twice will not verify, and should send the final addresses. Over its whole life a mention can be tried twice and put off three times, so thirty requests is the most one mention can ever cause. They go to the sender’s own server, whose address the sender chose, or to wherever it redirects.
• The request carries a User-Agent naming this plugin and your site’s address, as any HTTP request reveals your server’s IP address. No other data is sent.
• Requests are refused to private, loopback, link-local and other reserved network addresses, IPv4 and IPv6, at every redirect. Each is limited to 5 seconds and 1 MB, and a compressed response is refused rather than unpacked. A mention is tried twice at most, and put off at most three times when the destination has had its hour’s requests. Mentions are rate-limited by who sends them, by the publication they name and overall, and so are the requests your site makes to any one network: when that limit is reached a mention waits for the next hour instead of failing. The limits are counted per clock hour, so across the turn of an hour a sender can use one hour’s allowance and then the next.
• To apply the limit by sender, the plugin keeps a scrambled form of the network address each Webmention came from. It cannot be turned back into the address. If your site is behind a reverse proxy or a CDN, every sender appears to come from the proxy: use the soapbox_blyg_webmention_client filter to name the visitor’s address.
• There is no third-party service involved: no account, no API, no terms beyond those of whichever site sent the mention.

How a mention is handled. Your site answers the sender at once and looks nothing up while it does: the sender’s address is resolved, checked and fetched afterwards, by a background job. That job stops after twenty seconds in all, checked between each step; one step it cannot cut short is a DNS lookup, which can take as long as your server’s own resolver allows. Each sender is limited by the number of requests it makes, whether or not they are accepted.

Known limits. Each request is pinned to the address that was checked, which closes DNS rebinding. That needs PHP’s curl extension, which nearly every host has. A site without it, or one that sends its outbound requests through a proxy (WP_PROXY_HOST), cannot pin a request, so it verifies no Webmentions: they are accepted and then marked as failed. The templated layout’s manifest declares Blygger “0.3”, because the specification does not say what a manifest that uses the 0.4 construct should declare. IPv6-only senders are refused: a sender’s host must have an IPv4 address, and no address it has, in either family, may be private. WordPress only allows such requests on ports 80, 443 and 8080; a sender on another port is refused unless you add it with WordPress’s http_allowed_safe_ports filter. A mention is believed only when its item document is at exactly the address its own blyg gives for it, written plainly: an address with .. in it, or with characters percent-encoded that need not be, is refused. A document kept on another host than its blyg (a CDN, say) cannot be sent as the source itself; send the page that links to it, which must be on the blyg’s own host. In the templated layout, your own items are at addresses only your manifest names, so a receiver that implements Blygger 0.3 alone cannot verify a mention that names one of your items as its source; a receiver that reads the manifest’s templates can. That is one more reason the fixed layout is the one selected to begin with.

Caching. The plugin states its lifetime first, and relies on caches using the first of a repeated instruction, as RFC 9111 says and Varnish does. A cache that does not follow that rule could still use a lifetime your host appends. nginx’s proxy_cache and fastcgi_cache are one such case: when the host’s instruction arrives as a separate Cache-Control header line after the plugin’s, nginx uses the last line. This has not been tested with nginx in front. Site Health asks for the documents as a reader would and compares what it is given with the documents as they are now; if it is given an old copy, purge that cache and exclude the blyg’s paths from it as described above.

For developers

The source, with its tests, is at https://github.com/cyberscribe/soapbox-blyg. The scripts in build/ are compiled from assets/src/ there with npm run build.

The functions below are the stable API. Classes under src/ are internal. Guard calls with function_exists().

soapbox_blyg_register_origin( string $key, array $args ): void
soapbox_blyg_publish( string $origin, string $source_key, ?array $snapshot, array $opts = [] ): array
soapbox_blyg_reconcile( string $origin, array $snapshots_by_key, array $opts = [] ): array
soapbox_blyg_render_html( string $html, string $title, array $opts = [] ): array
soapbox_blyg_pin( string $id, int $version ): void
soapbox_blyg_get_status( string $origin, string $source_key ): ?array
soapbox_blyg_get_origin_url( string $origin ): string

Publishing your own post type. This is all it takes:

add_action( 'init', function () {
    register_post_type( 'book', array(
        'public'   => true,
        'label'    => 'Books',
        'supports' => array( 'title', 'editor', 'thumbnail', 'blyg' ),
    ) );
} );

Or, for a post type you do not control: add_post_type_support( 'book', 'blyg' );

Publishing something that is not a post. Register an origin, which is a second blyg at its own address, and hand the plugin snapshots. It decides what is a new version.

add_action( 'soapbox_blyg_register_origins', function () {
    soapbox_blyg_register_origin( 'notes', array(
        'path'   => '/notes/',
        'title'  => 'Notes',
        'author' => array( 'name' => 'Ada Example' ),
    ) );
} );

$rendered = soapbox_blyg_render_html( $html, $title );
soapbox_blyg_publish( 'notes', 'note:42', array(
    'kind'         => 'fragment',
    'title'        => $title,
    'permalink'    => 'https://example.com/notes/42/',
    'content_md'   => $rendered['content_md'],
    'content_html' => $rendered['content_html'],
    'media'        => $rendered['media'],
) );
soapbox_blyg_publish( 'notes', 'note:42', null ); // Withdraws it.

Rules for snapshots: permalink must be an absolute http or https URL, since it is served as the item’s page; a thread must carry a transclusions array and a fragment must not; content_hash is always computed by the plugin. Give each item a page of its own if you want Webmentions to be able to name it by page.

Actions: soapbox_blyg_register_origins, soapbox_blyg_event (event, id, version, origin), soapbox_blyg_pinned (id, version, origin), soapbox_blyg_mention (status, mention, fields).

Actions: soapbox_blyg_purge_urls (the addresses whose documents changed in this request; see “Which caches does Soapbox for Blygger purge?”), soapbox_blyg_event, soapbox_blyg_pinned, soapbox_blyg_forget_stale.

Filters: soapbox_blyg_origin_args, soapbox_blyg_is_public, soapbox_blyg_snapshot, soapbox_blyg_rendered_body, soapbox_blyg_allowed_html, soapbox_blyg_cache_lifetime, soapbox_blyg_cache_control, soapbox_blyg_discovery_origin, soapbox_blyg_webmention_limits, soapbox_blyg_webmention_client, soapbox_blyg_fingerprint_meta.

Withholding posts with soapbox_blyg_is_public. Return false for a post and it is kept off the blyg, and withdrawn if it is already there: at its next save, and whenever anything of it would be served. Three things to know when you write one:

• It is always run as a logged-out visitor, whoever is making the request. is_user_logged_in() and current_user_can() are false inside it. The question is whether the post is public, not whether the current user may read it.
• If it reads post meta, name the keys with the soapbox_blyg_fingerprint_meta filter, so that a change to one counts as a change to the post.
• The feed and the archive index check every post through your filter, and keep the answer for up to a minute. A change to a post, its meta or its terms is seen at once. If your filter depends on anything else (an option, a membership level), call do_action( 'soapbox_blyg_forget_stale' ) when that changes, or the feed and index may be up to a minute behind. An item’s own document is always checked afresh.

If a plugin adds its own text to every post through the_content (a “Share this” heading, a list of related posts), remove it in soapbox_blyg_rendered_body, or it will be published in every item.

Capabilities: soapbox_blyg_manage (maps to manage_options) and soapbox_blyg_pin_item (maps to being able to edit and publish the post). Change either through map_meta_cap.

Templates: copy templates/soapbox-blyg-withdrawn.php or templates/soapbox-blyg-origin.php into your theme to change the page shown for a withdrawn item, or at the bare address of an origin you registered.

JavaScript sources are in assets/src/; the built files in build/ are produced from them with @wordpress/scripts. The bundled library, league/html-to-markdown (MIT), is in vendor-prefixed/ under a prefixed namespace.