Waymender – Broken Link Checker, 404 Monitor & Redirects

Description

Broken links cost you readers, rankings and trust. Most link checkers either hammer your server and the sites you link to, or leave you with a spreadsheet and no way to fix anything. Waymender does three things, and does them gently:

1. Scan. Links and images in your posts, pages, custom post types, custom fields, custom-link menu items, Block, Text and Custom HTML widgets, term descriptions and page-builder content (Elementor, Bricks, Beaver Builder, Divi, WPBakery) are collected and checked in the background, in small batches with a politeness delay between link checks on the same domain, so no host is hammered and your own site never waits on a check. Menu items that point to a page, post or category are not scanned. Known anti-bot sites (LinkedIn, Instagram, Amazon…) are never requested; they are listed for manual review instead. A failure has to happen twice, at least six hours apart, before a link is called broken, so a hiccup never wakes you up. The exception is “Recheck now” on a link that has already failed once: that check confirms it immediately, so if it fails again the link is marked broken straight away.

2. Fix. From the Links screen you can:

• Edit the URL (the new one is checked right away, or its last result shown if Waymender already knows it)
• Unlink, keeping the text
• Remove or replace a broken image from the media library
• Use the closest Wayback Machine copy of a dead page
• Update every occurrence of a redirecting link to its final destination in one go (for this and “Fix everywhere”, a summary shows how many places were fixed, how many were not, and each distinct reason or note)
• Create a redirect for broken internal links

The editor sidebar lists the problem links in the post you are editing and offers Unlink and Remove right there; its “More options” link opens the Links screen for the other fixes. In the classic editor, a “Waymender: link problems” box links to the Links screen filtered to that post.

Fixes to post content and excerpts are saved through WordPress, so they create a normal post revision when the post type supports revisions. Menus, widgets, term descriptions, comments, custom fields and page-builder data have no revisions. For every fix, wherever the link is, Waymender’s fix log (the waymender_fix_log database table) records the old URL, the new URL (empty for Unlink and Remove), the action, the ID of the user who made it and the time, plus the revision ID when a revision was created. The log is only stored in the database: there is no screen for it and no undo. A menu item cannot be unlinked (edit or remove it under Appearance → Menus). A custom field that holds nothing but the URL gets the new URL, or is emptied when you unlink or remove it (you are told when that happens); an ACF link field keeps its title and target when its URL is replaced.

When an archived copy replaces a link, rel=”nofollow” (on by default) is added to the link’s existing rel value, never duplicated. That applies to HTML links in post content, excerpts, widgets, term descriptions, comments and HTML custom fields; menu items, custom fields that hold only a URL, and page-builder data keep their rel unchanged.

3. 404 log and redirects. Pages visitors could not find are logged with hit counts, referrers and a suggested target (fuzzy slug matching, date-prefix and case rules). One click turns an entry into a 301/302/307/410 rule. Wildcards, CSV import/export, and automatic hand-off when the Redirection plugin is active. Hit counts are sampled: each rule counts at most one hit per minute. To keep a flood of random URLs from filling the database, the 404 log records at most 120 hits per clock minute across the whole site (filterable with waymender_404_writes_per_minute); further 404s in that minute are not logged or counted.

Built to be light

• Custom tables, no post meta bloat, automatic pruning
• Action Scheduler queue with WP-Cron fallback
• Adaptive batch size, per-host throttling, Retry-After respected
• Detects blocked outbound HTTP and says so instead of reporting everything as broken
• WP-CLI: wp waymender scan, wp waymender links, wp waymender fix (run it with --user=<administrator>; it refuses to run without a user, and the fix is logged under that user), wp waymender 404, wp waymender redirect

Hooks for developers

waymender_http_args, `waymender_excluded_hosts`, `waymender_antibot_hosts`, `waymender_extract_sources`, `waymender_extracted_links`, `waymender_status_for_response`, `waymender_before_fix`, `waymender_after_fix`, `waymender_404_ignore`, `waymender_suggestions`, `waymender_scan_started`, `waymender_scan_complete`, `waymender_url_checked`, `waymender_capability`, `waymender_404_writes_per_minute`, `waymender_redirection_active`.

External services

Waymender runs entirely on your own server. It uses no third-party link-checking service and sends nothing to the plugin author. It does make the following outbound requests.

1. The websites you link to

To find out whether a link works, Waymender requests it from your server the way a browser would: a HEAD request first, then a GET when the site does not answer HEAD properly. Waymender sends the URL being checked with three headers: a User-Agent that identifies the plugin and your site address, Mozilla/5.0 (compatible; Waymender/<version>; +https://your-site/), so that site owners can see who is checking and why; Accept: text/html,application/xhtml+xml,*/*;q=0.8; and Accept-Language: en-US,en;q=0.8, so that sites answer as they would a browser. No cookies, referrer, visitor data or content from your site are sent.

Link checks are spaced out per domain: after one link on a domain is checked, the next link on that domain waits 2 seconds by default (configurable; at most 1 second for a few large sites that handle it easily: wikipedia.org, github.com, youtube.com, youtu.be, wordpress.org and google.com, subdomains included). Within a single check, the HEAD request, the GET fallback and any redirects it follows go out back to back. A 429 (Too Many Requests) or 503 (Service Unavailable) answer pauses checks of that domain for its Retry-After time (at most 7 days), or 1 hour when it sends none. Working links are cached for 7 days, and domains you list as excluded or anti-bot are never requested. Waymender does not read robots.txt. Checks happen in the background after you start a scan, in an hourly re-check of links that failed once or were rate limited, and, if you enable it in Settings, in a weekly automatic scan. Links pointing to your own site are resolved locally where possible and otherwise requested from your own server in the same way.

2. WordPress.org, once

If a link check fails with a DNS error, Waymender sends a single HEAD request to https://api.wordpress.org/core/version-check/1.7/ to find out whether your host blocks outbound connections altogether, so it can tell you instead of marking every link broken. It carries the same three headers as above. This happens once per site, and once more each time you tick the re-test option under Settings → Advanced (shown after outbound requests were found blocked). Privacy: https://wordpress.org/about/privacy/

3. Internet Archive, only when you ask

When you click “Use archived copy” on a broken link, Waymender asks the Wayback Machine availability API at https://archive.org/wayback/available for the closest snapshot of that URL. The broken URL is sent with a User-Agent naming the plugin; the archived URL and its date come back and are cached for 24 hours. If you then click “Use this copy”, the link in your content is rewritten to that web.archive.org address (with rel=”nofollow” by default, see Fix above) and the new address is checked like any other link: a HEAD request to web.archive.org, a GET when HEAD is not answered properly, and redirects followed up to five times. No page content is downloaded from the archive or inserted into your posts. Terms of use and privacy policy (the Internet Archive publishes both on one page): https://archive.org/about/terms

4. Email

If you enable the weekly report under Settings → Notifications (off by default), Waymender sends one email every Monday, around 08:00 in your site’s timezone, through your site’s normal wp_mail() to the addresses you enter, or to the site admin email. It contains counts plus the top broken links and 404 paths, and is skipped when there is nothing to report.

Privacy

The 404 log is on by default and records, for each page path visitors could not find: the path, a hit counter, first and last time seen, the domain name (not the full URL) of up to five referring sites, and a coarse browser family such as “chrome” or “bot”. It does not store IP addresses, full user-agent strings, cookies, or which logged-in user made the request. Query strings are dropped unless you turn on “Keep query strings”, in which case anything in the query string, including tokens or email addresses if your site puts them there, is stored as part of the path. Requests that look like bots are logged too, flagged as bots, and hidden on the 404 log screen until you tick “Include bots” (untick “Hide bot traffic by default” under Waymender → Settings → 404 log to show them from the start). An entry is deleted automatically once its path has not been requested for 30 days (configurable), and the log is capped at 5,000 paths (configurable), dropping the least recently seen first; both rules cover ignored entries too. You can turn the log off or add ignore patterns under Waymender → Settings → 404 log, and delete every entry, ignored ones included, with “Clear log” on the Waymender → 404 log screen.

Fixes and redirects record the ID of the user who made them, like any other WordPress edit. The plugin registers a suggested paragraph under Settings → Privacy → Policy Guide. Deleting the plugin removes all of its tables and options unless “Keep data” is enabled.