retnd

A backup producer somewhere writes a dump, an archive or a snapshot to disk. That machine has finite space, so something has to move the artifact off it and something has to delete the original. retnd is the half of that job that runs on the NAS: it discovers finished artifacts on the remote server over SFTP, pulls them, verifies them, commits them durably, records that it did, and only then removes the remote copy.

It is a standalone Go binary that embeds pinned rclone Go packages. It does not fork rclone, and it does not shell out to the rclone command line for normal data movement.

What 0.4.0 added

0.4.0 is one idea and two corrections. The idea is that a destination is an instance of a registered backend rather than one of a fixed pair, and nearly everything new here follows from taking that seriously. The corrections are what running 0.3.3 on real hardware turned up, and the first of them is a reason to upgrade rather than a nicety.

A backend is described by a manifest the build carries — data, not code — and the manifest is what decides which fields a destination has, which steps its connection test runs, and what the steps it skips say instead. Two are registered: local_volume, a directory on a disk this machine can see, and s3, a bucket on Amazon's service or anything that speaks its API. The drive backups already land on is an instance too, seeded on a fresh install as a declared destination called local, so it is a row in the same list with the same controls rather than an unnamed default the interface had no way to talk about. A deployment can hold two volumes and two buckets now, and the list can tell them apart.

Two wizards come with it. Add a destination is three steps — choose a backend, name the instance, confirm — and writes nothing at all, which the confirm step says in as many words. The backends it offers come from GET /api/v1/backends, so there is no list of backend names anywhere in the page, and a backend this build understands with no manifest behind it is drawn and refused rather than quietly left out. Configure is the manifest-driven half: one renderer, not a form per backend, and nothing in the page knows what a local volume asks for. It fills in what the manifest declares, proves it, and only then writes. That order is enforced rather than advised — editing any field makes an earlier pass stale, the screen says so where the edit was made, and Save configuration stays out of reach until a check against the values now on screen has passed — and the consequence an operator can use is that no retention tier can be pointed at a destination nobody has proved.

Handing the Default mark from one destination to another is its own confirmation, because only one of the two things it does is the thing that was clicked: the destination taking the mark stops being removable, and the one giving it up becomes removable. Nothing already written moves. Declaring a destination moves nothing either; backups arrive somewhere only once a retention tier names it, which stays a separate decision.

The first correction is #662, measured on 0.3.3 on the NAS this installer was proven on. A completed local copy could be recorded as zero bytes: whatever the backend reported for the read-back went into the journal and into the sidecar recovery manifest — the record a catalog rebuild trusts when the journal is gone — without ever being checked against the file. Reconciliation then found two journal fields disagreeing with each other and quarantined the intact file over it, without reading the file. And there was no way back out: retry and fetch looped, the artifact was re-discovered as already known, and the refusal named no verb that would resolve it. Commit now records the bytes it actually made durable, a transfer that reports less than its object does not carry the zero forward, a disagreement between two records is treated as a fault in the records rather than as a verdict about a file, and from a deployment already in that state some documented verb ends with the artifact at a durable restore point.

The second is the surface that reported it. The activity feed keyed a run's severity on the destination it was writing to, so a real failure could read as ordinary; it keys on the transition now, and a FAILED backup has a control that reaches a recovery verb instead of a message recommending a command the same window called nonexistent.

The last piece is the first ten minutes. What the installer prints is transcribed from the installer on every run of the gate rather than written beside it (#714), so the output blocks above are the real ones; and the enrolment link names this machine's own address rather than localhost, which on a laptop reading the log over SSH is the laptop (#688). enroll-link reissues one when the thirty minutes have passed.

The rule everything else serves

Every design decision below is either enforcing that rule or admitting where the enforcement does not exist yet. It is why the delete comes last, why the journal is written before the delete rather than after it, and why a changed SSH host key halts a backup set instead of carrying on.

What one artifact goes through

The stages below are the ones the interface names, and the order is fixed. Deletion of the remote source is the last thing that happens, after the local copy has been written, verified, fsynced and recorded.

Where a copy goes after that, and how long it is kept, is the retention policy's business. The reference page covers the destinations and the tiers that name them.

The command line

Everything the web interface does, and several things it does not, are reachable from retnd. Every command except version takes --config, defaulting to /etc/retnd/config/config.yaml, which is a file inside the config directory rather than a file mounted on its own; pass the directory and it resolves config.yaml inside it.

The README's copy of this table is checked against the binary's own dispatch table on every run of the repository's gate, so it cannot quietly go stale. The copy above is a transcription of it and is not checked, so if the two ever disagree, the README is right.

The reference page carries the same table with the subcommands and the interesting flags filled in, plus where a configuration change is written and what each exit code means. That copy is checked.

What has not been proven

This section exists because the README has one, and a documentation site that quietly drops the uncomfortable half is worse than no site.

• Nothing destructive can be done over HTTP, in any deployment that exists. The API's destructive operations are behind a deployment-level gate, and the only implementation of that gate this repository ships is NotYetImplementedGate, whose Passed() returns false unconditionally (apps/common/webhost/gate.go:114). There is no flag, environment variable or constructor argument that makes it true. So every destructive route fails closed everywhere, by construction rather than by configuration, until issue #92 supplies a real gate.
• No workflow run can be started from a browser, so none of the ones that were tested was. The gate above is why: run_backup_set is a destructive operation, so every workflow run the end-to-end suite observed was one the scheduler started, and the hook-bypass an administrator is allowed in principle lives on that same refused route. Remote hooks are worse than untested-looking: they had never executed at all between the feature landing and issue #919 being found by that suite and fixed, because the engine handed the remote-exec channel a step id its own token rule refused, and the refusal was mislabelled as a lost connection while a token-less capability probe went on reporting the channel healthy. The workflows guide lists the rest, including the two things the feature explicitly does not guarantee.
• The 0.4.0 image is not on the registry. This release is cut and not pushed: canonical.json records image.published: false and the release manifest records a null digest per architecture, which are held together so neither can move alone. 0.3.3 is the newest tag that resolves, and it is signed. Until 0.4.0 is pushed, install it by building the image yourself or side-loading one.
• No acceptance procedure has been run on real hardware. The procedures for TrueNAS, Unraid, OpenMediaVault, Synology and Proxmox VE are written, reviewed and specific, and nobody working on the repository has one of those machines to execute them on. Every one of those platforms is build-supported and uncertified.
• There is no restore execution. retnd restore asks a storage provider to make an archived copy readable again, which is a different thing. Getting a backup back onto a server is a documented manual procedure, and that procedure is the whole of it.
• There is no metrics endpoint. A Prometheus renderer exists in the tree and nothing imports it. No listener serves /metrics.
• Every picture on this site comes from a mock. Every screenshot and every clip was recorded against the development server's in-memory fixture API, not against a running engine. The layout, the copy, the flow and the interaction are the real ones. The data is not, and each page says so.
• The rename is complete on this site and nowhere a store reviewer looks. This product was called backupd until #885. Everything with a picture on this site was re-recorded against the renamed interface through the capture scripts in docs/site/tools/: the 20 first-run and enrolment stills, the 15 reference screens, the 8 web-interface clips, the 2 SSH clips and the 10 workflow clips — 55 files, every one of them reproducible by re-running the script that made it. What was not re-captured, and cannot be here, is the store material: the screenshots each provider's listing shows are of the application running on that provider's own hardware, so every one that has ever been captured shows the old name, and every Store screenshots row in the submission preflight is recorded outstanding rather than passing. Substituting one of the pictures above would be substituting a mock into a store listing, which docs/epic-checklist.md §10 forbids and which is the one place on this site where "the data is not real" could not be written under the image. No provider store listing has been re-reviewed since the rename either: the listing copy, the icons and the release notes were rewritten in the tree, and whether each store accepts the listing under a new product name is an external decision nothing in the repository can record. The wordmark itself was re-fitted rather than re-lettered — six glyphs became four, so the optical spacing and the aspect ratio were rebuilt and the accent stayed on the trailing d — and the reasoning, including the 16-pixel favicon decision, is in the brand-asset manifest. That manifest also records the two things about the art that are not proven: the lockup is a geometric construction rather than a designer's drawing, and the named human acceptance step — somebody looking at every asset and confirming none of them depicts the old name — has not been performed.