# Release v2.18.0 — server management

Released 2026-10-26 from `refactor` → `master`, tagged `v2.18.0`. About 1,060
commits on top of v2.17.0. Plan: `docs/plans/server-management.md`.

The gateway now sets up and keeps the servers around it — relays, bridges,
Splitting servers, other gateways — over SSH, from its own release cache, and
updates itself from the same cache.

**No release keys are built in yet.** This release does not carry the release
agent's public keys, so it trusts no signature from the release site: an
archive comes into the cache "unsigned" and an admin accepts it by hand, and
`auto` updates accept nothing. The keys come with v2.18.1; until then a
gateway on v2.18.0 is updated by uploading the archive (System → Updates) or
with the installer.

## Servers (System → Servers, admin only, off by default)

- **Adding a server**: managed over SSH with the gateway's own key (Gateway
  key tab: created here, kept in the encrypted storage or outside it,
  rotated with a finish step), the host key pinned on first use and checked
  on every connection; **Monitor only** — no key, its declared roles checked
  from outside; **Manual** — nothing logs in, the gateway writes a script you
  run on the server and upload the sealed result of.
- **Inventory**: system, architecture, access (root or sudo, the password
  asked on the job and never stored), ports, firewall, time sync, services
  installed by hand (offered to **Take over**: their secrets move into the
  gateway's registry, they keep running as found until reinstalled).
- **Owner marker** on every managed server: another gateway, or a newer copy
  of this one restored elsewhere, stops changes until an admin takes over.
- **Jobs**: a queue with per-server locks, steps, a live log (SSE, resumes
  after a reconnect), cancel / retry / abandon, a sudo prompt on the step,
  an alert when one fails. **Artifacts**: every key, token and certificate a
  job makes is filed and shown only after the password is asked again.
- **After a restore** changes on restored servers wait until they are checked
  and Apply is pressed.

### Roles

- **DERP relay** (own relays for the tailnet): installed with a pinned
  certificate from the gateway's CA or Let's Encrypt for its addresses,
  admission by node key pushed from the gateway, region in the map, devices
  at home and the month's traffic with a quota, certificate rotation that
  waits for the devices to hold the new map (and starts by itself 30 days
  before the end, in the server's maintenance window), Disconnect all, a
  shared relay with tenant gateways, a relay fallback while the own one is
  down, static endpoints.
- **Bridge** with PSK rotation in phases (consumers listed, fail-closed
  option), bound to the Bridges record naming its address.
- **Splitting**: reassembler, MASQUE proxy, echo server and obfs4, each on
  its own; the active server per component with an optional failover to a
  second server.
- **Gateway node**: a whole gateway installed on a server (sign-in on, the
  admin's password and its CA filed), paired as a federation peer unless
  asked not to; Logs and Check watch it.
- On every role: Update / Reinstall / Roll back from the release cache (its
  secrets stay), **Options…** — change its options and install it again from
  the release it runs (plan with "was → becomes"), Restart / Stop / Start,
  Logs, Check files (against the hashes its install recorded, drift alert),
  Remove (secrets kept or purged), Integrate / Unlink with the gateway's
  records. A release whose roles are of another compatibility level is never
  offered.
- **Update roles…**: one batch updates every role on another release, one
  job at a time, stopping at the first failure unless asked to go on.
- **Maintenance windows** per server (`HH:MM-HH:MM`, this gateway's time):
  a change — or a whole batch — waits queued until the server's window opens;
  Run now starts it at once.

### Installing

- **Set up a server…**: a guided setup — server and host key, inventory
  (✓ / ⚠ / ✗), roles with the plan of what the install changes on the server
  and on this gateway, the job, its artifacts, the bindings, the checks. The
  step survives a reload; the Bridges, Splitting and Nodes pages open it on
  the role chosen there.
- The install form: Advanced choices (release, certificate term, TLS key on
  the server only, SSH only from, sshd passwords off, journal in memory…),
  a draft kept on the gateway (one per server), Download plan, integrate
  later.

## Releases and the gateway's own update (System → Updates)

- **Release cache**: upload an archive in parts (resumable) or fetch it from
  the release site; SHA256SUMS and its signature checked; the cache keeps
  what roles run.
- **Self-update**: the new binary checks itself in a sandbox on a copy of
  the database first; a watcher written by the running version restarts the
  service and, through the confirmation window, checks the API, DNS, the
  fail-closed tables and the processes — and what worked outside the node
  before (tunnel links, the WAN, federation peers): an outside failure
  extends the window and raises an alert, it never rolls back. A failed
  window rolls back the binary and the database snapshot and carries over
  the security data written meanwhile.
- **Policy**: notify / manual / auto (auto in a maintenance window, only
  releases signed by a built-in key — none in this release), weekly or daily
  checks, latest or a pinned version, skip a version.

## Mesh, routing, DNS

- **Linked meshes**: two gateways that each host a headscale join their
  tailnets through the federation tunnel, with per-direction access policies
  and a "who can reach whom" matrix (`docs/linked-meshes.md`).
- **DERP over TLS** fronted by gatewayd on 443; a fresh install has no relay
  until one is set up.
- **Bridge steering by SNI / Host** on shared CDN addresses.
- **Encrypted storage** (LUKS2 vault, unlocked by a passphrase in the web UI)
  for backups and keys; tunnel keys stay outside so the VPN comes up.
- **Chain Routing retired**: multi-hop routing policies cover it; leftovers
  are cleaned up at boot.
- Fixes across the federation tunnel, mesh exit nodes and routes, the
  fail-closed kill switch after a mesh tun flap, tunnel DNS (Splitting's
  saved resolvers follow their tunnels while Splitting is off), the WAN
  watchdog, backups and the web UI.

## Upgrading from v2.17.0

- Database migrations 061–076 run forward on the first start.
- Server management stays off until an admin turns it on (Servers →
  Settings); it needs sign-in to be on.
- The release archive carries every binary for linux/amd64 and linux/arm64:
  gatewayd, gwctl, the server agent, the DERP admission helper, derper, the
  bridge, the reassembler, the MASQUE proxy and the echo server, with
  `install.sh`, `install-gateway.sh`, `BUILDINFO` and these notes.
- `install.sh` checks every file against the release's SHA256SUMS; it checks
  the signature from the release that builds the keys in.

Verified on the development gateway (pi4) through deploys r1–r135 of this
line: the full stage checks (332 at the last deploy), the leak tests, the
UI-truth and actions harnesses; the server roles on real servers (the relay,
gate-sup, gate-node and a test VM) and on systemd containers in the stand.
