Deployment

Apex domains and CDN-proxied records use the TXT fallback described in SPEC plus A/AAAA records (or ALIAS/flattening) pointing at the server, so publish the server's addresses to owners who need them.

Verification follows the owner's CNAME chain to its end and accepts it only when that end is <name>.<base domain> itself; if the wildcard were a CNAME to another name, every chain would end there and Wardlock could not tell which site a domain points at, so only the TXT record would verify.

WARDLOCK_DNS_RESOLVERS chooses where verification lookups go, in every TLS mode. Use public or authoritative-facing resolvers if the local one caches aggressively or answers split-horizon records. Wardlock's own DNS client keeps no cache; each lookup has 10 seconds.

The server checks domains in the background (a pass every minute): a new domain straight away and then after 1, 2, 4 … 32 minutes and hourly for 48 hours, then only when its owner asks, until it is deleted 7 days after it was added if it is still pending (logged as unverified custom domain expired; the hostname can then be claimed again); active and lapsed domains every 24 hours. An active domain that fails every check for 72 hours lapses and stops being served; one that passes again is restored without the owner doing anything. Status changes are logged as custom domain status changed with the hostname, the old and new status and the failure reason; verification tokens are never logged. Changes reach the edge, the on-demand certificate decision and /.wardlock/tls-ask on the next request, and a domain that lapses or is removed (or whose site is torn down) has its certificate deleted from <WARDLOCK_DATA_DIR>/certs in automatic TLS mode.

An unknown mode, local with public hosts, auto with localhost or IP hosts or without a DNS provider and token, or an auto-only variable (WARDLOCK_ACME_*, WARDLOCK_DNS_*) or WARDLOCK_TRUSTED_PROXIES set in a mode that ignores it.

• *.<base domain> through DNS-01. One wildcard covers every site and the auth origin auth.<base domain> (A2.5).
• The dashboard host: through DNS-01 with the same token when it is the base domain itself or deeper inside its zone (one label under it is already covered by the wildcard), otherwise through HTTP-01 or TLS-ALPN-01, so the token needs no access to the dashboard's zone.
• Custom domains (C1.9) on demand, at the first TLS handshake for the name, through HTTP-01 or TLS-ALPN-01, and only while the domain is verified and active. A name that has never been active (unverified or pending domains, nested names under the base domain, IP addresses) gets no certificate and the handshake fails. When a domain lapses or is removed (or its site is torn down, including by wardlockd admin teardown-site, unless the site was taken down and torn down within 2 seconds or while the server was stopped), Wardlock drops its certificate from memory and deletes it from storage, so the next handshake asks the decision again and fails; the edge answers the host with the same 404 as an unknown host meanwhile.

DNS-01 records are created under _acme-challenge.<base domain> and removed after validation.

ACME validation reaches the server on ports 80 and 443 from the internet, and the redirect and every URL Wardlock prints use the default HTTPS port, so publish ports 80 and 443. The listen addresses may differ when something forwards them, for example in Docker (WARDLOCK_HTTP_ADDR=:8080, WARDLOCK_HTTPS_ADDR=:8443, published as 80:8080 and 443:8443); the systemd unit can bind 80 and 443 directly. Both listeners drop idle keep-alive connections after 120 seconds and slow request headers after 10 seconds; the plain-HTTP listener also limits whole requests and responses to 30 seconds. The HTTPS listener has no whole-request limit, so slow uploads and downloads of large deploy files still complete. Certificates obtain and renew in the background, retrying with backoff; failures are logged with the name and the CA's reason. Until the first wildcard arrives, handshakes for site hosts fail.

Certificates, keys and the ACME account live in <WARDLOCK_DATA_DIR>/certs (certmagic file storage). They are re-issued if lost, but keep them across restarts to stay within the CA's rate limits.

Give the token the narrowest scope the provider allows and nothing else:

• Cloudflare: an API token (not the global API key) with the permissions Zone → DNS → Edit and Zone → Zone → Read, with Zone Resources set to the sites zone only. Optionally restrict Client IP Address Filtering to the server.
• deSEC: a token without permission to manage tokens. Restrict it with token policies: a default policy that denies writes, and one policy allowing writes to TXT records at subname _acme-challenge in the base domain (and at _acme-challenge.<dashboard label> if the dashboard host is deeper in that zone). Set allowed subnets to the server's address.

internal/tls tests run the whole flow against Pebble, Let's Encrypt's test CA, in process with a fake DNS server and DNS provider, as part of pnpm check; no Docker, network or credentials are involved. To try a real server without touching Let's Encrypt, run Pebble with pebble-challtestsrv as its DNS server, point WARDLOCK_ACME_CA at Pebble's directory and WARDLOCK_ACME_CA_ROOT at its test/certs/pebble.minica.pem, and resolve your test names to the server through the challenge test server. A real DNS provider is needed for DNS-01 outside the tests, so use the Let's Encrypt staging directory with a throwaway zone instead when you want to check provider credentials.

Each GitHub release carries wardlockd_<version>_<os>_<arch> and wardlock_<version>_<os>_<arch> archives for linux, darwin and windows on amd64 and arm64 (.zip on Windows), checksums.txt (SHA-256 of every archive and SBOM) and an SPDX SBOM per archive (<archive>.sbom.json). The wardlockd archive also contains wardlockd.service.

--base-url installs from a mirror or a local snapshot directory instead, for example --base-url file://$PWD/.local/goreleaser --version 0.0.1-snapshot.abc1234.

The publish job of release.yml makes two build provenance attestations with actions/attest-build-provenance: one for every file listed in checksums.txt (each archive and SBOM), and one for the image's multi-arch index digest, which is also pushed to ghcr.io beside the image. checksums.txt itself is not attested; verify the archive it describes. install.sh checks the archive against checksums.txt only, which catches a corrupted download but not a replaced release, so verify the archive as well when that matters.

To require the release workflow and the tag, not just this repository:

gh attestation verify wardlockd_0.1.0_linux_amd64.tar.gz --repo CubieCode/wardlock \
  --signer-workflow CubieCode/wardlock/.github/workflows/release.yml --source-ref refs/tags/v0.1.0

Attestations are made only when a vX.Y.Z tag publishes; snapshots and pull requests have none.

deploy/wardlockd.service runs as the wardlock user with StateDirectory=wardlock (so WARDLOCK_DATA_DIR=/var/lib/wardlock), AmbientCapabilities=CAP_NET_BIND_SERVICE for ports 80/443, and systemd hardening (ProtectSystem=strict, NoNewPrivileges, private /tmp, a @system-service system call filter, UMask=0077).

The setup URL (setup_url) is valid 60 minutes and single use, with a new one on every start until an owner exists. wardlockd admin create creates the owner and prints a one-time sign-in URL valid for 60 minutes. Owners add passkeys on the dashboard's Passkeys page; invitation links are valid for 7 days. Setup, sign-in and invitation URLs use https://<dashboard host> with no port, since production dashboards are reached over HTTPS on the default port; only a localhost or *.localhost dashboard host gets http:// and the listen port. Passkeys belong to the dashboard host: changing WARDLOCK_DASHBOARD_HOST (or reaching the dashboard at another address) makes existing passkeys unusable, so owners sign in by email code and add them again.

The per-address limit on sign-in emails (5 per hour) counts every request for that address, whoever sends it. Anyone who knows an owner's address can spend those sends, which delays that owner's sign-in by up to an hour; it cannot sign them in. Passkeys (C1.6) avoid this.

Release images will be multi-arch (linux/amd64, linux/arm64) at ghcr.io/cubiecode/wardlock:<version> and :latest. The example compose file has a read-only root filesystem, all capabilities dropped and one volume for data.

The sites wildcard is certified through DNS-01 with Cloudflare, the dashboard and owners' custom domains on demand. There is no reverse proxy, no Cloudflare Tunnel and no Cloudflare proxying.

The worked example is an OVHcloud VPS-1 (2 vCores, 4 GB RAM, NVMe) in the Singapore or Sydney region, but nothing in the files is specific to a provider: any VPS with a public IPv4 address, and preferably an IPv6 address, works. Below, example-sites.net is the sites domain (a zone on Cloudflare), wardlock.example.com the dashboard, and <account id> your Cloudflare account ID.

Docker 27 and later gives the Compose network its own private IPv6 subnet and forwards IPv6 connections with their source address, which per-IP limits and the access log need. Docker's published ports bypass ufw, which is why compose.yaml publishes the operator listener on the host's loopback only.

On OVHcloud:

• The VPS has a public IPv4 and an IPv6 address, both shown in the control panel. Some images do not configure the IPv6 address; if ip -6 addr shows no global address, add the address and gateway from the control panel to the network configuration (netplan or cloud-init), and check from another machine with ping -6 <server IPv6> before publishing AAAA records.
• The optional Network Firewall in the control panel filters traffic before it reaches the VPS. If you enable it, allow TCP 22, 80 and 443 there too, or certificate validation and visitors never arrive.
• Reverse DNS is optional: Wardlock sends mail through your SMTP provider, not from the VPS.

Proxied records would make Cloudflare terminate TLS instead of Wardlock, hide visitors' addresses, and break certificate validation for the dashboard and custom domains.

Keep the wildcard as A/AAAA records, not a CNAME (Custom domains). If either zone has CAA records, allow letsencrypt.org. See Domains and DNS for choosing the domains.

/readyz reports certificates failing until the wildcard and dashboard certificates arrive, usually within a minute or two; the log names the CA's reason when one fails. sudo docker compose up -d after the change recreates the container; staging and production certificates are stored apart under certs/certificates/, one folder per CA. The setup_url (https://wardlock.example.com/…) is valid 60 minutes, with a new one on every start until an owner exists, as Single binary with systemd describes. sudo docker compose exec wardlockd wardlockd admin create --email you@example.com works too.

Every certificate counts against Let's Encrypt's rate limits (per registered domain, per account and for failed validations), which matter mostly when many owners add domains at once or one keeps failing validation; renewals do not count against new-certificate limits. If you reach a limit, or Let's Encrypt is unavailable for a long time, WARDLOCK_ACME_CA can point Wardlock at another ACME CA's directory. It must issue without external account binding, which Wardlock does not support.

The data volume holds wardlock.db, certs/ (certificates, keys and the ACME account) and blob-cache/ (local copies of deploy files, see Blob cache); deploy files are in R2.

1. wardlockd backup writes a consistent copy of the database into the volume, while the server runs (Backups).
2. restic uploads the whole volume to the wardlock-backups bucket: the copy and certs/. The live wardlock.db, wardlock.db-wal and wardlock.db-shm are left out, since they cannot be copied safely while the server runs and the copy replaces them, and so is blob-cache/, which refills from R2.
3. The copy is deleted, and snapshots older than 14 daily, 8 weekly and 12 monthly are pruned.

Without systemd timers, a cron line does the same: 30 3 * * * root /opt/wardlock/deploy/vps/backup.sh in /etc/cron.d/wardlock-backup.

Keep the secrets out of the backup. WARDLOCK_SECRET lives in wardlock.env, outside the volume, so it is never in the backup. Store it, together with the restic password and both env files' credentials, somewhere that survives losing the VPS and is not the backup bucket: a password manager or an offline copy that only operators can reach. Without WARDLOCK_SECRET a restore loses every share link, API token and session; without the restic password the backups cannot be read at all.

Deploy files are not in this backup. Protect the wardlock-blobs bucket as Backups describes for blobs in a bucket, for example with an rclone sync to another bucket after each database backup.

WARDLOCK_TRUSTED_PROXIES takes the proxy's addresses, comma-separated IP addresses or CIDR ranges. A range covering every address (0.0.0.0/0, ::/0) is refused, and IPv4-mapped forms (::ffff:10.0.0.1) are read as IPv4. Wardlock believes forwarding headers only from those peers: it reads X-Forwarded-For from the right, skipping trusted hops, so the first untrusted address is the client and anything a client prepends is ignored; it keeps X-Forwarded-Proto only when it is http or https; and it removes X-Forwarded-*, X-Real-IP and Forwarded from every other peer. X-Forwarded-Host is never used: routing is by Host.

Without WARDLOCK_TRUSTED_PROXIES (Wardlock logs a warning at startup), Wardlock sees every client behind the proxy as the proxy's address, so every per-IP limit (owner and visitor sign-in, share-link guesses, API-token failures and CLI login) becomes instance-wide: one busy or hostile client can make sign-in wait for everyone until the window resets. Never list an address that untrusted clients can connect from, or they can choose their own address.

A deploy made with --immutable-assets sends Cache-Control: public, max-age=31536000, immutable for its hashed files under /assets/ on public sites (SPEC "Serving"), so a caching proxy or CDN in front may keep them for a year; everything else revalidates. Protected sites send private and must still not be cached at the proxy.

Raising the per-file limit means raising the proxy's request body limit to match.

With Caddy, use on-demand TLS with ask set to http://127.0.0.1:8080/.wardlock/tls-ask, so it only requests certificates for hostnames Wardlock has verified, and a catch-all https:// site with tls { on_demand }, as the example does. /.wardlock/tls-ask?domain=<hostname> answers 200 for an active custom domain and 404 for anything else, and only to a peer in WARDLOCK_TRUSTED_PROXIES (any Host) whose request carries no X-Forwarded-For, X-Forwarded-Proto, X-Forwarded-Host, Forwarded or X-Real-IP header: Caddy's own ask request has none, while every visitor request it forwards does. Every other request, and every TLS mode but proxy, gets 404. As a second guard, make the public site blocks answer /.wardlock/tls-ask themselves with a 404 before proxying, as the examples do (Caddy: @tlsask path /.wardlock/tls-ask* with respond @tlsask 404; nginx: location ^~ /.wardlock/tls-ask { return 404; }). Caddy keeps a certificate it already holds for a domain that later lapses or is removed until renewal, but Wardlock answers that host with the unknown-host 404. nginx cannot issue certificates on demand; use Caddy, or manage custom-domain certificates yourself.

• smtp://: a server that does not offer STARTTLS is refused; there is no option to send in plain text. The port defaults to 587.
• smtps://: the port defaults to 465.
• dev:// writes each message to .local/mail/ (relative to the working directory) as a readable .eml file and logs only the recipient, subject and file name, never the body or code. Outside local mode Wardlock logs a warning at startup (WARDLOCK_SMTP_URL is dev:// outside local mode).

The server certificate is always verified against the system roots. Percent-encode special characters in the user name and password (@ is %40). Credentials never appear in logs or errors, which show the server as smtp://[redacted]@host:port. WARDLOCK_SMTP_FROM defaults to Wardlock <wardlock@<dashboard host>>.

Providers never redirect to sites themselves, so one registration covers every site. The auth origin is covered by the sites' wildcard DNS record and certificate; WARDLOCK_DASHBOARD_HOST cannot be the auth origin, and auth cannot be a site name.

Wardlock asks GitHub for user:email, plus read:org only when the policy lists organisations; for an organisation with OAuth app access restrictions, an organisation owner must approve the app before memberships count. OIDC providers (Google included) are asked for openid email; an account counts only when the ID token (or the userinfo endpoint) says email_verified is true, so a provider that never asserts verification cannot be used. Discovery runs on the first sign-in, not at startup, so a provider outage does not stop the server.

Sign-ins in progress (up to 10 minutes at the provider) and the 60-second codes that hand them to sites are kept in memory: a restart drops the sign-ins in flight, and visitors simply sign in again. Behind a reverse proxy, keep code and state out of access logs as Wardlock does in its own.

The blob directory is laid out as sha256/ab/cdef…. Each upload is written to a temporary file in the same directory, fsynced and renamed into place, so put the blob directory on the same local filesystem as its parent.

Values are whole numbers of at least 1, without units (WARDLOCK_MAX_VIDEO_SIZE may also be 0). A deploy over a limit is rejected when it is created (413 limit_exceeded, naming the file where one is at fault), and an upload body longer than the per-file limit is rejected too. Raising the per-file limit means raising a reverse proxy's request body limit to match.

Video files are recognised by extension only (.mp4, .webm, .mov, .m4v, .mkv, .avi and the rest of the list in SPEC "Video files and blocked file types"), in any letter case, the same on every host; .ts is not on the list. Blocked extensions are written with or without the dot, in any letter case (exe, .MSI); a deploy listing such a file is rejected with 422 file_type_blocked naming it, and only a file's last extension counts. The suggested list leaves out .sh, .zip and other formats that sites often offer for download on purpose; add them if your instance should not host them. Both settings apply to new deploys only: deploys already made keep serving.

• Storage counts each file the owner's deploys list once, whatever its path, deploy or site, and counts a deploy as soon as it is created, before its upload. Deleting a site, or garbage collection pruning old deploys beyond WARDLOCK_DEPLOY_HISTORY, gives the space back; a lower WARDLOCK_DEPLOY_HISTORY keeps usage lower. The figure is the owner's share of what the instance stores, not the blob directory's size: content two owners publish is stored once but counts for both.
• Deploys in 24 hours counts deploys created, refused ones aside, in a rolling 24-hour window; deleting sites does not reset it.
• Refusals are 413 limit_exceeded when a deploy or site is created, with a message naming the quota and its limit, which the CLI shows as it is. Nothing is written, so a refused deploy does not count.
• Overrides. --storage takes bytes or a whole number with KiB, MiB, GiB or TiB; --sites and --deploys-per-day take a whole number. 0 is unlimited for that owner and default makes them follow the instance again. Flags you leave out keep their current value. The command prints the owner's usage and limits afterwards, and logs operator set owner quotas with the owner ID only. A lower limit deletes nothing: it refuses the owner's next deploy that adds files, or next site, until usage is under it again.
• Upgrading to A2.14 counts every existing deploy at once (migration 0009_owner_quotas), so the first start after the upgrade takes a little longer on a large instance.

The settings are checked at start, and a missing or malformed one stops the server with an error naming it.

Without WARDLOCK_S3_PATH_STYLE the client uses virtual-hosted addressing for AWS endpoints and path style for everything else. Use plain http:// only for a service on the same host or private network.

Credentials come from the standard AWS sources, in this order: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (with AWS_SESSION_TOKEN for temporary keys) in the environment; the shared credentials and config files (AWS_SHARED_CREDENTIALS_FILE, AWS_CONFIG_FILE, AWS_PROFILE; by default ~/.aws/credentials of the service user); then a container, web-identity or EC2 instance role. Put keys in the service's environment file (mode 0600) like WARDLOCK_SECRET. They are never logged: the startup line wardlockd started names the endpoint, bucket, region, prefix and which credential source is used, and nothing else.

Give the credentials only what Wardlock needs, scoped to the prefix where the service allows it: s3:ListBucket on the bucket (readiness and garbage collection), and s3:GetObject, s3:PutObject and s3:DeleteObject on <bucket>/<prefix>*. For AWS:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::wardlock-blobs"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::wardlock-blobs/wardlock/production/*"
    }
  ]
}

Keep the bucket private (no public access, no website hosting): every file is served through Wardlock's access control, and anyone who can read the bucket can read every protected site. Wardlock never creates presigned URLs.

Blobs are stored at <prefix>sha256/ab/cdef…, the same layout as on disk. Every upload is signed with the file's expected SHA-256 as its payload hash, and Wardlock hashes the bytes as they stream, failing the request before its last bytes are sent if they do not match, so a blob is either complete or absent.

How an upload reaches its key depends on whether the service itself rejects a body that does not match its signed payload hash. Once per server process, before the first upload, Wardlock asks: it uploads a small body to a temporary key <prefix>uploads/<random> signed with a deliberately wrong hash.

• Rejected with a checksum error (AWS S3 and MinIO do this): each later upload is one PUT straight to its final key.
• Accepted, or refused for another reason: the probe object is deleted, and each upload streams to a temporary key, is verified, is copied to its final key, and the temporary key is deleted. Unverified content never reaches a final key.
• No answer (rejected credentials, a server error or a network failure): that upload fails and the next one probes again.

The choice is logged once as s3 upload path with path (direct or temporary key) and the reason; it never depends on the provider's name and never logs credentials. Abandoned temporary keys older than WARDLOCK_GC_GRACE_HOURS are removed by garbage collection, as are blob objects with no database record. Each file is at most 5 GiB, the limit of one upload or copy, so WARDLOCK_MAX_FILE_SIZE may not exceed 5368709120.

Serving takes each file's size from the deploy's manifest, so opening a file makes no request. Wardlock reads with ranged GETs from the position the response needs, so a Range request never downloads the whole object. A file missing from the bucket shows up on that first read and answers 500, logged as open blob, before any of the response is sent. A HEAD request reads nothing, so it answers 200 even for a missing file.

• Deploys: one HEAD per manifest file the owner has uploaded before, when the deploy is created and again when it is activated. Each uploaded file is one PUT on a service that checks payload hashes, otherwise one PUT, one COPY and one DELETE. Either way, an upload makes the same requests when the content is already stored, so upload timing does not reveal what other owners have stored. Activation also reads each HTML file once (no request if the blob cache holds it) and stores its Markdown and text renditions the same way.
• Upload probe: one PUT per server process before its first upload, plus one DELETE when the service accepts the probe.
• Serving: one GET for each response that sends a body: a full file, a Range, or a 404.html. With the blob cache, a full response costs one GET the first time and none after; a Range the cache does not hold costs one ranged GET. A 304 Not Modified, a 412, a 416 or a HEAD request costs none, with or without the cache. Every byte served from the bucket is egress, so run Wardlock close to it, or pick a service without egress fees.
• Garbage collection: one LIST per 1,000 stored objects each run, plus one DELETE per removed blob. With S3, run it daily (WARDLOCK_GC_INTERVAL_MINUTES=1440) rather than at the hourly default: unreferenced blobs wait out a 24-hour grace period anyway, so hourly runs mostly repeat the listing.
• Readiness: one HEAD of the bucket per /readyz or /metrics scrape.

Each file the cache does not hold waits for at least one round trip to the bucket, so expect slower first bytes than from a local disk.

• Content-addressed. Copies are stored by SHA-256 (sha256/ab/cdef… under the directory), so a copy can never be stale: a new deploy's changed file has a new hash.
• Filled while serving. A response that reads a whole file from the start (a full GET, or Range: bytes=0-, which browsers send for media) streams it to a temporary file under fill/ as it is sent. The copy is kept only when its bytes hash to the file's SHA-256 at its size. A Range from anywhere else, or one that stops early, fills nothing, so it fetches only the bytes it asked for. Reading the rest of the object just to fill the cache would pay for bytes nobody requested.
• One fill at a time. Concurrent first requests for the same file all read from the bucket; one of them fills the cache.
• Least recently used out. Beyond the size limit, the copies used longest ago are deleted. A file bigger than the limit is never cached. Use is tracked in memory and stamped on each copy's modification time, so a restart rebuilds the order from the directory and removes unfinished fills.
• Safe to wipe. Deleting the directory, or any copy in it, while the server runs only means those files are fetched and cached again. A copy whose size is wrong is treated as missing.
• Garbage collection removes a blob's cached copy along with the blob, including one still filling. wardlockd gc removes copies from the same directory when it shares the server's configuration.

The cache works with the filesystem backend too, but there it only duplicates the blob directory, so it is meant for S3. Size it to hold the files visitors fetch most; one page view usually needs its HTML, stylesheets, scripts and images.

The active deploy is never pruned. Deploys not yet activated and bodies uploaded within WARDLOCK_GC_GRACE_HOURS are kept, so a slow upload is never cut short; keep the grace period longer than your longest deploy. wardlockd gc is safe to run while the server is running, for example from cron with WARDLOCK_GC_INTERVAL_MINUTES=0. With S3 blob storage, a daily run (WARDLOCK_GC_INTERVAL_MINUTES=1440) is enough and lists the bucket 24 times less often.

Every run logs garbage collection finished with deploys_pruned, blobs_deleted, files_deleted (stored files with no database record, left by a crash mid-upload, and abandoned temporary uploads) and bytes_freed, or garbage collection failed with the error; a failed run is retried at the next interval.

The defaults keep enough of an address to tell networks apart, not individual households; set both prefixes to their maximum to keep full addresses, or to 0 to store none. IPv4-mapped IPv6 addresses count as IPv4. The IP is the client's address after trusted-proxy resolution: behind a reverse proxy listed in WARDLOCK_TRUSTED_PROXIES it is the client from X-Forwarded-For; without that setting every entry carries the proxy's address. Paths are stored without their query string, and any token in the path or User-Agent keeps only its public ID, so the access log never holds a share link.

Entries are written in batches off the request path: serving never waits on the database. If the database falls behind and the buffer fills, new entries are dropped rather than slowing visitors; the server logs a warning (access log entries dropped, at most every 10 seconds) with the running total, and a failed batch write is logged and counted the same way. On shutdown the server writes what is buffered within the shutdown timeout. Retention runs at start and hourly beside the writer, deleting entries older than the retention period (including those of deleted sites) 1,000 at a time so the database is never locked for long. It never prunes the entries of a taken-down site or a suspended owner's sites, which are evidence (Operator controls); they are pruned by age again once the site is restored or the owner reinstated, or once the site is torn down.

• Suspending an owner revokes every dashboard session for good and denies any approved CLI login not yet collected, in one transaction. While suspended, their API tokens are refused, and sign-in by email code, sign-in link or passkey fails with the same neutral error as a wrong code or passkey (no code is sent, as for an unknown address), so the suspension is not revealed to anyone guessing addresses. Invitations to the address send nothing. Every site they own answers the unavailable page. Reinstating makes their tokens and sites work again; they sign in again, since the old sessions stay revoked.
• Taking a site down makes every path on every host of the site (base-domain and custom domains) answer 403 with a neutral "This site is unavailable" page, whatever the policy, cookie, share link or Bearer token. It says the operator of this server took the site offline, with no reason and no owner, and is not cached or indexed. Only /.wardlock/health still answers; a taken-down site with nothing published answers like a missing site (404). The page also sends Clear-Site-Data: "cache", "storage", so a service worker or cached pages the site left in a visitor's browser cannot keep answering in its place. The owner's dashboard shows the site as "Taken down"; they can still read it, create and update policies, and create, rotate and revoke share links, so they can change who has access. They cannot deploy (or upload to a pending deploy), activate, roll back or delete it, add or remove custom domains, or delete a policy (409 site_unavailable): domains stay as they were reported, and deleting a policy would delete the sessions that record who had access. The command's output says what is kept and what the owner can no longer do.
• Nothing is deleted. Garbage collection skips every deploy of a taken-down site or a suspended owner's site, and access-log retention skips their entries, so restoring serves the same deploy, and evidence (files, custom domains, share-link and session records and the access log) stays in place. A sign-in or CLI-login approval is stored only if the owner is not suspended, checked in the same SQL statement, so nothing created while a suspension races it survives reinstatement.
• Tearing down works only on a taken-down site, so a typo cannot delete a serving one, and deletes only while the site is still taken down (a restore-site racing it wins; the command then says the site is not taken down). It deletes the site's domains, deploys, policies and share links as an owner's teardown does, and frees the name; garbage collection later removes files nothing references, and its access-log entries are pruned by age again. In automatic TLS mode the running server's watch, which remembers each unavailable site's custom domains, removes their certificates once the site is gone. Only a site taken down and torn down within the watch's 2-second interval, or while the server was stopped, keeps its certificates until they expire, unrenewed.
• How a running server notices. Sessions and tokens are checked against the database on every request; a running server also polls for sites that became unavailable or available again every 2 seconds and drops its cached copy of them.
• Logs name IDs only: operator suspended owner with owner_id, operator took down site with site_id, and so on.
• Invites. With WARDLOCK_INVITES=operator the invite API answers 403 invites_restricted. The dashboard and CLI have no invite action today, so nothing changes there.

• The link never appears on the site's own pages, only on Wardlock's. The form is on the dashboard host, not the site's origin, because the reported site's own scripts and service workers run there and could script, fake or flood a form. It is rendered by the server (not the dashboard app) with the login pages' headers.
• The form exists only for a host that serves a published site, taken down or not; any other host, a site with nothing published, and the dashboard and auth hosts answer like an unknown host (404), so it reveals nothing beyond visiting the host.
• The form names the site and asks for the page (prefilled with host and path, editable, at most 2,000 characters), a reason (Phishing or a scam; Malware or a harmful download; Illegal content; Sexual content involving a child; Copyright or trademark infringement; Something else), optional details (at most 2,000 characters) and an optional contact address (at most 254 characters). It accepts only same-origin posts, runs no scripts and is never cached. "Back to the site" links to the site's origin. The thank-you page says the report went to the operator.
• Only valid reports count against the limits (an invalid one is refused first): 5 per IP, 20 per network (IPv4 /24, IPv6 /48) and 200 across the instance, per hour; over a limit it answers 429 with Retry-After and "Too many reports have been sent. Try again in …". There is no per-site limit, which a reported site could spend to silence its reporters.
• Before sending, any t= query parameter value in the page or details becomes [removed] and anything shaped like a Wardlock token (wl_<id>_<secret>) becomes [token removed], so a reporter cannot forward a working share link by mistake.
• The plain-text email starts with your lines: the site, site ID, time received, the takedown-site command and a pointer to this section. Below a rule come the reporter's reason, page, contact address and details, introduced as their own words (which may be wrong and contain no instruction from Wardlock) and every line quoted with > , so a reporter cannot forge instruction lines. The reporter's IP is never sent or logged; the server logs abuse report sent with site_id and reason only.

• Triage. Open the reported page in a private browser window. A protected site shows only its login page (reports have share-link tokens removed); then rely on the report and, if needed, ask the reporter for more detail through the contact address they left, never for copies of suspected child sexual abuse material. Do not open such material beyond what you must to confirm what it is. Check that the site still exists (wardlockd admin list-sites) and who owns it. Reports can be mistaken or malicious: a rival may report a site to have it removed, so decide on what the site shows, not on what the report says. When in doubt about material that may be illegal, take the site down while you decide; restore-site undoes it.
• Speed. Act on child sexual abuse material, phishing and malware the same day you see the report: they harm visitors every hour they stay up. Other reports can wait for a reply from the owner.
• Contacting the owner. Email the owner's address from list-owners. Say which site and page, which rule of the policy it breaks, what you have done (taken down, suspended) and what they can do (remove the content and ask for a restore, or reply if they think the report is wrong). Never forward the reporter's contact address or words without their consent; summarise instead. For material you have reported to the authorities, ask them before contacting the owner: warning a suspect can destroy evidence. A suspended owner cannot sign in, so email is the only channel.
• Restoring. If the owner fixes a site, they cannot deploy while it is taken down: restore-site it, ask them to deploy the fix straight away, and check it. Reinstate a suspended owner with reinstate-owner only once you are satisfied.
• Records. Keep a private log of each report: when it arrived, the site name and ID, the owner ID, the reason, what you saw, what you did and when (with the commands' output), every message to or from the owner and the authorities, and the outcome. Log IDs and summaries, not the reported material or copies of it. The server logs each operator action with IDs only (operator took down site, site_id), which you can match to your notes. Keep the records at least as long as the evidence, and store them with your backups, not on the server alone.
• Repeat offenders. An owner whose sites are taken down more than once for clear breaches should be suspended; with WARDLOCK_INVITES=operator, nobody else can invite them back.

Garbage collection already skips every deploy of a taken-down site or a suspended owner's site, so its files stay in the blob directory or bucket however long the investigation takes, and access-log retention skips its entries until the site is restored, the owner reinstated or the site torn down. Custom domains are frozen and policies cannot be deleted, so the hosts in a report and the session records of who had access stay too. With the S3 backend, check that no bucket lifecycle rule expires or deletes objects (current or noncurrent versions) during that time, and keep the database backup from step 3 with the bucket: the backup records which files the site's deploys list. Restoring a site with restore-site serves the same deploy again if the report turns out to be unfounded.

• What is sent is decided from each file's bytes, so renaming a file changes nothing. Images: a PNG, JPEG or WEBP of at most 20 MB as it is; a GIF as PNG images of up to 10 frames, half evenly spaced across the whole animation and half picked at random each time (every frame is decoded; more than 2,000 frames, or a minute of decoding, is unscreenable); a PNG, JPEG or GIF frame over 20 MB scaled down to at most 2,048 pixels a side; an icon's largest PNG. Text: when a path serving the bytes ends .html, .htm, .shtml, .xhtml, .xht, .xhtm, .svg, .txt, .text, .md, .markdown, .csv, .json or .xml (the types the edge serves as displayed text, pinned in Wardlock rather than read from /etc/mime.types), or the bytes are HTML, XML or SVG, the extracted plain text (SVG text included), in pieces of at most 10,000 characters, at most 10 (100,000 characters); text beyond that is not screened. Also up to 16 images or SVGs (64 MiB in all) embedded as data: URIs of any media type, base64, percent-encoded or plain, in those files and in any other text file (a payload is screened when its bytes are an image or markup; empty, unclosed and script-assembled ones are skipped, so minified bundles are not flagged for them): stylesheets and scripts are searched for embedded images, though their source is not sent as text. A file with nothing of these (scripts and stylesheets without embedded images, fonts, videos, archives) is skipped, and a page with no text is allowed, without anything being sent. A file first screened without its text is screened again when a path serving it as text appears.
• Unscreenable images. AVIF, HEIF, BMP, TIFF and JPEG XL images, bitmap-only icons, a WEBP over 20 MB, any image over 64 MiB or with a canvas over 40 million pixels, an image the endpoint refuses as input, and a page embedding more than 16 images are served but recorded as flagged with reason unscreenable, listed by admin moderation list and included in the flagged digest. Look at the file and release it, or take the site down.
• Precompressed siblings. While screening applies to a site (screening on, owner not exempt), a browser is offered a page.html.gz only once screening has decompressed it and found exactly page.html's bytes, and never a .br (Go's standard library has no brotli decoder to check one, and Wardlock adds no dependency for it), so the file itself or a checked .gz is sent instead. A sibling that holds other bytes is never offered. Exempt owners' sites negotiate as before. The bandwidth cost: browsers that would have taken brotli get gzip or the uncompressed file.
• Caching while unscreened. Until a file has its verdict it is sent with Cache-Control: public, no-cache (private, no-cache on a protected site), never the year of long-term caching a deploy may opt into, so shared caches revalidate it and stop serving it once it is withheld. Browsers or caches may still hold a copy fetched before the verdict. A change to an owner's exemption reaches a running server's caching and sibling choices when the site is next deployed or its files are next screened, or on restart.
• Once per content hash, ever. Files are queued by SHA-256, and no other path, deploy, site or owner listing the same bytes sends them again. Only active deploys on serving sites are screened: not pending deploys, taken-down sites or suspended owners' sites (they are screened once they serve again). An exempt owner's sites are not queued; a file both an exempt and a non-exempt owner serve is screened as the non-exempt owner's.
• When. An activation, rollback or other change to what a site serves queues that site's files at once; every pass (on each such change, and every 30 seconds) also sweeps the next 100 sites for files not yet queued, so a full sweep of a large instance takes a while but never rescans every deploy at once. Two files are screened at a time, and the image decoding screening does itself (GIF frames, scaling, icons) runs one image at a time. The queue is in the database (moderation_results), so a restart resumes where it stopped. wardlock_moderation_backlog on /metrics counts files waiting, retries included.
• Failures. A server error or network failure is retried after 30 seconds, doubling each time up to 6 hours, with jitter, and never sooner than a Retry-After the endpoint sends (which also pauses every other file, for at most 6 hours). A failure every file would hit (a refused key, a rate limit, or a 400 that is not about the input, such as an unknown model) pauses screening with the same back-off, logs screening provider refused the request; screening paused (or ... refused the API key ...) at error, sets wardlock_moderation_paused to 1 and counts in wardlock_moderation_provider_errors_total; the file is not marked as failing. Only a 400 naming the input or a 413 makes that input unscreenable. Retries continue until they succeed, so a wrong key shows as a paused worker and a growing backlog. Serving and deploys never wait on any of this.
• Outcomes. A file is quarantined if any flagged category is in WARDLOCK_MODERATION_QUARANTINE, else flagged if any is in WARDLOCK_MODERATION_FLAG (or any at all when that is unset), else allowed. Wardlock uses the endpoint's own flagged decision per category; it does not apply score thresholds. Changing the categories applies to files screened afterwards; rescan applies them to one already screened.
• What a withheld file answers. Every request that would serve it, on every host of every site listing the same bytes, answers 451: HTML pages show a neutral "This page is unavailable" page linking to the site's home page, other files a line of text. That covers clean URLs, 404.html and the SPA fallback, ?format=md|txt, precompressed siblings, HEAD, Range and conditional requests, and share links; the policy is checked first, so a visitor without access still sees the login page. The answer is no-store, so release takes effect on the next request. The server loads the list before it listens and does not start if it cannot, so a restart never serves a withheld file; it reloads the list every 5 seconds, so release, confirm and discard from wardlockd admin reach it within that.
• Evidence. Garbage collection keeps every quarantined file's stored copy until you discard or release it, even if its owner deletes the deploy or the site, so evidence survives until you decide. list shows such a file as quarantined (held) once no site serves it.
• Rescans. A rescan, or a later text check, that the endpoint cannot judge (nothing to send, rejected, unscreenable) keeps the verdict and your confirm or discard, and a rescan never lowers a quarantine while any part of the file could not be judged (a page withheld for an embedded image stays withheld if the image is refused on the rescan); otherwise a new verdict replaces the old one and keeps your decision only while the status stays the same.
• Notices and logs. Notices go out once per change of status: a new quarantine emails you at once, newly flagged files go in one digest per pass, and a rescan that keeps the status emails nobody. Owners are emailed when a file of theirs is newly withheld, never for a taken-down site or while suspended. The log records screened file with sha256, kind, status, any categories and what was unscreenable, and operator decisions with the hash or owner ID; never file content, page text, the endpoint's error messages or the API key.
• Memory. Screening holds at most a 64 MiB read of each of the two files in progress, the images it sends (each at most 20 MB, plus its base64 request), up to 64 MiB of a file's embedded images, and one decode at a time: a canvas of at most 40 million pixels (160 MB, or 320 MB for a 16-bit PNG) plus a scaled copy. The worst case is about 800 MB above the server's usual footprint, so a 4 GB VPS has room; typical pages and images take a few tens of MB.
• Turning it off stops sending; verdicts and the withheld list stay, so withheld files stay withheld until you release them.

It must differ from WARDLOCK_HTTP_ADDR and WARDLOCK_HTTPS_ADDR. It answers on any Host and serves nothing else (404); site, dashboard and unknown hosts on the main listeners never reach it, so /metrics and /readyz there are ordinary site paths or the dashboard. Its requests are neither logged nor counted in the request metrics.

GET /readyz runs the checks concurrently and answers 200 with {"status":"ready","checks":{"database":"ok","blobs":"ok"}} when all pass, or 503 with "status":"unready" and the failing check's reason in place of ok. The checks: database reads from the database; blobs creates and removes a probe file (.ready-*) in WARDLOCK_BLOB_DIR, so a missing, read-only or full blob directory fails it, or with the S3 backend sends HEAD to the bucket, so an unreachable service, a missing bucket or rejected credentials fail it; certificates (automatic TLS mode only) needs an unexpired certificate for every managed name (the sites wildcard and the dashboard host), so it fails until the first certificates arrive and if renewal falls behind until expiry. A check that has not finished within 5 seconds reports timeout, so /readyz and /metrics answer promptly even when, say, the blob directory sits on a hung network mount; it keeps running in the background, and later probes wait on that same run rather than starting another. Once graceful shutdown begins (SIGTERM or SIGINT), /readyz answers 503 with "status":"shutting_down" without running the checks; the operator listener is the last to stop, so a load balancer sees the drain while in-flight requests finish (up to 10 seconds). A check that starts failing is logged once as readiness check failing with check and error, and readiness check recovered when it passes again.

GET /metrics is the Prometheus text format (0.0.4). Labels never name hosts, paths, sites or owners:

A Prometheus scrape job needs nothing more than the address: static_configs: [{targets: ['127.0.0.1:9740']}]. Alert on wardlock_ready == 0, a rising wardlock_access_log_dropped_total, and wardlock_certificate_expiry_timestamp_seconds - time() < 14 * 86400 (renewal starts well before that). For systemd add WARDLOCK_OPERATOR_ADDR=127.0.0.1:9740 to the env file; the Docker example sets :9740 inside the container and publishes it on the host's loopback only. Inside Compose, :9740 listens on the container's network interface, so every other container on the same Docker network can reach wardlockd:9740 (that is how a Prometheus container would scrape it). Keep untrusted containers off that network, or give Wardlock and its monitoring a network of their own.

Each request on the main listeners logs request with method, host, path, status and duration. The path keeps its query string with credential parameters (t, token, code, state, password) replaced by [redacted], and any Wardlock token anywhere in the path or query reduced to its public ID (wl_<public id>_[redacted]). Events about resources carry IDs (site_id, owner_id, policy_id, public_id, token_id, session_id), never tokens, cookies, passwords, codes or Authorization headers; secrets in configuration print as [redacted], SMTP URLs without their credentials. The exceptions are deliberate: the one-time setup_url while no owner exists, and with WARDLOCK_SMTP_URL=dev:// the recipient and subject of each message (never the body or code). No other line names an email address; invitations, for example, log the new owner's ID and the inviter's. Certificate events from certmagic (automatic TLS) keep only reviewed fields, such as names, ACME URLs, states, times and a shortened, scrubbed error; any other field is dropped and named in dropped, so even at debug they carry no keys, challenge values, headers or the ACME contact address (SECURITY-MODEL, Third-party log output).

Never copy wardlock.db (or its -wal and -shm files) while the server runs. Use wardlockd backup /path/to/backup.db with the same environment as the server (it reads WARDLOCK_DATA_DIR). It uses SQLite's online backup API to copy the whole database inside one read transaction, so the copy is the database as of one moment while the server keeps reading and writing (WAL mode does not block writers meanwhile). The copy is written to a temporary file beside the destination, checked with PRAGMA quick_check, made a single self-contained file (no -wal beside it), synced and moved into place with mode 0600. The destination must not exist yet: the command never overwrites a file. Moving into place is a hard link (which fails rather than replace a file that appeared meanwhile), so the destination's filesystem must support hard links: local filesystems do, but some network shares (SMB/CIFS), FAT/exFAT and some FUSE mounts do not, and the command then fails with move backup into place. In that case write the backup to a local directory and copy it to the share afterwards. A failed backup removes its temporary file and any -journal, -wal or -shm beside it. It needs no WARDLOCK_SECRET. As an option, Litestream can replicate the database continuously to object storage.

• Blobs: content-addressed and write-once, so a plain rsync (after the database backup) is consistent: every blob the backed-up database references already exists, provided garbage collection does not run in between. Schedule backups away from collections, or set WARDLOCK_GC_INTERVAL_MINUTES=0 and run wardlockd gc after each backup.
• Blobs in a bucket: wardlockd backup copies only the database, never the bucket. Protect the bucket with the service's own tools: turn on versioning, with a lifecycle rule that expires noncurrent versions no sooner than your oldest database backup, so a blob that garbage collection deleted after a backup can be brought back; or replicate the bucket to another bucket or region. A copy with rclone sync or aws s3 sync taken after each database backup works too, under the same rule about garbage collection as rsync.

The server's tests (TestBackupAndRestore in apps/server/cmd/wardlockd) do exactly this against a running server while it takes writes, and check that the restored instance serves the site, keeps the owner's session and takes new writes.

For go get / go install to work, https://wardlock.nz/<module>?go-get=1 must return HTML with a go-import meta tag mapping the module path to the repository and its subdirectory:

<!-- https://wardlock.nz/cli?go-get=1 -->
<meta
  name="go-import"
  content="wardlock.nz/cli git https://github.com/CubieCode/wardlock apps/cli"
/>
<!-- https://wardlock.nz/server?go-get=1 -->
<meta
  name="go-import"
  content="wardlock.nz/server git https://github.com/CubieCode/wardlock apps/server"
/>
<!-- https://wardlock.nz/apiclient?go-get=1 -->
<meta
  name="go-import"
  content="wardlock.nz/apiclient git https://github.com/CubieCode/wardlock packages/apiclient-go"
/>

The fourth field is the module's directory inside the repository; Go supports it from 1.25. home-site/public/index.html carries all three tags on one page, and the site's SPA fallback answers it for every package path below a module (such as wardlock.nz/cli/cmd/wardlock?go-get=1) with status 200; Go uses the tag whose module path is a prefix of the package. scripts/home-site.test.mjs checks the tags against the modules in go.work, and scripts/test-module-tags.sh runs go install through the page.

Module tags are prefixed with the module's directory, the fourth go-import field; a plain vX.Y.Z tag is not a version of any module. go install pkg@version refuses a module whose go.mod has replace directives, so the tagged commit cannot be the development state. The release steps are:

1. On the release branch, tag packages/apiclient-go/vX.Y.Z at the commit to release.
2. In apps/cli/go.mod and apps/server/go.mod, drop the replace wardlock.nz/apiclient directives, require wardlock.nz/apiclient vX.Y.Z where it is used, and run GOWORK=off go mod tidy so go.sum records the client's hash (the release owner holds the lockfile gate for this). Commit.
3. Tag that commit apps/cli/vX.Y.Z and apps/server/vX.Y.Z and push those tags (with packages/apiclient-go/vX.Y.Z). Module tags never start the publish job.
4. Tag the same commit vX.Y.Z and push that tag on its own (git push origin vX.Y.Z). GitHub creates no push events when more than three tags are pushed at once, so pushing every tag together can skip the release run. The tag push starts the release workflow: snapshot and module-tags run first, then publish logs in to ghcr.io with the workflow's GITHUB_TOKEN, pushes ghcr.io/cubiecode/wardlock:X.Y.Z and :latest, creates a draft GitHub release with the archives, SBOMs and checksums.txt, and signs the archives, SBOMs and image with artefact attestations (Verifying a release).
5. Check the draft release, add the release notes and publish it by hand. After the first image push, set the ghcr.io package's visibility.

Main keeps its replace directives; only the release commit drops them.

scripts/test-module-tags.sh proves this scheme without publishing: it makes the release commit and tags in a temporary clone, serves that clone with git daemon and answers wardlock.nz/<module>?go-get=1 from a local proxy, then runs go install for both commands. It also checks that an unprefixed tag and a tag on a commit with replace directives fail. The tags exist only in the deleted clone. The release workflow runs it on every pull request.

.goreleaser.yaml builds every artefact, and .github/workflows/release.yml runs goreleaser release --snapshot --clean on pull requests and main, then scripts/release-smoke.sh checks the snapshot: all 12 archives and SBOMs match checksums.txt, install.sh installs them and rejects a bad checksum, and the image runs as uid 65532 and answers /healthz. Pull requests, main and manual runs never publish: snapshots never push, the image and GitHub release are skipped outside snapshots unless WARDLOCK_PUBLISH=true, and the publish job that sets it runs only when github.event_name == 'push' and the ref starts with refs/tags/v in CubieCode/wardlock. The workflow triggers on tags matching v*; GitHub matches tag filters against the whole tag name and * does not match /, so apps/cli/vX.Y.Z, apps/server/vX.Y.Z and packages/apiclient-go/vX.Y.Z do not start it. Only publish gets contents: write and packages: write; every other job has contents: read. GoReleaser is told the triggering tag (GORELEASER_CURRENT_TAG), since the release commit also carries the module tags. The GitHub release is always a draft.