Skip to content

ADR-010 — Secrets outside the repository and outside the database, in an encrypted institutional vault

Status

Accepted (2026-07-03, phase 3). Amended for quiz: the principle (secrets in the environment, never in the repository, an image or the database; no secret in the logs) applies as written, but the GitHub App private key, the GitHub OAuth and webhook secrets and the runner registration PAT (ADR-007) are heig-classroom's and do not exist here. Quiz's secrets are the database password (POSTGRES_PASSWORD), the OIDC client secret or private key (OIDC_CLIENT_SECRET, OIDC_PRIVATE_KEY_PATH), the cookie secret, the runner's shared token (RUNNER_TOKEN, ADR-016), the metrics token, and the notification credentials (SCW_SECRET_KEY, TEAMS_CLIENT_SECRET, ADR-030); .env.prod.example and apps/api/src/config.ts are the full list.

Amended again (2026-09-30, ADR-035, the classroom merge): the GitHub secrets come back, as those of Quiz's own GitHub App (D23), not classroom's: the App's private key (GITHUB_APP_PRIVATE_KEY_PATH, the PEM mounted read-only), the webhook secret (GITHUB_WEBHOOK_SECRET) and the App's OAuth client secret (GITHUB_APP_CLIENT_SECRET), beside GITHUB_APP_ID, GITHUB_APP_SLUG and GITHUB_APP_CLIENT_ID. Staging holds its own App's, never production's (ADR-028). Point 3's two-key rotation applies to Quiz's App as written. The online workspace (ADR-047) adds the codespace launch secret (CODESPACE_LAUNCH_SECRET, at least 32 characters, the same on both sides, refused without CODESPACE_URL and as change-me), set only when phase M6 ships. The runners PAT of ADR-007 stays on classroom's runner VM and never reaches Quiz. All of them go into the age-encrypted vault of point 2.

Amended again (2026-09-30, ADR-051, attested kiosk stations): the Chrome Verified Access service account's JSON key (KIOSK_VA_KEY_FILE, mounted read-only like the edu-ID PEM) joins the list, set only when KIOSK_ATTESTATION=google; config.ts refuses to start in production when it is unreadable. The access token it buys lives in memory only and is never logged (N-SEC-19). Staging may hold its own service account's key, never production's. It goes into the vault of point 2 like the others; rotation is Google's two-key procedure (create a new key, deploy, delete the old one).

Amended again (2026-10-01, ADR-058, the LLM gateway): one secret lives in the database, the LLM provider's API key, entered by an administrator from the console, and only ENCRYPTED (AES-256-GCM) under a master key, LLM_KEY_SECRET, which is a secret of this ADR: environment file, age vault, never the repository nor the database. Rejected alternative 3 still holds for every other secret. Losing the master key loses only the provider key, which an administrator enters again. The key is never returned by the API and the log redaction knows sk-ant- and x-api-key.

Amended again (2026-10-09, #235, the off-site backups): the borg passphrase of the off-site repository and the Storage Box keys are secrets of this ADR, outside the repository and the database. The passphrase lives on the VM, srv's ~/.config/borg-offsite/passphrase (mode 600), because the VM encrypts its archives; the box's full-access key does not, and the VM's own key can only append (ADR-009, amendment of decision 3). The passphrase, the full-access key and the exported repository key go into the vault of point 2.

Context

Server secrets: the PEM private key of the GitHub App, the OIDC and GitHub OAuth client secrets, the webhook secret, the cookie secret, the runner registration PAT (ADR-007). AU-43 requires that they come from the environment or from a secret manager, "never from the repository nor from the database". The restore runbook (RTO 4 h, NFR-16) must be able to reinject them reproducibly.

Decision

  1. At runtime: environment files on the VM (owned by root, permissions 600), the PEM key mounted read-only in the container; never in an image, a git repository or the database.
  2. For recovery: an age-encrypted backup copy of each secret in the institutional vault (HEIG Vaultwarden or equivalent), referenced by the runbook; restoring is a script that decrypts and puts the files back.
  3. Rotation documented in the runbook: the GitHub App accepts two active private keys during the switchover (generate, deploy, revoke the old one); the same interruption-free switchover procedure applies to the teacher API keys (AU-40) and to the runners PAT (12-month expiry).
  4. No secret in the logs: dedicated pino serializers mask keys beyond their prefix, the OAuth code, cookies and Authorization headers (AU-41).

Consequences

  • A strict reading of AU-43 is satisfied: nothing in the repository, not even encrypted.
  • Restoring depends on no human memory: the vault and the script make the RTO reproducible (the "manual KeePass" weakness is fixed).
  • The compromise of a secret has a written response: immediate revocation on the GitHub or IdP side, rotation through the two-key procedure.

Rejected alternatives

  1. sops/age secrets committed to the infrastructure repository (productivity and robustness proposals): practical and versioned, but in literal tension with AU-43 ("never from the repository"); the encrypted copy therefore lives in a separate vault, not in git.
  2. A dedicated Vault (HashiCorp or equivalent): one more stateful service to operate and back up, out of proportion for about a dozen secrets.
  3. Secrets in the database: forbidden by AU-43 and pointless — the database is backed up off site, which would widen the exposure surface. Except the LLM provider key, encrypted under a master key kept out of the database (ADR-058).