System Design Cases
Secrets management: Vault, KMS, sealed secrets
Secrets management: Vault, KMS, External Secrets Operator, Sealed Secrets, SOPS. Five scenarios: app fetches static secret via Vault (k8s SA JWT auth → token → KV read with KMS-backed envelope encryption + audit log), Vault dynamic DB credentials (auto-generated postgres user with TTL=1h, auto-revoke makes leaked password useless), External Secrets Operator (GitOps sync from Vault into K8s Secret, plus Sealed Secrets alternative), SOPS encrypted YAML in git (Mozilla SOPS with KMS/age, no runtime Vault dependency), and ADR comparing Vault vs cloud-native Secrets Manager vs ESO+Sealed Secrets with hybrid recommendation.
Secrets management: identity bootstrap, short leases, rotation, and response
A secret is high-value authentication or cryptographic material with an owner, purpose, scope, version, lifetime, and revocation path. Workloads bootstrap with identity, retrieve only authorized short-lived material, and never treat storage encryption as lifecycle management.
Security invariants
- SECRET-C1: Central secret management applies access control, lifecycle, audit, availability, and incident policy instead of scattering plaintext values through code and configuration.
- SECRET-C2: Workload identity is the bootstrap for secret access so static credentials are not embedded in images or repositories.
- SECRET-C3: Short-lived dynamic credentials provide per-workload scope, attribution, expiry, and revocation where the target supports them.
- SECRET-C4: Rotation creates a new version, reloads consumers, verifies it, and revokes the old version under bounded overlap.
- SECRET-C5: Outage continuity is explicit and bounded by existing lease validity; required secret authorization does not fail open.
- SECRET-C6: Exposure response revokes or rotates material, investigates use, removes unsafe copies, repairs the leak path, and audits break-glass access.
Trust-boundary map
| Component | Responsibility |
|---|---|
workload | Application Workload |
workload-identity | Attested Workload Identity |
secret-manager | Central Secret Manager |
policy-engine | Secret Access Policy |
dynamic-issuer | Dynamic Credential Issuer |
target-service | Protected Database or Service |
rotation-controller | Rotation and Revocation Controller |
audit-log | Secret Metadata Audit |
break-glass | Controlled Break-Glass Path |
emergency-operator | Approved Emergency Operator |
secret-scanner | Repository and Log Secret Scanner |
attacker | Compromised Workload or Operator |
Topology edges represent authenticated or otherwise explicit communication paths. Responses reuse those physical paths in reverse; no response-only or bypass edges are added.
Executable scenarios
identity-bootstrap — Bootstrap without a static application secret
The platform attests workload identity and the secret manager authorizes that identity; no long-lived bootstrap credential is baked into an image or repository.
least-privilege-fetch — Fetch a least-privilege short lease
A workload receives only the exact secret version or dynamic credential it needs, with a bounded lifetime and no value in logs.
dynamic-credential — Prefer dynamic credentials when supported
Per-workload database credentials reduce sharing, enable attribution, and expire automatically instead of rotating one global password everywhere.
versioned-rotation — Rotate with explicit version overlap
Consumers reload a new version before the old version is revoked; overlap is bounded and rollback does not resurrect an already compromised value.
lease-expiry — Expired lease fails closed
A credential past its lease or cryptoperiod is not extended by a client clock or cached indefinitely; renewal requires a fresh authorized exchange.
manager-outage — Secret-manager outage has an explicit continuity policy
A workload may use an already issued unexpired lease if policy permits, but cannot obtain or silently extend credentials after expiry.
secret-exposure-response — Respond to a detected secret exposure
Detection starts containment: identify scope, revoke or rotate, remove the value from logs and history, investigate use, and repair the leak path.
break-glass-access — Break-glass access is exceptional and accountable
Emergency access is narrowly scoped, strongly authenticated, time bounded, independently alerted, and followed by rotation and review.
Operational and failure rules
- Every rejection path fails closed at the component that owns the decision; infrastructure failure is never converted into authentication, authorization, or integrity success.
- Retries preserve stable transaction identity, replay detection, bounded freshness, and audit context. A retry does not erase a prior success or compromise signal.
- Concurrent validation branches are independent checks. Completion requires every mandatory branch, and no check substitutes for another.
- Logs contain decision metadata and stable identifiers, not credentials, bearer tokens, cryptographic keys, or sensitive payloads.