Rotate YourBOX Secrets

When To Use

Use this runbook when:

A YourBOX credential leaks (gitleaks finding, accidental copy in a chat, exposed in a screenshot).
A credential reaches its expiry window (cosign key, OIDC client secret).
An operator joins or leaves the YourBOX age recipient list.

Preconditions

You hold the appropriate OpenBao role for the environment being rotated.
You can run the YourBOX CI pipeline manually.
The matching application is healthy: the runbook does not introduce a rotation during an active incident.

Procedure

Identify the credential and its scope.
Terminal window
```
grep -rn "<credential_key>" \
  portfolio-docs/src/content/docs/services/yourbox/secret-management.mdx
```
Expected result: the env contract table lists the key, the environments it applies to and the OpenBao KV path.
Generate the new credential through the upstream provider.

For database passwords, generate 32 random bytes:
Terminal window
```
openssl rand -base64 32
```
For OIDC client secrets, request a rotation in Authentik admin under the YourBOX application.

For the cosign key, run cosign generate-key-pair against an OpenBao path:
Terminal window
```
COSIGN_PASSWORD=$(openssl rand -hex 32) \
  cosign generate-key-pair --kms hashivault://yourbox/cosign-key
```
Write the new value to OpenBao.
Terminal window
```
bao kv patch kv/yourbox/<env> "<CREDENTIAL_KEY>=<NEW_VALUE>"
```
Expected result: bao kv get kv/yourbox/<env> shows the new value with the updated created_time.
Refresh the External Secrets projection.
Terminal window
```
kubectl -n yourbox annotate externalsecret yourbox-env \
  force-sync=$(date +%s) --overwrite
```
Expected result: kubectl -n yourbox describe externalsecret yourbox-env shows SyncedSuccessfully in the Status field within 30 seconds.
Roll the YourBOX deployment to pick up the new value.
Terminal window
```
kubectl -n yourbox rollout restart deployment/yourbox
kubectl -n yourbox rollout restart deployment/yourbox-worker
```
Expected result: both Deployments report Ready within two minutes.
Verify the application health.
Terminal window
```
curl --fail --silent --show-error https://yourbox.site/health
```
Expected result: HTTP 200 with {"status":"ok"}.
For OIDC rotations, exercise a fresh login flow against Authentik.
Terminal window
```
curl --silent --output /dev/null --write-out "%{http_code}\n" \
  https://yourbox.site/oidc/login
```
Expected result: HTTP 302 redirecting to the Authentik authorize URL.

Update the SOPS encrypted dev and VPS file.

sops --decrypt prod.env.enc > prod.env
sed -i "s|^<CREDENTIAL_KEY>=.*|<CREDENTIAL_KEY>=<NEW_VALUE>|" prod.env
sops --encrypt --input-type dotenv --output-type dotenv prod.env > prod.env.enc
shred -u prod.env

Expected result: git diff prod.env.enc shows the encrypted file changed; no plaintext file remains.

Trigger a VPS redeploy so the new credential lands on the smoke test.
Terminal window
```
ansible-playbook -i inventory/staging playbooks/yourbox-vps.yml \
  --tags yourbox
```
Expected result: the play converges with zero failed tasks and the smoke test passes.

Record the rotation in the audit log channel.

bao audit-log write \
  reason="rotated <CREDENTIAL_KEY> on $(date -Iseconds)" \
  actor="$USER"

Validation

kubectl -n yourbox logs deployment/yourbox --since=5m \ | grep -i 'auth\|connect\|secret' | grep -i fail returns no results.
Grafana panel YourBOX > Overview > Auth Failure Rate stays at baseline.

If This Does Not Work

If the External Secrets projection does not refresh within 60 seconds, check the ClusterSecretStore status:

kubectl describe clustersecretstore openbao-prod

Escalate to the platform on call with the ClusterSecretStore status, the External Secrets controller logs and the OpenBao audit log entries for the past hour.