Skip to content
DevSecOps Homelab

Bump A YourBOX Container Image Version

When To Use

Section titled “When To Use”

Use this runbook when:

  • Renovate or Dependabot opens an MR that bumps a container image referenced by YourBOX (Dockerfile FROM, CI image, Helm chart values, Ansible compose template).
  • Trivy reports a CVE on the current image and a fixed upstream version exists.
  • The maintainer needs to evaluate a new upstream release for inclusion.

Do NOT use this runbook to “refresh the digest” of a tag that has not changed version. A digest that changes for an unchanged tag is a supply chain incident: follow the Digest Mismatch Incident Path at the end of this runbook instead.

Preconditions

Section titled “Preconditions”
  • The Renovate or Dependabot configuration is active on the YourBOX repository (see the Digest Pin Policy procedure for the schedule and the scope).
  • The maintainer can run docker buildx imagetools inspect against the target registry.

Procedure For A Version Bump

Section titled “Procedure For A Version Bump”

  1. Confirm the MR is a version bump, not a digest mutation under the same version.

    Terminal window
    git diff origin/main..HEAD -- Dockerfile .gitlab-ci/ k8s/charts/ \
      roles/yourbox/runtime/templates/compose.yml.j2

    Expected result: the diff shows the version tag changing and the digest changing in the same hunks. A diff that touches only the digest is the incident path.

  2. Read the upstream release notes for the new version.

    Renovate adds a release notes link in the MR body. Follow the link. Confirm the change set covers the CVE fix or the feature you expect.

  3. Independently re-resolve the digest of the new tag.

    Terminal window
    docker buildx imagetools inspect <registry>/<image>:<new_tag> \
      --format '{{.Manifest.Digest}}'

    Expected result: the digest matches the value Renovate proposed. A mismatch means the registry returned a different digest than the one the bot pinned. Stop and investigate.

  4. Trigger the YourBOX pipeline on the MR.

    The build stage rebuilds the application image on top of the new base, the scan stage runs Trivy with HIGH and CRITICAL gating, and Plumber validates that every reference still pins by digest.

    Expected result: the pipeline succeeds with no new HIGH or CRITICAL findings.

  5. Compare the Trivy report against the previous baseline.

    Terminal window
    jq '.Results[].Vulnerabilities | length' trivy.json
    jq '.Results[].Vulnerabilities | length' trivy.previous.json

    Expected result: the new vulnerability count is lower or equal. A higher count is acceptable only if every new finding has a fix tracked under the next bump candidate.

  6. Run the smoke test against the new image.

    Terminal window
    bash tests/smoke/dockerfile-smoke.sh

    Expected result: exit 0.

  7. Approve and merge the MR.

    The Argo CD reconciliation picks up the new image digest within five minutes (production), and the deploy stages of the pipeline push the bump to the VPS and Azure staging.

Validation

Section titled “Validation”
  • grep -nE '(image: ?|FROM )[^@]+$' Dockerfile .gitlab-ci/ k8s/charts/ returns no matches.
  • Plumber score on the merge request stays at 100.
  • Trivy reports zero HIGH or CRITICAL findings on the new image.
  • The Argo CD Application status reports Synced and Healthy after the merge.

Digest Mismatch Incident Path

Section titled “Digest Mismatch Incident Path”

If a diff shows a digest change under an unchanged tag, treat the event as a possible supply chain compromise.

  1. Stop the merge. Do not pin the new digest.

  2. Re-resolve the digest independently from a clean environment.

    Terminal window
    docker buildx imagetools inspect <registry>/<image>:<tag> \
      --format '{{.Manifest.Digest}}'

    If the registry returns the same new digest, the registry itself served a different image under the same tag.

  3. Check the upstream advisory channel for an announced retag.

    Some maintainers retag for a benign reason (manifest list update, metadata correction). Confirm by reading the upstream release notes, the maintainer’s mailing list and the matching CVE feeds.

  4. Compare image content if the registry exposes both manifests.

    Terminal window
    docker buildx imagetools inspect --raw \
      <registry>/<image>@<old_digest> > /tmp/old.json
    docker buildx imagetools inspect --raw \
      <registry>/<image>@<new_digest> > /tmp/new.json
    diff /tmp/old.json /tmp/new.json

    Expected result: the diff is bounded to documented changes (architecture additions, annotation updates). An unexplained diff is the incident signal.

  5. Escalate. Open a ticket in the security channel with:

    • The old digest, the new digest, the tag, the registry.
    • The upstream advisory you found, or the absence of one.
    • The diff between the manifests when available.

  6. Pin the previous digest explicitly in the repository until the investigation concludes. Add a comment next to the pin describing the incident reference and the date.

  7. Keep the deployment running on the previous image. Argo CD continues reconciling against the known good digest.

If This Does Not Work

Section titled “If This Does Not Work”

If the new version introduces a regression that the smoke test catches, revert the MR. Renovate raises the same bump again once a newer version is published. Do not silence the bot.

If Trivy reports a new HIGH or CRITICAL CVE on the bumped version, hold the bump until upstream releases a fix, or document an exception in infra/security/trivy-allowlist.yaml with a CVE identifier, a justification and an expiry date.

If Renovate proposes a bump that violates the workspace registry allowlist (the new tag moved to a non allowed registry), reject the MR and open a follow up to update the allowlist explicitly with a written rationale.