feat: add centralized trust bundle cert management #2167

joelmccoy · 2025-11-21T19:55:32Z

Description

Implements the Centralized Trust Bundle management feature according to design doc.

Related Issue

Fixes #2126

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Other (security config, docs update, etc)

Steps to Validate

uds run -f tasks/test.yaml uds-core-private-pki
- can toggle/swap out values in the ClusterConfig manually to ensure expected reconciliation for configmaps in cluster

Checklist before merging

Test, docs, adr added or updated as needed
Contributor Guide followed

src/pepr/operator/crd/sources/cluster-config/v1alpha1.ts

joelmccoy · 2025-12-03T20:15:25Z

src/pepr/operator/crd/validators/clusterconfig-validator.ts

+    // Validate each certificate in the bundle
+    certBlocks.forEach((certPem, index) => {
+      try {
+        new X509Certificate(certPem);


note: add in a validation with the crypto library to ensure that provided values for CA_BUNDLE_CERTS are proper certs

joelmccoy · 2025-12-03T20:16:28Z

docs/reference/configuration/trust-management/private-pki.md

note to self: add a uds docs redirect link for this

joelmccoy · 2025-12-03T20:19:07Z

tasks/test.yaml

          uds zarf tools kubectl label namespace authservice-ambient-test-app zarf.dev/agent=ignore
          uds zarf tools kubectl apply -f src/test/app-ambient-authservice-tenant.yaml
      - task: private-pki:private-pki-tests
+      - task: trust-bundle:trust-bundle-tests


note: I decided to keep these trust bundle e2e tests in the private-pki nightly workflow. if we feel that we want to run them as standard e2e tests that is fine too, just lmk. We will just have to update our k3d-standard bundle deploy to enable public CA certs so we can test this.

joelmccoy · 2025-12-03T20:20:14Z

test/vitest/network.spec.ts

 }

-// Execute commands inside a pod
-async function execInPod(


note: moved this into a helper function folder so it can be reused

defenseunicorns/uds-docs#326

joelmccoy · 2025-12-03T20:21:26Z

test/vitest/vitest.config.js

    environment: 'node',
    globalSetup: ['./vitest.setup.js'],
    include: ['**/*.spec.ts'],
+    exclude: ['trust-bundle/**'],


note: e2e vitest flow was running the trust-bundle ones as well, so had to explicitly exclude this

Copilot

Pull request overview

This PR implements centralized trust bundle management for UDS Core, enabling automatic distribution of CA certificates (custom, DoD, and public) across the cluster through ConfigMaps. The feature introduces new CRD fields, a dedicated CA bundle controller, and comprehensive test coverage.

Key Changes

Added caBundle configuration to ClusterConfig and Package CRDs with support for custom, DoD, and public CA certificates
Implemented automatic CA bundle ConfigMap generation and distribution in namespaces with UDS Packages
Enhanced ClusterConfig with status subresource tracking (Pending/Ready/Failed phases)
Deprecated CA_CERT variable in favor of CA_BUNDLE_CERTS for improved clarity

Reviewed changes

Copilot reviewed 42 out of 44 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
src/pepr/operator/controllers/ca-bundles/ca-bundle.ts	New controller for creating and updating CA bundle ConfigMaps across the cluster
src/pepr/operator/controllers/ca-bundles/ca-bundle.spec.ts	Comprehensive unit tests for CA bundle controller functionality
src/pepr/operator/controllers/config/config.ts	Enhanced config reconciliation with CA bundle support, status management, and update logic
src/pepr/operator/controllers/config/config.spec.ts	Expanded test coverage for config handling including CA bundle scenarios
src/pepr/operator/crd/sources/package/v1alpha1.ts	Added caBundle field to Package CRD schema
src/pepr/operator/crd/sources/cluster-config/v1alpha1.ts	Added caBundle field and status subresource to ClusterConfig CRD
src/pepr/operator/crd/validators/clusterconfig-validator.ts	Added validation for CA bundle certificates with X.509 verification
src/pepr/operator/crd/validators/clusterconfig-validator.spec.ts	Test coverage for CA bundle certificate validation
src/pepr/operator/reconcilers/package-reconciler.ts	Integrated CA bundle ConfigMap creation into package reconciliation
test/vitest/trust-bundle/	New E2E test suite validating trust bundle functionality with live containers
test/vitest/helpers/k8s.ts	Refactored execInPod helper for reuse across test suites
docs/reference/configuration/trust-management/central-trust-bundle-management.md	Comprehensive documentation for the centralized trust bundle feature
schemas/package-v1alpha1.schema.json	JSON schema for Package CRD with caBundle field
schemas/clusterconfig-v1alpha1.schema.json	JSON schema for ClusterConfig CRD with caBundle and status fields
src/pepr/zarf.yaml	Added new Zarf variables and ClusterConfig status validation wait action
src/pepr/uds-operator-config/values.yaml	Updated Helm values with CA bundle configuration

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/pepr/operator/controllers/config/config.ts

src/pepr/operator/crd/sources/cluster-config/v1alpha1.ts

src/pepr/operator/crd/validators/clusterconfig-validator.ts

src/pepr/operator/controllers/ca-bundles/ca-bundle.ts

schemas/package-v1alpha1.schema.json

src/pepr/operator/controllers/ca-bundles/ca-bundle.spec.ts

src/pepr/operator/controllers/ca-bundles/ca-bundle.ts

mjnagel

Initial review - need to deploy locally and validate the deprecated value at least but this looks pretty great overall.

bundles/k3d-standard/uds-private-pki-config.yaml

src/pepr/operator/controllers/ca-bundles/ca-bundle.ts

src/pepr/operator/controllers/config/config.ts

src/pepr/uds-operator-config/values.yaml

src/pepr/operator/controllers/config/config.ts

src/pepr/uds-operator-config/templates/clusterconfig.yaml

chance-coleman

this is looking good to me, only really one place that tripped me up a bit.

src/pepr/operator/controllers/config/config.ts

src/pepr/operator/controllers/ca-bundles/ca-bundle.ts

slaskawi · 2025-12-08T11:49:48Z

bundles/k3d-standard/uds-private-pki-config.yaml

 variables:
  core:
-    # CA certificate for Authservice
-    CA_CERT: "PLACEHOLDER_CA_CERT"


Shall we call this out as a breaking change?

We're leaving CA_CERT around, just deprecated so I don't think we need to. This removal is just in a CI only config file so not a downstream breaking change here.

slaskawi · 2025-12-08T11:56:35Z

hack/dev-manifests/uds-ca-certs.yaml

+  name: uds-ca-certs
+  namespace: pepr-system
+data:
+  dodCACerts: ""


ConfigMap has a limitation to 1 MB. I did some quick math and for a Base64 encoded certs that are quite large (2 KB, 2.72 KB after encoding), we can store around 370 of these.

Even though it appears quite large, shall we mention somewhere in the docs that there is certain ceiling there and administrators should be aware of it?

Yea, we did some math in the design doc around this. Realistically, we don't imagine users hitting the upper limit on the configmap. We could put a callout in the docs about the upper limit, but I feel it is adding a caution to something that most likely won't happen? I lean towards leaving it out but could be convinced to put it in

slaskawi · 2025-12-08T11:59:34Z

src/pepr/zarf.yaml

      - name: module
        valuesFiles:
          - values.yaml
+    actions:


Is this necessary? I though Zarf automatically checks the Ready field.

Zarf will check for the Ready condition, but not the phase field by default. Think in the design doc the status stuff popped up towards the end of review/approval but we were leaning towards keeping status simple and not using conditions here. That would be a way to get rid of this action though if we wanted to.

joelmccoy added 2 commits November 21, 2025 13:54

feat: add ca trust bundle management

f73205f

chore: more tests and update private-pki test

37b6de7

joelmccoy commented Nov 21, 2025

View reviewed changes

src/pepr/operator/crd/sources/cluster-config/v1alpha1.ts Show resolved Hide resolved

joelmccoy added 7 commits December 2, 2025 10:31

fix: don't skip initial startup

ea88d5d

chore: update kfc/crds/tsdocs

2494c98

fix: update kfc/proper crds

a11fbb9

chore: add e2e test for trust-bundle-management

dc3551a

Merge branch 'main' into ca-trust-bundle

787b366

fix: testing issues

c95b13f

chore: add docs

b67855a

joelmccoy commented Dec 3, 2025

View reviewed changes

chore: update tsdocs

939b48c

joelmccoy changed the title ~~feat: add ca trust bundle management~~ feat: add centralized trust bundle cert management Dec 3, 2025

joelmccoy marked this pull request as ready for review December 3, 2025 21:22

joelmccoy requested a review from a team as a code owner December 3, 2025 21:22

Copilot AI review requested due to automatic review settings December 3, 2025 21:22

Copilot started reviewing on behalf of joelmccoy December 3, 2025 21:23 View session

Copilot finished reviewing on behalf of joelmccoy December 3, 2025 21:25

Copilot AI reviewed Dec 3, 2025

View reviewed changes

joelmccoy added 2 commits December 3, 2025 16:15

chore: address pr feedback

e69385c

fix: proper typing

ee48af0

mjnagel reviewed Dec 8, 2025

View reviewed changes

chance-coleman reviewed Dec 8, 2025

View reviewed changes

src/pepr/operator/controllers/config/config.ts Outdated Show resolved Hide resolved

mjnagel reviewed Dec 8, 2025

View reviewed changes

src/pepr/operator/controllers/ca-bundles/ca-bundle.ts Show resolved Hide resolved

chore: address PR feedback

5e879ea

slaskawi reviewed Dec 9, 2025

View reviewed changes

joelmccoy added 3 commits December 9, 2025 10:25

chore: refactor to not create empty configmaps

c8ade8b

Merge branch 'main' into ca-trust-bundle

f4d6436

chore: update docs links & crd schema

c037e91

joelmccoy mentioned this pull request Dec 9, 2025

Add CA Bundle Optional Volume Mounts for Grafana/Keycloak #2203

Open

joelmccoy mentioned this pull request Dec 9, 2025

chore: add redirect for private-pki doc defenseunicorns/uds-docs#326

Draft

mjnagel approved these changes Dec 9, 2025

View reviewed changes

feat: add centralized trust bundle cert management #2167

Are you sure you want to change the base?

feat: add centralized trust bundle cert management #2167

Uh oh!

Conversation

joelmccoy commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue

Type of change

Steps to Validate

Checklist before merging

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mjnagel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chance-coleman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

joelmccoy commented Nov 21, 2025 •

edited

Loading