Skip to content

Support per-build registry credentials - #572

Open
sjmiller609 wants to merge 4 commits into
mainfrom
hypeship/build-registry-auth
Open

sjmiller609 wants to merge 4 commits into
mainfrom
hypeship/build-registry-auth

Conversation

@sjmiller609

@sjmiller609 sjmiller609 commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Add optional registry_credentials to POST /builds: registry-host-keyed username/password credentials with optional expiry, borrowed for one build.
  • Keep upstream credentials out of persisted host metadata/config and deliver them over vsock to the ephemeral builder. Reuse the existing Docker auth config alongside local-registry auth.
  • Bound borrowing to 30 minutes from submission (including queue time), or caller expiry. Release host references on completion/cancellation/expiry and fail interrupted builds rather than replaying without credentials.
  • Bypass shared base-image mirroring for credentialed builds so private bases stay isolated. Reject local-registry overrides and older builders without the credential handshake.
  • Redact raw/encoded credentials from guest stdout, streamed logs, stored logs, and errors. Clean up the guest auth files after the build.
  • Update the generic builder Dockerfile and regenerate the OpenAPI server/spec. The large generated diff is the compressed embedded specification.

Validation

  • Build package short suites pass: go test -short -tags containers_image_openpgp ./lib/builds/...
  • Targeted API/lifecycle/vsock/config/redaction tests pass with the race detector; go vet passes for builds and API packages.
  • Real local BuildKit test rejects an unauthenticated private FROM and succeeds with the generated borrowed-auth config; output/image metadata checks pass.
  • Generic builder image builds successfully locally.
  • Broader API short suite was attempted but four VM-dependent tests fail locally on bridge/KVM permission errors.

Experimental validation

Tested af90b44 with isolated Hypeman, the rebuilt builder image, real KVM guests, and a private TLS registry:

  • Correct credentials built successfully (10.1s); missing/wrong credentials failed.
  • In-flight expiry (21s) and cancellation (11.5s) cleaned up the builder VMs; zero remained.
  • No raw or Docker-auth-encoded credentials found in 17 persisted build-record/log files.
  • Test resources removed; existing services untouched. AWS token issuance and platform integration were not tested.

Detailed results

Rollout / scope

Rebuild and deploy the builder image before enabling credentialed callers. No refresh protocol: callers resubmit interrupted/expired builds with fresh credentials. Underlying registry/IAM permissions still determine token scope; callers should use pull-only access to required repositories.

This PR is Hypeman only. Platform issuance, API FROM references, CLI flags, and SDK caller updates are separate follow-ups. No deployment or image publication performed.


Note

High Risk
Touches credential handling, vsock delivery, and build lifecycle/recovery paths; mistakes could leak secrets or allow builds with stale or overly broad upstream access.

Overview
Adds optional registry_credentials on POST /builds: a bounded JSON map of upstream registry username/password pairs (with optional expiry) so builds can pull private FROM images without host-wide registry secrets.

The API and build manager validate input (host shape, size limits, no local-registry override), keep credentials only in memory for up to 30 minutes, and do not persist them to build metadata or config disks. Credentialed builds skip shared base-image mirroring; the manager hands credentials to the builder over a new vsock handshake and rejects old builder images that never request them.

The builder agent waits for borrowed credentials, merges them into ephemeral 0600 Docker auth config beside the local push token, omits the Docker Hub mirror when credentialed, redacts secrets in logs/errors, and deletes auth files after the build. Interrupted credentialed builds fail closed on recovery (no replay without resubmit). OpenAPI/docs and builder image packaging include the new registryauth module.

Reviewed by Cursor Bugbot for commit 93d1461. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
-->

✱ stlc build

✅ go code · compare

Your SDK build was successful.

generate ✅ → bootstrap ✅ → format ✅

116 files generated at 60cfbaa (pushed)

go get github.com/kernel/hypeman-go-staging@60cfbaa1c94a69805661600aec4207c6183584eb
✅ python code · compare

Your SDK build was successful.

generate ✅ → bootstrap ✅ → format ✅

233 files generated at 46e9347 (pushed)

✅ typescript code · compare

Your SDK build was successful.

generate ✅ → bootstrap ✅ → format ✅

138 files generated at 0083204 (pushed)

Diagnostics: ❗ 0 new / 1 total error, 💡 0 new / 5 total note
LevelCodeMessageTargets
Build metadata
Buildbd_76m7fCs8-rich-ice
Timestamp2026-11-16T19:46:31.464Z
stlc8413509
Spec hash9efe0f4156f2
Config hash659c3687c3f0

This comment is auto-generated by stlc and is kept up to date as you push.
If you push new commits, re-run this workflow to update this comment.
Last updated: 2026-11-16 19:46:52 UTC

@sjmiller609

Copy link
Copy Markdown
Collaborator Author

Additional end-to-end validation at af90b44: ran an isolated Hypeman API with the rebuilt builder image on a KVM-capable Linux host, using a private TLS registry and synthetic per-request credentials.

  • Anonymous registry access returned 401.
  • Real builder VM: no credentials failed; correct borrowed credentials built successfully (10.1s); wrong credentials failed.
  • In-flight credential expiry failed the build and cleaned up the VM (21.0s for an 18s expiry).
  • In-flight cancellation reached cancelled and cleaned up the VM (11.5s).
  • Zero builder instances remained. Scanned 17 persisted build-record/log files for raw and Docker-auth-encoded credentials; no matches.
  • Removed the isolated controller, network, and test data. Existing services were not restarted.

This validates the full API → vsock → guest Docker config → BuildKit private-FROM path, not AWS-specific token issuance or the platform caller integration. No production ECR credentials used and no code changes were needed.

@sjmiller609
sjmiller609 marked this pull request as ready for review November 16, 2026 19:25

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f9e7b1f. Configure here.

Comment thread lib/builds/builder_agent/main.go Resolved
Comment thread lib/builds/manager.go Resolved
@sjmiller609

Copy link
Copy Markdown
Collaborator Author

Picked hiroTamada to review: strongest familiarity with the changed build code; 3 pending reviews.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant