Skip to content

Extract the protected CloudFront Geo baseline - #1532

Draft
ChristianPavilonis wants to merge 24 commits into
baseline/native-aws-20261110from
baseline/protected-geo-20261110
Draft

ChristianPavilonis wants to merge 24 commits into
baseline/native-aws-20261110from
baseline/protected-geo-20261110

Conversation

@ChristianPavilonis

@ChristianPavilonis ChristianPavilonis commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Current review follow-through

Current head: 7d6b7a1b9.

  • Restored shared jurisdiction behavior and real Geo-to-auction/publisher-credential coverage in 60617bd9a.
  • Normalized native profile/hosting errors to typed reports and derived Display, preserving safe diagnostics and excluding private inputs. Added diagnostic/redaction regression tests.
  • Clarified optional Geo-header guarantees and added 50 descriptive messages without changing existing Geo assertion predicates or expected data.
  • Local Core, native adapter, parity, Fastly, default/Geo/all-feature Axum, all eight target-matched Clippy lanes, formatting and changed-lines domain checks pass. Existing ignored cases remain explicit. A fresh independent final review found no findings.
  • The Geo-enabled hosted test step is present and passed at 60617bd; the historical artifact-upload outage is no longer current. Hosted workflows for this head are still pending or running; their results are not counted as local proof.
  • All 12 inline threads now have explanations and are resolved. Re-review is requested from aram356 and dhruv8sh; their verdicts were not dismissed.

This PR remains draft. The temporary TS Geo-profile retirement waits for TS #1573 and EdgeZero's manifest binding #482/integration tag #491. The shared header-clearing refactor and pre-existing Fastly absent-Geo behavior remain explicitly deferred. Native EC-creation proof, final main-bound integration and live cloud qualification are not claimed. Issue #1529 remains open.

Historical baseline and verification receipts follow.


Summary

  • Default-off CloudFront boot profile, one immutable request outcome and absent optional detail.
  • Reorganize the existing working demo using EdgeZero83d7a629, not current-main forwarding compatibility.

Draft baseline review unit. Not a current-main merge candidate, deployment approval or completed current-head CI claim.

Baseline and sequence

Head: affb0d0a4c0a1f3b9ac00d11c12de1f45080a8ce. Base: baseline/native-aws-20261110.

Historical main anchor: 2dd968aeebeb2cc0d5110115f64e0ad58e2e1d13. Original demo: TS2423a2ed plus separately preserved dirty inputs. Matching EdgeZero: draft431, exact revision 83d7a6296d285e135c4ea70021b19e6c8dca7e9f.

The user selected compatible-baseline cleanup. Current-main integration with EdgeZero462 and the authenticated-forwarder incompatibility are deferred. Preserve current-main PR1525/1526 ready and1527/1528 draft, with their existing bases. No excluded1499 or independent1506 implementation, relay/privacy experiment, new transport fallback, group/codec/phase design or operational deployment is included.

Changes

Area Change
This unit Default-off CloudFront boot profile, one immutable request outcome and absent optional detail.
Unchanged work Original checkouts, dirty Prebid inputs, recovery references and existing PRs remain preserved.

Closes

Refs #1530. This unit completes one part of the baseline extraction and does not close the overall tracker on its own.

Spec

Not a new feature. This is extraction of existing demo behavior and bounded build/caller corrections. Original native and EC storage design/review records remain in the baseline where applicable; they are not presented as approval of later combined source. No new forwarding, consent, timer or provider policy is designed here.

Test plan

Core3002 and native all-feature suites pass, including10 real startup cases. No production KV Geo, local Geo database, new consent authority or cloud acceptance is added.

  • Required current-head hosted checks complete before ready-state transition.
  • Final baseline combined-check results recorded with exact input revisions and limitations.
  • Focused failures and corrections retained rather than assertions relaxed.

The b67ffc6 baseline checkpoint has passing documented local gates: Fastly/default/reuse/native/Cloudflare/Spin suites, all adapter strict lint lanes, 17 cross-adapter parity cases, 1252 JS cases, CLI suites, formatting, clock/digest/proposal checks and changed-lines domain lint. Passing checks at121ebcc36 are reused only for unchanged relevant inputs; the final follow-up covers the CLI test-import and documentation-only corrections. These results are not a substitute for this PR's current-head hosted checks.

Checklist

  • Existing behavior extracted into a scoped review unit.
  • Public examples use fictional inputs and no licensed Geo data.
  • No deployment, main merge, old-RC closure or recovery deletion.
  • No new worker launched or paused experiment resumed.
  • Review readiness confirmed independently from later units.

Operational Terraform remains in trusted-server-deployer. Local checks and prototype tests do not qualify cloud access, fleet durability or a live publisher.

Hosted docs-build correction

The first hosted format-docs job caught an additional site-build gate that the original local formatter checkpoint had not run: a relative link from the native EC guide to a Terraform example outside the site. Native EC correction8b11dbad3 replaces it with the plain repository path; no dead-link check is disabled. That docs-only correction is carried through the existing upper PR identities. Native EC and final integration8985b111 both pass docs ESLint and VitePress build; the final docs formatter also passes. Runtime, dependency, image-build-context and test inputs are unchanged from b67ffc6, so their prior exact-input proofs are reused. Hosted current-head checks still control readiness.

Startup negative-probe correction

A hosted native EC case failed the no-listener probe while its child logged the expected startup validation failure. The old log has no socket endpoints, so the failure's exact subtype is not claimed confirmed. A separate controlled Linux reproduction demonstrates a successful TCP self-connection with no listening server. Native correction8490c691d distinguishes equal local/peer endpoints from a real listener; actual socket controls prove the self-connection is excluded and a real listener is still detected. Nonzero exit, expected diagnostic, secret-redaction and the original startup deadline remain required. This changes only the acceptance fixture, not production admission, transport or trust.

The earliest native unit and final9356c557 default/all-feature acceptance checks and strict lint pass, including both controls and the original failure case. Ignores remain explicit; earlier separately executed celld proof is not relabeled as a new ignored-case run. Correction is merged upward through existing PR identities without force-pushing or retargeting. Current-head hosted checks must pass; the old failure is retained rather than rerun-only waived. Runtime/API/dependency inputs remain unchanged. A fresh exact9356 producer build and real runtime smoke now pass: image sha256:65a735d38e9dbc0573d0f6068a8b261bd202c43aea05085c2b2d6bccc91ebdbd, linux/amd64 UID/GID10001, including startup rejection and in-flight SIGTERM drain. Prior b67 image receipts remain separate; no image was pushed. All local parent runners are complete.

Browser CI host compatibility

Preserved older browser failures show Playwright1.58.2 rejecting ubuntu-latest's new Ubuntu26.04 image before browser installation; later missing-executable failures are consequences, not passing browser tests. Nativeb1216763a pins only the browser job to ubuntu-24.04, leaving dependency versions, tests and assertions unchanged. Direct Actionlint1.7.12 passes at the owning native and final integration points; the first shim-only attempt could not run because no global version was configured, and is not counted as lint proof. No global tooling was changed. The only9356 ->ee7 delta is this workflow host/comment; runtime, Cargo, docs, tests and Docker-context inputs are unchanged. Actual browser execution on the supported hosted runner remains pending and must pass before readiness. No waiver or browser skip.

Reconciliation refresh above the new PR 1506 head

Merge-forward head 9075b4b391c84bc5954d7b8a92f810251d2657be retains original public 852b875fe617048ea8aae39d0b74ce9a999fb1bc, previous local candidate 8f517932294472b768928528c6d40db03806978d and updated parent bf3569dd9b9d30290b7b18e284642edf9f67b336. The original feature patch is unchanged. Aram's #1516 code/tests/spec are preserved upstream rather than duplicated as another owned production port. The existing PR identity, feature boundary and readiness remain intact.

Fresh shared-root and combined RC qualification passes on the refreshed source. Reused JS/Chrome/storage/wrapper/Terraform evidence is explicitly fingerprinted, not claimed as a new execution. Source-image proof is bound to refreshed #1539, not a new RC release. Fresh hosted/macOS, digest-backed wrapper/RC images, other external browser/framework lanes and live AWS/publisher/CMP acceptance remain unverified. Ignored/unexecuted cases and original failed receipts are not counted as passes. Historical claims above remain bound to their named old inputs. No tag, registry publication or deployment is included.

Published snapshot; newer upstream integration pending

Published deliberately for review from the qualified #1506 checkpoint 57d7c6ad8db43601ef31c1e1d1615e6dd2aa6a6a. At publication the moving author branch was at 628474ec4a8cee8361a6927ae6b79b68fdd33a55. Newer upstream bounded-response handling, per-profile RTB caps and Sourcepoint changes are not incorporated or qualified in this snapshot. Further merge-forward reconciliation and conflict resolution may be needed before merge. Local results above describe this exact qualified source, not a merge with the newer base or a release.

@prk-Jr prk-Jr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed affb0d0a4c0a1f3b9ac00d11c12de1f45080a8ce against baseline b1216763a3c2a38df1a2909285330e818823423a. No actionable findings.

Reviewed native Geo admission and startup, request isolation, response finalization, optional-field serialization, consent handling, and affected auction consumers.

Local verification: cargo test --locked -p trusted-server-adapter-axum --all-features --test startup passed all 10 tests, including CloudFront trust rejection and error-response finalization, using Rust 1.98.1 and Node 24.12.0. Head and base were rechecked and remain unchanged.

This approval covers the historical baseline unit. Current-main compatibility and live AWS qualification remain outside scope. Main-only CodeQL did not run.

CI status: all 17 reported checks PASS at the reviewed head:

  • integration tests
  • integration tests (Fastly EC lifecycle)
  • browser integration tests
  • docs-proposal scripts
  • cargo test (cross-adapter parity)
  • cargo test (axum native)
  • cargo fmt
  • CLAUDE.md symlink guard
  • cargo test (ts CLI, native) (macos-latest)
  • cargo check/build/test (spin native + wasm32-wasip2)
  • vitest
  • cargo test (ts CLI, native) (ubuntu-latest)
  • cargo check (cloudflare native + wasm32-unknown-unknown)
  • cargo test
  • format-docs
  • format-typescript
  • prepare integration artifacts

@ChristianPavilonis
ChristianPavilonis marked this pull request as ready for review November 11, 2026 13:47
Retain the existing Geo feature head while projecting private managed outcomes into partial request-local geography. Preserve current-main trace privacy, direct outbound APIs, startup router ownership and ingress diagnostics. Do not restore backend or HTTP compatibility bridges.

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Reviewed 0ae96ef5f against baseline/native-aws-20261110 (merge base 65ba4f3df). The CloudFront path itself is well bounded: EdgeZero handles admission, credential stripping and assertion parsing; TS reads only the immutable RequestContext::geo(); and populated fields keep their flat wire names. Changes are needed because:

  • The jurisdiction rule change reaches every adapter, so Cloudflare US traffic loses server-side auctions and EC creation.
  • The CloudFront tests never run in CI, and no test proves geography reaches consent.
  • Native startup lost its environment diagnostic.
  • At this head, the integration, Fastly EC lifecycle and browser jobs were skipped.

2 of the inline comments below carry a one-click GitHub suggestion. Use Commit suggestion (or Add suggestion to batch for both) to apply them. The other comments describe the fix in prose because it touches several files or lines outside the diff.

Blocking

🔧 wrench

  • US without a region now fails closed on every adapter (Cloudflare US auctions and EC blocked): see inline at crates/trusted-server-core/src/consent/jurisdiction.rs:71-74
  • Geography-to-consent wiring is untested (mutation survives): see inline at crates/trusted-server-adapter-axum/src/platform.rs:166
  • Native startup drops EdgeZero's environment diagnostic (suggestion): see inline at crates/trusted-server-adapter-axum/src/main.rs:60
  • Use the full TRUSTED_SERVER__ env prefix, one name everywhere: see inline at crates/trusted-server-adapter-axum/src/native_geo.rs:14
  • CloudFront tests never run in hosted CI: see Cross-cutting below
  • Failed CI check skipped the integration and browser jobs at this head: see Cross-cutting below

Non-blocking

  • ♻️ One core helper for geo response headers (Axum duplicates the clearing, Fastly's None branch skips it): see inline at crates/trusted-server-adapter-axum/src/middleware.rs:183-193
  • 🤔 Credential-absence assertions check the response, where the request header can't appear: see inline at crates/trusted-server-adapter-axum/tests/startup.rs:350-351
  • ⛏ derive_more::Display and error-stack for the new errors: see inline at crates/trusted-server-adapter-axum/src/native_geo.rs:27
  • ⛏ Imports inside test function bodies: see inline at crates/trusted-server-core/src/consent/jurisdiction.rs:236
  • ⛏ set_response_headers doc overclaims for Fastly (suggestion): see inline at crates/trusted-server-core/src/geo.rs:24
  • ⛏ Assertions without messages: see Cross-cutting below

Cross-cutting / body-level findings

  • 🔧 CloudFront tests never run in hosted CI. cargo test-axum (.github/workflows/test.yml, test-axum job) runs with default features. From the new tests it runs only compiled_out_geo_selection_fails_before_store_preparation_and_listen and invalid_geo_profile_fails_before_store_preparation_and_listen. Three tests, and the geo-cloudfront branch of selected_implementation_is_explicitly_validated, are only compiled by clippy-axum --all-features, never run:

    • managed_geo_is_partial_immutable_per_request_and_finalized_on_auth_errors
    • admitted_body_error_keeps_managed_geo_without_losing_framework_status
    • nonmatching_real_socket_peer_cannot_supply_geo

    They pass locally with cargo test -p trusted-server-adapter-axum --features geo-cloudfront. Please add that as a step in the test-axum job.

  • 🔧 prepare integration artifacts failed, so the downstream jobs never ran at this head. The failure is in actions/upload-artifact: Failed to CreateArtifact ... Unexpected token '<', "<!DOCTYPE "... is not valid JSON after 5 attempts. The artifacts built fine, so this is GitHub-side. As a result, integration tests, integration tests (Fastly EC lifecycle) and browser integration tests were skipped. The previous approval covered affb0d0a4, before the two EdgeZero tag merges. Please re-run the workflow so those jobs execute at the current head. No required checks are configured on this base branch, so this is a gate AGENTS.md treats as required rather than one branch protection enforces.

  • ⛏ Assertions without messages. AGENTS.md asks for a descriptive message on every assertion. Examples among the new tests:

    • crates/trusted-server-adapter-axum/src/platform.rs:241-245
    • crates/trusted-server-core/src/platform/mod.rs:305, :311-312
    • crates/trusted-server-core/src/consent/jurisdiction.rs:221-227, :247-248
    • crates/trusted-server-core/src/geo.rs:243-244, :251
    • crates/trusted-server-core/src/integrations/prebid.rs:6667
    • crates/trusted-server-core/src/integrations/aps.rs:2437
    • crates/trusted-server-adapter-cloudflare/src/platform.rs:893-894, :934
    • crates/trusted-server-adapter-axum/tests/startup.rs:342-344

CI Status

  • browser integration tests: SKIPPED (upstream artifact job failed)
  • integration tests (Fastly EC lifecycle): SKIPPED (upstream artifact job failed)
  • integration tests: SKIPPED (upstream artifact job failed)
  • prepare integration artifacts: FAIL
  • cargo check (cloudflare native + wasm32-unknown-unknown): PASS
  • cargo test (cross-adapter parity): PASS
  • cargo test (axum native): PASS
  • vitest: PASS
  • cargo test: PASS
  • cargo test (ts CLI, native) (ubuntu-latest): PASS
  • cargo test (ts CLI, native) (macos-latest): PASS
  • cargo fmt: PASS
  • format-typescript: PASS
  • docs-proposal scripts: PASS
  • format-docs: PASS
  • CLAUDE.md symlink guard: PASS
  • cargo check/build/test (spin native + wasm32-wasip2): PASS

No checks are marked required on baseline/protected-geo-20261110.

Comment thread crates/trusted-server-core/src/geo.rs Resolved

@dhruv8sh dhruv8sh left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by a follow-up review with inline comments: #1532 (review)

@dhruv8sh dhruv8sh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Makes GeoInfo detail optional (city, continent, coordinates, metro), adds the opt-in native CloudFront Geo boot profile for Axum, and tightens jurisdiction detection. The partial-geo plumbing into OpenRTB/APS/Prebid and the response headers looks right and is well tested. The jurisdiction change, though, reaches every adapter, not just CloudFront.

1 of the inline comments below carries a one-click GitHub suggestion. Use Commit suggestion to apply it. The other comments describe the fix in prose.

Blocking

🔧 wrench

  • US geography without a region now resolves to Unknown on every adapter — see inline at crates/trusted-server-core/src/consent/jurisdiction.rs:72-74

Non-blocking

🤔 thinking

  • CountryCode::new now gates every adapter — see inline at crates/trusted-server-core/src/consent/jurisdiction.rs:54-56

⛏ nitpick

  • New assertions lack "should ..." messages — see inline at crates/trusted-server-core/src/consent/jurisdiction.rs:221-230

Cross-cutting / body-level findings

  • 🌱 Stale x-geo-* headers when geo is absent on Fastly — Axum's apply_finalize_headers (trusted-server-adapter-axum/src/middleware.rs:146-155) now strips every upstream x-geo-* header before writing, and GeoInfo::set_response_headers clears them when geo is present. Fastly's apply_finalize_headers (trusted-server-adapter-fastly/src/middleware.rs:201-208) only sets x-geo-info-available: false when geo is None, so origin-supplied x-geo-city etc. pass through next to false. This was already the case before this PR, but the PR now handles the two adapters differently. A follow-up could share the clearing step.
  • ♻️ native_geo errors bypass error-stack — load_selection / parse_selection (trusted-server-adapter-axum/src/native_geo.rs) return a bare Result<_, NativeGeoProfileError>. AGENTS.md asks for Report<E>. The bounded, redacted categories are a good design and can stay; wrap them in Report with change_context at the call site so the error stays consistent with the rest of the crate.

CI Status

All 25 reported checks PASS: worker-deadlines (spin), worker-deadlines (cloudflare), vitest, ts dev lint domains (changed lines), trace runtime and browser acceptance, release scripts, prepare integration artifacts, integration tests (Fastly EC lifecycle), integration tests, HTML EOF runtime (Fastly), format-typescript, format-docs, Fastly response diff script, docs-proposal scripts, CLAUDE.md symlink guard, CI Node pin guard, cargo test (ts CLI, native) (ubuntu-latest), cargo test (ts CLI, native) (macos-latest), cargo test (cross-adapter parity), cargo test (axum native), cargo test, cargo fmt, cargo check (cloudflare native + wasm32-unknown-unknown), cargo check/build/test (spin native + wasm32-wasip1), browser integration tests. Branch protection reports no required checks for this base branch.

aram356 added a commit that referenced this pull request Nov 12, 2026


Per the user, every feature is built once and works on every adapter in
platform order, with platform gaps sent to EdgeZero (DEC-16, SH-5). T21
moves the identity routes, EC lifecycle, providers and write-back into one
router in core that each adapter mounts; A2, A10, T8, T9, T17 and T18
shrink to turning it on per adapter and passing the conformance suite, and
T7, T14 and A11 fold into A2 and A10. EZI-2a adds EdgeZero's local store
for Axum development and EZI-4 binds Axum's geo provider from the manifest.

Restack the Axum RC units on #1506: #1531 carries A1 in place of #1528,
#1533 and #1534 close as copies of #1525 and #1526, and the image and
Terraform units sit directly on #1532 so A6 and A7 stop waiting on A2.
Record #1566, refresh the provider stack's fixes at their current heads,
and flag #1535 as unplanned.
aram356 added a commit that referenced this pull request Nov 12, 2026
Per the user, the RC line is forward-ported to main (DEC-6). Rechecking the
proposed stack against DEC-3 and DEC-16: #1536 and #1537 are mostly
Axum-only stores, store selection and wiring that EZI-2b, T6 and T21
replace, so A2 becomes a new pull request that takes their tests, and both
close with #1535 once their code moves. The image and Terraform units
restack on #1543, which publishes the image and shares .dockerignore with
#1539. #1531 goes on #1506 for A1 and #1532 on #1531 for #1529.
@aram356

aram356 commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

CREATOR: From the server-side identity plan (#1559, c7dc9de77): When this unit is ported to main: drop the core change that maps a US request with no region to Unknown, which fails every US Cloudflare request closed (AX-3), and show that the same trusted country and region reach EC creation and auction consent on Axum. All units: replace TS_* names (AX-2).

Part of #1529

@ChristianPavilonis
ChristianPavilonis removed this pull request from stack #1564 November 13, 2026 18:32
@aram356 aram356 linked an issue Nov 13, 2026 that may be closed by this pull request
@aram356
aram356 marked this pull request as draft November 13, 2026 19:47
Restore the reviewed shared jurisdiction policy and add Cloudflare compatibility coverage. Prove managed Geo reaches auction consent and bid requests, and capture credential stripping at the publisher origin.

Run protected Geo tests in CI, retain default-off coverage, and remove duplicated documentation. SDK integration and native EC lifecycle work remain deferred.

Refs #1529.
Use typed redacted reports while preserving existing profile and SDK startup messages. Add diagnostic and report-redaction regression coverage.

Clarify optional Geo header guarantees and add descriptive messages to existing Geo assertions without changing their predicates. Profile retirement and shared header clearing remain deferred.

Refs #1529.
@ChristianPavilonis

Copy link
Copy Markdown
Collaborator Author

Review follow-through at 7d6b7a1b9

All 12 inline threads now have a fix explanation or an explicit agreed deferral and are resolved. Re-review is requested from aram356 and dhruv8sh. The PR remains draft; no reviewer verdict was dismissed.

  • 60617bd restored the shared consent policy, added actual bidder/publisher capture tests, moved the cited imports and enabled the Geo feature tests in CI.
  • 7d6b7a1 handles the remaining error-stack/derived Display conventions, preserves SDK diagnostics without retaining raw inputs, clarifies the Geo-header documentation and finishes descriptive Geo assertions.
  • The summary-only CI concerns are accounted for: the protected CloudFront step executed successfully at 60617bd, and all 25 checks passed there, including the previously skipped artifact/integration/browser jobs. Hosted workflows for this head are still pending or running; their results are not counted as local proof.
  • The aram356 summary and superseding dhruv8sh summary are covered by those changes and the individual replies.

Local checks passed for Core, all native adapters, parity, Fastly, Axum default/Geo/all features, all eight Clippy lanes, formatting and changed-lines domain lint. The final independent review found no findings. Existing ignored cases were not relabeled as passes. The temporary build-cache quota failures were corrected before the final checks.

Still deferred, not claimed implemented: retiring the TS profile through #1573 and EdgeZero #482/#491, the shared Geo-header clearing follow-up, actual native EC-creation proof and final main-bound integration. These dispositions also account for the CREATOR main-port note. #1529 stays open; no live AWS/deployment qualification or merge approval is implied.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extract the protected CloudFront native Geo integration

4 participants