Skip to content

feat(axum): add safe diagnostics and trusted proxy metadata - #409

Open
ChristianPavilonis wants to merge 17 commits into
issue-393-buffered-body-limit-specfrom
issue-397-diagnostics-proxy-trust-spec
Open

ChristianPavilonis wants to merge 17 commits into
issue-393-buffered-body-limit-specfrom
issue-397-diagnostics-proxy-trust-spec

Conversation

@ChristianPavilonis

@ChristianPavilonis ChristianPavilonis commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Current review readiness — 2026-11-10

Ready for review at the publication head below. The required workspace and test aggregate have now passed in run42916832735; all 29 latest checks pass. Reviewers: aram356, prk-Jr, dhruv8sh.

Review transitions are independent of TS work and the combined integration audit; #462 remains draft. This current status supersedes historical draft-only prerequisites below without claiming unexecuted deployment/manual acceptance complete. All three reviewers are requested. Stack465 is the current grouping; #453 is excluded.

Current review-stack checkpoint

Publication head: 5bbff2bf6d6d853b470f76e7293b7ecfa55476f5. This cleanup preserves the existing feature unit and PR identity. The additional change is the completed generated-native build correction: image-only code now follows the scaffold's existing Fastly-example selection, with no request-policy change. Its focused generator/native proof is retained; fresh per-branch hosted checks are not assumed green.

Review order: #275 → #406 → #407 → #409 → #414 → #422 → #439 → #454. Draft integration checkpoint: #462. New SDK/group/codec/phase development and unrelated TS work are not prerequisites or additions to this PR. Earlier notes below retain their original tested revisions and limits; they are not a whole-tree pass for this new head.

Earlier implementation and proof notes

RC refresh checkpoint, 2026-11-10

Current head: 3b52b804e5570564cc95cdaaf4b9efc6307875ae.

Checked forwarding trusts the actual socket peer, validates the selected family and preserves original normalized-head accounting before stripping assertions and entering application callbacks. Diagnostics retain the existing terminal owner/time and redact raw values. Independent source review found no unresolved correctness defect.

The CLI-only diagnostics gate is fixed, with 57 CLI-only tests passing. Native ingress trust is now under edgezero_adapter_axum::trusted_proxy, leaving the current outbound legacy-API guard unchanged. Parent namespace proof passes 370 Axum unit, 28 contract and 32 production-hosting tests plus axum-only strict Clippy. Nested demo resolution matches existing exact workspace getrandom0.4.2/ipnet2.12.0 requirements; no manifest pin was relaxed.

This layer does not strip a future protected-CloudFront credential or prove a Geo reader identity.

Draft pending fresh hosted checks and the combined RC audit. Two unchanged native default-parallel deadline misses remain recorded and unclassified; serial passes and untouched-foundation parallel passes do not clear them. No timing bound was relaxed. No deployment, main merge, security-alert dismissal or review-thread resolution occurred. Historical design/review context follows.


Summary

  • Add opt-in, socket-peer-gated proxy trust and checked native ingress metadata. No proxies are trusted by default; managed admission strips both forwarding families.
  • Add safe correlated native records, request IDs, coherent work snapshots and an optional observer at the existing response terminal owner. Preserve application logging, completions, observers and custom rendering.
  • Use fixed native framework routing and JSON-deserialization error text, publish operator guidance, and add real proxy/TLS/socket regression coverage.

Changes

Crate / file Change
edgezero-adapter-axum Bounded literal policy/parser, owned metadata, strict settings, identity/counters/records, hosting integration and selected-status capture
edgezero-core Checked host witness, original normalized-head summary, timestamp-aware joined completion and bounded failure provenance; no report-shape or portable Tokio/Send change
Native fixtures Source-bound .2/.3 relays, verified test TLS with wrong-name control, 32 production-hosting cases, callback/panic/privacy/counter checks and defensive fallback tests
Docs Operator/migration guide, immutable reviewed specification and implementation/acceptance record

The approved JSON extension retains HTTP 400 and unmanaged serde detail. Custom renderers still own their response, but native framework JSON-deserialization errors now reach them as invalid JSON payload. Application-authored EdgeErrors are not globally sanitized. Managed raw-forwarding-header removal is documented as a compatibility change; the low-level converter remains a trust-policy bypass.

Dependency changes are four direct Axum edges to existing locked packages. Rustls and Tokio Rustls are dev-only. The published TLS private key is an explicitly public test asset, never a deployment credential.

Closes

Closes #397.

Stack and acceptance

Part of #391. This PR remains draft until the required integration and acceptance gates are satisfied.

Stacked above #407 using gh stack, at base 3308041b910b648af746c15e809ee86b96adc2e9. The user authorized implementation on this open stack. That exception does not satisfy accepted #275/#393 integration or authorize bypassing #394/#395 upload experiments. Accepted-stack, final-image and deployed-ingress acceptance remain deferred.

Test plan

Local checks passed for implementation commit 66a05e2:

  • cargo test --workspace --all-targets — 1,938 passed, one existing ignored generated-workspace compile smoke
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • cargo fmt --all -- --check and git diff --check
  • cargo check --workspace --all-targets --features "fastly cloudflare spin"
  • All three adapter WASM checks: Fastly wasm32-wasip1, Cloudflare wasm32-unknown-unknown, Spin wasm32-wasip2
  • cargo test -p edgezero-adapter-axum --all-targets --all-features — 255 library, 13 contract and 32 hosting cases
  • cargo test -p edgezero-core — 731 library tests plus integrations/doctests; 13 existing ignored doc examples
  • Locked docs install, lint, format and build; compiled guide API/options/snapshot smoke
  • Excluded examples/app-demo workspace
  • Final image and deployed proxy/provider acceptance

docs/superpowers/plans/2026-10-06-axum-diagnostics-trusted-proxy.md records commands, timings, historical failures, source fingerprints, all 22 specification rows and proof boundaries. Final runtime-input manifest SHA256: ac40934b17bb6c521951f15090e2accf3059abcb4d386c6da74a69d95de8c9d7.

Local wire tests directly prove owned proxy peers, consumer agreement, verified TLS, selected 200 plus failure after an observed head/prefix, hook/cardinality preservation and graceful/grace-forced work release. Missing-peer, exhaustive parser/identity bounds, precise injected timing and defensive fallback branches remain service/unit evidence. Environment-derived secret/config error privacy is tested; declared-store/backend and actual outbound-origin experiments are not claimed. Explicit diagnostic stop fixtures do not prove the convenience second-signal-plus-native-snapshot combination. The existing supported convenience second-signal test is retained.

Independent source reviews found and resolved lost Abort failure classification and formatting drift. A maintenance-only duplicated IPv6 bracket split remains unchanged; node and authority port grammars differ. Docs installation reports seven existing audit vulnerabilities, two moderate and five high; no docs package changes were made.

Checklist

  • Follows CLAUDE.md; shared Cargo/global formatting centrally scheduled
  • No portable Tokio dependencies or Send migration
  • Brace-style route parameters and core HTTP types
  • No exporters, public diagnostics endpoints, tracing propagation or new terminal owner
  • Application hook/logger/renderer ownership preserved
  • Test keys are explicitly public fixtures; no deployment secrets committed
  • Distinguish local proof from downstream acceptance; keep the PR draft until its merge gates are satisfied

TS cutover follow-up, November 11

Native hosting invokes application head preparation after structural validation and before stripping forwarding fields. Protected Geo paths also capture facts before credential sanitation; invalid heads still skip application preparation.

Current head: 3708ffa418778b8ec3499daad132b1a3f9c7962f.

Local combined-RC qualification at 36f8e9c9c2c93661aaa73f2ca956cec3bc9e4c92: workspace tests 2,947 passed / 6 ignored; all-feature Axum 568 passed, including 36 production-hosting, 6 AWS startup and 4 celld startup cases; core doctests 19 passed / 13 ignored; strict workspace Clippy, formatting, feature compilation and Spin WASM check passed under Rust 1.98.1. Narrow TS consumer proof under Node 24.20.0: all four real proxy tests and typed prepared config/secrets/router startup passed. This is not final TS-stack/image/deployment qualification. Fresh hosted checks are still required.

Owned follow-up review checkpoint

  • Branch advanced through an ordinary merge; every previous public head remains an ancestor.
  • Current head: b8c65667e643f77dfa1cd61b79cee5bd7a95f552. Source tree a461a3dfd5ffa17698d452c034c935f59f365bea is byte-identical to previous head 31f46bde222ce3e942c1c801966ecc053a0305c9.
  • Related owned draft: #499. It is intentionally a placeholder until the author cleans feat: implement portable outbound HTTP and request lifecycle contracts #275 and the owned baseline/follow-up are reconciled.
  • Review base and ready/draft state are unchanged. The feature-only review diff is byte-identical to the pre-update diff.
  • Existing qualification remains tied to its recorded inputs. New hosted outcomes are reported separately; this history update does not qualify new source-version-bound binaries or move a release tag.

@ChristianPavilonis
ChristianPavilonis force-pushed the issue-397-diagnostics-proxy-trust-spec branch from 66a05e2 to 9fa1666 Compare November 10, 2026 19:54

@prk-Jr prk-Jr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review

Summary

Reviewed head 31f46bde against stack base d6f74308. Since the previous review at 5bbff2bf, the only PR-specific change is the received-head preparer call in service.rs; every file carrying the earlier blocking findings is unchanged, and those threads have no replies. Requesting changes until they are addressed or answered.

Findings

Blocking

  • 🔧 Earlier blocking findings remain open at this head: the trust walk collapsing a client to the trusted proxy's address on a malformed prefix (trusted_proxy.rs:417), duplicate Host selecting the first value (trusted_proxy.rs:369), the dropped post-commit body-source failure category (response.rs:516), and the two open questions on redaction scope (context.rs:666, axum-diagnostics.md:427). Please fix or reply on each thread.
  • 🔧 Proxy/TLS hosting tests fail on macOS (reproduced): lo0 has only 127.0.0.1, so relay() cannot bind 127.0.0.2/127.0.0.3, and the bind error is swallowed. cargo test -p edgezero-adapter-axum --all-features --test production_hosting -- native_source_bound native_two_proxy native_verified_tls gives 0 passed / 3 failed, with misleading messages (response head at production_hosting.rs:390, TLS UnexpectedEof at :871). CI passes only because it runs on Linux. Suggestion: skip or gate these tests when the extra loopback addresses are not routable, and surface the relay error.

Non-blocking (inline)

  • 🤔 Guide pipeline omits the received-head preparer, which sees unchecked forwarding headers: service.rs:408

CI Status

  • Hosted checks at 31f46bde: 29/29 pass
  • New received_head_preparation_precedes_stripping_but_not_head_validation: PASS locally
  • Targeted proxy/TLS hosting tests on macOS: FAIL (3/3), as above. Local toolchain was rustc 1.99.1 rather than the pinned 1.98.1; the failure is an address-bind issue, not toolchain-dependent.
  • Full workspace gates not rerun for this head.

.await;
}
};
app.prepare_received_head(&parts.headers, &mut parts.extensions);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 The guide says application code runs after stripping, but the preparer runs before the trust check

prepare_received_head runs before normalize and strip_forwarding_headers, so the callback sees client-forgeable Forwarded / X-Forwarded-* even when the peer is untrusted. That is useful when the app authenticates its own credential first (as the new test does), and the rustdoc warns about it. However, the guide's flowchart and the "Remove both raw forwarding families" step (docs/guide/axum-diagnostics.md:89-98) still show application code running only after stripping. An app author reading the guide could trust X-Forwarded-Host inside a preparer without a credential check.

Suggestion: add the preparer step between head validation and proxy-policy matching in the diagram, and state that it receives unchecked headers and must authenticate before retaining any forwarded value.

aram356 added a commit that referenced this pull request Nov 14, 2026
aram356 added a commit that referenced this pull request Nov 14, 2026
Brings #275 at e9ffaa8 and the refreshed heads of #499, #460, #406,
#407, #409, #414, #422 and #439. Each PR merged the one below it, so
this merge has no conflicts.

Two edits fit #469 to #275, as #469's open review asks:

- #469's ingress JA4 test calls into_core_request with one argument.
  #275 now passes the Axum outbound executor through that function, so
  the test gets an executor handle the way the other ingress tests do.
- The Axum guide names EdgeZeroAxumService, which #275 removes and
  check_outbound_legacy_api.sh rejects. The guide now names only
  AxumDevServer::with_trusted_ja4_header.
# Conflicts:
#	crates/edgezero-core/src/app.rs
aram356 added a commit that referenced this pull request Nov 16, 2026
Brings #275 at b65e714 and the refreshed heads of #499, #460, #406,
#407, #409, #414, #422 and #439. Each PR merged the one below it, so
this merge has no conflicts. #409 resolved the one conflict on the way
up, in edgezero-core's app.rs imports, by keeping both.
aram356 added a commit that referenced this pull request Nov 17, 2026
Brings #275 at ea2549f and the refreshed heads of #499, #460, #406,
#407, #409, #414, #422 and #439, including #422's stock-celld store
service and its merge into #439. Each PR merged the one below it, with
no conflicts on the way up, so this merge has no conflicts.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Axum: add safe production diagnostics and trusted-proxy handling

4 participants