Skip to content

Expose JA4 through a standard request interface - #469

Open
dhruv8sh wants to merge 12 commits into
mainfrom
spec/283-ja4-request-interface
Open

dhruv8sh wants to merge 12 commits into
mainfrom
spec/283-ja4-request-interface

Conversation

@dhruv8sh

@dhruv8sh dhruv8sh commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Adds a provider-neutral JA4 TLS fingerprint interface (Expose JA4 through a standard request interface #283): ctx.ja4() -> Option<&CapturedJa4>, backed by the adapter-owned RequestIngress metadata from Add pre-dispatch hooks and explicit request ingress fidelity #403. Today the only way to get JA4 is the Fastly-only raw-request hook, and apps can't tell "this platform never supplies JA4" from "this request has none".
  • Fastly (get_tls_ja4()) and Cloudflare (Bot Management ja4) populate it. Spin reports NotExposed because it never sees the TLS handshake. Axum reports NotExposed by default. It can instead read JA4 from a TLS-terminating proxy's header (FoxIO's nginx module, HAProxy, ATS), but only when the direct TCP peer is in an operator-configured CIDR allowlist. List only proxies you operate: shared provider ranges such as CloudFront's would let any of that provider's customers pass the check, so CloudFront needs Add pluggable native Geo providers and protected CloudFront ingress #439's protected-CloudFront check instead. Client-supplied headers are never trusted. The Fastly raw-request hook can't overwrite the value; pre-dispatch hooks and middleware receive &mut Request and can (spec §5.7).
  • Includes the design spec, the implementation plan, the implementation, tests on every runtime, and docs. Only JA4 (BSD-3-Clause) is in scope; the other JA4+ methods are under the FoxIO License 1.1.

Changes

Crate / File Change
crates/edgezero-core/src/ja4.rs (new) Ja4Fingerprint (strict 36-byte a_b_c validation, summary / cipher_hash / extension_hash, redacted Debug, no Display/Serialize), CapturedJa4 (Captured { fingerprint, source } or Unavailable(reason)), Ja4Source::RuntimeTls, Ja4Unavailable::{Malformed, Missing, NotExposed}, CapturedJa4::from_runtime_tls, Ja4Source::TrustedProxyHeader, CapturedJa4::from_trusted_proxy_header.
crates/edgezero-core/src/ingress.rs RequestIngress gains with_ja4 / ja4(). None means the adapter made no claim. Debug shows the state, never the value.
crates/edgezero-core/src/context.rs RequestContext::ja4().
edgezero-adapter-fastly capture_request_ingress records get_tls_ja4().
edgezero-adapter-cloudflare into_core_request records cf().bot_management().ja4().
edgezero-adapter-spin Records Unavailable(NotExposed).
edgezero-adapter-axum New ja4.rs: TrustedJa4Header (header name + CIDR allowlist via ipnet 2, already in the lockfile). Configured by EDGEZERO__ADAPTER__JA4_HEADER + EDGEZERO__ADAPTER__JA4_TRUSTED_PROXY_CIDRS, or AxumDevServer / EdgeZeroAxumService::with_trusted_ja4_header. Fails closed on partial config, invalid entries and /0. Without config it records NotExposed as before.
tests/fixtures/request-fidelity{,-runner} Hook reports a fixed ja4_state label. The spoof case sends ja4 and cloudfront-viewer-ja4-fingerprint headers. The runner asserts the per-platform state. Observations re-recorded; the only change is the new field.
docs/guide/adapters/*.md New "Client TLS fingerprint (JA4)" section in the overview (example, capability matrix, trust model, FoxIO attribution). The Fastly raw-request example now uses the H2 fingerprint. Cloudflare and Spin notes. New Axum section "JA4 from a trusted proxy" with an nginx example and a warning that the proxy must overwrite the header.
docs/superpowers/{specs,plans}/ Design spec and implementation plan with verification log.

Closes

Closes #283

Test plan

  • cargo test --workspace --all-targets: 1566 passed
  • cargo clippy --workspace --all-targets --all-features -- -D warnings on the pinned Rust 1.98.1, plus clippy on wasm32-unknown-unknown (Cloudflare), wasm32-wasip1 (Fastly) and wasm32-wasip2 (Spin)
  • cargo fmt --all -- --check, and for both fixture workspaces
  • cargo check --workspace --all-targets --features "fastly cloudflare spin"
  • WASM runtime tests: Fastly lib under Viceroy (98 passed); Cloudflare contract under wasm-bindgen-test-runner + Chromium (14 passed); Spin contract under wasmtime 48.0.1 (12 passed)
  • examples/app-demo workspace: 33 passed
  • Docs build: cd docs && npm run lint && npm run format && npm run build
  • Axum trusted proxy: unit tests for the peer matrix (IPv4, bare IP, IPv4-mapped IPv6, IPv6, untrusted, unknown), Missing / Malformed, and config rejection; a real-TCP dev-server test where a loopback client is captured under 127.0.0.0/8 and NotExposed under 192.0.2.0/24
  • Manual testing: edgezero demo (Axum) serves GET / → 200 with and without the JA4 config, and refuses to start with only one JA4 variable set or with 0.0.0.0/0
  • Other: ./scripts/test-request-fidelity.sh all against real Spin 4.1.0, Viceroy 0.21.0 and workerd. JA4 state is missing on Fastly and Cloudflare (local runtimes serve plain HTTP) and not_exposed on Spin, including the spoofed-header case. This covers Spin's into_core_request path, which the plain-wasmtime suite can't construct.
  • Live provider check (needs credentials and a deployment). Fastly's Captured/Malformed path can't be exercised locally (Viceroy can't build a TLS client request), so this is its only coverage; Cloudflare's is now covered by injecting cf.botManagement.ja4 in the browser contract tests. Fastly over HTTPS → Captured, over HTTP → Missing; Cloudflare zone with Bot Management → Captured, without → Missing. Also confirms providers emit lowercase hex.

Checklist

  • Changes follow CLAUDE.md conventions
  • No Tokio deps added to core or adapter crates
  • Route params use {id} syntax (not :id)
  • Types imported from edgezero_core (not http crate)
  • Store wiring goes through KvRegistry / ConfigRegistry / SecretRegistry (not the legacy single-handle setters) — see spec §6.6
  • New code has tests
  • No secrets or credentials committed

Specify a provider-neutral JA4 accessor (ctx.ja4()) backed by adapter-owned
RequestIngress metadata: Fastly and Cloudflare populate it from their trusted
runtime APIs, Spin and Axum report NotExposed, and no adapter derives JA4 from
request headers.

Refs #283
@dhruv8sh dhruv8sh self-assigned this Nov 12, 2026
aram356 added a commit to IABTechLab/trusted-server that referenced this pull request Nov 12, 2026
Per the user, ID-7 relies on a trusted JA4 where the adapter supplies one
and otherwise does not block record creation (DEC-18), and JA4 is an
EdgeZero abstraction, ctx.ja4(), filled per adapter: Fastly from runtime
TLS, Cloudflare from Bot Management, Axum from a trusted proxy header such
as CloudFront's, Spin never. Trusted Server reads JA4 only through it, so
T12 deletes the direct Fastly read. IABTechLab/edgezero#469's draft leaves
Axum NotExposed and defers the trusted proxy header to its section 13.1;
the plan asks for that section in R2.
@aram356

aram356 commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

CREATOR: From the server-side identity plan (IABTechLab/trusted-server#1559, c7dc9de77): Bring section 13.1, JA4 from a trusted proxy header recorded as Ja4Source::TrustedProxyHeader, into the work that ships in R2b. Axum's ID-7 needs JA4 from a hop that #409 or #439 checks, such as CloudFront's CloudFront-Viewer-JA4-Fingerprint; an unchecked peer stays NotExposed (EZI-3, DEC-18).

Part of #283

@dhruv8sh
dhruv8sh marked this pull request as ready for review November 13, 2026 16:15
@dhruv8sh dhruv8sh linked an issue Nov 13, 2026 that may be closed by this pull request
4 tasks
Add edgezero_core::ja4 with a validated, Debug-redacted Ja4Fingerprint
and CapturedJa4 provenance/absence states. RequestIngress carries an
optional capture and RequestContext::ja4() reads it.

Fastly and Cloudflare populate it from their runtime TLS APIs; Spin and
Axum report NotExposed. No adapter derives JA4 from request headers.

Includes the implementation plan and adapter documentation.

Refs #283
Add a fixed ja4_state label to the request-fidelity fixture hook and send
spoofed ja4 / cloudfront-viewer-ja4-fingerprint headers in its spoof case.
The runner asserts Fastly and Cloudflare report missing and Spin reports
not_exposed on the real local runtimes, covering Spin's into_core_request
path that the plain-wasmtime contract suite cannot construct.

Replace magic section offsets in ja4.rs with named constants shared by
parse and the accessors, use matches! byte ranges, share one FoxIO sample
constant across core tests, and move test imports to module level.

Refs #283
@dhruv8sh dhruv8sh changed the title Add JA4 request interface design spec Expose JA4 through a standard request interface Nov 13, 2026
Axum never sees the TLS handshake, but a TLS-terminating proxy in front of
it (FoxIO's nginx module, HAProxy, ATS, CloudFront) can compute JA4. Add
Ja4Source::TrustedProxyHeader and CapturedJa4::from_trusted_proxy_header,
and an Axum TrustedJa4Header that records the configured header only when
the direct TCP peer is inside an operator CIDR allowlist. Any other peer
stays NotExposed, so clients cannot forge it.

Configured with EDGEZERO__ADAPTER__JA4_HEADER and
EDGEZERO__ADAPTER__JA4_TRUSTED_PROXY_CIDRS, or programmatically via
AxumDevServer / EdgeZeroAxumService::with_trusted_ja4_header. Config fails
closed on partial settings, invalid entries and /0 networks. Default
behaviour is unchanged.

Spec §5.8, plan task 6 and the Axum guide document the design, the
operator requirement to overwrite the header, and an nginx example.

Refs #283

@prk-Jr prk-Jr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review

Summary

A well-structured design: one adapter-owned ctx.ja4() with explicit provenance and explicit "no fingerprint" states, and request headers are never trusted by default. The new Axum trusted-proxy path is careful and fails closed.

I'm requesting changes for three reasons:

  • The docs present CloudFront as a source for a plain IP allowlist, and any CloudFront distribution can pass that check.
  • The Fastly/Cloudflare capture path still has no test that could fail if it broke.
  • The separate JA4 proxy allowlist departs from the requested #409/#439 trust check, and that needs a maintainer decision.

😃 Praise

  • The Axum trust check is well protected. I applied five deliberate breaks, and each was caught:
    • Trust any peer: caught by the peer-matrix test and the real-TCP dev-server test.
    • Trust an unknown peer: caught.
    • Skip IPv4-mapped canonicalization: caught.
    • Allow /0 networks: caught.
    • Ignore the config in into_core_request: caught by the real-TCP test.
  • Config fails closed on partial settings, invalid entries and /0. A repeated header field is treated as Malformed rather than picking one.
  • Ja4Fingerprint::parse is tight. It uses named offsets, never echoes its input, and has good rejection cases.
  • The request-fidelity spoof case is strong real-runtime evidence. At a3afa231, the CI logs show ja4_state asserted in 16 Fastly, 12 Cloudflare and 13 Spin results, checked against the committed observations.json.
  • On every production Fastly path, the ingress snapshot is inserted after hook values are merged (request.rs:330, :453).

Findings

Blocking

  • 🔧 CloudFront is listed as a source for the IP allowlist, which any CloudFront distribution can pass. See inline on docs/guide/adapters/axum.md.
  • 🔧 The tests can't tell real Fastly/Cloudflare JA4 wiring from a constant Missing. See inline on crates/edgezero-adapter-cloudflare/tests/contract.rs.
  • ❓ A separate JA4-only proxy allowlist. aram356 asked for JA4 "from a hop that #409 or #439 checks". #409 adds EDGEZERO__ADAPTER__TRUSTED_PROXY_CIDRS, and this PR deliberately adds a second EDGEZERO__ADAPTER__JA4_TRUSTED_PROXY_CIDRS. Spec §5.8 explains why: it's a narrower claim, and it avoids depending on when #409 merges. That's reasonable, but operators would maintain two IP lists for the same proxies, and the lists can drift. It also departs from the stated requirement. @aram356, is the separate setting acceptable, or should this path wait for #409's CheckedProxyMetadata?

Non-blocking

  • 🤔 Every Fastly request now loads the full client TLS metadata bundle. See inline.
  • 🤔 An empty provider value is classified as Malformed. See inline.
  • ⛏ Some parser rules and the redaction of section a are not protected by tests. See inline.
  • 🤔 Spec §10 lists Fastly and Spin tests that weren't added. See inline.
  • ⛏ Nothing tests that run_app reads the JA4 environment variables. See inline.
  • 📝 The nginx warning misses the proxy_set_header inheritance trap. See inline.
  • ⛏ The overview matrix is slightly wrong for configured Axum. See inline.
  • 📝 The PR description says "app hooks can't overwrite the value". That holds for the Fastly raw-request hook only. Pre-dispatch hooks receive &mut Request and can replace extensions. The overview doc and spec §5.7 already state this correctly.

CI Status

  • Hosted checks for 304f5a42 were still queued when this review was written.
  • I ran these locally on 304f5a42:
    • cargo test -p edgezero-core --lib: 511 passed.
    • cargo test -p edgezero-adapter-axum --all-targets --features axum: 131 passed.
    • My local Rust is 1.99.1; the repo pins 1.98.1.
  • I did not re-run the wasm suites locally. Their code is unchanged since a3afa231, where they passed in CI.
  • No live Fastly or Cloudflare Captured result has been verified yet.

Comment thread docs/guide/adapters/axum.md Resolved
Comment thread crates/edgezero-core/src/ja4.rs Resolved
Comment thread crates/edgezero-core/src/ja4.rs Resolved
Comment thread docs/guide/adapters/axum.md Resolved
Comment thread docs/guide/adapters/overview.md Resolved
Both workspaces depend on edgezero-adapter-axum by path, so adding ipnet
to it requires their lockfiles to record the new edge. CI runs them with
--locked. No new packages are resolved.

Refs #283

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

One item, found while bringing this PR into rc/identity (#495). It is the only reason #495's CI is red.

The inline comment carries a one-click GitHub suggestion. Use Commit suggestion to apply it.

Blocking

  • 🔧 The Axum guide names EdgeZeroAxumService, which #275 removes (docs/guide/adapters/axum.md:350-351)

Comment thread docs/guide/adapters/axum.md Resolved
aram356 added a commit that referenced this pull request Nov 14, 2026
Brings #275 at e9ffaa8 and the refreshed heads of #499, #460, #406,
#407, #409, #414, #422 and #439. Each PR merged the one below it, so
this merge has no conflicts.

Two edits fit #469 to #275, as #469's open review asks:

- #469's ingress JA4 test calls into_core_request with one argument.
  #275 now passes the Axum outbound executor through that function, so
  the test gets an executor handle the way the other ingress tests do.
- The Axum guide names EdgeZeroAxumService, which #275 removes and
  check_outbound_legacy_api.sh rejects. The guide now names only
  AxumDevServer::with_trusted_ja4_header.
EdgeZeroAxumService does not exist once this meets #275, and #275's check_outbound_legacy_api.sh rejects the name in docs/guide. AxumDevServer::with_trusted_ja4_header is public on both main and #275.
@dhruv8sh
dhruv8sh requested review from aram356 and prk-Jr November 16, 2026 10:01
dhruv8sh added a commit that referenced this pull request Nov 16, 2026
Conflicts resolved by keeping rc/identity's layout and porting the #469
test changes to where those tests now live:
- core ingress JA4 redaction assert -> ingress/fidelity.rs
- Cloudflare cf.botManagement.ja4 test -> tests/ingress_fidelity.rs
- Axum run_app env test -> capture_dev_environment unit test (the
  serve_app refactor from #469 does not apply to rc/identity's run_local)

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Review of 61fd14d7274b694aca1d3cb18ea1a219d6dacefb against e11ea3082f66ec864aabdd6007b15c4926c73b82.

Summary

The provider-neutral capture model is sound, and the trusted-proxy path works with a correctly configured FoxIO nginx module. Requesting changes for the reproducible mapped-IPv6 allowlist mismatch; the other inline comments are non-blocking.

Praise

😃 Clear capture semantics and meaningful regression coverage. Explicit unsupported, missing and malformed states, redacted fingerprint debugging, the Cloudflare injected-metadata test, and Axum's trusted/untrusted TCP tests provide a strong foundation. Real nginx-to-Axum tests also confirmed that TLS 1.2/1.3 fingerprints reach ctx.ja4(), client assertions are replaced, and plain HTTP reports Missing when the module is installed.

Existing discussion

📝 Checked-ingress scope remains an existing maintainer question, not a new duplicate thread. The previous review asks whether the separate JA4 allowlist is acceptable. The linked trusted-server design at c7dc9de77 explicitly requires Axum JA4 from a hop EdgeZero's checked ingress trusts (EZI-3); its plan names #409/#439 and CloudFront. This PR deliberately provides an independent operator-owned-proxy path and excludes protected CloudFront. I found no explicit maintainer answer reconciling that scope with R2b.

Verification

All 29 changed files and the relevant unchanged adapter entrypoints/converters/tests were reviewed. The linked identity design and implementation plan were read.

  • PASS: cargo fmt --all -- --check
  • PASS: cargo clippy --workspace --all-targets --all-features -- -D warnings
  • PASS: cargo test --workspace --all-targets — 1,567 passed, 1 ignored
  • PASS: cargo check --workspace --all-targets --features "fastly cloudflare spin"
  • PASS: cargo check -p edgezero-adapter-spin --target wasm32-wasip2 --features spin
  • PASS: Fastly WASM library tests under Viceroy 0.21.0 — 98 passed; Fastly WASM contracts — 7 passed
  • PASS: Cloudflare browser contracts with the matching wasm-bindgen runner — 15 passed
  • PASS: Spin contracts under Wasmtime — 12 passed
  • PASS: locked request-fidelity runner tests — 20 passed, 1 ignored; formatting checks for both fixture workspaces
  • PASS: ./scripts/test-request-fidelity.sh all — all 56 probes matched the committed observations, including documented unsupported outcomes
  • PASS: locked app-demo workspace tests — 33 passed
  • PASS: locked reusable fixture-axum compilation
  • PASS: docs npm ci, npm run lint, npm run format, npm run build
  • PASS: git diff --check; generated-file comparison confirmed that the three lockfiles change only the Axum ipnet dependency edge and observations change only the 41 expected JA4-state fields
  • Additional local wire probes covered real nginx TLS, spoof replacement, an untrusted TCP peer, mapped-IPv6 policies, and nginx inheritance/module-absence negative controls.

Verification limits

📝 Live-provider acceptance remains unverified. No Fastly or Cloudflare deployment was performed; local/browser tests do not establish deployed TLS output or Bot Management entitlement. The full transport replay initially stopped during npm installation because sharp detected the host's global libvips; its supported SHARP_IGNORE_GLOBAL_LIBVIPS=1 switch resolved that without dependency changes. Mise shim limitations were bypassed using matching installed runners. Current-head hosted checks pass. Scratch verification sources/binaries and servers were cleaned up, and the worktree is clean.

/// Whether `peer` is a configured proxy. IPv4-mapped IPv6 peers from
/// dual-stack listeners match their IPv4 networks.
fn trusts(&self, peer: IpAddr) -> bool {
let canonical = peer.to_canonical();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 P2: Mapped-IPv6 allowlist entries cannot match their own peer. Only the peer is canonicalized; configured networks retain their IPv6 representation. TrustedJa4Header::parse("x-ja4", "::ffff:127.0.0.1") succeeds, but a request whose actual TCP peer is [::ffff:127.0.0.1]:<port> becomes IPv4 before IpNet::contains, so its header is ignored and JA4 reports NotExposed.

I reproduced this through the real Axum listener with both the bare mapped IP and ::ffff:127.0.0.1/128. Changing only the policy to 127.0.0.1/32 produced Captured { source: TrustedProxyHeader, .. }. These are accepted configurations that silently fail to trust the exact peer they name.

Suggested fix: normalize mapped configuration entries consistently with peers, preserve original-peer matching where appropriate, or explicitly reject unsupported mapped notation at configuration time. Add bare-IP and CIDR regression tests alongside the existing mapped-peer/IPv4-policy case.

/// Trust `header` only from peers inside `trusted_proxies`.
#[inline]
#[must_use]
pub const fn new(header: HeaderName, trusted_proxies: Vec<IpNet>) -> Self {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Clarify the unchecked constructor policy (non-blocking). new stores the typed list verbatim, whereas parse rejects an empty list and /0 networks. A programmatic new(..., vec!["0.0.0.0/0".parse()?]) therefore permits capture from every IPv4 peer, and neither service builder revalidates it.

There are no current in-repository callers using this bypass, and the typed constructor may intentionally permit operator-selected policies, so I do not consider this a demonstrated blocking defect. Suggestion: document that new intentionally omits the parser's policy checks, or share validation between the public construction paths if the restriction is intended to apply uniformly.


Handlers and middleware read the client's JA4 fingerprint through one
provider-neutral accessor, `RequestContext::ja4()`. Adapters record it on
`RequestIngress`; it is never derived from request headers.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⛏ Update the remaining unconditional no-header claims (non-blocking). This sentence contradicts the opt-in Axum trusted-proxy path documented below. The same stale claim remains in the core JA4 module documentation and in the spec/implementation plan.

Suggested wording: JA4 is never inferred from unchecked client-supplied headers; Axum may capture an explicitly configured header from a trusted proxy peer. This preserves the actual trust guarantee without describing the implemented proxy path as impossible.

omits it when the value is empty:

```nginx
proxy_set_header X-JA4 $http_ssl_ja4;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Verify that the FoxIO module is loaded; nginx syntax validation alone is insufficient (non-blocking). I built nginx 1.30.0 with FoxIO's module at 116d2b27a63c8b3251b1f03a0b81758ad6b754fe: this directive correctly replaced client assertions with computed JA4 on TLS 1.2/1.3 and removed the field on plain HTTP.

A stock-nginx negative control exposed an operational trap: without the module, nginx -t still accepts $http_ssl_ja4, but nginx resolves it as the ordinary incoming Ssl-Ja4 header. Sending a valid fingerprint in that client header then caused the trusted upstream Axum request to report Captured.

This violates the example's module-installed precondition rather than exposing a defect in the correctly configured path. Suggestion: explicitly warn that a missing/unloaded module can silently turn this variable into client input, and recommend a deployment smoke test proving that client Ssl-Ja4 cannot determine ctx.ja4(), including over plain HTTP.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose JA4 through a standard request interface

4 participants