Skip to content

MLP phase 3: merge-base Beacon, ASIC 1 GHz, typed fence, CTA any-slot placement, SimX fidelity - #478

Open
tinebp wants to merge 10 commits into
masterfrom
feature/mlp-phase3
Open

tinebp wants to merge 10 commits into
masterfrom
feature/mlp-phase3

Conversation

@tinebp

@tinebp tinebp commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Ten commits on master 26d2bc4. Each commit message gives its own design notes, Beacon deltas and measurements.

Commits

Commit Change
logic_gate: compare against the merge base, add ASIC Beacon, tie lint Beacon gates every build against the merge base, not against baselines recorded on the PR's own RTL. Adds an ASAP7 flow (--tool beacon_asic) and a harness tie count.
pe_serializer: select the bypass buffer through an OUT_BUF code, drop OUT_SKID One parameter instead of two. Same netlist.
tcu: assert the TCU_LD ordering that the per-block AGU relies on Simulation-only assert. No area.
asic_gate: raise the target to 1 GHz and close timing on gfx and tcu All 14 ASIC DUTs pass at 1 GHz. earlyz splits its Z sum (+128 flops); ASIC builds use the 2-stage TFR multiply.
cache flush: one typed fence operation from runtime to every cache level fence_op = {ACQ, REL, SYS, IO} replaces five encodings. Fixes a release fence that left dirty lines invisible to TEX/RTU/raster/OM/DXA next to a single-core write-back dcache.
cta_dispatch: place single CTAs in any free slot A single CTA takes the lowest free slot instead of waiting for the round-robin slot. Clusters keep aligned windows.
simx: issue a TCU metadata load's first request in the cycle the AGU takes it SimX timing follows the RTL.
simx: hold TCU admission on the result port's registered ready, as the RTL does SimX timing follows the RTL.
simx: model the dcache core-response path with the RTL's buffers and stalls SimX timing follows the RTL.
ldst_matrix: 8-byte offsets, unshared base group, issue yield, column-major uops Default-off extension. Default kernels are bit-identical.

Gate results (run locally on this tree)

Gate Result
Logic (Beacon, FPGA and ASIC flows, against the merge base) 14/14 pass in both flows
ASIC (Yosys + OpenSTA, 1 GHz) 14/14 pass
Perf (rtlsim, 49 cases) 49/49 pass; identical instruction counts; geomean -1.54 %
Model parity 89/89 pass
FPGA (Vivado 2024.2.2, U55C) 13/14 pass; core below target, see below
Verilator -Wall lint, XLEN 32 and 64 Clean on 16 configurations; SimX functional build passes at both XLENs

Perf, cycles against master:

  • raycast: nt16 -32.6 %, nt4 -12.1 %
  • rt_raycast: -3.6 % to -9.4 %
  • sgemm_tcu fp16: multi-core nt16 -2.3 %, wide-issue -1.3 %

All 16 measured perf changes bisect to the CTA placement commit; a 17th, draw3d nt16, moved by 5 cycles. Four cases are slower: fp16 128^3 +0.37 %, fp16 multi-core nt4 +0.17 %, draw3d +0.12 % and +0.05 %. The commit message shows that each comes from CTA admission timing, not a systematic cost. The fp16 config is 2.32 % faster in geomean over 10 sizes.

FPGA core

core builds at 286.9 MHz against its 300 MHz target (floor 294 MHz). This is not new with this PR: master 26d2bc4 builds core at 289.5 MHz with the same Vivado 2024.2.2.

A Vivado bisect over master puts the drop on #473 (fpu fields): its parent eef0dd1 builds at 296.6 MHz, matching the nightly; #473 builds at 281.0 MHz; #472 moves it to 289.5 MHz without touching the failing nets. The two netlists are nearly identical: #473 adds 61 LUTs of decode legality, the core shrinks overall (169455 -> 169280 LUT, -828 FF), and Beacon finds no path deeper. The new failing paths are four nets that already passed by only 0.006 to 0.016 ns on the parent: the divide and square-root serializer enables (fanout 800 to 1400 clock-enable pins inside the vendor IP), the SFU dispatch ready into the 1610-bit dispatcher queue output (fanout up to 2273), and the LSU scheduler empty flag. #473 moved their placement; it did not change them. The fix for these high-fanout enables goes to a separate change, so core keeps margin against any small netlist change rather than recovering one placement.

This PR's further 2.6 MHz is on the same nets.

Not included

A mem_scheduler change (one request ready for reads and writes) was dropped. It made writes wait on a full index buffer: sgemv nt16 +2.37 % in rtlsim, with no gain to offset it.

🤖 Generated with Claude Code

tinebp and others added 10 commits November 15, 2026 19:50
Beacon (logic_gate) stayed green through every synthesis-gate failure
the MLP merge caused. Its baselines had been recorded on the PR's own
final RTL, so the CI cell compared the head with itself (+0 % by
construction). It also ran only the Xilinx flow, where the STD FPU and
the ASIC multiplier never elaborate, and its verdict table was swallowed
by pytest's capture.

- --against-rev REV builds a reference from REV's RTL under the
  change's own flow and gates every build against it. On pull requests,
  ci.yml passes the merge base. References are cached by merge-base SHA
  and flow fingerprint, locked against concurrent runs, and trimmed to
  finished builds. A reference that fails to build is REF-FAIL, not a
  pass. NO-REFERENCE is reserved for a DUT absent at that revision. The
  `beacon-skip-reference` label is a maintainer-granted escape for flow
  changes the merge base cannot elaborate under.
- `--update-baseline --against-rev` writes the accepted regressions into
  the baseline. They lapse once the merge base carries the same entry.
- gate_asic runs Beacon's ASAP7 model over asic_gate.yaml's builds. It
  gates FF and cells, registers that got deeper against the reference,
  and a predicted-Fmax backstop 32 % below clock_mhz. The band comes
  from 84 design/target pairs against OpenSTA: any band in (25.1 %,
  38.5 %) has no false positive at 800 or 1000 MHz and still catches the
  core/top misses. Macros are reported, not gated.
- `beacon.py ties` counts the instance-input bits each DUT harness ties
  to constants, gated on any change. The tex wrapper that hid 128 LOD
  bits from Vivado reads 129 against 1.
- The verdict table streams into the CI log and the job summary, and
  the Beacon reports are uploaded as artifacts. p_fmax is advisory only.

Validated against known-bad and known-good trees: core/top DEEPER on
fdivsqrt nrm_reg (29 -> 75) and fma pipe_rnd1 (26 -> 51); cache FF
+17.3 %; lmem FF +16.2 %; old tex wrapper HARNESS; and vm FF +11 % and
gfx BRAM +15 % against the merge base. There are no false positives on
master. A forced reference failure fails the run, and a resumed trimmed
cache rebuilds.

Baselines recorded on this tree with the CI pins (CIRCT 892a65a01,
foldsim d8350bc0f): the new beacon_asic set (14 builds) and the beacon
set, which gains only tied_bits/tied_ports. Both gates pass 14/14.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… OUT_SKID

VX_pe_serializer had two parameters for one choice, OUT_BUF and a
separate OUT_SKID flag, and every FPU core passed OUT_BUF (0) with
OUT_SKID (1). OUT_BUF already encodes the output buffer kind (size,
register depth, LUTRAM), so the bypass buffer becomes one more code
in that encoding: bit 4, `OUT_BUF_BYPASS = (16|1)`. A module without
a bypass buffer reads the low bits and falls back to one registered
entry.

The FPU cores pass `OUT_BUF_BYPASS. The generated netlist is the same:
the same VX_bypass_buffer under the same g_out_skid generate block,
whose nets the synthesis baselines record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moving the TCU_LD AGU into each TCU block (fc0a35b) dropped the
gather buffer: load responses now write the metadata RAM directly,
while MMAs read it. That is safe only if no MMA of the loading warp
that reads the loaded region (sparse or MX metadata) is presented to
the core while the load is in flight. Older MMAs have left the in-order
execute stream; younger ones wait on the scoreboard.

A simulation-only assert now checks it, qualified on a valid MMA and a
busy AGU. sgemm_tcu_wg_sp-4 passes with it in rtlsim. With sparse MMAs
no longer waiting on the TCU_LD's scoreboard bit (a temporary decode
fault), the same case fails on the assert at cycle 91283. No area: the
check is outside synthesis.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ASIC gate's 800 MHz target sat well below what most DUTs reach on
ASAP7. Raising it to 1 GHz (floor 980 MHz) left two misses, both fixed at
their source:

- gfx 971.7 MHz: VX_raster_earlyz's EVAL1 formed za*2qx + zb*2qy + zc in
  one cycle. zc now goes into four per-pixel offsets
  (zc + i0*za + i1*zb), computed in parallel, and EVAL2's per-pixel sum
  becomes a two-input add. Results are bit-identical, and instruction
  and cycle counts are unchanged; the cost is 128 flops. gfx: 1046.8 MHz.
- tcu 874 MHz: VX_config.toml gave ASIC builds the simulation-only
  single-cycle TFR multiply (tcu_latency_tfr 4). ASIC now uses the
  2-stage multiply that FPGA builds already use (latency 5, a product
  register): tcu 1101.7 MHz, tensor 1125.0 MHz, tcu area +2.6 %.
  Simulation and FPGA builds are unchanged. The latency-5 path passed
  rtlsim on fp8, bf8, fp16, sparse, WGMMA and sparse WGMMA.

All 14 ASIC DUTs pass at 1 GHz (tfu 1031.5, top 1040.4 and core 1036.3
are the closest). The yosys baselines are re-recorded at the new target,
because clock_mhz is part of each build's config hash.

Beacon is re-recorded against the parent with --against-rev, accepting two
deltas:
- gfx: earlyz's new sum register `quad_zprod` at 14 levels (the unit's
  deepest path drops from 16 to 14);
- tcu/tensor ASIC: FF 47,446 -> 66,855 from the product register.
The ASIC Beacon thresholds follow the new clock.

The toml edit changes the generated config headers, and with them every
perf case's config hash, although no simulation build sees ASIC defined.
The 49 perf entries keep their cycle and instruction counts; only
config_hash is refreshed, and only where the stored hash matched the
parent's config.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The work a fence or flush owes the caches was encoded five ways: the
runtime's CP_FLUSH_FLAG_L1_ONLY (two copies), bit 16 of the CACHE_FLUSH
DCR write, byteen bit 0 of the core flush token, the byteen/is_addr_io/
is_noalloc overloads of the LSU fence token, and three bools in SimX.
The token kinds were readable only through a 5-row table, each level
re-derived the kind it forwarded, and the names described cache levels
("L1-only") rather than meaning.

As in the PTX, Vulkan and HSA memory models, the operation is now
ordering x scope: fence_op = {ACQ, REL, SYS, IO}, defined once in
VX_types.toml. The same field rides every hop: the CMD_CACHE_FLUSH
header flags [5:2], the CACHE_FLUSH DCR write value, the flush token
(in amo_op, with amo_valid 0; no new wires), and SimX. Each cache
applies one rule: above the scope's point of coherence an acquire
walks, a release drains, and the level forwards the same operation;
the point of coherence answers. ACQ_FORWARD becomes the structural
NEXT_IS_POC. A cache flush is ACQ|REL at device or system scope;
asserts in the CP, the DCR receiver and SimX reject anything else.

Every single-kind fence and flush does the same cache work as before.
Two multi-token rounds change: IO plus a memory fence now drains the
point of coherence instead of walking it, and full plus IO keeps the IO
bit, so every level waits for its non-cacheable path.

Also fixes a release fence on a write-back level above the point of
coherence (a single-core dcache next to TEX, RTU, raster, OM or DXA):
it only quiesced, leaving dirty lines that those memory clients could
not see. It now walks, in RTL and SimX.

Without the A extension the LSU has no amo_op lane field, so the op
crosses the LSU in is_noalloc/is_addr_io and VX_lsu_mem_path restores
the typed field before the request leaves; a dedicated field would cost
two flip-flops. Beacon against the parent: core -23 LUT, cache -12 LUT,
gfx -55 LUT -9 FF, core without A -18 LUT, tensor -25 LUT; no FF
increase and no path deeper. The gfx and tensor harness tie counts
drop (removed flush_l1_only ports), so their FPGA and ASIC Beacon
baselines are re-recorded against the merge base.

The VX_types.toml edit changes the generated config headers, and with
them every perf case's config hash. The 49 perf entries keep their cycle
and instruction counts; only config_hash is refreshed.

The cp_dcr_proxy unittest encodes flushes with the new constants (user
approved).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A single-CTA launch now takes the lowest free usable slot (priority encode of
the slot map), the way a GPU places a CTA wherever an SM has room, instead of
waiting for the round-robin slot. Clusters with K > 1 keep their window at the
round-robin pointer, which only cluster members advance, and keep the window
alignment rules. The slot map updates as one set/clear mask, decoded from
registers, so the late base_zero only selects between decoded masks.

SimX follows the RTL: the slot pointer and window size persist across
launches, as the RTL's tail and window registers do, and clear only on a
device reset (launch_resetting). The device resets once; a launch keeps cache
lines and skips the init walk. A CACHE_FLUSH decides whether to walk with the
typed fence rule (owes_walk): levels above the point of coherence walk and
drop their lines, and the shared LLC answers a device-scope flush without a
walk. The runtime's device-scope flush ahead of every launch after a kernel
keeps the kept lines coherent.

Beacon vs parent: core LUT 143953 -> 143935, FF -11, deepest path 15 -> 15;
the dispatcher's slot_valid_r and tail_r paths 8 -> 7 levels; no register
deeper.

rtlsim cycles over the 49 perf cases, identical instruction counts, geomean
-1.54 %: raycast nt16 -32.6 %, nt4 -12.1 %; rt_raycast -3.6 % to -9.4 %;
sgemm_tcu fp16 multi-core nt16 -2.3 %, wide-issue -1.3 %; sgemmx and raster
nt16 -0.4 %; four more faster by under 0.05 %; 32 cases unchanged. Four
are slower, each by chance alignment rather than a systematic cost:
- sgemm_tcu fp16 128^3, 8 threads, one core: +0.37 % (245149 -> 246051).
  The config has 4 single-warp CTAs resident, one issue slice and one TCU
  block; the warp comes from the free-warp list, and the kernel uses no
  shared memory, so the slot number changes nothing else. 253 of 256
  admissions see exactly one free slot, so the slot choice is forced; only
  admission timing differs. Idle warp time between CTAs falls (1512 vs 4548
  warp-cycles), but the warps line up differently and collide more in the
  dcache banks (bank stalls 35920 -> 39647, misses 31925 -> 31454, load
  latency flat), so each CTA runs 0.9 % longer. Over 10 sizes of the same
  config the new placement is faster at 8 (64^3 -5.7 %, 192x192x128
  -3.8 %), slower at 128^3 (+0.37 %) and 128x128x64 (+0.19 %); geomean
  -2.32 %.
- sgemm_tcu fp16 on 2 cores x 4 threads: +0.17 % (219517 -> 219890). CTAs go
  to whichever core admits first; the split, exactly even before (167456
  instructions per core), ends 168110 / 166802, and the kernel ends with the
  busier core. The same kernel at 16 threads is 2.3 % faster.
- gfx_draw3d 4 threads, one core and 2 cores: +0.12 % and +0.05 %. The
  dcache counters are identical and load latency is flat; of the two
  kernels, the first runs 66 cycles faster and the second 700 slower.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…takes it

The RTL path from VX_tcu_agu through VX_tcu_unit and VX_lsu_mem_path to the
LSU scheduler's request queue is plain wiring behind the AGU's busy register,
so the next metadata request enters the queue three cycles after the previous
response lands. SimX sent the request at the start of the cycle after the AGU
took the load and the channel added one more, two cycles late. The extra cycle
let a warp's TCU_LD fall behind another warp's flw and reordered the warps from
there on.

agu_step is split: agu_collect takes responses at the start of the tick, and
agu_issue sends requests after the input loop. The metadata turnaround is now
the same in both models, and SimX's LSU and TCU requests leave in the RTL's
order. Only MX and sparse configurations (TCU_META_ENABLE) change.

model_parity, SimX vs rtlsim: mxfp8 7.18 % -> 0.2 % (5527 / 5516),
mxfp8-lsu-lanes 4.1 % (5256 / 5047), wgmma-sp-dxa 1.46 % (47743 / 47056).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e RTL does

VX_tcu_core admits a micro-op while ~fedp_done || result_if.ready, and that
ready is the lane-gather stream buffer's registered ready (in TCU_META builds
also low while the AGU presents a TCU_LD result). SimX instead held admission
when a result completed while its landing queue was non-empty. The TCU ticks
before the core's commit, so that count included the result commit takes in
the same cycle: SimX held micro-ops the RTL admits. In model_parity-mxfp8 one
warp-2 MMA micro-op committed at 22 cycles of latency instead of 7, which let
warp 3's micro-ops issue first and reordered the warps.

The core now sends each TCU block, one cycle ahead, the cycles its lane-gather
queue ends full; the block holds admission when a result completes on such a
cycle or while the AGU result is pending. A static_assert keeps one TCU block
per issue slot, which VX_tcu_unit requires. RTL is unchanged.

model_parity SimX vs rtlsim, CI LLVM: mxfp8-lsu-lanes -2.12 % -> -0.90 %,
sparse-fp16 -0.89 % -> +0.20 %, mxfp8 -0.92 % -> -0.97 %, fp16 -0.19 % ->
-0.34 %, wgmma-sp-dxa unchanged at 1.35 %; ~/tools LLVM: mxfp8-lsu-lanes
+4.14 % -> +1.57 %. MMA dispatch-to-commit latency now matches for 83 % of
micro-ops on mxfp8 (was 71 %).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stalls

Each RTL dcache port's responses cross the bank's 2-entry response skid, the
per-port arbiter's output register (VX_stream_arb OUT_BUF=1) and the bypass
switch's 2-entry skid (VX_cache_bypass OUT_BUF=3); a skid's ready is a
register, and a bank whose head cannot retire freezes its whole pipeline
(pipe_stall), input and fill forward included. SimX differed three ways:
- CacheRspMcast granted into a 2-entry channel the RTL does not have, which
  the cache tick then forwarded to the port;
- the bank response queue and the last output skid freed a slot in the cycle
  it drained, not the next;
- a bank whose head could not retire still took a new request into the empty
  stage behind it and still ran the fill forward.

The read-merge arbiter now grants straight into the cache's core-response
port with the existing delay, the bank response queues, tree slices and the
coalescer and lane-adapter inputs (the last skid, also in passthrough) have a
registered ready, and a head stall blocks that tick's input and fill forward.
RTL is unchanged.

In model_parity-mxfp8 the bank-1 request that entered a cycle early now enters
at the RTL's cycle, and SimX and RTL commit in the same order from cycle ~1097
to ~2247 (it broke at 1271). SimX vs rtlsim: parity-sgemm-mc64 +2.3 % ->
-0.06 %, fp16 -128 -> -77 cycles, mxfp8 unchanged, mxfp8-lsu-lanes -46 -> -70
(its next difference is the fill replay cadence), parity-vecadd and
sgemm-dcache-16-banks unchanged. SimX perf at perf_gate args: sgemv nt16
318580 -> 287142 (-9.9 %; RTL 285869), sgemm nt16 -6507, sgemv nt4 -183,
sgemm nt4 -57, vecadd nt16 +192, vecadd nt4 unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-major uops

Four fixes to the default-off matrix fragment load/store extension
(vx_ldm/vx_stm), each cutting the cost of the macro-op or its effect on
the other warps. The extension stays off by default.

- Encoding: the shape moves to imm[2:0] as an enumeration (codes 0-5;
  6 and 7 reserved) and the byte offset to imm[11:3]. Before, the shape
  took imm[4:0] and the offset imm[11:5], so the K tiles of an unrolled
  loop (8 bytes apart at the narrowest, a sparse A at NT=2 and 4) could
  not fold into the immediate, and decode had to map a 5-bit code to a
  3-bit index. tests/regression/ldst_matrix encodes the instructions by
  hand with .insn, so its shape constants follow the new codes, and it
  gains one offset check at the smallest 8-byte multiple of the column
  span, which the old encoding could not express. A reserved shape now
  trips a decode assertion in RTL simulation, where SimX already stops.
- vx_tensor.h ldm_group derives row groups 1..J-1 from the uncopied base
  with non-volatile asm adds. The compiler cannot strength-reduce them
  into J induction variables, the base enters the pinned group without a
  mv, and unrolled tiles with the same base share the adds.
- Every fragment uop but the last carries issue_yield, the rule wide
  ld/st already uses, so one warp's 8 uops no longer issue back to back
  while another warp waits. The plumbing in VX_ibuffer/VX_scoreboard
  moves under EXT_LDST_ANY.
- Uops run column-major: uop e takes row group = low bits of e and
  column = high bits, with the register given by the TCU permutation. A
  closed form in VX_uop_packld and fragment_layout.h replaces the search.

SimX A/B over 114 knob-off/knob-on pairs: geomean -1.3 % (was -0.3 %),
57 pairs still slower with the knob on (was 87). The commit-port
contention behind the rest stays open; the TCU-over-LSU commit priority
that removes it on TCU-bound configurations slows down shared-LSU ones,
so it is not part of this change.

Verified: SimX and rtlsim parity of the matrix ld/st cases
(parity-ldst-matrix, -notcu, model_parity fp16/fp16-u4/sgemm2/sp/wgmma
-ldm); default kernels build bit-identical; Beacon core knob off
identical, knob on nothing deeper, +158 LUT over the previous
encoding; Verilator lint clean at XLEN 32 and 64, knob on and off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tinebp
tinebp force-pushed the feature/mlp-phase3 branch from ecbf2e7 to 573964c Compare November 16, 2026 05:36

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant