[yugt]

What this adds

Rows in docs/SUPPORTED_GPUS.md for four data-center NVIDIA cards, none of which were in the table: A100 80 GB and A30 (sm_80), L40S (sm_89), H100 NVL and H100 80 GB (sm_90), each on CUDA and Vulkan. The raw numbers are in docs/datacenter-nvidia-results.md.

It also changes one test helper: is_default_model compared the whole path with DEFAULT_MODEL, so on any machine where the reference GGUF is not at /mnt/ollama/... the default-model checks were skipped (the 0.05 tolerance in gpu_forward_matches_cpu_reference, the type-coverage counts in tests/kernels.rs, the shapes in tests/quant.rs and tests/gguf.rs). It now compares the file name, so a copy of Qwen3.6-35B-A3B-UD-Q2_K_XL.gguf under STRATA_MODEL runs the same checks as on the reference machine. Nothing in src/ changes.

Validation

Rocky Linux 8.10, driver 615.71.09, NVRTC 13.0, Rust 1.99.1, one GPU per Slurm job, Qwen3.6-35B-A3B-UD-Q2_K_XL.gguf (fits in VRAM on all of them).

• STRATA_MODEL=$M cargo test --release --no-fail-fast -- --nocapture with the helper change: 185 passed, 0 failed, 5 ignored on A100, A30, L40S and H100 80GB HBM3. Both backends were open in every run (48 cuda device and 47 vulkan: device lines per log); the only skips are no glslc and STRATA_TEST_BACKENDS does not select cuda. H100 NVL ran the suite at 7264e57 without the change: same counts.
• rustingstrata check --backend vulkan: 96 pairs, largest 49152 of 49152 bytes, on each card.
• rustingstrata bench --gpu --pp 512,4096 --tg 128 --reps 3, no calibration, MTP off, pp 4096 prefill / decode in tok/s:

GPU	CUDA	Vulkan
A100 80 GB	6203 / 174	5589 / 132
A30	3544 / 132	3326 / 104
L40S	9030 / 213	8571 / 185
H100 NVL	9131 / 246	6428 / 192

The tests and benchmarks ran on 7264e57; the branch is rebased onto ddf33db and I did not rerun them after the rebase.

One thing I ran into: parallel tests can hang in vkDestroyDevice

One cargo test --release on the A30 stopped in tests/kernels.rs with the GPU idle. gdb shows a test thread in vkDestroyDevice joining a driver thread that spins in its thread-local destructor (libnvidia-eglcore.so.615.71.09), while the tests that start next wait on a loader mutex in vkCreateInstance. The trace is in the results doc.

Running the kernels test binary 15 times per card:

	A30	A100
parallel (default)	2 of 15 hung	3 of 15 hung
--test-threads=1	0 of 15	0 of 15

Every run that finished passed, so this is about test teardown, not results. I did not loop L40S or H100.

I also tried a process-wide mutex around device creation and destruction in src/vulkan/mod.rs (7 lines): 0 of 30 parallel runs hung with it, at the same run time. 30 runs is not much, and the patch as I have it would deadlock if Vk::new failed after the device exists, so it is not in this PR. I can send it separately if you want it.

🤖 Generated with Claude Code

[Vasyl-Trefilov]

Hi hi, looks very very interesting and cool, thank you so much for that. Looks nice and save, so I merge immediately

[Vasyl-Trefilov]

@yugt why haven't you calibrated? You would get much better performance, also can you please do me a favor and run some bigger models not just Q2 35b, if it's possible, pls run DeepSeekV4.1 1017 or just full precision Qwen3.6:35b-a3b model, this benchmark will help me a lot and with our hardware, you don't need to run Q2 as I do. Thanks again!

[yugt]

Thanks for the review. Done in #26:

• Calibrated all four cards and benched each before and after calibrate (plus --mtp 0), CUDA and Vulkan. Calibration mostly turns on MTP: decode after 512 on the H100 went 272 → 524 tok/s with Q2_K_XL, 298 → 532 with Q8_0.
• Bigger files: Q8_0 and full-precision BF16 of Qwen3.6-35B-A3B. BF16 fits on the A100 and H100 NVL (248 and 330 tok/s calibrated). On the A30 and L40S its experts do not fit, and calibrate picks MTP 0 there.
• DeepSeek-V4.1-Flash-1017 IQ2_XXS on an A100 and an H100 NVL: correct output, and 4.5-8.9 tok/s end-to-end decode once the experts are in the page cache, depending on the VRAM expert budget.

Tables and the caveats are in docs/datacenter-nvidia-results.md on that PR.

[Vasyl-Trefilov]

Wow, tysm, I still need to work on DeepSeek speed, bc your performance is pretty low for this hardware, but anyway, tysm again for benchmark, this is very helpful!