[yugt]

Eight more NVIDIA cards from a second Slurm cluster, each benchmarked before and after calibrate on CUDA and Vulkan, plus the test suite, and one card that does not run. Docs only.

Runs at f47e370, Qwen3.6-35B-A3B-UD-Q2_K_XL.gguf, one GPU per job with 8 cores and 48 GB RAM, cargo test with --test-threads=1.

pp 4096 prefill / decode, tok/s ("MTP off" is the calibrated run with --mtp 0; "calibrated" uses the MTP that calibrate chose):

Card (driver)	CUDA, MTP off	CUDA calibrated	Vulkan, MTP off	Vulkan calibrated	cargo test
A100-PCIE 40 GB (615.71.09)	5877 / 169	5510 / 275	5639 / 136	5267 / 223	185 passed
RTX A6000 (615.71.09)	5094 / 188	4814 / 329	5052 / 165	4782 / 252	185 passed
A40 (615.71.09)	4805 / 159	4579 / 289	4796 / 141	4539 / 222	185 passed
RTX 4500 Ada (615.71.09)	4934 / 153	4631 / 273	4979 / 142	4645 / 221	185 passed
Quadro RTX 6000 (615.71.09)	2795 / 122	2660 / 179	2719 / 109	2558 / 143	181 passed, 4 failed
RTX 2080 Ti 11 GB (615.71.09)	2714 / 140	2595 / 197	2630 / 126	1864 / 158	181 passed, 4 failed
V100-SXM2 16 GB (580.95.05)	3145 / 128	2965 / 193	1676 / 120	1598 / 170	181 passed, 4 failed
V100-SXM2 32 GB (580.95.05)	3118 / 121	2951 / 183	1665 / 116	1585 / 169	181 passed, 4 failed

The cards sit in hosts with different CPUs, so compare within a row. The model fits in VRAM everywhere except the 2080 Ti (8213-8712 experts resident, hit rate 99.7-99.9%).

Two things I ran into

1. Four Vulkan kernel tests fail on Turing too, not only on the V100. On the RTX 2080 Ti, the Quadro RTX 6000 and both V100s, f16_weight_uploads_as_f32, gemm_matches_reference_for_every_type, gemv_partial_superblocks_and_q2_0 and vulkan_type_matrix fail with relative errors around 5e-4. With STRATA_VK_F16_TRUNC=1 all four cards pass 185 of 185. The two Turing cards report cooperative matrices (f16 and int8) and run driver 615.71.09, the same driver on which the Ampere and Ada cards pass without the variable. So the truncating f16 conversion is not specific to the Volta driver; the comment at tests/kernels.rs:339 says it is. The Turing and V100 Vulkan rows say "Runs, test suite passes with STRATA_VK_F16_TRUNC=1" and note that the variable only changes what the tests expect.

2. A GTX 1080 Ti (Pascal, driver 580.95.05) runs on neither backend.

• --backend vulkan: no usable Vulkan device 0 (0 usable; NVIDIA GeForce GTX 1080 Ti: missing shaderFloat16).
• --backend cuda with NVRTC 12.6: the startup line is cuda device 0: NVIDIA GeForce GTX 1080 Ti, sm_61 (Volta, m8n8k4), then NVRTC compile of kernels.cu failed with default_program(2321): error: identifier "__nanosleep" is undefined.

The README said "Pascal and older use Vulkan" and SUPPORTED_GPUS.md said "Use --backend vulkan; not tested"; both now say what happens on this card, and the README's Vulkan requirements list shaderFloat16. cargo test on that card reports 185 passed only because every GPU test skipped (no usable Vulkan device, and that job had no NVRTC loaded), so I did not count it as tested.

Would you like a small separate PR for this: an early "sm_61 is not supported" error for devices below sm_70 instead of the NVRTC compile failure and the "Volta" label? I have not written it.

What is not in here

• The STRATA_VK_F16_TRUNC=1 runs got the variable through sbatch --export=ALL; the logs do not echo it. Three of the four reruns landed on another node of the same card type than the run without it (the Quadro RTX 6000 was the same node).
• All test runs were serial, so nothing new on the parallel vkDestroyDevice hang from A100, A30, L40S and H100: tested rows, results, and default-model test checks by file name #24.
• Only Q2_K_XL: no Q8_0 or BF16 on this cluster.
• An RTX A4000 and a Quadro RTX 5000 are queued; I will send them separately if the jobs get to run.

Full tables (mean, min-max of 3 repetitions), what calibrate chose, hosts and startup lines are in docs/more-nvidia-results.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_014eNPMupWL2tifApEBfcCgi