ctpop/ctlz/cttz results are never cast back to the operand width — u64::count_ones() returns 0 on device #401

rustc_codegen_nvvm omits the result cast that rustc_codegen_llvm performs on the bit-counting intrinsics. Every width except 32 is silently wrong on device: u64::count_ones() returns 0 for all inputs, and the same source is correct on the host. It compiles with no error or warning.

Reproducer: https://github.com/nh13/rust-cuda-nvvm-repros (01-ctpop/)

Mechanism

core::intrinsics::ctpop returns u32 for every operand width — library/core/src/intrinsics/mod.rs:1646 at rustc f34ba774c (= nightly-2025-08-04, the nightly this backend pins). Same for ctlz (:1687), ctlz_nonzero (:1709), cttz (:1750), cttz_nonzero (:1772). (bswap/bitreverse return T and are unaffected.)

rustc_codegen_llvm therefore casts the result back:

// compiler/rustc_codegen_llvm/src/intrinsic.rs:397-400
sym::ctpop => {
    let ret = self.call_intrinsic("llvm.ctpop", &[llty], ...);
    self.intcast(ret, result.layout.llvm_type(self), false)
}

rustc_codegen_nvvm does not:

// crates/rustc_codegen_nvvm/src/intrinsic.rs:481-484  @ 7fa76f3d
sym::ctpop => self.call_intrinsic(&format!("llvm.ctpop.i{width}"), &[args[0].immediate()]),

Same omission for ctlz/cttz/ctlz_nonzero/cttz_nonzero at :471-479. Nothing casts downstream either — the value is stored via OperandRef::from_immediate_or_packed_pair(...).val.store(...) (:633-635), and check_store (builder.rs:1415-1427) bitcasts the pointer, never truncates the value.

Where the oversized value lands depends on the destination (rustc_codegen_ssa/src/mir/block.rs:935-956):

• Common case — a wrong answer. The destination is a fresh alloca of the intrinsic's return layout, i.e. 4 bytes (:948; :942 reuses an existing alloca of the same size). An i64 result is then an 8-byte store into a 4-byte alloca, which is deleted outright (below), so the load reads undef → 0.
• Destination is a place projection (:955) — potential memory corruption. For a struct field, array element, or Deref of a device pointer, the 8-byte store is in bounds of the enclosing object. SROA does not delete it; it instead clobbers 4 bytes of adjacent memory — for a *mut u32 destination, a straight 4-byte device-buffer overrun.

I have only observed the first case: count_ones is an #[inline] wrapper, so the destination is normally a temp, and reaching the second would require destination propagation to forward it into a place. I flag it because if it is reachable the impact is memory corruption rather than a wrong answer, which changes the severity of this bug considerably.

In the common case that out-of-bounds store is deleted outright by SROA (LLVM 7, lib/Transforms/Scalar/SROA.cpp:809-816 — "If this memory access can be shown to statically extend outside the bounds of the allocation, it's behavior is undefined, so simply ignore it"). The subsequent load reads undef → 0, and the llvm.ctpop.i64 call becomes dead. This is what makes the failure a clean 0 rather than a truncation, and it independently explains the PTX evidence below. (I read LLVM 7's SROA but could not confirm which pass instance in libNVVM's pipeline fires; the PTX makes that moot.)

PTX evidence: the emitted PTX contains zero popc.b64 instructions — the call is eliminated, not lowered. After switching to a u32-halves workaround, popc.b32 appears as expected.

Width 32 is the only width that works, because its cast is a no-op: llvm.ctpop.i32 returns i32, store size equals alloca size, nothing overruns.

Scope

• count_ones, leading_zeros, trailing_zeros on u8/u16/u64/u128 (and signed equivalents).
• ctlz_nonzero/cttz_nonzero too, so NonZero::leading_zeros, ilog2, etc.
• u128 goes through emulate_i128.rs:626-640, which zexts to i128 — a 16-byte store into the same 4-byte slot.
• u8/u16 fail differently: their store is in-bounds (1–2 bytes into 4), so SROA keeps it and the upper bytes of the i32 load are undef — garbage rather than a clean 0.

Still present on master

Unchanged at crates/rustc_codegen_nvvm/src/intrinsic.rs:630-633 on master (103a8d5). git log -S 'intcast(ret' on that file returns nothing — the cast has never been there. The backend predates the rustc signature change: ctpop<T> -> T in Rust 1.75, -> u32 by 1.80.

Suggested fix

Mirror rustc_codegen_llvm. Rather than wrapping each of the five arms (and handle_128_bit_intrinsic, a free fn with no result in scope, so its cast has to land at the call site anyway), a single intcast around the whole if width == 128 { … } else { match name { … } } result is strictly better:

let llval = self.intcast(llval, result.layout.llvm_type(self), false);

An intcast to an identical type is a no-op (LLVMRustBuildIntCast → CreateIntCast), so the intrinsics in that arm which correctly return T — bswap, bitreverse, rotate_*, saturating_* — are unaffected, and any intrinsic added later is covered for free. result: PlaceRef is already the parameter at intrinsic.rs:219 and LayoutLlvmExt is imported at :23, so this resolves in scope as written.

Worth considering alongside it: a debug assertion in builder.rs's store_with_flags that the stored value's store-size does not exceed the destination place's size. This bug class — a silent oversized store into an intrinsic result slot, then silently deleted — then becomes a codegen-time error for every intrinsic rather than a per-symbol fix. A regression test asserting popc.b64 appears in the emitted PTX would pin this specific case.

Environment

Rust-CUDA 7fa76f3d717038a92c90bf4a482b0b8dd3259344, cuda_builder 0.3.0, nightly-2025-08-04, CUDA 13.2 libNVVM, driver 595.71.05. Wrong answers observed on Tesla T4 (sm_75) and A10G (sm_86). Kernel crates no_std, -Z saturating_float_casts=false, --override-libm on (default).

Note on provenance: the wrong answers and the PTX evidence come from the original project where this was found. The standalone reproducer in 01-ctpop/ was written afterwards on a machine without a CUDA host and has not itself been built — it mirrors the same three-way comparison (u64 intrinsic vs u32-halves split vs SWAR, all reading the same value) that isolated the defect originally.

Prepared with the assistance of Claude (Anthropic). Every source citation above was checked against the referenced revisions — please verify them directly. The one claim I could not confirm from source, the place-projection corruption case, is marked as unobserved in the text.