Skip to content

Support cupy arrays: subsetting, dtype preservation, and standardization matching the Rust path - #25

Open
CarlKCarlK wants to merge 9 commits into
mainfrom
cupy-sub-matrix
Open

CarlKCarlK wants to merge 9 commits into
mainfrom
cupy-sub-matrix

Conversation

@CarlKCarlK

@CarlKCarlK CarlKCarlK commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Lets SnpData and other PstData objects hold CuPy (GPU) arrays end to end, as FaST-LMM's single_snp(xp="cupy") needs, and makes CuPy results and errors match the numpy (Rust) path.

  • sub_matrix: a CuPy array used to reach the numpy-only Rust helper and fail with TypeError, so any subsetting of a CuPy-backed SnpData failed. CuPy arrays are now detected directly (sys.modules.get("cupy"), so cupy is never imported) and subset with CuPy indexing, with the same validation as the numpy path and no extra GPU copy. Because CuPy silently wraps negative and out-of-range indexes, they are checked and raise IndexError. Non-CuPy inputs take the numpy path exactly as before.
  • PstData._fixup_input_val: a val moved between numpy and CuPy now keeps the dtype that same-module input keeps: float32 or float64, or any dtype (for example, int8) when _require_float32_64=False; anything else (for example, a list) still becomes float64. Before, numpy float32 and int8 were upcast to float64 on CuPy, and a CuPy val under numpy raised CuPy's implicit-conversion TypeError.
  • Standardization of CuPy arrays (the Python standardizer; numpy uses Rust): an all-missing SNP now raises ValueError("No individual observed for the SNP.") like the Rust path, instead of silently becoming zeros. Beta standardization failed on CuPy (SciPy received a CuPy array); it now computes the per-SNP weights on the host.

Validation

  • New tests (skipped when CuPy is not installed), each failing without its fix; all pass with CuPy 14.2 on an RTX 3060:
    • test_sub_matrix_cupy: exact equality with the Rust path across 2-D/3-D shapes, F/C input orders, A/F/C output orders, float32/float64 in and out, repeated indexes, and rejection of bad indexes.
    • test_snpdata_subset_cupy: subsetting a CuPy-backed SnpData.
    • test_snpdata_cupy_keeps_dtype: both directions, with and without _require_float32_64, against the same-module baseline.
    • test_standardize_cupy_matches_rust: Unit and Beta, float64 and float32, missing/constant/single-value SNPs (within 8 ulp), and the all-missing error.
  • tests/test.py (legacy suite) at 2640a5e: 745 OK, 4 skipped (the CuPy tests, in an environment without CuPy).
  • FaST-LMM's GPU tests (branch gpu-f32) pass with this branch.

PR #27 adds the same _checked_index helper to the numpy path; whichever PR merges second keeps one copy.

FaST-LMM's GPU path will need a PySnpTools release containing this change, followed by a raised lower bound in FaST-LMM.

@CarlKCarlK CarlKCarlK changed the title Support cupy arrays in sub_matrix and keep float32 when moving data to cupy Support cupy arrays: subsetting, dtype preservation, and standardization matching the Rust path Nov 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant