Skip to content

BitPacking 0.3: one bit model - #18

Merged
AntonOresten merged 13 commits into
mainfrom
v0.3
Nov 7, 2026
Merged

AntonOresten merged 13 commits into
mainfrom
v0.3

Conversation

@AntonOresten

@AntonOresten AntonOresten commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

BitPacking 0.3: one bit model for every container, width and Julia version.

This is a breaking release. The new bit model docs page describes the design, and its last section lists what changes for 0.2 users.

The bit model

  1. Value bits and padding. A value of T has bitwidth(T) value bits. They are stored LSB-first, element 1 lowest, and padding is always zero.
  2. reinterpret never changes width. It keeps every bit, and throws when the bit counts don't match. Width changes go through raw_bits and from_raw_bits.
  3. The array rule. As for Base arrays, reinterpret on a packed array keeps the total bit count and rescales dimension 1.
  4. Three layouts. NArray is the value layout. For FP4, FP8 and BF16 pairs it matches PTX's packed registers (e2m1x2, e4m3x2, bf16x2). NarrowArray is the memory layout, a dense byte stream. SArray or a tuple is the compute layout.

Changes

  • Element protocol. bitwidth, raw_bits and from_raw_bits have one contract on every Julia version, including 1.14's narrow primitive types. Any bitstype of up to 16 bytes is an element, NArray included.

  • Widths. Elements may be 1 to 128 bits wide. Storage that doesn't fit an unsigned integer is a tuple of the widest limbs that divide its bytes, so a chunk of four FP6 values is exactly 3 bytes.

  • NArray.

    • reinterpret follows the array rule.
    • Constructors always take values.
    • The storage word is raw_bits(v), and from_raw_bits(NVector{T,L}, bits) goes the other way.
  • NarrowArray.

    • reinterpret works to and from any width.
    • copy(x) and T.(x) return a dense array on the same backend, with one allocation on the GPU.
    • New: setindex! and NarrowArray{T}(undef, dims...).
    • dest .= x packs on the device for GPU sources, where it used to write element by element.
  • Removed. NarrowTuple, @NarrowTuple, Pad, ZeroPad, OnePad, and the deprecated Narrow{T} forms. Use Narrow(T).

  • Microfloats compat. Microfloats is a weak dependency with no extension, and compat 0.2.9. Microfloats 0.2.1 and earlier don't depend on BitPacking, so without this the resolver could pair them with 0.3, where their sub-byte types would pack as 8 bits.

  • Speed. These are 1M-element Bool timings on Julia 1.12, against 0.2.5. Every listed operation is now faster than BitVector too.

    0.2.5 0.3
    pack 106 µs 35 µs
    copy 94 µs 36 µs
    scalar read loop 2374 µs 432 µs
    dest .= x 350 µs 55 µs

    For a 4-bit type, packing is 14× faster.

Tests

  • CPU: pass on Julia 1.10, 1.12, 1.13 and 1.14.0-alpha2, with no method ambiguities, no unbound type parameters and no boxed variables, which GPU code can't compile.
  • GPU: the cuTile extension suite passes on a GB10 with Julia 1.12 and cuTile 1.0.0.
  • Docs: they build, and the doctests pass.
  • Microfloats: a branch moving Microfloats to 0.3 passes its full suite on 1.12 and 1.14, including its CUDA tests. The cuTile suite here passes against it too.

CI

  • Matrix: CI tests 1.10, 1.11 and the latest release instead of pre.
  • cuTile pin: the extension tests pin cuTile 1.0.0, since 0.3.2 can't be installed on 1.14.
  • cuTile tests: off in CI until Microfloats 0.2.9, the first release on BitPacking 0.3, is registered.

After merging

  1. Register 0.3. Until Microfloats 0.2.9 exists, environments with Microfloats stay on BitPacking 0.2.5 and Microfloats 0.2.8.
  2. Release Microfloats 0.2.9 from Real widths on Julia 1.14, and BitPacking 0.3 Microfloats.jl#37, which moves it to BitPacking 0.3.
  3. Turn the cuTile tests back on in CI.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GqhLuuGKG92oXMxGJxVHDf

AntonOresten and others added 11 commits November 6, 2026 14:12
NarrowTuple, @NarrowTuple and the Pad, ZeroPad and OnePad layout markers
are removed. Packed records of mixed field types are covered by
PackedStructs.jl, which the README now points to. BitPacking keeps the
homogeneous packed containers, NArray and NarrowArray.

The Narrow{T} type forms, `Narrow{T}.(x)` and `reinterpret(Narrow{T}, a)`,
deprecated in favor of `Narrow(T)`, are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
raw_bits(x) now returns the value in the low bitwidth(x) bits with zeros
above, also for types declared with whole bytes but a narrower bitwidth, such
as Bool or a 4-bit float stored in a byte, whose high bits it used to pass
through. from_raw_bits(T, bits) uses only the low bitwidth(T) bits of an
unsigned of any width, so from_raw_bits(Bool, 0x03) is a valid true and
from_raw_bits(UInt8, 0x0102) is 0x02, where they used to make an invalid Bool
and throw.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e array rule

pack, unpack and packed_getindex take raw elements as unsigned_type(sizeof(T))
words and split fields over the limbs of any storage, so every element width
from 1 to 128 bits packs; wider elements throw an ArgumentError.

NArray storage is now determined by the total bit count: the unsigned
integer of that many bytes when one exists, else a byte tuple. reinterpret(T,
::NArray) returns an NArray with the same bits and dimension 1 rescaled,
throwing when the bits don't divide, as for Base arrays and NarrowArray.
reinterpret(Unsigned, v) returns the storage word, and
reinterpret(NVector{T,L}, word) is its inverse, checking the storage type and
the padding bits. The public storage constructor is gone, so constructor
arguments are always values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Storage that is not exactly 1, 2, 4, 8 or 16 bytes is a tuple of the widest
of UInt64, UInt32, UInt16 and UInt8 that divides its byte count, rather than
always a tuple of bytes. Sizes don't change, so NarrowArray chunks still
reinterpret to and from plain bytes, but an NVector{Float32,8} is four UInt64
words and its elements are read with one shift instead of four byte loads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… device

reinterpret(T, ::NarrowArray) gives a plain view when T fills its bytes and a
NarrowArray{T} otherwise, for any width from 1 to 128 bits, rather than only
whole-byte types and types narrower than a byte.

dest .= x into a NarrowArray now takes over materialize!, evaluating the
broadcast into a dense temporary on dest's backend and packing it. Before,
a GPU source gave the broadcast a GPU style whose copyto! wrote the
NarrowArray element by element, which fails on the device. The broadcast
also follows the usual shape rules now, so scalars and singleton dimensions
expand.

The Narrow docstring's table no longer says plain T selects the unpacked
form: reinterpret never unpacks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NArray now implements the element protocol: raw_bits(v) returns its bits as
one unsigned integer, joining tuple storage into the next unsigned type, and
from_raw_bits(NVector{T,L}, bits) builds one, ignoring bits above its width.
Before, the generic raw_bits went through reinterpret, which on an NArray
follows the array rule and returned an NArray, so an NArray could not be an
element of an NArray or a NarrowArray. NVector{T,L} and the like, with the
storage parameter free, stand for their concrete type as reinterpret targets;
a non-concrete element type throws.

This replaces reinterpret(Unsigned, v) and reinterpret(NVector{T,L}, word):
the first widened padded arrays, which is raw_bits' job, and both were the
only reinterprets of an NArray that did not follow the array rule.

raw_bits and from_raw_bits now handle any bitstype of up to 16 bytes: a
primitive type whose sizeof is not a power of two, such as a 24-bit type
before Julia 1.14 (sizeof 3, array stride 4), is zero-extended and truncated,
and a struct such as three UInt8s goes through its bytes. NarrowArray takes
its raw-bits path for primitive types whose sizeof is below their array
stride, where Base's array reinterpret refuses them. Non-bitstype targets
throw an ArgumentError.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
copy(narr) and T.(narr) return a plain array of the parent's backend, an
Array on the host and a CuArray on the device, instead of a ReinterpretArray
over a temporary array of SVector chunks. The result is allocated once and
the unpacked chunks are written straight into it, viewed as runs of values.

setindex! writes one element by repacking its chunk, so the in-place parts of
the AbstractArray interface work on the host: copyto!, fill!, sort, reverse.
getindex checks bounds against the logical size.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A "Bit model" docs page states the rules every container follows (LSB-first
values, zero padding, reinterpret keeping bits and width, the array rule),
the element protocol, the NArray storage table, NArray as the packed
register layout PTX instructions take and NarrowArray as the dense memory
layout, map staying packed while broadcasting unpacks, known consumers of the
bit order, how PackedStructs.jl differs (first field in the highest bits,
widths from EmulatedBitIntegers.bits), and what changed from 0.2.

The README describes the value and memory layouts and shows the 0.3 API.
Docstrings on internal packing functions become comments, as Documenter
requires every docstring to be in the manual.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ructors

Scalar getindex and setindex! called fldmod1, which was not inlined; the chunk
index is now an unsigned divrem, a shift and a mask for power-of-two chunk
lengths. NArray's Base.setindex on word storage replaces the element's bits in
place rather than repacking the chunk.

Packing and unpacking a CPU Array went through Base's ReinterpretArray, which
indexes about 25 times slower than an Array; it now views the Array's memory
as an Array of NTuple runs. Other arrays, including GPU arrays, are unchanged.

On a 1M-element Bool array, against 0.2: packing 3x, copy 2.6x, scalar reads
5x and dest .= x 6x faster, all now faster than BitVector; for a 4-bit type
packing is 14x faster and copy is back to 0.2's speed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e in CI

cuTile 0.3.2 does not resolve on Julia 1.14, which failed the pre job before
any test ran; the extension tests pass unchanged against cuTile 1.0.0. CI tests
1.10, 1.11 and the latest release instead of pre, and leaves the cuTile tests
off until a Microfloats release supports BitPacking 0.3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GqhLuuGKG92oXMxGJxVHDf
@codecov

codecov Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.30516% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.80%. Comparing base (54da004) to head (6cffa5d).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/NarrowArray.jl 88.88% 7 Missing ⚠️
src/NArray.jl 96.66% 2 Missing ⚠️
src/packing.jl 98.52% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main      #18       +/-   ##
===========================================
- Coverage   97.47%   86.80%   -10.67%     
===========================================
  Files           7        6        -1     
  Lines         593      326      -267     
===========================================
- Hits          578      283      -295     
- Misses         15       43       +28     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

AntonOresten and others added 2 commits November 7, 2026 00:28
from_raw_bits(::Type{<:NArray}, bits) reassigned a variable that the closure
splitting it into limbs captures, so Julia boxed it. On the CPU that only costs
speed; in GPU code it does not compile, which Microfloats' CUDA extension hits
when it reads a pair register as an NVector. The variable is now assigned once,
and a test checks that no BitPacking method boxes a variable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GqhLuuGKG92oXMxGJxVHDf
Microfloats 0.2.1 and earlier don't depend on BitPacking, so nothing stopped
the resolver from pairing them with BitPacking 0.3, where their sub-byte
types would silently pack as 8 bits. Listing Microfloats as a weak dependency,
with no extension, applies a compat bound to it whenever it is in the
environment. 0.2.9 is the first Microfloats release on BitPacking 0.3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GqhLuuGKG92oXMxGJxVHDf
@AntonOresten
AntonOresten merged commit d3769a7 into main Nov 7, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant