Repository navigation
BitPacking 0.3: one bit model - #18
Merged
Merged
Conversation
NarrowTuple, @NarrowTuple and the Pad, ZeroPad and OnePad layout markers
are removed. Packed records of mixed field types are covered by
PackedStructs.jl, which the README now points to. BitPacking keeps the
homogeneous packed containers, NArray and NarrowArray.
The Narrow{T} type forms, `Narrow{T}.(x)` and `reinterpret(Narrow{T}, a)`,
deprecated in favor of `Narrow(T)`, are removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
raw_bits(x) now returns the value in the low bitwidth(x) bits with zeros above, also for types declared with whole bytes but a narrower bitwidth, such as Bool or a 4-bit float stored in a byte, whose high bits it used to pass through. from_raw_bits(T, bits) uses only the low bitwidth(T) bits of an unsigned of any width, so from_raw_bits(Bool, 0x03) is a valid true and from_raw_bits(UInt8, 0x0102) is 0x02, where they used to make an invalid Bool and throw. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e array rule
pack, unpack and packed_getindex take raw elements as unsigned_type(sizeof(T))
words and split fields over the limbs of any storage, so every element width
from 1 to 128 bits packs; wider elements throw an ArgumentError.
NArray storage is now determined by the total bit count: the unsigned
integer of that many bytes when one exists, else a byte tuple. reinterpret(T,
::NArray) returns an NArray with the same bits and dimension 1 rescaled,
throwing when the bits don't divide, as for Base arrays and NarrowArray.
reinterpret(Unsigned, v) returns the storage word, and
reinterpret(NVector{T,L}, word) is its inverse, checking the storage type and
the padding bits. The public storage constructor is gone, so constructor
arguments are always values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Storage that is not exactly 1, 2, 4, 8 or 16 bytes is a tuple of the widest
of UInt64, UInt32, UInt16 and UInt8 that divides its byte count, rather than
always a tuple of bytes. Sizes don't change, so NarrowArray chunks still
reinterpret to and from plain bytes, but an NVector{Float32,8} is four UInt64
words and its elements are read with one shift instead of four byte loads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… device
reinterpret(T, ::NarrowArray) gives a plain view when T fills its bytes and a
NarrowArray{T} otherwise, for any width from 1 to 128 bits, rather than only
whole-byte types and types narrower than a byte.
dest .= x into a NarrowArray now takes over materialize!, evaluating the
broadcast into a dense temporary on dest's backend and packing it. Before,
a GPU source gave the broadcast a GPU style whose copyto! wrote the
NarrowArray element by element, which fails on the device. The broadcast
also follows the usual shape rules now, so scalars and singleton dimensions
expand.
The Narrow docstring's table no longer says plain T selects the unpacked
form: reinterpret never unpacks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NArray now implements the element protocol: raw_bits(v) returns its bits as
one unsigned integer, joining tuple storage into the next unsigned type, and
from_raw_bits(NVector{T,L}, bits) builds one, ignoring bits above its width.
Before, the generic raw_bits went through reinterpret, which on an NArray
follows the array rule and returned an NArray, so an NArray could not be an
element of an NArray or a NarrowArray. NVector{T,L} and the like, with the
storage parameter free, stand for their concrete type as reinterpret targets;
a non-concrete element type throws.
This replaces reinterpret(Unsigned, v) and reinterpret(NVector{T,L}, word):
the first widened padded arrays, which is raw_bits' job, and both were the
only reinterprets of an NArray that did not follow the array rule.
raw_bits and from_raw_bits now handle any bitstype of up to 16 bytes: a
primitive type whose sizeof is not a power of two, such as a 24-bit type
before Julia 1.14 (sizeof 3, array stride 4), is zero-extended and truncated,
and a struct such as three UInt8s goes through its bytes. NarrowArray takes
its raw-bits path for primitive types whose sizeof is below their array
stride, where Base's array reinterpret refuses them. Non-bitstype targets
throw an ArgumentError.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
copy(narr) and T.(narr) return a plain array of the parent's backend, an Array on the host and a CuArray on the device, instead of a ReinterpretArray over a temporary array of SVector chunks. The result is allocated once and the unpacked chunks are written straight into it, viewed as runs of values. setindex! writes one element by repacking its chunk, so the in-place parts of the AbstractArray interface work on the host: copyto!, fill!, sort, reverse. getindex checks bounds against the logical size. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A "Bit model" docs page states the rules every container follows (LSB-first values, zero padding, reinterpret keeping bits and width, the array rule), the element protocol, the NArray storage table, NArray as the packed register layout PTX instructions take and NarrowArray as the dense memory layout, map staying packed while broadcasting unpacks, known consumers of the bit order, how PackedStructs.jl differs (first field in the highest bits, widths from EmulatedBitIntegers.bits), and what changed from 0.2. The README describes the value and memory layouts and shows the 0.3 API. Docstrings on internal packing functions become comments, as Documenter requires every docstring to be in the manual. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ructors Scalar getindex and setindex! called fldmod1, which was not inlined; the chunk index is now an unsigned divrem, a shift and a mask for power-of-two chunk lengths. NArray's Base.setindex on word storage replaces the element's bits in place rather than repacking the chunk. Packing and unpacking a CPU Array went through Base's ReinterpretArray, which indexes about 25 times slower than an Array; it now views the Array's memory as an Array of NTuple runs. Other arrays, including GPU arrays, are unchanged. On a 1M-element Bool array, against 0.2: packing 3x, copy 2.6x, scalar reads 5x and dest .= x 6x faster, all now faster than BitVector; for a 4-bit type packing is 14x faster and copy is back to 0.2's speed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e in CI cuTile 0.3.2 does not resolve on Julia 1.14, which failed the pre job before any test ran; the extension tests pass unchanged against cuTile 1.0.0. CI tests 1.10, 1.11 and the latest release instead of pre, and leaves the cuTile tests off until a Microfloats release supports BitPacking 0.3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GqhLuuGKG92oXMxGJxVHDf
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #18 +/- ##
===========================================
- Coverage 97.47% 86.80% -10.67%
===========================================
Files 7 6 -1
Lines 593 326 -267
===========================================
- Hits 578 283 -295
- Misses 15 43 +28 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
from_raw_bits(::Type{<:NArray}, bits) reassigned a variable that the closure
splitting it into limbs captures, so Julia boxed it. On the CPU that only costs
speed; in GPU code it does not compile, which Microfloats' CUDA extension hits
when it reads a pair register as an NVector. The variable is now assigned once,
and a test checks that no BitPacking method boxes a variable.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GqhLuuGKG92oXMxGJxVHDf
Microfloats 0.2.1 and earlier don't depend on BitPacking, so nothing stopped the resolver from pairing them with BitPacking 0.3, where their sub-byte types would silently pack as 8 bits. Listing Microfloats as a weak dependency, with no extension, applies a compat bound to it whenever it is in the environment. 0.2.9 is the first Microfloats release on BitPacking 0.3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GqhLuuGKG92oXMxGJxVHDf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BitPacking 0.3: one bit model for every container, width and Julia version.
This is a breaking release. The new bit model docs page describes the design, and its last section lists what changes for 0.2 users.
The bit model
Thasbitwidth(T)value bits. They are stored LSB-first, element 1 lowest, and padding is always zero.reinterpretnever changes width. It keeps every bit, and throws when the bit counts don't match. Width changes go throughraw_bitsandfrom_raw_bits.reinterpreton a packed array keeps the total bit count and rescales dimension 1.NArrayis the value layout. For FP4, FP8 and BF16 pairs it matches PTX's packed registers (e2m1x2,e4m3x2,bf16x2).NarrowArrayis the memory layout, a dense byte stream.SArrayor a tuple is the compute layout.Changes
Element protocol.
bitwidth,raw_bitsandfrom_raw_bitshave one contract on every Julia version, including 1.14's narrow primitive types. Any bitstype of up to 16 bytes is an element,NArrayincluded.Widths. Elements may be 1 to 128 bits wide. Storage that doesn't fit an unsigned integer is a tuple of the widest limbs that divide its bytes, so a chunk of four FP6 values is exactly 3 bytes.
NArray.reinterpretfollows the array rule.raw_bits(v), andfrom_raw_bits(NVector{T,L}, bits)goes the other way.NarrowArray.reinterpretworks to and from any width.copy(x)andT.(x)return a dense array on the same backend, with one allocation on the GPU.setindex!andNarrowArray{T}(undef, dims...).dest .= xpacks on the device for GPU sources, where it used to write element by element.Removed.
NarrowTuple,@NarrowTuple,Pad,ZeroPad,OnePad, and the deprecatedNarrow{T}forms. UseNarrow(T).Microfloats compat. Microfloats is a weak dependency with no extension, and compat
0.2.9. Microfloats 0.2.1 and earlier don't depend on BitPacking, so without this the resolver could pair them with 0.3, where their sub-byte types would pack as 8 bits.Speed. These are 1M-element
Booltimings on Julia 1.12, against 0.2.5. Every listed operation is now faster thanBitVectortoo.copydest .= xFor a 4-bit type, packing is 14× faster.
Tests
CI
pre.After merging
🤖 Generated with Claude Code
https://claude.ai/code/session_01GqhLuuGKG92oXMxGJxVHDf