Repository navigation
fix: constrain MutSparseArray lookup hints - #27
Open
iAmMichaelConnor wants to merge 1 commit into
Open
iAmMichaelConnor wants to merge 1 commit into
iAmMichaelConnor wants to merge 1 commit into
Conversation
iAmMichaelConnor
force-pushed
the
mike/04-fix-mut-lookup-hints
branch
from
November 2, 2026 18:16
60cfc14 to
efe70eb
Compare
TomAFrench
force-pushed
the
mike/04-fix-mut-lookup-hints
branch
from
November 3, 2026 03:53
efe70eb to
765c2fe
Compare
iAmMichaelConnor
force-pushed
the
mike/04-fix-mut-lookup-hints
branch
from
November 10, 2026 20:11
765c2fe to
5f280da
Compare
This was referenced Nov 11, 2026
iAmMichaelConnor
force-pushed
the
mike/04-fix-mut-lookup-hints
branch
from
November 11, 2026 12:45
5f280da to
eb76733
Compare
get() and set() checked keys[found_index] <= idx <= keys[next] when the hint claimed the key was found, rather than keys[found_index] == idx. A prover could therefore read or overwrite the value of a smaller key. The same check indexed past the end of the key list for the end sentinel, so reading or writing `maximum` failed unless it was an input key. Use the bound check of SparseArray::get, whose upper bound is the found slot itself when the key is found and the next slot otherwise, so a found lookup must match its key exactly and never follows the end sentinel's link. Key 0 and `maximum` can also occupy several slots (unused slots hold key 0, and either may be an input key besides its sentinel), so a found hint could select a slot holding a stale or default value. Require those two keys to resolve to their sentinel slots, and have the hint return them. get() and set() now only fetch the hint and pass it to get_with_hint() and set_with_hint(), which hold every constraint on it, so tests can call those with a hint the search would never return.
iAmMichaelConnor
force-pushed
the
mike/04-fix-mut-lookup-hints
branch
from
November 11, 2026 13:16
eb76733 to
292be9d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack
masterEach PR is based on the one above it. Please review and merge in order from the top.
What
MutSparseArray::get/settake(found, found_index)from an unconstrained search and thencheck
keys[found_index] + 1 - found <= idx <= keys[linked_keys[found_index]] + found - 1.For
found = truethat iskeys[found_index] <= idx <= keys[next], notkeys[found_index] == idx(the immutable array useskeys[found_index + 1 - found], whichdoes reduce to equality). Consequences on
master:value of the next smaller key (e.g.
get(3)returns key 2's value).input key as well as the start sentinel. A found hint can select a slot with a stale value
after
set(0, ...), or an unused slot's default value.keys[linked_keys[...]]even when found, and the endsentinel links to
0xFFFFFFFF, soget/setofmaximumfail with an out-of-boundsaccess unless
maximumwas an input key.The fix mirrors
SparseArray::get: the upper bound is read fromfound_indexitself when thekey is found and from the next slot otherwise, so a found lookup must match its key exactly and
never follows the end sentinel's link (issues 1 and 3). In addition, a found lookup of 0 must
use slot 0, and a found lookup of
maximummust use the end sentinel, which is the only slotwhose link is
0xFFFFFFFF(issue 2). The search hint returns those sentinel slots. Pinningmaximumhas to land with the bound fix: onmasterthe end sentinel's slot cannot beselected at all (issue 3), so an input key equal to
maximumhas only one usable slot; oncethe sentinel slot is readable there are two, and a hint could choose between them.
Cost (bb 6.0.0, a
MutSparseArray<16, Field>created from 8 entries): onegetcosts 130gates, +14 over #26 (116); one
setcosts 365 gates, +22 over #26 (343). This form reuses theexisting bound check of
SparseArray::getand needs no new struct field; an if/else check witha separate message per case measured 4 gates cheaper per
getand 10 perset, too little tojustify a second pattern.
getandsetnow only fetch the hint and pass it toget_with_hint/set_with_hint,which contain everything that was in their bodies after the search, including the call to
assert_valid_lookup, which holds the checks above. The split exists so that a test can handthe constrained code a hint that the search never returns.
Testing
The honest search never returns an invalid lookup, so no test that goes through
getorsetalone can tell whether the hint is constrained. The forged-hint tests therefore callget_with_hint/set_with_hint, which are the whole ofget/setapart from fetchingthe hint. Each is pinned to its assertion message with
should_fail_with.test_get_rejects_absent_key_marked_found,test_set_rejects_absent_key_marked_foundidx is above the hinted rangetest_get_rejects_stale_zero_slot,test_set_rejects_stale_zero_slotkey 0 must use the start sentineltest_get_rejects_unused_slotkey 0 must use the start sentineltest_get_rejects_stale_maximum_slot,test_set_rejects_stale_maximum_slotmaximumfound in its input slot, not the sentinelmaximum must use the end sentineltest_get_rejects_present_key_marked_absent,test_set_rejects_present_key_marked_absentidx is below the hinted rangetest_get_rejects_wrong_gap,test_set_rejects_wrong_gapidx is above the hinted rangeassert_valid_lookupmakes the tests pinned toits message fail ("Test passed when it should have failed").
the forged hints of the first four rows ("Test passed when it should have failed";
test_get_rejects_stale_maximum_slotfails earlier, in its setupset(9, 99), because ofissue 3). Rows 1 to 3 are issues 1 and 2; row 4 is not a bug on
master, where the inputslot is the only usable one for
maximum, and tests the pin that the fix makes necessary.test_maximum_not_in_input, through the public API, fails on this PR's base ("Index out ofbounds") and passes here: the evidence for issue 3.
test_overwrite_boundary_keysis a regression test that the honest hint still passes when 0and
maximumare input keys and are overwritten.nargo test(Noir 1.0.0): 31/31 passed.nargo fmt --checkis clean.