Skip to content

fix: constrain MutSparseArray lookup hints - #27

Open
iAmMichaelConnor wants to merge 1 commit into
mike/03-fix-boundary-valuesfrom
mike/04-fix-mut-lookup-hints
Open

iAmMichaelConnor wants to merge 1 commit into
mike/03-fix-boundary-valuesfrom
mike/04-fix-mut-lookup-hints

Conversation

@iAmMichaelConnor

@iAmMichaelConnor iAmMichaelConnor commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Stack

PR Change Builds on
1 of 4: #26 fix: read boundary sentinel values from the sorted keys master
➡ this PR 2 of 4: #27 fix: constrain MutSparseArray lookup hints #26
3 of 4: #28 fix: support creating sparse arrays from empty inputs #27
4 of 4: #29 chore: fix README usage example #28

Each PR is based on the one above it. Please review and merge in order from the top.

What

MutSparseArray::get/set take (found, found_index) from an unconstrained search and then
check keys[found_index] + 1 - found <= idx <= keys[linked_keys[found_index]] + found - 1.
For found = true that is keys[found_index] <= idx <= keys[next], not
keys[found_index] == idx (the immutable array uses keys[found_index + 1 - found], which
does reduce to equality). Consequences on master:

  1. Soundness: a prover can claim an absent key is present and read, or overwrite, the
    value of the next smaller key (e.g. get(3) returns key 2's value).
  2. Soundness: key 0 can occupy several slots: unused slots hold key 0, and 0 can be an
    input key as well as the start sentinel. A found hint can select a slot with a stale value
    after set(0, ...), or an unused slot's default value.
  3. Completeness: the check reads keys[linked_keys[...]] even when found, and the end
    sentinel links to 0xFFFFFFFF, so get/set of maximum fail with an out-of-bounds
    access unless maximum was an input key.

The fix mirrors SparseArray::get: the upper bound is read from found_index itself when the
key is found and from the next slot otherwise, so a found lookup must match its key exactly and
never follows the end sentinel's link (issues 1 and 3). In addition, a found lookup of 0 must
use slot 0, and a found lookup of maximum must use the end sentinel, which is the only slot
whose link is 0xFFFFFFFF (issue 2). The search hint returns those sentinel slots. Pinning
maximum has to land with the bound fix: on master the end sentinel's slot cannot be
selected at all (issue 3), so an input key equal to maximum has only one usable slot; once
the sentinel slot is readable there are two, and a hint could choose between them.

Cost (bb 6.0.0, a MutSparseArray<16, Field> created from 8 entries): one get costs 130
gates, +14 over #26 (116); one set costs 365 gates, +22 over #26 (343). This form reuses the
existing bound check of SparseArray::get and needs no new struct field; an if/else check with
a separate message per case measured 4 gates cheaper per get and 10 per set, too little to
justify a second pattern.

get and set now only fetch the hint and pass it to get_with_hint / set_with_hint,
which contain everything that was in their bodies after the search, including the call to
assert_valid_lookup, which holds the checks above. The split exists so that a test can hand
the constrained code a hint that the search never returns.

Testing

The honest search never returns an invalid lookup, so no test that goes through get or
set alone can tell whether the hint is constrained. The forged-hint tests therefore call
get_with_hint / set_with_hint, which are the whole of get / set apart from fetching
the hint. Each is pinned to its assertion message with should_fail_with.

Test Forged hint Must fail with
test_get_rejects_absent_key_marked_found, test_set_rejects_absent_key_marked_found key 3 "found" in key 2's slot idx is above the hinted range
test_get_rejects_stale_zero_slot, test_set_rejects_stale_zero_slot key 0 found in its input slot, not the sentinel key 0 must use the start sentinel
test_get_rejects_unused_slot key 0 found in an unused slot key 0 must use the start sentinel
test_get_rejects_stale_maximum_slot, test_set_rejects_stale_maximum_slot maximum found in its input slot, not the sentinel maximum must use the end sentinel
test_get_rejects_present_key_marked_absent, test_set_rejects_present_key_marked_absent key 2 "not found" after its own slot idx is below the hinted range
test_get_rejects_wrong_gap, test_set_rejects_wrong_gap key "not found" in a gap that does not contain it idx is above the hinted range
  • Removing any one of the four assertions in assert_valid_lookup makes the tests pinned to
    its message fail ("Test passed when it should have failed").
  • The old check, put behind the same split as a manual check outside the test suite, accepts
    the forged hints of the first four rows ("Test passed when it should have failed";
    test_get_rejects_stale_maximum_slot fails earlier, in its setup set(9, 99), because of
    issue 3). Rows 1 to 3 are issues 1 and 2; row 4 is not a bug on master, where the input
    slot is the only usable one for maximum, and tests the pin that the fix makes necessary.
  • test_maximum_not_in_input, through the public API, fails on this PR's base ("Index out of
    bounds") and passes here: the evidence for issue 3.

test_overwrite_boundary_keys is a regression test that the honest hint still passes when 0
and maximum are input keys and are overwritten.

nargo test (Noir 1.0.0): 31/31 passed. nargo fmt --check is clean.

get() and set() checked keys[found_index] <= idx <= keys[next] when
the hint claimed the key was found, rather than keys[found_index] == idx.
A prover could therefore read or overwrite the value of a smaller key.
The same check indexed past the end of the key list for the end
sentinel, so reading or writing `maximum` failed unless it was an input key.

Use the bound check of SparseArray::get, whose upper bound is the found
slot itself when the key is found and the next slot otherwise, so a found
lookup must match its key exactly and never follows the end sentinel's link.

Key 0 and `maximum` can also occupy several slots (unused slots hold
key 0, and either may be an input key besides its sentinel), so a found
hint could select a slot holding a stale or default value. Require those
two keys to resolve to their sentinel slots, and have the hint return them.

get() and set() now only fetch the hint and pass it to get_with_hint()
and set_with_hint(), which hold every constraint on it, so tests can
call those with a hint the search would never return.
@iAmMichaelConnor
iAmMichaelConnor force-pushed the mike/04-fix-mut-lookup-hints branch from eb76733 to 292be9d Compare November 11, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant