[gburd]

What

In a btree whose bt_compare equates distinct byte strings (e.g. a
case-insensitive comparator, "abc" == "ABC"), a duplicate set whose members
are all deleted while a cursor still pins one of them, followed by a put of
an equal-comparing key, revived the set under the deleted key's bytes
instead of the caller's. R1 (50ef9c73f) fixed the non-duplicate slot but
deliberately skipped a key item shared by a duplicate set. This finishes
R1 for every duplicate shape (KNOWN-ISSUES R2, RFC 0014 §8.3).

The fix: rewrite the set's shared key item in place

The members of a duplicate set share one physical key item (every member
slot's inp[] points at it). So when the whole set is deleted and its stored
bytes differ from the caller's, a single in-place rewrite of that one key item
changes the key for the entire set at once, and the new member then joins by
the ordinary duplicate path. The set stays one set.

__bamc_put, on an exact match in a DB_AM_DUP database where
__bam_dup_alldeleted reports the set entirely deleted with differing stored
bytes, calls __bam_dup_rekey and falls through to the normal duplicate path:

• on-page, non-overflow key: __bam_ritem replaces the shared item and
  re-aims every inp[] offset that referenced the moved region — including
  each sharing member — so all members follow. Its __bam_repl log record is
  position-indexed and replays through the same __bam_ritem, so recovery
  is correct with no new or changed log record. A growing key that will not
  fit returns DB_NEEDSPLIT, and __bamc_put's existing retry splits and
  re-drives.
• off-page duplicate tree: the key is a lone (unshared) parent-leaf slot,
  rewritten by delete-and-re-add as R1 does for a non-dup slot (covering
  overflow keys and type transitions there).

While any member is live, the set's key does not move (the documented
shared-key semantics). No on-disk / log / region format change; no public API
or ABI change (the three new helpers are static).

Why the first version was replaced

The first fix redirected the put to a fresh DB_KEYFIRST insert next to the
still-pinned dead set. A second equal-key put then found the dead set
again and redirected again, producing a second standalone key item — two
equal-comparing keys as two separate non-dup key items, which DB->verify
rejects ("non-dup dup key"); DBC->count read 1 instead of 2 and a reverse
walk dropped a record. That was structural corruption (lead-reproduced with
r2_walk_corruption.c). Equal-comparing keys must form one set, so the
existing set's key must move — an insert next to it cannot be correct.

Documented residual (not corruption)

An on-page set whose shared key is an overflow item is left under its old
key bytes: there is no logged in-place primitive that replaces a leaf item with
an arbitrary overflow header while keeping the shared on-page offset. It stays
a single valid set (DB->verify clean, DBC->count correct), identical to
master's behaviour, differing only in the key bytes. Narrow exposure (a dup set
of keys each large enough to be an overflow key, all deleted while pinned).
Recorded in test/KNOWN-ISSUES.md R2.

Tests (fail before, pass after)

• test/db/r2_dup_revive.c (run_r2_dup_revive.sh): every case does two
  equal-key puts and asserts, always — DB->verify clean, DBC->count = live
  set size, forward (DB_NEXT) and reverse (DB_LAST+DB_PREV) walks returning the
  same mirrored complete set, two new records under one key — and the key bytes
  ("ABC", or the documented "abc" for the overflow residual). Matrix: DB_DUP /
  DB_DUPSORT, on-page / off-page, pin / no-pin, no-env / txn, a tall off-page
  tree, overflow keys on-page (residual) and off-page (rewritten), a growing
  key that forces a split, the live-member control, and recovery from the log
  alone (DB_RECOVER_FATAL, the .db removed). A sabotage arm
  (-DR2_REVIVE_SABOTAGE) disables the rewrite and must FAIL.
• test/db/r2_fuzz.c (run_r2_fuzz.sh): randomised — a case-insensitive
  DB_DUP/DB_DUPSORT btree driven with random pins, equal-comparing puts and
  deletes; after every op, DB->verify passes, the per-folded-key live count
  matches a model, and every live record of a folded key carries one spelling
  (one set). 60 seeds × 300 ops (~30 s).

Teeth, three libraries: the first R2 fix FAILs (DB_VERIFY_BAD + wrong
count; fuzz 5/60 seeds), master FAILs the key-bytes check, the new fix PASSes
(fuzz 60/60). Master passes the fuzz too — it keeps a valid single set, so the
fuzz isolates the structural corruption from the benign key-bytes difference.

Evidence

• --enable-diagnostic test/db tier: 65 PASS, 4 SKIP, 0 FAIL, including
  run_r2_dup_revive (normal PASS, sabotage FAIL 26 = teeth).
• r2_walk_corruption.c on the fix: verify 0, count 2, both walks complete,
  key ABC.

No EC2 of my own (shared dev box only).

[github-actions]

ABI diff vs v2026.10.8 (libabigail — authoritative)

abidiff exit bitmask: 0

Removed exported symbols (nm -D, _NNNN version suffix normalized)

None.

---

Advisory: libabigail/nm is the authoritative binary-ABI check; Coccinelle is complementary source-level early warning. See dist/cocci/README.md.

[github-actions]

Coccinelle convention checks

No new violations. ✅

Resolved since baseline (3) -- update dist/cocci/baseline.txt to lock these in.

rule_mutex_unbalanced|MUTEX_UNBALANCED|src/btree/bt_curadj.c|return (ret);
rule_mutex_unbalanced|MUTEX_UNBALANCED|src/crypto/mersenne/mt19937db.c|return (ret);
rule_mutex_unbalanced|MUTEX_UNBALANCED|src/mp/mp_register.c|return (ret);

[gburd]

Do not merge (lead review). With this change, two equal-key puts over an all-deleted, pinned DUPSORT set create two separate key items instead of one duplicate set:

• DB->verify returns DB_VERIFY_BAD (BDB1067 Page 1: non-dup dup key at entry 4);
• DBC->count is 1 where it should be 2;
• a DB_PREV walk skips the following key.

The reproducer is /scratch/libdb/r2-work/r2_walk_corruption.c. Without this change the tree stays valid (verify 0, count 2); only the R2 key-bytes bug remains. A fix is being reworked.

[gburd]

Ready for re-review. The corrupting redirect is replaced by an in-place rewrite of the duplicate set's shared key item (commits 07a240a fix, 2d24d21 multi-put test, 1711687 docs, d816880 fuzz).

• r2_walk_corruption.c (the lead's reproducer) on the fix: verify 0, count 2, both walks complete, key ABC. On master: valid tree but key abc. On the first R2 fix: DB_VERIFY_BAD "non-dup dup key".
• --enable-diagnostic test/db tier: 65 PASS, 4 SKIP, 0 FAIL, including run_r2_dup_revive (normal PASS, sabotage FAIL 26 = teeth).
• Fuzz teeth (60 seeds × 300 ops): new fix 0 FAIL, first R2 fix 5 FAIL (DB_VERIFY_BAD / "two spellings in one walk"), master 0 FAIL (valid single set — the fuzz isolates the structural corruption from the benign key-bytes difference).
• No on-disk / log / region / ABI change; the three new helpers are static. One documented residual: an on-page set whose shared key is an overflow item keeps its old bytes — still one valid set (verify clean, count correct), recorded in KNOWN-ISSUES R2.

Built and tested --enable-diagnostic on Debian 12.

[gburd]

Lead re-verification of v2: merge-ready.

• r2_walk_corruption.c: verify 0, count 2, both walks complete.
• My randomised fuzz: 601/601 seeds × 500 ops pass on v2. The same fuzz fails v1 on 20 of 301 seeds.
• Extra edge test, the shared key item at the lowest page offset. This hits __bam_ritem's p == t fast path, which re-aims only inp[indx]. Keys of a different length (3→1 and 3→8 bytes), with the set inserted first and last: 6/6 verify 0, one set with the new spelling, and the pinned deleted cursor returns DB_KEYEMPTY. The fast path is safe because every member shares the one offset.
• Documented residual (shared key that is an overflow item keeps the old bytes, valid set, same as master): accepted.