Skip to content

perf(hermitcrab): share trailing analysis template slot batches - #546

Open
johnml1135 wants to merge 4 commits into
masterfrom
perf/shared-template-slot-batches
Open

johnml1135 wants to merge 4 commits into
masterfrom
perf/shared-template-slot-batches

Conversation

@johnml1135

@johnml1135 johnml1135 commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Start here: src/SIL.Machine.Morphology.HermitCrab/AnalysisAffixTemplatesRule.cs -- the slot trie and the shared walk; AnalysisStratumRule now builds it instead of a plain RuleBatch.

Status: open since 2026-10-22, rebased onto master after #522 (merged). CI green. Review of the design is what it waits on.

FieldWorks consumer: FLEx's parser worker adopts this through ParseWords on a degree-1 Morpher, tracked in LT-22885.

What it does. Templates whose trailing slots are identical (same rule objects, same order, same optionality) now run each shared slot batch once per battery call instead of once per template. Output order and first-seen words per template are unchanged. On a rebased-head benchmark the shared walk cut slot batches by 33.1% (Sena sample) and 21.9% (Mbugwe sample).

The unknown you arrive with: does a shared walk lose or reorder parses? No. Each trie edge is the same ordered rules and optionality beneath the same preceding slots, so every template under it reaches the edge with the word its own walk would. Outputs are still unioned per template, in template order, so the same representatives reach analysis folding. The question worth your time is the guard list, not the walk.

Where to look

  • Order and identity: SharedWalkMatchesPerTemplateWalkInOrder (4 grammar variants) compares the shared walk with the per-template RuleBatch, by ordered Word equality and by AnalysisStratumRule.AnalysisMergeKey sequence.
  • One batch, not two: UntracedBatteryRunsSharedSlotBatchOnce pins two identical tails to one execution, and the next call executing again.
  • Fallbacks to the original walk (Apply guard): TracedAndParallelBatteriesKeepPerTemplateWalk, SharedWalkPreservesBatteryAlternativeLimit, AlternativeLimitStopsBeforeLaterTemplateAdmission, ChangedSlotOptionalityPreservesPerTemplateWalk.
  • Admission: AnalysisAffixTemplateRule.CanApply is now split out so the trie checks each template once.

Not here. The shared walk runs only when Morpher.MaxDegreeOfParallelism == 1. The default degree is 0, which keeps ParallelApplySlots, so the speedup applies to degree-1 morphers. FieldWorks HCParser and the HermitCrab tool use the default. Throughput there comes from #582 (open, Morpher.ParseWords), which parses several words at once on the degree-1 path. Also not here: tracing, finite MaxAlternatives, custom compiled templates, changed slot optionality. Full-corpus parity is not verified.

Stack and verification. Independent of the other open HermitCrab PRs; #577 stacks on this. Ran earlier at an older head: HermitCrab test project, git diff --check, restore and CSharpier. Not run: ./local_check.sh build step (native SentencePiece build needs cmake, absent locally; CI builds pass). The conformance suite (#480 head) was not rerun after the rebase; its older results are in Evidence.

Next: review the guard list in AnalysisAffixTemplatesRule.Apply, then approve or name a missing fallback.


Reading this a year from now -- start here

This change shares trailing slot batches across affix templates inside one battery call. It is separate from #490's battery-level memo, which #522 removed: that memo cut battery invocations, this cuts repeated slot batches inside one invocation. The precedent is sillsdev/PanGloss ledger entry 095, docs/divergences/095-shared-template-slot-batches.md, and its stratum.rs slot trie.

Decisions and why
  • Shared walk only at degree 1. The parallel path uses a layered breadth-first queue (ParallelApplySlots); sharing it needs its own design. Parallelism across words (Add bounded batch parsing to HermitCrab #582) is the planned route for default callers.
  • Positive MaxAlternatives uses the original walk to keep early termination and per-template admission.
  • Optionality is snapshotted when the trie is built; any later change in slot count or optionality falls back to the original walk.
  • Intermediate words and outputs are local to one battery call. Only topology, configuration snapshots and compiled rules persist; trace events are unchanged.
Parity argument

Recursing through batch outputs in their existing order, then taking the optional unchanged-word continuation, preserves each template's depth-first emission sequence and admits or loses no parses. Each template keeps its own output set, unioned in template order with the existing comparer, so first-seen representatives and ordered words reach deduplication and analysis folding unchanged.

Evidence

Measured at an earlier head, before #522 removed the memo, so absolute batch counts predate the rebase. Single-threaded, sharing disabled versus enabled, new analysis scope per parse, temporary counters only in the output-directory build. The two later figures in the pitch come from a benchmark on the rebased head.

Input Completed segmentable pairs Slot batches before -> after Fewer batches
shared-template-unconstrained-suffix (10 words x 100) 1,000 2,000 -> 1,000 50.0%
Sena (200 sampled rows) 172 47,072 -> 32,287 31.4%
Mbugwe (200 sampled rows) 139 72,467 -> 58,486 19.3%

All completed segmentable pairs had identical morpheme-ID/surface signature lists after ordinal sorting, including multiplicity: zero differences, zero errors. Sena had 28 matching invalid-shape outcomes and no capped pairs; Mbugwe excluded 61 pairs that hit a two-second watchdog in either mode. Samples are evenly spaced nonempty rows from each word list. No corpus words, parse examples or grammar contents are included.

Conformance (#480 head, 60 fixtures, --include-pathological, older head): this branch and the unchanged base both failed 2. The requested 60/60 baseline could not be reproduced: both fail traced-rule attribution in boundary-leftover-in-narrowing-lhs-strrep, and deep-optional-affix-nesting crosses its 15-second budget, using the original walk because it has no shared slot tails.

Local runs used WSL ~/.dotnet, cached NuGet packages and bounded workers, no network. ./local_check.sh and --agent-strict failed only at the native SentencePiece build (missing cmake); restore, CSharpier and the comment scan passed.

🤖 Generated with Claude Code


This change is Reviewable

@codecov-commenter

codecov-commenter commented Oct 22, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.70992% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.69%. Comparing base (c245476) to head (17a6894).

Files with missing lines Patch % Lines
...orphology.HermitCrab/AnalysisAffixTemplatesRule.cs 97.29% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #546      +/-   ##
==========================================
+ Coverage   74.57%   74.69%   +0.12%     
==========================================
  Files         453      454       +1     
  Lines       38342    38459     +117     
  Branches     5284     5308      +24     
==========================================
+ Hits        28593    28727     +134     
+ Misses       8583     8574       -9     
+ Partials     1166     1158       -8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@claude

This comment has been minimized.

@claude

This comment has been minimized.

if (
!_hasSharedSlots
|| _morpher.TraceManager.IsTracing
|| _morpher.MaxDegreeOfParallelism != 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: F4. A default Morpher never takes the shared walk: maxDegreeOfParallelism defaults to 0 (Morpher.cs:29), so this guard sends every call to base.Apply and only callers who pass degree 1 get fewer batches. The PR body defers parallel sharing but presents the speedup as general; say in the description that it needs degree 1, or say why the default should not be covered.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we do 4 at once? I did a test and it worked better. What do you think? See #582.

@claude

claude Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor
  1. Verdict: approve with fixes. The branch was rebased onto Remove HermitCrab's analysis-cascade memo #522, so I re-reviewed the whole PR. The shared walk still matches the per-template walk in output order, optional-slot skips and per-template admission.
  2. Most important: F4. A Morpher built with the default maxDegreeOfParallelism = 0 never reaches the shared walk (src/SIL.Machine.Morphology.HermitCrab/AnalysisAffixTemplatesRule.cs:62), so the speedup only reaches callers who pass degree 1. Separately, after the rebase the PR body still cites AnalysisScope.TemplateMemo and AnalysisStratumRule.ApplyTemplateBattery. Remove HermitCrab's analysis-cascade memo #522 removed both, and a grep of src/SIL.Machine.Morphology.HermitCrab finds neither, so the "Memo and folding" bullet and the parity argument describe code that no longer exists.
  3. Critical 0, Important 1, Low 0 new.
  4. Ran dotnet test tests/SIL.Machine.Morphology.HermitCrab.Tests -c Release --filter FullyQualifiedName~AnalysisAffixTemplatesRuleTests: 11 passed, 0 failed. From gh pr checks 546: Build on ubuntu-22.04 and on windows-latest passed, and Create NuGet package is pending. The Comment hygiene checks passed, but they are advisory and do not stand in for --agent-strict.
  5. Not verified: the corpus parse-time and batch-count figures, which came from temporary counters that are not in the branch, and whether those figures still hold on the rebased base without the Remove HermitCrab's analysis-cascade memo #522 memo.

Public API, target framework, package, machine.py parity: None verified. AnalysisAffixTemplatesRule and CanApply are internal.

Findings: F1 addressed, F2 addressed, F3 addressed, F4 new.

Reviewed at 5ed55e0

johnml1135 added a commit that referenced this pull request Oct 23, 2026
…round

AnalysisScope.TemplateMemo no longer wraps the battery on master, so the statement that #546 composes with it is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
johnml1135 added a commit that referenced this pull request Oct 26, 2026
…round

AnalysisScope.TemplateMemo no longer wraps the battery on master, so the statement that #546 composes with it is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@johnml1135
johnml1135 force-pushed the perf/shared-template-slot-batches branch from 5ed55e0 to 536686d Compare October 26, 2026 11:50
johnml1135 added a commit that referenced this pull request Oct 28, 2026
R1, R2, R5 and T1 are kept with their net-of-own-cost numbers in steps, R4 is a recorded dead end, R6 is unbuilt, R3 and R1 on #546 are in progress. Adds ledger rows 38-46.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
johnml1135 added a commit that referenced this pull request Oct 28, 2026
Rows 43 and 44 move from in progress to done. R1 on #546 is draft PR #577
(commit bbb81a0, shadow 0 violations); R3 ordered look-ahead is draft
PR #576 (shadow 0 violations). Adds the round-doc section for both and
the PanGloss port issues (sillsdev/PanGloss #65, #66).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@johnml1135
johnml1135 requested a review from jtmaxwell3 October 28, 2026 18:08
johnml1135 and others added 4 commits October 28, 2026 21:42
…k guard, avoid per-node closures

The shared walk is only taken when MaxAlternatives is 0, so its limit check
could never fire; the guard comment now names all five fallback conditions;
Walk checks admitted templates with a loop instead of a capturing lambda.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#522 removed AnalysisStateKey along with the analysis-cascade memo and
kept the same identity as AnalysisStratumRule.AnalysisMergeKey, so the
shared-walk equivalence test now compares outputs under that key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants