Skip to content

Filter synthesis by the allomorphs used - #548

Draft
jtmaxwell3 wants to merge 5 commits into
masterfrom
filter-synthesis-by-allomorph
Draft

jtmaxwell3 wants to merge 5 commits into
masterfrom
filter-synthesis-by-allomorph

Conversation

@jtmaxwell3

@jtmaxwell3 jtmaxwell3 commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Parsing "kanna" in the DolganShared project takes forever to synthesize because many of the morphemes have lots of unconstrained allomorphs. For instance, the -LA: morpheme has more than 30 unconstrained allomorphs. The synthesizer tries all possible combinations of allomorphs in order to find the combination that produces the input word. One analysis of "kanna" had 5 morphemes that together had ~10,000 allomorph combinations. This was just one of 315 analyses for "kanna".

This pull request solves the problem by recording the allomorphs used during analysis and restricting the synthesizer to only use those allomorphs. This reduces the number of syntheses for "kanna" by four orders of magnitude.

Filtering by allomorphs wasn't as simple as you might imaging because some allomorphs block other free fluctuation allomorphs, so I couldn't just filter allomorphs at the top of the loop.


This change is Reviewable

@jtmaxwell3
jtmaxwell3 marked this pull request as draft October 22, 2026 16:29
Comment thread src/SIL.Machine.Morphology.HermitCrab/Word.cs
Comment thread src/SIL.Machine.Morphology.HermitCrab/Word.cs
Comment thread src/SIL.Machine.Morphology.HermitCrab/Word.cs
Comment thread src/SIL.Machine.Morphology.HermitCrab/Word.cs
appliedAllomorphIndices.Add(i);

if (_rule.Blockable && outWord.CheckBlocking(out Word newWord))
if (input.IsAllomorphApplicable(allo))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: F6. An allomorph that matches but was not used in analysis is dropped with no trace event, neither MorphologicalRuleApplied nor MorphologicalRuleNotApplied. In a Try-a-Word trace it just disappears, which hides cases like F1.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F6 still applies at bb7211a: the reworked block at SynthesisAffixProcessRule.cs:199 still drops a matched but unused allomorph with no trace event, and the MPR failures for unused allomorphs are now untraced too.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

F6 still applies at f7aea42: the MPR failures at SynthesisAffixProcessRule.cs:150 and :167 are traced again, but an allomorph that matches and was not used in analysis (the IsAllomorphApplicable check at line 199) still produces neither MorphologicalRuleApplied nor MorphologicalRuleNotApplied, so it vanishes from a trace.

@claude

This comment has been minimized.

@codecov-commenter

codecov-commenter commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.86207% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.59%. Comparing base (2599204) to head (f7aea42).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
...ab/MorphologicalRules/SynthesisAffixProcessRule.cs 66.66% 2 Missing and 2 partials ⚠️
src/SIL.Machine.Morphology.HermitCrab/Word.cs 76.92% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #548      +/-   ##
==========================================
+ Coverage   74.57%   74.59%   +0.01%     
==========================================
  Files         453      453              
  Lines       38314    38332      +18     
  Branches     5280     5283       +3     
==========================================
+ Hits        28573    28593      +20     
+ Misses       8576     8574       -2     
  Partials     1165     1165              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@claude

This comment has been minimized.

@claude

claude Bot commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

Approve. The new head adds a test for the allomorph filter and traces MPR failures for every allomorph again. Only a Low trace gap remains.

  1. Most important: F6, SynthesisAffixProcessRule.cs:199. When analysis did not use an allomorph that matches, synthesis still drops it without a trace event, so it vanishes from a Try-a-Word trace.
  2. Counts: 0 Critical, 0 Important, 1 Low (F6). No new findings.
  3. Ran: gh pr checks 548. The ubuntu-22.04 and windows-latest builds and claude-review are pending. Comment hygiene passes, but that check is advisory. Diffed bb7211a..f7aea42. Ran dotnet test filtered to AffixProcessRuleTests on the merge commit: 22 passed, 0 failed.
  4. Not verified: no fixture covers the F1 case (the disjunctive break) or the F2 equality case, so those fixes were traced by hand only. I did not check the DolganShared speedup.
  5. Design: the earlier suggestion still applies. A single list of (rule, allomorph) pairs would keep _allomorphApps in step with _mruleApps. Only the test uses Morpher.SynthesizeWord and LexicalLookup, which is now internal.

Public API, target framework, package, or sillsdev/machine.py parity changed: None verified. The changed members are internal, and machine.py has no HermitCrab.

Findings: F1 addressed; F2 addressed; F3 withdrawn; F4 addressed; F5 addressed; F6 open.

Reviewed at f7aea42

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants