Repository navigation
Add Arabic, and read macOS keyboards from the OS - #83
Merged
Merged
Conversation
describe_keymaps on macOS walks the character block of every selectable keyboard through UCKeyTranslate, the counterpart of the Windows ToUnicodeEx walk, so a variant whose keys differ from the bundled table (Apple's Arabic against Arabic 101) is corrected on the keys the user has. The current keyboard comes first, as the DB keeps the first of a language. Lam-alef keys fold to their presentation form; tatweel is left out so `_` never reads as a letter.
HunspellSource gains Extras: prefixes attached to every form, respellings for a key that types one code point where the dictionary spells two, and forms dropped last. Every existing source uses NO_EXTRAS. encoding_of_aff now finds a tab-separated SET past the head of the file, and a stems-only dictionary no longer parses affix rules it never uses.
ar.toml read off macOS ArabicPC, a 2.5 M-form Ayaspell wordlist (stems, plus article forms and the B-key lam-alef spelling), a stop list checked against en-US, Arabic's long vowels for plausibility, and presentation forms counted as Arabic script. Mapped on macOS (all Apple Arabic variants), Linux (ara) and in the MSI.
Arabic and Hebrew leave short vowels unwritten, so a correctly typed word with no long vowel (بسرعه) scored 0.0 as one consonant cluster while its en-US rendering (fsnui) scored 0.75, and plausibility switched it away. Text wholly in the current layout's abjad is now kept by plausibility; leaving it takes a dictionary hit. Found in daily use on macOS.
Member
|
Merged and released in 0.39.0, thank you. Your commits are in as they were. On top I fixed what live typing showed: Could you try |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Adds Arabic (Arabic 101) as a bundled layout, and gives the macOS backend a
describe_keymapsso the keyboard actually installed outranks the bundled table, as on Windows.The second part matters because Apple ships three Arabic keyboards (
Arabic,Arabic - PC,Arabic - North Africa) that differ on most of the bottom row, and all three map to oneLayoutId. On my Mac (Arabic - North Africa) the log showsadopted the OS keymap … layout=ar … keys=47 replaced=41.Trade-offs:
Btypesلا, which is two code points, and a key entry is onechar. The table carries the presentation forms (ﻻU+FEFB, plusﻷ ﻹ ﻵon the shifted keys), and the macOS reader folds the OS's two-code-point output the same way. Leaving the key out was worse: an unmapped key is dropped from a translation, sogdbread asلي, a real word, andshell_vocabulary_shadows_no_other_layouts_wordcaught it.Script::ofnow counts the presentation-form blocks as Arabic..affusesFLAG longwith anAFalias table, soarisStemsOnly, like Hebrew. Stems carry no article, though, andالstarts a large share of running text. SoHunspellSourcegains anExtrasfield: prefixes (ال وال بال فال كال لل), respellings (لا→ﻻ, sohgsblstill findsالسلام), and a drop list (حصي, whichpwdtypes as). Every other source usesNO_EXTRAS. The result is 2.5 M forms, 6.3 MB gz, mid-pack among the bundled lists. A bare one-letter clitic (وكتاب) still misses and falls to plausibility.derive_vowels("ar")now returns the written long vowels (alef forms, waw, yeh, alef maksura, the lam-alef forms). Hebrew has the same empty set today; I left it alone.ـ, alphabetic) is kept off Shift+-in both the TOML and the macOS reader, so_never reads as a letter and the snake_case heuristic stays on.encoding_of_affnow finds a tab-separatedSETon line 360 ofar.aff, and a stems-only dictionary no longer parses rules it never uses.New test
arabic_and_english_typed_in_the_wrong_layout_are_switchedruns the real detectors both ways.السلام عليكم كتاب مدرسةtyped on en-US switch toar, includingالسلامtyped withB.hello world table schooltyped onarswitch to en-US. Neither word changes in its own layout.Known gap (Windows), follow-up: an Arabic HKL resolves to
ar-EG/ar-SA/ …, notar, so the bundled layout is not matched there yet. This needs a language-subtag fallback in the id mapping. I did not want to guess at it without a Windows box to test on. Linux mapsara, and the MSI ships the files.Platforms touched
macOS (layout backend), all (data,
poltertype-detectscript ranges, xtask), Windows (MSI file entries only).Checklist
cargo fmt --allandcargo clippy --workspace --all-targets --all-features -- -D warningsare cleancargo test --workspacepassesdata/wordlists/CREDITS.md)UCKeyTranslate,TISGetInputSourceProperty/kTISPropertyUnicodeKeyLayoutData,LMGetKbdType