Skip to content

Type text on the simulator's hardware keyboard layout in FBSimulatorControl - #1001

Open
joshuayoes wants to merge 1 commit into
meta:mainfrom
joshuayoes:keyboard-layout-text-framework
Open

joshuayoes wants to merge 1 commit into
meta:mainfrom
joshuayoes:keyboard-layout-text-framework

Conversation

@joshuayoes

@joshuayoes joshuayoes commented Nov 10, 2026 •

Copy link
Copy Markdown

Motivation

SimulatorHIDEvent key presses are USB HID usages, which name key positions. The helpers that turn text into key presses (the REPL's text command here, and idb ui text in the client) map characters to positions with a US table, so on a simulator whose hardware keyboard layout is not US the guest types different characters: "azerty" arrives as "qwerty" on French. See #666.

This adds the framework support for typing text on a given layout:

  • SimulatorHIDEvent.text(_:layout:table:) builds the key presses for a string from a layout table. Every character is resolved before any event is built, so text with a character the layout has no key for throws SimulatorHIDError.characterNotOnKeyboardLayout and sends nothing. The error names the character's offset and the layout, not the character, so typed text stays out of logs.
  • SimulatorUSKeyboardLayoutTable is the US table that was private to the REPL's command dispatcher. A test pins it entry by entry to KEY_MAP in idb/common/hid.py.
  • SimulatorUCHRKeyboardLayoutTable reads every other layout from the runtime's own USBKeyboardLayouts.bundle, the data the guest types with, including option and dead-key sequences. The planned key presses are replayed through UCKeyTranslate and refused if they would not reproduce the text.
  • Simulator.keyboard.typingEvent(for:layout:) types on a named layout, or on the one the simulator's keyboard settings give a newly launched app (AppleKeyboards, AppleKeyboardsExpanded, KeyboardLastUsed, and AppleLanguages when the keyboards have not been written yet). A layout that cannot be determined is assumed to be US, and the reason is logged.

The REPL's text command now goes through this, so it follows the simulator's layout. On a simulator set to US it sends the same key presses as before for every character the US table has; characters outside that table that the runtime's US layout can type, such as "é", are now typed instead of refused.

Known limit: an app keeps the hardware layout it was launched with. The layout is read from the simulator's keyboard settings, so after changing the simulator's keyboards an app has to be relaunched before the two agree.

No change to idb.proto, the gRPC server or the Python client.

FBSimulatorControl now imports CoreServices for UCKeyTranslate. The Xcode build links it through autolinking; the generated project.yml and LibraryPackage.json do not list it and I have not edited them.

Test Plan

Run on this change alone:

  • ./build.sh test FBSimulatorControlUnitTests: 513 tests in 70 suites pass. New suites: SimulatorUSKeyboardLayoutTableTests, SimulatorHIDEventTextTests, SimulatorKeyboardLayoutTests, SimulatorUCHRKeyMapTests.
  • ./build.sh test CompanionLibTests: 163 tests in 27 suites pass.
  • ./build.sh build idb_companion succeeds.

Run with the follow-up change applied as well:

  • ./build.sh test FBSimulatorControlSmokeTests on fresh iOS 27.2 and iOS 26.2 simulators: 16 tests run, 0 failures (1 and 2 skipped; the skips are existing tests). SimulatorKeyboardSmokeTests (14 tests) loads every layout the runtime ships, checks French, German, Canadian French and US key presses against the runtime's data, and checks that the layout follows the simulator's keyboard settings.
  • REPL, simulator set to French: IDB.ui.text("azerty mobiletest@example.com €ê ñ^") into a text field arrives as that text.
  • ./build.sh build all succeeds.

The unit, CompanionLib and build results are from main at 60beb1c with both changes applied. This change was built and tested alone, and the smoke tests were run, on 5f34870 before rebasing; the rebase touched none of these files beyond a clean merge in IDBCommandExecutor.swift.

Related PRs

The gRPC event, the idb ui text --layout option, the end-to-end test and the website docs follow in #1002, which builds on this one. FBSimulatorControl/README.md is updated here.

🤖 Generated with Claude Code

…ontrol

SimulatorHIDEvent key presses are USB HID usages, which name key positions.
The helpers that turn text into key presses map characters to positions with
a US table, so on a simulator whose hardware keyboard layout is not US the
guest types different characters: "azerty" arrives as "qwerty" on French.

Add framework support for typing text on a given layout:

- SimulatorHIDEvent.text(_:layout:table:) builds the key presses for a
  string from a layout table, resolving every character before any event is
  built.
- SimulatorUSKeyboardLayoutTable is the US table that was private to the
  REPL's command dispatcher.
- SimulatorUCHRKeyboardLayoutTable reads every other layout from the
  runtime's USBKeyboardLayouts.bundle and confirms the planned key presses
  with UCKeyTranslate.
- Simulator.keyboard.typingEvent(for:layout:) types on a named layout, or on
  the one the simulator's keyboard settings give a newly launched app.

The REPL's text command now goes through this.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant