Skip to content

fix(tools): stop importing Hermes to read reserved tool names - #49

Merged
offendingcommit merged 1 commit into
mainfrom
fix/reserved-names-without-host-import
Oct 31, 2026
Merged

offendingcommit merged 1 commit into
mainfrom
fix/reserved-names-without-host-import

Conversation

@offendingcommit

Copy link
Copy Markdown
Owner

Fixes the plugin-load deadlock behind offendingcommit/infra#1558, where the standalone Hermes gateways crashlooped for 15 h.

Cause. In 9a4f3c7, validate_tool_name() → _agent_loop_tool_names() imports model_tools. Hermes v0.22.5 runs discover_plugins() from inside model_tools' own import (line 163) and registers plugins on worker threads. Each worker blocked on the module lock until its 10 s load deadline. This was confirmed with a faulthandler dump in the live pod: the plugin-load:* threads sit in _lock_unlock_module under _agent_loop_tool_names, while the main thread waits in run_with_load_deadline. Python start went from 2.4 s to 51–71 s.

The live read also could not have worked: _AGENT_LOOP_TOOLS is defined at line 607, after discovery.

Change.

  • _AGENT_LOOP_TOOL_NAMES is the runtime source again, with a comment explaining why it must not import the host.
  • test_reserved_tool_names_match_the_host in the pinned contract lane still pins the static set to the deployed Hermes.

Proof.

  • just test: 276 tests OK, EXIT=0.
  • just test-contract-pinned: 20 tests OK, EXIT=0, including the reserved-name contract.
  • New test_tool_name_validation_never_imports_the_host: fails on main with AssertionError: tool-name validation imported model_tools, and passes with this change.

After merge, the infra plugin snapshot needs re-promoting with this kit revision.

https://claude.ai/code/session_01B6XeAUeJ9KK4jQJCuufXYV

9a4f3c7 made validate_tool_name() run `from model_tools import ...`.
Hermes registers plugins from loader threads while model_tools is still
importing (model_tools calls discover_plugins() at import time), so every
registering thread waited on the module lock until its 10 s plugin-load
deadline. Every Python start in a gateway went from about 2 s to 51-71 s,
and the standalone gateways crashlooped on their startup probe until
offendingcommit/infra#1558 rolled the snapshot back.

The live read could never succeed during registration anyway: v0.22.5
defines _AGENT_LOOP_TOOLS after it runs discovery. The static set is now
the runtime source, and the pinned contract lane keeps it equal to the
deployed host's names.

A regression test fails if tool-name validation imports model_tools.

Refs: offendingcommit/infra#1558
Claude-Session: https://claude.ai/code/session_01B6XeAUeJ9KK4jQJCuufXYV
@offendingcommit
offendingcommit merged commit ca9e1c9 into main Oct 31, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant