Two amplifying optimizations to per-turn overhead in the gateway:
1. get_tool_definitions() memoization (model_tools.py)
Keyed on (frozenset(enabled), frozenset(disabled),
registry._generation, config.yaml mtime+size). Only active when
quiet_mode=True (which is every hot-path caller — gateway,
AIAgent.__init__); quiet_mode=False keeps the existing print side
effects. Cached path returns a shallow-copy list sharing read-only
schema dicts.
Measured: 7.5 ms → 0.01 ms per call (~750× speedup). Gateway
constructs fresh AIAgent per message, so this saves ~7 ms/turn before
any LLM work.
2. check_fn() TTL cache (tools/registry.py)
check_fn callables like check_terminal_requirements probe external
state (Docker daemon, Modal SDK, playwright binary). For a long-lived
process, hitting them on every get_definitions() pass was pure waste
— external state changes on human timescales. 30 s TTL so env-var
flips (hermes tools enable X) propagate within a turn or two without
explicit invalidation.
Measured: first call 7.5ms → 1.6ms (check_fn probes now dominate);
subsequent calls ~0.01ms via the upstream memoization.
Invalidation surface:
- registry._generation bumps on register/deregister/register_toolset_alias,
invalidating the memoized definitions automatically.
- config.yaml mtime in the cache key captures user-visible config edits
affecting dynamic schemas (execute_code mode, discord allowlist).
- invalidate_check_fn_cache() exposed for explicit flushes (e.g. after
hermes tools enable/disable).
- tests/conftest.py autouse fixture clears both caches before every test
so env-var monkeypatches don't see stale results.
Also fixes a regression from PR #17046 that I missed:
- tools/web_tools.py — Firecrawl was removed from module scope by the
lazy import, breaking 8 tests that patch 'tools.web_tools.Firecrawl'.
Applied the same _FirecrawlProxy pattern used in auxiliary_client/
run_agent for OpenAI (module-level proxy that looks like the class
but imports the SDK on first call/isinstance; patch() replaces the
attribute as usual).
Verified:
- 49/49 tests/tools/test_web_tools_config.py pass (was 8 failing on main)
- 68/68 tests/tools/test_homeassistant_tool.py pass (was 1 failing in
the full suite due to check_fn TTL cross-test pollution; fixed by
the autouse fixture)
- 3887/3895 tests/tools/ (8 pre-existing fails: 2 delegate, 1 mcp
dynamic discovery, 5 mcp structured content — all confirmed on main)
- 2973/2976 tests/agent/ + tests/run_agent/ (3 pre-existing fails)
- 868/868 tests/run_agent/ (excluding test_run_agent.py which has
pre-existing suite-level issues)
- Live smoke: 2 turns + /model switch + tool calls, zero errors in
agent.log session window.
Co-authored-by: teknium1 <teknium@users.noreply.github.com>
The original tree-wide ast.walk() would match registry.register() calls
inside functions too. Restrict to top-level ast.Expr statements so helper
modules that call registry.register() inside a function are never picked
up as tool modules.
Add BudgetConfig dataclass to centralize and make overridable the
hardcoded constants (50K per-result, 200K per-turn, 2K preview) that
control when tool outputs get persisted to sandbox. Configurable at
the RL environment level via HermesAgentEnvConfig fields, threaded
through HermesAgentLoop to the storage layer.
Resolution: pinned (read_file=inf) > env config overrides > registry
per-tool > default. CLI override: --env.turn_budget_chars 80000
When a connected MCP server sends a ToolListChangedNotification (per the
MCP spec), Hermes now automatically re-fetches the tool list, deregisters
removed tools, and registers new ones — without requiring a restart.
This enables MCP servers with dynamic toolsets (e.g. GitHub MCP with
GITHUB_DYNAMIC_TOOLSETS=1) to add/remove tools at runtime.
Changes:
- registry.py: add ToolRegistry.deregister() for nuke-and-repave refresh
- mcp_tool.py: extract _register_server_tools() from
_discover_and_register_server() as a shared helper for both initial
discovery and dynamic refresh
- mcp_tool.py: add _make_message_handler() and _refresh_tools() on
MCPServerTask, wired into all 3 ClientSession sites (stdio, new HTTP,
deprecated HTTP)
- Graceful degradation: silently falls back to static discovery when the
MCP SDK lacks notification types or message_handler support
- 8 new tests covering registration, refresh, handler dispatch, and
deregister
Salvaged from PR #1794 by shivvor2.
When a tool plugin registers a schema without an explicit 'name' key,
get_definitions() crashes with KeyError:
available_tool_names = {t["function"]["name"] for t in filtered_tools}
Fix: always merge entry.name into schema so 'name' is never missing.
Refs: #3729
Co-authored-by: ekkoitac <ekko.itac@gmail.com>
* feat: show estimated tool token context in hermes tools checklist
Adds a live token estimate indicator to the bottom of the interactive
tool configuration checklist (hermes tools / hermes setup). As users
toggle toolsets on/off, the total estimated context cost updates in
real time.
Implementation:
- tools/registry.py: Add get_schema() for check_fn-free schema access
- hermes_cli/curses_ui.py: Add optional status_fn callback to
curses_checklist — renders at bottom-right of terminal, stays fixed
while items scroll
- hermes_cli/tools_config.py: Add _estimate_tool_tokens() using
tiktoken (cl100k_base, already installed) to count tokens in the
JSON-serialised OpenAI-format tool schemas. Results are cached
per-process. The status function deduplicates overlapping tools
(e.g. browser includes web_search) for accurate totals.
- 12 new tests covering estimation, caching, graceful degradation
when tiktoken is unavailable, status_fn wiring, deduplication,
and the numbered fallback display
* fix: use effective toolsets (includes plugins) for token estimation index mapping
The status_fn closure built ts_keys from CONFIGURABLE_TOOLSETS but the
checklist uses _get_effective_configurable_toolsets() which appends plugin
toolsets. With plugins present, the indices would mismatch, causing
IndexError when selecting a plugin toolset.
- Registry now warns when a tool name is overwritten by a different
toolset (silent dict overwrite was the previous behavior)
- MCP tool registration checks for collisions with non-MCP (built-in)
tools before registering. If an MCP tool's prefixed name matches an
existing built-in, the MCP tool is skipped and a warning is logged.
MCP-to-MCP collisions are allowed (last server wins).
- Both regular MCP tools and utility tools (resources/prompts) are
guarded.
- Adds 5 tests covering: registry overwrite warning, same-toolset
re-registration silence, built-in collision skip, normal registration,
and MCP-to-MCP collision pass-through.
Reported by k_sze (KONG) — MiniMax MCP server's web_search tool could
theoretically shadow Hermes's built-in web_search if prefixing failed.
- Add 'emoji' field to ToolEntry and 'get_emoji()' to ToolRegistry
- Add emoji= to all 50+ registry.register() calls across tool files
- Add get_tool_emoji() helper in agent/display.py with 3-tier resolution:
skin override → registry default → hardcoded fallback
- Replace hardcoded emoji maps in run_agent.py, delegate_tool.py, and
gateway/run.py with centralized get_tool_emoji() calls
- Add 'tool_emojis' field to SkinConfig so skins can override per-tool
emojis (e.g. ares skin could use swords instead of wrenches)
- Add 11 tests (5 registry emoji, 6 display/skin integration)
- Update AGENTS.md skin docs table
Based on the approach from PR #1061 by ForgingAlex (emoji centralization
in registry). This salvage fixes several issues from the original:
- Does NOT split the cronjob tool (which would crash on missing schemas)
- Does NOT change image_generate toolset/requires_env/is_async
- Does NOT delete existing tests
- Completes the centralization (gateway/run.py was missed)
- Hooks into the skin system for full customizability
get_definitions() already wrapped check_fn() calls in try/except,
but is_toolset_available() did not. A failing check (network error,
missing import, bad config) would propagate uncaught and crash the
CLI banner, agent startup, and tools-info display.
Now is_toolset_available() catches all exceptions and returns False,
matching the existing pattern in get_definitions().
Added 4 tests covering exception handling in is_toolset_available(),
check_toolset_requirements(), get_definitions(), and
check_tool_availability().
Closes#402
logger.error() only records the exception message string, silently
discarding the stack trace. Switch to logger.exception() which
automatically appends the full traceback to the log output.
Without this change, when a tool handler raises an unexpected error
the log shows only the exception type and message, making it
impossible to determine which line caused the failure or trace
through nested calls.
