mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-04-25 00:51:20 +00:00
2ff1ef6ae6
Byte-level reasoning models (xiaomi/mimo-v2-pro, kimi, glm) can emit lone surrogates in reasoning output. The proactive sanitizer walked content/ name/tool_calls but not extra fields like reasoning or the nested reasoning_details array. Surrogates in those fields survived the proactive pass, crashed json.dumps() in the OpenAI SDK, and the recovery block's _sanitize_messages_surrogates(messages) call also didn't check those fields — so 'found' was False, no retry happened, and after 3 attempts the user saw: API call failed after 3 retries. 'utf-8' codec can't encode characters in position N-M: surrogates not allowed Changes: - _sanitize_messages_surrogates: walk any extra string fields (reasoning, reasoning_content, etc.) and recurse into nested dict/list values (reasoning_details). Mirrors _sanitize_messages_non_ascii coverage added in PR #10537. - _sanitize_structure_surrogates: new recursive walker, mirror of _sanitize_structure_non_ascii but for surrogate recovery. - UnicodeEncodeError recovery block: also sanitize api_messages, api_kwargs, and prefill_messages (not just the canonical messages list — the API-copy carries reasoning_content transformed from reasoning and that's what the SDK actually serializes). Always retry on detected surrogate errors, not only when we found something to strip — gate on error type per PR #10537's pattern. Tests: extended tests/cli/test_surrogate_sanitization.py with coverage for reasoning, reasoning_content, reasoning_details (flat and deeply nested), structure walker, and an integration case that reproduces the exact api_messages shape that was crashing. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_branch_command.py | ||
| test_cli_approval_ui.py | ||
| test_cli_background_tui_refresh.py | ||
| test_cli_browser_connect.py | ||
| test_cli_context_warning.py | ||
| test_cli_extension_hooks.py | ||
| test_cli_file_drop.py | ||
| test_cli_image_command.py | ||
| test_cli_init.py | ||
| test_cli_interrupt_subagent.py | ||
| test_cli_loading_indicator.py | ||
| test_cli_mcp_config_watch.py | ||
| test_cli_new_session.py | ||
| test_cli_plan_command.py | ||
| test_cli_prefix_matching.py | ||
| test_cli_preloaded_skills.py | ||
| test_cli_provider_resolution.py | ||
| test_cli_retry.py | ||
| test_cli_save_config_value.py | ||
| test_cli_secret_capture.py | ||
| test_cli_skin_integration.py | ||
| test_cli_status_bar.py | ||
| test_cli_status_command.py | ||
| test_cli_tools_command.py | ||
| test_compress_focus.py | ||
| test_cwd_env_respect.py | ||
| test_fast_command.py | ||
| test_manual_compress.py | ||
| test_personality_none.py | ||
| test_quick_commands.py | ||
| test_reasoning_command.py | ||
| test_resume_display.py | ||
| test_session_boundary_hooks.py | ||
| test_stream_delta_think_tag.py | ||
| test_surrogate_sanitization.py | ||
| test_tool_progress_scrollback.py | ||
| test_worktree.py | ||
| test_worktree_security.py | ||