hermes-agent/tests
Teknium cb38ce28cb
refactor(codex): drop SDK responses.stream() helper; consume events directly (#33042)
* refactor(codex): drop SDK responses.stream() helper; consume events directly

The OpenAI Python SDK's high-level `client.responses.stream(...)` helper
does post-hoc typed reconstruction from the terminal
`response.completed.response.output` field.  The chatgpt.com Codex
backend has been observed (today, gpt-5.5) to ship `response.output =
null` on terminal frames, which crashes the SDK with `TypeError:
'NoneType' object is not iterable` mid-iteration.

Carlton's #32963 patched the symptom by wrapping the helper in
try/except and recovering from the same per-event accumulator the SDK
was supposed to populate.  This PR removes the helper from the call
path entirely: we now use `client.responses.create(stream=True)` (raw
AsyncIterable of SSE events) and assemble the final response object
ourselves from `response.output_item.done` events as they arrive.  The
terminal event's `output` field is never read for content.  Same
strategy OpenClaw uses for the same backend.

This makes Hermes structurally immune to the bug class, not patched.
The next time OpenAI ships a shape change to chatgpt.com's terminal
frame, our consumer keeps working because it doesn't read that frame
for content — only for usage/status/id.

Changes
- `agent/codex_runtime.py`: new `_consume_codex_event_stream()` shared
  consumer; `run_codex_stream()` uses `responses.create(stream=True)`;
  `run_codex_create_stream_fallback()` collapses into a thin alias
  since the primary path now does what the fallback used to do.
- `agent/auxiliary_client.py`: `_CodexCompletionsAdapter` uses the
  same consumer; old null-output recovery helpers deleted as
  unreferenced.
- Tests migrated: fixtures that mocked `responses.stream` now mock
  `responses.create` returning a raw iterable.  New regression test
  asserts the auxiliary path returns streamed items even when the
  terminal event's `output` is literally `null`.

Validation
- Live: tested against fresh OAuth on `chatgpt.com/backend-api/codex`
  with `gpt-5.5` — response built correctly with `response.output=null`
  on the terminal frame, all events consumed, usage/reasoning tokens
  propagated.
- `tests/run_agent/test_run_agent_codex_responses.py` +
  `tests/agent/test_auxiliary_client.py`: 242 passed.

* test+fix(codex): migrate streaming tests, raise on truncated streams

CI surfaced 10 test failures across tests/run_agent/test_streaming.py
and tests/run_agent/test_codex_xai_oauth_recovery.py — both files had
their own `responses.stream(...)` mocks I missed in the first sweep.

agent/codex_runtime.py: _consume_codex_event_stream() now raises
"Codex Responses stream did not emit a terminal response" when the
stream ends without any terminal frame AND no usable content. This
preserves the signal callers used to get from the SDK's high-level
helper, which they distinguished from "completed with empty body"
in error handling.

Tests migrated:
- test_streaming.py: text-delta callback, activity-touch, and
  remote-protocol-error tests all switch from mocking responses.stream
  to responses.create returning an iterable of events.
- test_codex_xai_oauth_recovery.py: prelude-error tests are recast as
  wire-error-event tests (the new path raises _StreamErrorEvent
  directly when the wire emits type=error, which is strictly better
  than the old two-phase "SDK RuntimeError → retry → fallback"). The
  retry-on-transport-error test moves from responses.stream side-effect
  to responses.create side-effect.

Verified live against chatgpt.com Codex with gpt-5.5 — AIAgent.chat()
through the full codex_responses path returns correctly, 319/319
targeted tests passing.
2026-05-27 00:30:06 -07:00
..
acp test(acp): drop flaky runtime_calls[-1] tail-position assertion 2026-05-24 23:23:12 -07:00
acp_adapter feat(azure-foundry): add Microsoft Entra ID auth 2026-05-18 10:14:38 -07:00
agent refactor(codex): drop SDK responses.stream() helper; consume events directly (#33042) 2026-05-27 00:30:06 -07:00
cli feat(cli): show live background terminal-process count in status bar (#32061) 2026-05-25 05:35:02 -07:00
cron test(cron): guard schedule-required description text on CRONJOB_SCHEMA 2026-05-26 14:09:37 -07:00
docker test(docker): fix svstat 'want up' assertion in profile-gateway lifecycle test 2026-05-25 12:25:06 +10:00
e2e refactor(gateway): migrate Discord adapter to bundled plugin (full Teams parity) 2026-05-22 14:21:41 -07:00
fakes
gateway fix(gateway): refresh cached agent tools on /reload-mcp 2026-05-26 14:28:51 -07:00
hermes_cli Merge remote-tracking branch 'origin/main' into jq/hermes-update-branch-flag 2026-05-27 00:48:25 -04:00
hermes_state feat(session_search): single-shape tool with discovery, scroll, browse — no LLM (#27590) 2026-05-17 23:28:45 -07:00
honcho_plugin chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) 2026-05-17 02:29:41 -07:00
integration refactor(gateway): migrate Discord adapter to bundled plugin (full Teams parity) 2026-05-22 14:21:41 -07:00
openviking_plugin
plugins fix: parse Codex image generation SSE directly 2026-05-26 20:40:29 -07:00
providers fix(custom): pass custom provider extra body 2026-05-21 07:48:53 -07:00
run_agent refactor(codex): drop SDK responses.stream() helper; consume events directly (#33042) 2026-05-27 00:30:06 -07:00
scripts feat(acp-registry): switch to uvx distribution, drop npm launcher 2026-05-14 22:27:09 -07:00
skills fix(skills): add timeout to Google OAuth urlopen calls 2026-05-19 00:11:44 -07:00
stress docs: align kanban readiness docs and smoke tests 2026-05-18 21:07:03 -07:00
tools Merge pull request #22534 from wesleysimplicio/fix/voice-mode-docker-respect-pulse-pipewire 2026-05-27 13:59:12 +10:00
tui_gateway chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) 2026-05-17 02:29:41 -07:00
website docs(skills): explain restoring bundled skills 2026-05-05 13:46:20 -07:00
__init__.py
conftest.py test: isolate API server env in gateway tests 2026-05-25 14:54:02 -07:00
run_interrupt_test.py
test_account_usage.py
test_atomic_replace_symlinks.py
test_base_url_hostname.py
test_batch_runner_checkpoint.py
test_bitwarden_secrets.py perf(cli): cut hermes startup 63% — flip head-to-head vs codex (#31968) 2026-05-25 03:06:39 -07:00
test_cli_file_drop.py
test_cli_manual_compress.py fix(tests): catch up six stale tests after compression/aux/kanban changes (#28465) 2026-05-18 21:43:59 -07:00
test_cli_skin_integration.py
test_ctx_halving_fix.py fix(cache): kill long-lived prefix layout — system prompt is now byte-static within a session (#24778) 2026-05-12 20:46:04 -07:00
test_empty_model_fallback.py
test_env_loader_secret_sources.py fix(secrets): only apply external secrets once per HERMES_HOME per process (#32271) 2026-05-25 15:18:55 -07:00
test_evidence_store.py
test_gateway_streaming_nested_config.py fix(gateway): load streaming config from nested gateway.streaming key 2026-05-14 14:51:07 -07:00
test_get_tool_definitions_cache_isolation.py
test_hermes_bootstrap.py fix(entry-points): guard hermes_bootstrap import so partial updates don't brick hermes (#22091) 2026-05-08 14:43:13 -07:00
test_hermes_constants.py fix(security): guard os.chmod(parent) against / and top-level dirs 2026-05-20 22:56:55 -07:00
test_hermes_home_profile_warning.py fix(constants): warn once when get_hermes_home() falls back under an active profile (#18746) 2026-05-02 01:49:55 -07:00
test_hermes_logging.py fix(tests): catch up 25 stale tests after recent merges (#28626) 2026-05-19 01:28:32 -07:00
test_hermes_state.py fix(gateway): separate observed Telegram group context 2026-05-23 01:33:42 -07:00
test_hermes_state_wal_fallback.py fix(sqlite): fall back to journal_mode=DELETE on NFS/SMB/FUSE (#22043) 2026-05-09 02:09:35 -07:00
test_honcho_client_config.py
test_install_sh_browser_install.py fix(install): support non-sudo service-user installs on apt distros (#25814) 2026-05-14 09:05:31 -07:00
test_install_sh_pythonpath_sanitization.py fix: harden install.sh against inherited Python env leakage 2026-05-06 04:02:02 -07:00
test_install_sh_setup_wizard_tty_probe.py
test_install_sh_symlink_stomp.py fix(install): preserve pip entry point when re-running on symlinked install 2026-05-14 07:08:45 -07:00
test_install_sh_termux_network_prereqs.py fix: strengthen termux install network prerequisites 2026-05-07 13:04:08 -07:00
test_ipv4_preference.py
test_lazy_session_regressions.py fix: resolve lazy session creation regressions (#18370 fallout) (#20363) 2026-05-06 01:11:49 +05:30
test_lint_config.py lint: enable PLW1514 as a blocking ruff rule 2026-05-08 14:27:40 -07:00
test_live_system_guard_self_test.py chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) 2026-05-17 02:29:41 -07:00
test_mcp_serve.py fix(mcp): unwrap platforms key in channels_list 2026-05-07 13:41:16 -07:00
test_mini_swe_runner.py
test_minimax_model_validation.py
test_minimax_oauth.py fix(minimax-oauth): refresh short-lived access tokens per request (#30619) 2026-05-22 15:16:15 -07:00
test_minisweagent_path.py
test_model_picker_scroll.py
test_model_tools.py chore: remove Atropos RL environments and tinker-atropos integration (#26106) 2026-05-15 10:36:38 +05:30
test_model_tools_async_bridge.py
test_ollama_num_ctx.py
test_package_json_lazy_deps.py fix(update): make Camofox lazy-installed instead of eager (#27055) 2026-05-16 12:15:45 -07:00
test_packaging_metadata.py
test_plugin_skills.py fix(skills): support category-qualified local skill names 2026-05-05 10:15:31 -07:00
test_process_loop_event_loop_warning.py fix(cli): replace get_event_loop() with get_running_loop() to silence RuntimeWarning in process_loop thread (#19285) 2026-05-07 06:35:54 -07:00
test_project_metadata.py fix(packaging): ship dashboard plugin assets in wheel 2026-05-18 20:35:00 -07:00
test_retry_utils.py
test_run_tests_parallel.py test: use subprocesses for each test file (#29016) 2026-05-21 16:40:04 +05:30
test_sanitize_tool_error.py security: sanitize tool error strings before injecting into model context (#26823) 2026-05-16 00:57:39 -07:00
test_sql_injection.py
test_subprocess_home_isolation.py fix: avoid process-wide cron profile home mutation 2026-05-18 17:39:50 +00:00
test_termux_all_extra_compat.py fix: add termux-all install profile and safe fallbacks 2026-05-07 13:04:08 -07:00
test_timezone.py chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) 2026-05-17 02:29:41 -07:00
test_toolset_distributions.py
test_toolsets.py test(toolsets): lock web search into default platform coverage 2026-05-14 08:03:33 -07:00
test_trajectory_compressor.py
test_trajectory_compressor_async.py
test_transform_llm_output_hook.py test+docs: cover transform_llm_output hook + release author map 2026-05-07 05:46:05 -07:00
test_transform_tool_result_hook.py
test_tui_gateway_server.py feat: add TUI session orchestrator 2026-05-26 20:51:59 -07:00
test_utils_truthy_values.py
test_yuanbao_integration.py
test_yuanbao_markdown.py
test_yuanbao_pipeline.py
test_yuanbao_proto.py