mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-06-09 08:21:50 +00:00
16786f3bb3
Desktop connected to a remote gateway can now attach images and PDFs and
display agent-written images. Previously the desktop passed a LOCAL file path
to image.attach; on a remote gateway that path doesn't exist, so the image was
silently dropped ("skipped unreadable path") and the vision model never saw it.
The reverse direction was also broken — images the agent wrote on the gateway
rendered as dead links in the remote client.
Gateway (tui_gateway/server.py):
- image.attach_bytes: base64 byte upload written into the gateway's own images
dir and queued via the existing native-image-attach pipeline. Magic-byte
extension sniffing, data-URL prefix + whitespace tolerance, 25 MB cap,
structured error codes. Accepts content_base64/filename (canonical) and
data/ext (older-desktop aliases).
- pdf.attach: renders each page to PNG via pdftoppm (poppler-utils) at 150 DPI
and queues the pages as images; 50 MB / 25-page caps. Accepts host path or
base64 upload.
- Shared helpers (_decode_attach_base64, _sniff_image_ext, _queue_attached_image)
so the two methods and the existing image.attach don't duplicate logic.
Gateway (hermes_cli/web_server.py):
- GET /api/media: returns a gateway-local image as a base64 data URL so remote
clients can display it. Auth-gated like every /api route, extension
allowlist + size cap, AND confined to the gateway's own media roots
(images/screenshots/cache, resolved symlink-safe) so an authed caller can't
read image-extension files anywhere on disk.
Desktop (apps/desktop):
- syncImageAttachmentsForSubmit uploads bytes via image.attach_bytes when the
connection mode is 'remote'; the local fast path is unchanged.
- media.ts gains isRemoteGateway() + gatewayMediaDataUrl(); directive-text and
markdown-text fetch images over /api/media in remote mode.
Consolidates the competing remote-media PRs (#38876, #40317, #21908, #39437)
into one coherent implementation, taking the strongest parts of each and adding
shared-helper cleanup plus the /api/media root-confinement hardening on top.
The per-profile gateway switching from #38876 is intentionally left out as a
separable feature. TUI file uploads (#40492) remain a separate surface.
Tested: 11 new tui_gateway tests + 5 /api/media endpoint tests + desktop
media.remote unit tests; full tui_gateway + web_server suites green (472
passed); tsc -b clean; E2E verified the full attach→disk→queue and
gateway-path→data-URL display round-trip plus the out-of-root security block.
Co-authored-by: Max Mitcham <maxmitcham@mac.home>
Co-authored-by: Justlrnal4 <Justlrnal4@users.noreply.github.com>
Co-authored-by: Chris Cook <ccook@nvms.com>
Co-authored-by: Thomas Paquette <thomas.paquette@gmail.com>
|
||
|---|---|---|
| .. | ||
| acp | ||
| acp_adapter | ||
| agent | ||
| cli | ||
| cron | ||
| docker | ||
| e2e | ||
| fakes | ||
| fixtures/plugins/example-dashboard/dashboard | ||
| gateway | ||
| hermes_cli | ||
| hermes_state | ||
| honcho_plugin | ||
| integration | ||
| openviking_plugin | ||
| plugins | ||
| providers | ||
| run_agent | ||
| scripts | ||
| skills | ||
| stress | ||
| tools | ||
| tui_gateway | ||
| website | ||
| __init__.py | ||
| conftest.py | ||
| run_interrupt_test.py | ||
| test_account_usage.py | ||
| test_atomic_replace_symlinks.py | ||
| test_base_url_hostname.py | ||
| test_batch_runner_checkpoint.py | ||
| test_bitwarden_secrets.py | ||
| test_cli_file_drop.py | ||
| test_cli_manual_compress.py | ||
| test_cli_skin_integration.py | ||
| test_ctx_halving_fix.py | ||
| test_desktop_mac_entitlements.py | ||
| test_docker_home_override_scripts.py | ||
| test_docker_stage2_browser_discovery.py | ||
| test_dockerfile_tini_compat_shim.py | ||
| test_empty_model_fallback.py | ||
| test_env_loader_secret_sources.py | ||
| test_evidence_store.py | ||
| test_gateway_streaming_nested_config.py | ||
| test_get_tool_definitions_cache_isolation.py | ||
| test_hermes_bootstrap.py | ||
| test_hermes_constants.py | ||
| test_hermes_home_profile_warning.py | ||
| test_hermes_logging.py | ||
| test_hermes_state.py | ||
| test_hermes_state_compression_locks.py | ||
| test_hermes_state_wal_fallback.py | ||
| test_honcho_client_config.py | ||
| test_honcho_session_context.py | ||
| test_honcho_startup_fail_open.py | ||
| test_install_sh_browser_install.py | ||
| test_install_sh_pythonpath_sanitization.py | ||
| test_install_sh_root_fhs_uv_python_path.py | ||
| test_install_sh_setup_wizard_tty_probe.py | ||
| test_install_sh_symlink_stomp.py | ||
| test_install_sh_termux_network_prereqs.py | ||
| test_ipv4_preference.py | ||
| test_lazy_session_regressions.py | ||
| test_lint_config.py | ||
| test_live_system_guard_self_test.py | ||
| test_mcp_serve.py | ||
| test_mini_swe_runner.py | ||
| test_minimax_model_validation.py | ||
| test_minimax_oauth.py | ||
| test_minisweagent_path.py | ||
| test_model_picker_scroll.py | ||
| test_model_tools.py | ||
| test_model_tools_async_bridge.py | ||
| test_ollama_num_ctx.py | ||
| test_output_cap_parsing.py | ||
| test_package_json_lazy_deps.py | ||
| test_packaging_metadata.py | ||
| test_plugin_skills.py | ||
| test_process_loop_event_loop_warning.py | ||
| test_project_metadata.py | ||
| test_retry_utils.py | ||
| test_run_tests_parallel.py | ||
| test_sanitize_tool_error.py | ||
| test_sql_injection.py | ||
| test_subprocess_home_isolation.py | ||
| test_termux_all_extra_compat.py | ||
| test_timezone.py | ||
| test_toolset_distributions.py | ||
| test_toolsets.py | ||
| test_trajectory_compressor.py | ||
| test_trajectory_compressor_async.py | ||
| test_transform_llm_output_hook.py | ||
| test_transform_tool_result_hook.py | ||
| test_tui_gateway_server.py | ||
| test_utils_truthy_values.py | ||
| test_wheel_locales_e2e.py | ||
| test_yuanbao_integration.py | ||
| test_yuanbao_markdown.py | ||
| test_yuanbao_pipeline.py | ||
| test_yuanbao_proto.py | ||