From e2c2e41137d541e2b45fab562e972922f8af1385 Mon Sep 17 00:00:00 2001 From: Teknium <127238744+teknium1@users.noreply.github.com> Date: Wed, 10 Jun 2026 05:12:37 -0700 Subject: [PATCH] =?UTF-8?q?fix(egress):=20v4=20round=20=E2=80=94=20bridge?= =?UTF-8?q?=20bind=20on=20Linux,=20listener-role=20split,=20fallback=20gat?= =?UTF-8?q?e,=20audit.log=20truth?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses GodsBoy's May 30 follow-up review on PR #30179. P0 — Linux sandboxes could not reach the proxy: - _default_http_listen now binds the docker bridge gateway on Linux (host.docker.internal resolves to the bridge gateway there; the old loopback-only bind was unreachable from containers). Loopback stays the default on Docker Desktop platforms; bridge-less Linux falls back to loopback with a warning. - Live-testing the fix against the real v0.39 binary surfaced a second latent bug the host-side E2E had masked: v0.39's http_listen does NOT terminate CONNECT — tunnel_listen does. HTTPS_PROXY traffic through http_listen got 400s from upstream. build_proxy_config now binds tunnel_listen (CONNECT/MITM) on tunnel_port and http_listen (plain-HTTP forwards) on tunnel_port+1; docker.py points HTTPS_PROXY at tunnel_port and HTTP_PROXY at tunnel_port+1. - Liveness probes (start_proxy poll loop, get_status) now probe the CONFIGURED bind host via _read_http_listen_from_config() instead of hardcoded loopback, which would have killed a healthy bridge-bound daemon as 'never came up'. P2 — allow_env_fallback dead on the partial-secret path: - The missing-secret branch in _build_proxy_subprocess_env now honors proxy.allow_env_fallback exactly as its own error message promises. - cmd_start refuses (or warns, with the fallback flag) when credential_source=bitwarden but secrets.bitwarden is disabled/missing — closing the silent-degrade-to-host-env hole. P2 — audit.log decoy on v0.39: - Wizard pre-create failure downgraded to a warning (file is non-load-bearing until the version bump); success line qualified as 'reserved'; user + dev docs stop telling operators to wire monitoring to the path today. P3 — metrics comment no longer claims 'hermes egress status' surfaces the ephemeral metrics port (it can't; :0 is random and unrecorded). Validation: 180 unit tests pass (9 new), gated E2E passes, and a live end-to-end run against the real binary verified CONNECT-MITM with Authorization swap on tunnel_listen, plain-HTTP swap on tunnel_port+1, 403 on non-allowlisted hosts, and bind-host-aware status probing. Docs gain a Linux firewall troubleshooting section (container→docker0 INPUT drops, e.g. ufw default-deny). --- agent/proxy_sources/iron_proxy.py | 239 ++++++++++++------ hermes_cli/proxy_cli.py | 51 +++- tests/test_iron_proxy.py | 173 +++++++++++-- tests/test_iron_proxy_cli.py | 85 +++++++ tests/test_iron_proxy_e2e.py | 19 +- tools/environments/docker.py | 7 +- .../docs/developer-guide/egress-internals.md | 12 +- website/docs/user-guide/egress/iron-proxy.md | 42 ++- 8 files changed, 500 insertions(+), 128 deletions(-) diff --git a/agent/proxy_sources/iron_proxy.py b/agent/proxy_sources/iron_proxy.py index b9c54428e99..f650e52bbf9 100644 --- a/agent/proxy_sources/iron_proxy.py +++ b/agent/proxy_sources/iron_proxy.py @@ -37,8 +37,10 @@ Design summary at proxy startup instead. * The proxy runs as a managed subprocess (``hermes egress start``), pidfile - at ``/proxy/iron-proxy.pid``, structured audit log at - ``/proxy/audit.log``. + at ``/proxy/iron-proxy.pid``. Daemon output (including + per-request records on v0.39) goes to ``/proxy/iron-proxy.log``; + ``audit.log`` is pre-created but reserved for a future pin that supports + ``log.audit_path``. * Failures (binary missing, port collision, bad config) emit a one-line warning and do *not* block agent startup. The Docker backend refuses to @@ -686,26 +688,44 @@ def mint_proxy_token(prefix: str = "hermes-proxy") -> str: def _default_http_listen(tunnel_port: int) -> List[str]: - """Build the list of host:port pairs the proxy should bind on. + """Build the single host:port bind the proxy should listen on. - Always binds loopback (``127.0.0.1``) so host-side test tooling can hit - the proxy directly. On Linux we also bind the docker bridge gateway - (``172.17.0.1`` by default) so containers can reach the proxy via - ``host.docker.internal:host-gateway``. We do NOT bind ``0.0.0.0`` — - that would expose the proxy (and, with a leaked sandbox token, the - user's API quota) to anyone on the local network. + iron-proxy v0.39 supports exactly ONE ``proxy.http_listen`` bind per + daemon process, so this returns a one-element list and the choice of + host matters: - On macOS / Windows Docker Desktop the bridge gateway is managed by - Desktop itself and ``host.docker.internal`` resolves via VPNkit, so - a single loopback bind is enough. + * **Linux:** bind the docker bridge gateway (``172.17.0.1`` by + default). Sandboxes reach the proxy via + ``host.docker.internal:host-gateway``, which Docker resolves to + exactly this bridge gateway IP on Linux — a loopback-only bind is + unreachable from inside containers there. The bridge IP is still + host-local (it's an address on the host's ``docker0`` interface), + so host-side tooling and the status probe can reach it too. When + no docker bridge is detected (docker not installed / not started), + fall back to loopback — there are no sandboxes to serve in that + state, and the operator gets a warning. + * **macOS / Windows Docker Desktop:** ``host.docker.internal`` + resolves via VPNkit to the host, so a loopback bind is reachable + from containers and is the least-exposed choice. + + We never bind ``0.0.0.0`` — that would expose the proxy (and, with a + leaked sandbox token, the user's API quota) to anyone on the local + network. The bridge-gateway bind is reachable by other containers + on the default bridge network, which is unavoidable given v0.39's + single-bind limit; requests still require a minted proxy token and + an allowlisted upstream. """ - binds = [f"127.0.0.1:{tunnel_port}"] if platform.system() == "Linux": bridge_ip = _detect_docker_bridge_ip() if bridge_ip and bridge_ip != "127.0.0.1": - binds.append(f"{bridge_ip}:{tunnel_port}") - return binds + return [f"{bridge_ip}:{tunnel_port}"] + logger.warning( + "No docker bridge (docker0) detected — binding iron-proxy to " + "loopback only. Docker sandboxes will NOT be able to reach " + "the proxy until it is restarted with docker running." + ) + return [f"127.0.0.1:{tunnel_port}"] def _detect_docker_bridge_ip() -> Optional[str]: @@ -792,13 +812,14 @@ def build_proxy_config( real secrets from its OWN environment via ``source: {type: env, var: ...}``; the sandbox never sees them. - Bind policy: by default we bind loopback (``127.0.0.1``) plus the - docker bridge gateway IP on Linux (``172.17.0.1`` or whatever - ``docker0`` resolves to). Sandboxes use ``host.docker.internal`` which - Linux Docker maps to the bridge gateway via ``--add-host``; macOS / - Windows Docker Desktop manage their own gateway. We do NOT bind - ``0.0.0.0`` — a LAN peer with a leaked sandbox token could otherwise - spend the operator's API quota against any allowlisted upstream. + Bind policy: the sandbox-facing listeners (``tunnel_listen`` on + ``tunnel_port``, plain-HTTP ``http_listen`` on ``tunnel_port + 1``) + bind the docker bridge gateway on Linux (``172.17.0.1`` or whatever + ``docker0`` resolves to — that's what ``host.docker.internal`` + resolves to inside containers there) and loopback on macOS / Windows + Docker Desktop. We do NOT bind ``0.0.0.0`` — a LAN peer with a + leaked sandbox token could otherwise spend the operator's API quota + against any allowlisted upstream. SSRF policy: ``upstream_deny_cidrs`` defaults to a conservative deny list covering loopback, link-local (incl. AWS/GCP/Azure IMDS at @@ -851,16 +872,34 @@ def build_proxy_config( else: deny_cidrs = list(upstream_deny_cidrs) - # Listen address. iron-proxy v0.39 takes a single string at - # ``proxy.http_listen`` — there is no plural ``http_listens`` field, - # despite earlier drafts of this module claiming v0.39 accepts both. - # An empirical strings(1) audit + a live "start the binary and - # observe the YAML unmarshal error" confirms the singular form is - # the only one the binary accepts. We bind loopback by default; the - # docker bridge listener is added as a second binary invocation if - # we ever need multi-bind, but a single bind is what v0.39 supports. + # Listen addresses. iron-proxy v0.39 takes a single string per + # listener field — there is no plural ``http_listens`` form, despite + # earlier drafts of this module claiming v0.39 accepts both. An + # empirical strings(1) audit + a live "start the binary and observe + # the YAML unmarshal error" confirms the singular form is the only + # one the binary accepts. + # + # LISTENER ROLES (verified live against the v0.39 binary): + # * ``tunnel_listen`` is the CONNECT + MITM listener. HTTPS through + # ``HTTPS_PROXY`` issues CONNECT — this is the listener sandboxes + # must reach. A CONNECT sent to ``http_listen`` is NOT terminated: + # v0.39 forwards it upstream as a regular request and the upstream + # responds 400. + # * ``http_listen`` is the absolute-form plain-HTTP forward listener + # (``HTTP_PROXY`` for ``http://`` URLs). Transforms fire here too. + # Both get the sandbox-facing bind host: tunnel on ``tunnel_port``, + # plain HTTP on ``tunnel_port + 1``. + # + # The bind host comes from _default_http_listen: the docker bridge + # gateway on Linux (containers reach the proxy via + # host.docker.internal, which maps to the bridge gateway there — + # loopback would be unreachable from inside sandboxes) and loopback + # on macOS/Windows Docker Desktop (where host.docker.internal routes + # to the host via VPNkit). listens = list(http_listen) if http_listen else _default_http_listen(tunnel_port) primary_listen = listens[0] if listens else f"127.0.0.1:{tunnel_port}" + bind_host = primary_listen.rsplit(":", 1)[0] or "127.0.0.1" + plain_http_listen = f"{bind_host}:{tunnel_port + 1}" log_block: Dict = {"level": "info"} # NOTE: ``log.audit_path`` is NOT a field in iron-proxy v0.39's @@ -888,19 +927,20 @@ def build_proxy_config( "proxy_ip": "127.0.0.1", }, "proxy": { - # http_listen is the HTTP-proxy listener that handles both plain - # HTTP forwards AND CONNECT tunnels for HTTPS. Sandboxes set - # `HTTPS_PROXY=http://host:tunnel_port` and the same listener - # serves both protocols. We bind loopback by default — NOT - # 0.0.0.0. LAN peers with a leaked sandbox token would - # otherwise be able to spend the operator's API quota against - # any allowlisted upstream. - "http_listen": primary_listen, - # The HTTPS-listener (direct TLS termination, no CONNECT) and - # the SOCKS5/CONNECT-only tunnel listener get loopback ephemeral - # ports — we don't expose them. + # tunnel_listen is the CONNECT/MITM listener — what sandboxes + # hit via `HTTPS_PROXY=http://host:tunnel_port` for HTTPS + # upstreams (curl/requests/node issue CONNECT through it). + # http_listen handles absolute-form plain-HTTP forwards + # (`HTTP_PROXY` for http:// URLs) on tunnel_port+1. Both + # bind the docker bridge gateway on Linux / loopback on + # Docker Desktop — NEVER 0.0.0.0. LAN peers with a leaked + # sandbox token would otherwise be able to spend the + # operator's API quota against any allowlisted upstream. + "tunnel_listen": primary_listen, + "http_listen": plain_http_listen, + # The HTTPS-listener (direct TLS termination, no CONNECT) + # gets a loopback ephemeral port — we don't expose it. "https_listen": "127.0.0.1:0", - "tunnel_listen": "127.0.0.1:0", "max_request_body_bytes": 16 * 1024 * 1024, "max_response_body_bytes": 0, "upstream_response_header_timeout": "120s", @@ -914,9 +954,11 @@ def build_proxy_config( # startup. Pin the metrics listener to an ephemeral loopback # port (``127.0.0.1:0``) so the metrics binding can't collide # with the proxy listener regardless of what tunnel_port the - # operator chose. The daemon still emits metrics for any local - # scraper that knows where to look (via ``hermes egress - # status``); we just don't expose them to the public default. + # operator chose. NOTE: ``:0`` means the kernel picks a fresh + # random port each start and nothing records it — metrics are + # effectively disabled/undiscoverable at this pin. If we want + # scrapable metrics later, allocate a fixed port and surface it + # in ``ProxyStatus`` / ``hermes egress status``. "metrics": { "listen": "127.0.0.1:0", }, @@ -943,20 +985,19 @@ def build_proxy_config( def ensure_audit_log(audit_path: Path) -> None: """Create the audit log file with private permissions (0o600). - Called from the wizard right before ``start_proxy``. Without this, - iron-proxy creates the file under the default umask the first time it - writes — meaning every host-side request history is potentially - world-readable. We pre-create the file empty with 0o600 so the daemon - inherits the tight permissions. + Called from the wizard right before ``start_proxy``. On the pinned + v0.39 the daemon never writes this file (no ``log.audit_path`` + config field), so the pre-create is purely forward-compat: when the + pin moves to a version that supports a dedicated audit stream, the + file already exists with tight permissions and the daemon inherits + them instead of creating it under the default umask. Raises :class:`RuntimeError` on any OSError (planted symlink, - immutable parent dir, full disk). The previous version of this - function logged a warning and returned silently — that's the bad - option, because the docstring's promise of "iron-proxy inherits - tight permissions" depends on us actually creating the file. When - the create fails, the caller MUST know about it so the wizard can - surface a clear error instead of printing "✓ audit log: ..." for a - file the daemon will create under the default umask. + immutable parent dir, full disk) so the caller can decide how to + surface it. The wizard treats this as a WARNING on v0.39 — the + file is non-load-bearing until the version bump — but the qualified + message keeps operators from wiring monitoring to a path that can't + exist. """ try: @@ -1509,7 +1550,16 @@ def start_proxy( # We scope a Ctrl-C handler around the poll loop so an operator who # hits Ctrl-C while waiting for ``hermes egress start`` doesn't leak # an orphan with the port bound. - tunnel_port = _read_tunnel_port_from_config() or _DEFAULT_TUNNEL_PORT + # + # Probe the CONFIGURED bind host, not loopback unconditionally — on + # Linux the daemon binds the docker bridge gateway, where a loopback + # connect never succeeds and we'd kill a healthy daemon as "never + # came up". + listen_hp = _read_http_listen_from_config() + if listen_hp is not None: + probe_host, tunnel_port = listen_hp + else: + probe_host, tunnel_port = "127.0.0.1", _DEFAULT_TUNNEL_PORT listening = False def _interrupt_handler(_signum, _frame): # pragma: no cover - signal path @@ -1546,7 +1596,7 @@ def start_proxy( f"iron-proxy exited immediately (code {proc.returncode}). " f"Last log lines:\n{tail}" ) - if _port_listening("127.0.0.1", tunnel_port): + if _port_listening(probe_host, tunnel_port): listening = True break if time.time() >= deadline: @@ -1582,7 +1632,7 @@ def start_proxy( except FileNotFoundError: pass raise RuntimeError( - f"iron-proxy did not bind 127.0.0.1:{tunnel_port} within " + f"iron-proxy did not bind {probe_host}:{tunnel_port} within " f"{_STARTUP_GRACE_SECONDS}s. Process was killed. " f"Last log lines:\n{tail}" ) @@ -1758,14 +1808,24 @@ def _build_proxy_subprocess_env( # selected. An operator on credential_source=bitwarden # picked it specifically to get rotation; falling back # to parent env reintroduces the bug class the mode - # is supposed to defeat. - raise RuntimeError( - f"Bitwarden refresh did not return secrets for " - f"{missing}. Either add the secrets to your BWS " - f"project, switch to credential_source: env via " - f"`hermes egress setup --no-bitwarden`, or set " - f"`proxy.allow_env_fallback: true` in config.yaml " - f"to opt into the legacy host-env fallback." + # is supposed to defeat. ``allow_env_fallback`` is the + # documented, deliberate opt-out — honor it here exactly + # as the empty-token branch below does (the error + # message tells operators to set it, so it must work). + if not (bitwarden_config or {}).get("allow_env_fallback"): + raise RuntimeError( + f"Bitwarden refresh did not return secrets for " + f"{missing}. Either add the secrets to your BWS " + f"project, switch to credential_source: env via " + f"`hermes egress setup --no-bitwarden`, or set " + f"`proxy.allow_env_fallback: true` in config.yaml " + f"to opt into the legacy host-env fallback." + ) + logger.warning( + "Bitwarden refresh did not return secrets for %s — " + "falling back to host env for those names " + "(allow_env_fallback=true).", + missing, ) # bws warnings are non-secret status messages (e.g. "no # project found", "rate limited"), but the taint analyzer @@ -1897,7 +1957,12 @@ def get_status() -> ProxyStatus: """ status = ProxyStatus() - status.tunnel_port = _read_tunnel_port_from_config() or _DEFAULT_TUNNEL_PORT + listen_hp = _read_http_listen_from_config() + if listen_hp is not None: + probe_host, status.tunnel_port = listen_hp + else: + probe_host = "127.0.0.1" + status.tunnel_port = _DEFAULT_TUNNEL_PORT binary = find_iron_proxy(install_if_missing=False) if binary: @@ -1918,12 +1983,33 @@ def get_status() -> ProxyStatus: pid = _read_pid() if pid and _pid_alive(pid): status.pid = pid - status.listening = _port_listening("127.0.0.1", status.tunnel_port) + # Probe the configured bind host — on Linux that's the docker + # bridge gateway, where a loopback connect would report a healthy + # daemon as "not listening". + status.listening = _port_listening(probe_host, status.tunnel_port) return status def _read_tunnel_port_from_config() -> Optional[int]: + listen = _read_http_listen_from_config() + if listen is None: + return None + return listen[1] + + +def _read_http_listen_from_config() -> Optional[Tuple[str, int]]: + """Return ``(host, port)`` of the configured sandbox-facing listener. + + Reads ``proxy.tunnel_listen`` — the CONNECT/MITM listener sandboxes + hit via ``HTTPS_PROXY`` — falling back to ``proxy.http_listen`` for + configs written before the tunnel/http listener-role split. + + The bind host matters for liveness probes: on Linux the daemon binds + the docker bridge gateway (e.g. ``172.17.0.1``), where a loopback + connect would report "not listening" for a perfectly healthy daemon. + """ + cfg = _proxy_state_dir_ro() / "proxy.yaml" if not cfg.exists(): return None @@ -1935,17 +2021,20 @@ def _read_tunnel_port_from_config() -> Optional[int]: data = yaml.safe_load(cfg.read_text(encoding="utf-8")) except (OSError, yaml.YAMLError): return None + proxy_block = (data or {}).get("proxy") or {} # The CLI/Docker side calls this "the tunnel port" because that's how - # sandboxes use it (HTTPS_PROXY), but on the iron-proxy side it's the - # http_listen — the HTTP-proxy listener handles both plain HTTP and the - # CONNECT method for HTTPS upstreams. - listen = ((data or {}).get("proxy") or {}).get("http_listen") or "" + # sandboxes use it (HTTPS_PROXY) — on the iron-proxy side it's the + # tunnel_listen (CONNECT + MITM). http_listen is the plain-HTTP + # forward listener on tunnel_port+1. + listen = proxy_block.get("tunnel_listen") or proxy_block.get("http_listen") or "" if not isinstance(listen, str) or ":" not in listen: return None + host, _, port_s = listen.rpartition(":") try: - return int(listen.rsplit(":", 1)[1]) + port = int(port_s) except ValueError: return None + return (host or "127.0.0.1", port) def _port_listening(host: str, port: int) -> bool: diff --git a/hermes_cli/proxy_cli.py b/hermes_cli/proxy_cli.py index 3f4582cb82c..63ecad6ac35 100644 --- a/hermes_cli/proxy_cli.py +++ b/hermes_cli/proxy_cli.py @@ -351,17 +351,19 @@ def cmd_setup(args: argparse.Namespace) -> int: ] audit_log_path = ip._proxy_state_dir() / "audit.log" - # Pre-create the audit log with 0o600 so iron-proxy inherits private - # perms instead of letting the daemon create it under the default - # umask (potentially world-readable). Raises on failure (planted - # symlink, immutable parent, full disk) — the wizard must surface - # that rather than print "✓" for a file the daemon will create - # under a slacker umask. + # Pre-create the audit log with 0o600. On the pinned v0.39 the + # daemon does NOT write to this file (no ``log.audit_path`` field in + # its config schema) — it's reserved for the v0.40+ upgrade where + # per-request records start flowing. Because the file is + # non-load-bearing today, a pre-create failure (immutable parent, + # pre-existing foreign-owned file, full disk) is a WARNING, not a + # setup abort. + audit_log_ok = True try: ip.ensure_audit_log(audit_log_path) except RuntimeError as exc: - console.print(f" [red]✗ {exc}[/red]") - return 1 + audit_log_ok = False + console.print(f" [yellow]⚠ {exc}[/yellow]") # Allow operator override of the deny list via # ``proxy.upstream_deny_cidrs`` — but the default (None) gives a safe @@ -381,7 +383,12 @@ def cmd_setup(args: argparse.Namespace) -> int: mappings_path = ip.write_mappings(mappings) console.print(f" [green]✓[/green] config: {cfg_path}") console.print(f" [green]✓[/green] mappings: {mappings_path}") - console.print(f" [green]✓[/green] audit log: {audit_log_path}") + if audit_log_ok: + console.print( + f" [green]✓[/green] audit log: {audit_log_path} " + f"[dim](reserved — not written by iron-proxy v0.39; " + f"per-request records land in iron-proxy.log)[/dim]" + ) # ------------------------------------------------------------------ enable proxy_cfg["enabled"] = True @@ -451,6 +458,32 @@ def cmd_start(args: argparse.Namespace) -> int: and bw_cfg is not None and bool(bw_cfg.get("enabled")) ) + # Silent-degrade guard: the operator explicitly chose + # ``credential_source: bitwarden``, but secrets.bitwarden has since + # been disabled or removed. Proceeding would quietly start on host + # env — exactly the bug class the BW mode is meant to defeat. Refuse + # unless the documented escape hatch is set. + if credential_source == "bitwarden" and not refresh_bw: + if bool(proxy_cfg.get("allow_env_fallback", False)): + console.print( + "[yellow]⚠ credential_source=bitwarden but " + "secrets.bitwarden is disabled or missing — falling back " + "to host-env secrets (allow_env_fallback=true). Rotated " + "Bitwarden keys will NOT propagate.[/yellow]" + ) + else: + console.print( + "[red]✗ Refusing to start: proxy.credential_source is " + "'bitwarden' but secrets.bitwarden is disabled or " + "missing.[/red]" + ) + console.print( + " Re-enable it (`secrets.bitwarden.enabled: true`), switch " + "back to env credentials with `hermes egress setup " + "--no-bitwarden`, or set `proxy.allow_env_fallback: true` " + "to opt into the host-env fallback." + ) + return 1 # Pass the proxy-side allow_env_fallback opt-in through to # start_proxy. This is a deliberate, documented escape hatch: when # set, the daemon silently falls back to host env if BWS is diff --git a/tests/test_iron_proxy.py b/tests/test_iron_proxy.py index 617f3583251..21e0ddc6f52 100644 --- a/tests/test_iron_proxy.py +++ b/tests/test_iron_proxy.py @@ -212,8 +212,8 @@ def test_wizard_rendered_yaml_contains_deny_list(hermes_home, tmp_path): def test_default_bind_is_loopback_not_zero_zero(tmp_path): - """``http_listen`` must NOT be ``0.0.0.0:PORT`` or ``:PORT`` (latter is - INADDR_ANY). Loopback only by default.""" + """The sandbox-facing listeners must NOT be ``0.0.0.0:PORT`` or + ``:PORT`` (latter is INADDR_ANY).""" cfg = ip.build_proxy_config( mappings=[_sample_mapping()], @@ -222,12 +222,16 @@ def test_default_bind_is_loopback_not_zero_zero(tmp_path): tunnel_port=12345, http_listen=["127.0.0.1:12345"], # explicit so test is deterministic ) - primary = cfg["proxy"]["http_listen"] - assert primary == "127.0.0.1:12345" - # Sentinel: confirm we didn't accidentally serialize a bare-port form - # like ":12345" (that's INADDR_ANY). - assert not primary.startswith(":") - assert "0.0.0.0" not in primary + # tunnel_listen is the CONNECT/MITM listener sandboxes hit via + # HTTPS_PROXY; http_listen is the plain-HTTP forward on port+1. + assert cfg["proxy"]["tunnel_listen"] == "127.0.0.1:12345" + assert cfg["proxy"]["http_listen"] == "127.0.0.1:12346" + for key in ("tunnel_listen", "http_listen", "https_listen"): + val = cfg["proxy"][key] + # Sentinel: confirm we didn't accidentally serialize a bare-port + # form like ":12345" (that's INADDR_ANY). + assert not val.startswith(":") + assert "0.0.0.0" not in val # iron-proxy v0.39 doesn't support http_listens (plural). We # deliberately do NOT emit that key — re-emitting it would cause # the daemon to fail YAML unmarshal at start time. @@ -237,14 +241,14 @@ def test_default_bind_is_loopback_not_zero_zero(tmp_path): ) -def test_default_bind_uses_loopback_on_linux(tmp_path, monkeypatch): +def test_default_bind_uses_docker_bridge_on_linux(tmp_path, monkeypatch): """When http_listen isn't passed AND we're on Linux, the singular - http_listen field is the loopback bind. iron-proxy v0.39 only - supports one bind per daemon process — earlier versions of this - code emitted a plural http_listens list with the docker bridge - appended, but v0.39 rejects that key so we keep loopback only. - Sandboxes still reach the daemon via host.docker.internal which - Docker maps to the host gateway, so loopback-only is functional.""" + http_listen field is the DOCKER BRIDGE bind — not loopback. + iron-proxy v0.39 only supports one bind per daemon process, and on + Linux ``host.docker.internal:host-gateway`` resolves to the bridge + gateway (172.17.0.1 by default), which a loopback-only daemon never + answers. Sandboxes must be able to reach the proxy from the + container's vantage point.""" monkeypatch.setattr(ip.platform, "system", lambda: "Linux") monkeypatch.setattr(ip, "_detect_docker_bridge_ip", lambda: "172.17.0.1") @@ -254,12 +258,48 @@ def test_default_bind_uses_loopback_on_linux(tmp_path, monkeypatch): ca_key=tmp_path / "ca.key", tunnel_port=9090, ) - # Primary http_listen is loopback. - assert cfg["proxy"]["http_listen"] == "127.0.0.1:9090" + # The sandbox-facing CONNECT/MITM listener binds the bridge gateway — + # reachable from containers via host.docker.internal. Plain-HTTP + # forward listener rides on port+1, same host. + assert cfg["proxy"]["tunnel_listen"] == "172.17.0.1:9090" + assert cfg["proxy"]["http_listen"] == "172.17.0.1:9091" # No http_listens (plural) — v0.39 rejects that key. assert "http_listens" not in cfg["proxy"] +def test_default_bind_falls_back_to_loopback_without_bridge(tmp_path, monkeypatch): + """On Linux without a detectable docker0 bridge (docker not + installed / not running), fall back to loopback rather than + refusing to bind.""" + + monkeypatch.setattr(ip.platform, "system", lambda: "Linux") + monkeypatch.setattr(ip, "_detect_docker_bridge_ip", lambda: None) + cfg = ip.build_proxy_config( + mappings=[_sample_mapping()], + ca_cert=tmp_path / "ca.crt", + ca_key=tmp_path / "ca.key", + tunnel_port=9090, + ) + assert cfg["proxy"]["tunnel_listen"] == "127.0.0.1:9090" + assert cfg["proxy"]["http_listen"] == "127.0.0.1:9091" + + +def test_default_bind_is_loopback_on_macos(tmp_path, monkeypatch): + """On Docker Desktop platforms host.docker.internal routes to the + host via VPNkit, so loopback is reachable from containers and is + the least-exposed bind.""" + + monkeypatch.setattr(ip.platform, "system", lambda: "Darwin") + cfg = ip.build_proxy_config( + mappings=[_sample_mapping()], + ca_cert=tmp_path / "ca.crt", + ca_key=tmp_path / "ca.key", + tunnel_port=9090, + ) + assert cfg["proxy"]["tunnel_listen"] == "127.0.0.1:9090" + assert cfg["proxy"]["http_listen"] == "127.0.0.1:9091" + + def test_metrics_listener_pinned_to_loopback_ephemeral(tmp_path): """iron-proxy v0.39's default metrics_listen is ``:9090``, which collides with our default tunnel_port=9090. build_proxy_config MUST @@ -1455,3 +1495,102 @@ def test_persisted_nonce_roundtrip(hermes_home, monkeypatch): def test_persisted_nonce_returns_none_when_missing(hermes_home): """No nonce file → None, callers fall back to argv0 basename.""" assert ip._read_persisted_nonce() is None + + +# --------------------------------------------------------------------------- +# v4 round (GodsBoy follow-up): bind-host-aware liveness probes + +# allow_env_fallback on the partial-secret path +# --------------------------------------------------------------------------- + + +def test_read_http_listen_from_config_returns_host_and_port(hermes_home): + """_read_http_listen_from_config must surface the BIND HOST, not just + the port — on Linux the daemon binds the docker bridge gateway and a + hardcoded loopback probe would report a healthy daemon as dead.""" + + state = ip._proxy_state_dir() + (state / "proxy.yaml").write_text( + "proxy:\n http_listen: 172.17.0.1:9090\n", encoding="utf-8" + ) + assert ip._read_http_listen_from_config() == ("172.17.0.1", 9090) + assert ip._read_tunnel_port_from_config() == 9090 + + +def test_read_http_listen_from_config_missing_file(hermes_home): + assert ip._read_http_listen_from_config() is None + + +def test_get_status_probes_configured_bind_host(hermes_home, monkeypatch): + """get_status must probe the configured bind host (e.g. the docker + bridge IP), not loopback unconditionally.""" + + state = ip._proxy_state_dir() + (state / "proxy.yaml").write_text( + "proxy:\n http_listen: 172.17.0.1:9123\n", encoding="utf-8" + ) + (state / "ca.crt").write_text("cert") + ip._write_pidfile_safely(ip._pidfile(), 99999) + monkeypatch.setattr(ip, "_pid_alive", lambda pid: True) + monkeypatch.setattr(ip, "find_iron_proxy", lambda **kw: None) + + probed = {} + + def fake_probe(host, port): + probed["host"] = host + probed["port"] = port + return True + + monkeypatch.setattr(ip, "_port_listening", fake_probe) + status = ip.get_status() + assert probed == {"host": "172.17.0.1", "port": 9123} + assert status.listening is True + assert status.tunnel_port == 9123 + + +def test_partial_bitwarden_secrets_honor_allow_env_fallback( + hermes_home, monkeypatch, +): + """The missing-secret branch's own error message tells operators to + set proxy.allow_env_fallback — so the flag must actually work there + (previously only the empty-token branch honored it).""" + + ip.write_mappings([_sample_mapping("OPENROUTER_API_KEY")]) + monkeypatch.setenv("OPENROUTER_API_KEY", "sk-host-fallback") + + import agent.secret_sources.bitwarden as bw + monkeypatch.setattr( + bw, "fetch_bitwarden_secrets", lambda **kw: ({}, []), + ) + monkeypatch.setenv("BWS_ACCESS_TOKEN", "tok") + bw_cfg = { + "project_id": "proj", + "access_token_env": "BWS_ACCESS_TOKEN", + "allow_env_fallback": True, + } + + env = ip._build_proxy_subprocess_env( + refresh_from_bitwarden=True, bitwarden_config=bw_cfg, + ) + # Falls back to the host env value instead of raising. + assert env.get("OPENROUTER_API_KEY") == "sk-host-fallback" + + +def test_partial_bitwarden_secrets_raise_without_fallback( + hermes_home, monkeypatch, +): + """Strict default: missing BWS secrets raise.""" + + ip.write_mappings([_sample_mapping("OPENROUTER_API_KEY")]) + monkeypatch.setenv("OPENROUTER_API_KEY", "sk-host") + + import agent.secret_sources.bitwarden as bw + monkeypatch.setattr( + bw, "fetch_bitwarden_secrets", lambda **kw: ({}, []), + ) + monkeypatch.setenv("BWS_ACCESS_TOKEN", "tok") + bw_cfg = {"project_id": "proj", "access_token_env": "BWS_ACCESS_TOKEN"} + + with pytest.raises(RuntimeError, match="did not return secrets"): + ip._build_proxy_subprocess_env( + refresh_from_bitwarden=True, bitwarden_config=bw_cfg, + ) diff --git a/tests/test_iron_proxy_cli.py b/tests/test_iron_proxy_cli.py index e2a64110048..9476c322f7d 100644 --- a/tests/test_iron_proxy_cli.py +++ b/tests/test_iron_proxy_cli.py @@ -425,3 +425,88 @@ def test_setup_has_rotate_tokens_flag(): assert args.rotate_tokens is False args = parser.parse_args(["setup", "--rotate-tokens"]) assert args.rotate_tokens is True + + +# --------------------------------------------------------------------------- +# v4 round: credential_source=bitwarden with secrets.bitwarden disabled +# must NOT silently degrade to host-env secrets +# --------------------------------------------------------------------------- + + +def test_cmd_start_refuses_when_bitwarden_mode_but_disabled(hermes_home, monkeypatch): + """config keeps credential_source: bitwarden but secrets.bitwarden.enabled + later flips to false — cmd_start must refuse, not silently start on + host env (the silent-degrade class strict mode is meant to close).""" + + from hermes_cli.config import load_config, save_config + cfg = load_config() + cfg.setdefault("proxy", {})["enabled"] = True + cfg["proxy"]["credential_source"] = "bitwarden" + cfg["proxy"]["fail_on_uncovered_providers"] = False + cfg.setdefault("secrets", {})["bitwarden"] = {"enabled": False} + save_config(cfg) + + def must_not_call(**kw): + pytest.fail("start_proxy must not run when bitwarden mode is broken") + monkeypatch.setattr(ip, "start_proxy", must_not_call) + monkeypatch.setattr(ip, "discover_uncovered_providers", lambda **kw: []) + monkeypatch.setattr(ip, "discover_blocked_providers", lambda **kw: []) + + rc = proxy_cli.cmd_start(_args()) + assert rc == 1 + + +def test_cmd_start_bitwarden_disabled_proceeds_with_env_fallback( + hermes_home, monkeypatch, +): + """Same scenario but proxy.allow_env_fallback=true is the documented + escape hatch — start proceeds (with a warning).""" + + from hermes_cli.config import load_config, save_config + cfg = load_config() + cfg.setdefault("proxy", {})["enabled"] = True + cfg["proxy"]["credential_source"] = "bitwarden" + cfg["proxy"]["allow_env_fallback"] = True + cfg["proxy"]["fail_on_uncovered_providers"] = False + cfg.setdefault("secrets", {})["bitwarden"] = {"enabled": False} + save_config(cfg) + + captured: dict = {} + + def fake_start_proxy(**kw): + captured.update(kw) + s = ip.ProxyStatus() + s.pid = 4242 + s.listening = True + return s + + monkeypatch.setattr(ip, "start_proxy", fake_start_proxy) + monkeypatch.setattr(ip, "discover_uncovered_providers", lambda **kw: []) + monkeypatch.setattr(ip, "discover_blocked_providers", lambda **kw: []) + + rc = proxy_cli.cmd_start(_args()) + assert rc == 0 + # BW refresh is off (bitwarden disabled), running on env secrets. + assert captured.get("refresh_secrets_from_bitwarden") is False + + +def test_cmd_setup_audit_log_failure_is_warning_not_abort(hermes_home, monkeypatch): + """On the pinned v0.39 the daemon never writes audit.log, so a + pre-create failure must not abort the wizard.""" + + monkeypatch.setattr(ip, "find_iron_proxy", lambda **kw: hermes_home / "iron-proxy") + monkeypatch.setattr(ip, "discover_provider_mappings", lambda **kw: [ + ip.TokenMapping( + proxy_token="hermes-proxy-deadbeef", + real_env_name="OPENROUTER_API_KEY", + upstream_hosts=("openrouter.ai",), + ), + ]) + monkeypatch.setattr(ip, "discover_uncovered_providers", lambda **kw: []) + + def boom(path): + raise RuntimeError("could not pre-create audit log (synthetic)") + monkeypatch.setattr(ip, "ensure_audit_log", boom) + + rc = proxy_cli.cmd_setup(_args()) + assert rc == 0 diff --git a/tests/test_iron_proxy_e2e.py b/tests/test_iron_proxy_e2e.py index 9e0e47f4479..6be3ac7697f 100644 --- a/tests/test_iron_proxy_e2e.py +++ b/tests/test_iron_proxy_e2e.py @@ -105,6 +105,11 @@ def test_iron_proxy_swaps_authorization_header_end_to_end(hermes_home, monkeypat # Test target is on loopback — clear the default IMDS+loopback # deny list so iron-proxy will dial 127.0.0.1. upstream_deny_cidrs=[], + # Hermetic: pin the bind to loopback. Without this, Linux + # hosts with docker0 present would bind the bridge gateway + # (the production default) and the loopback curl below would + # never reach the proxy. + http_listen=[f"127.0.0.1:{tunnel_port}"], ) ip.write_proxy_config(cfg) ip.write_mappings([mapping]) @@ -125,19 +130,17 @@ def test_iron_proxy_swaps_authorization_header_end_to_end(hermes_home, monkeypat pytest.fail("iron-proxy never started listening on the tunnel port") # ----- request through the proxy ---------------------------------- - # The fake upstream listens on plain HTTP (not HTTPS), so we use the - # proxy's tunnel for the CONNECT but talk plaintext to upstream via - # `--proxy-insecure` semantics: iron-proxy accepts HTTPS_PROXY-style - # CONNECT to any host on its allowlist. For a clean E2E we hit - # http://127.0.0.1:/ which goes through the proxy as a plain - # HTTP forward (no MITM needed) and the secrets transform still fires - # on the Authorization header. + # The fake upstream listens on plain HTTP (not HTTPS). Plain-HTTP + # absolute-form forwards are served by the http_listen listener on + # tunnel_port + 1 (tunnel_port itself is the CONNECT/MITM listener + # that HTTPS_PROXY traffic hits). The secrets transform fires on + # the plain forward too, swapping the Authorization header. result = subprocess.run( [ "curl", "--silent", "--max-time", "10", - "-x", f"http://127.0.0.1:{tunnel_port}", + "-x", f"http://127.0.0.1:{tunnel_port + 1}", "-H", f"Authorization: Bearer {proxy_token}", f"http://127.0.0.1:{upstream_port}/", ], diff --git a/tools/environments/docker.py b/tools/environments/docker.py index f5981c8082c..4deb19f1ec1 100644 --- a/tools/environments/docker.py +++ b/tools/environments/docker.py @@ -449,15 +449,18 @@ def _egress_proxy_args_for_docker() -> tuple[list[str], dict[str, str], list[str container_ca = "/etc/ssl/certs/hermes-egress-ca.crt" volume_args = ["-v", f"{status.ca_cert_path}:{container_ca}:ro"] + # tunnel_port serves CONNECT (HTTPS); the plain-HTTP forward listener + # is on tunnel_port + 1 (see build_proxy_config's listener-role notes). proxy_url = f"http://host.docker.internal:{status.tunnel_port}" + plain_http_url = f"http://host.docker.internal:{status.tunnel_port + 1}" env_overrides: dict[str, str] = { # HTTPS_PROXY / HTTP_PROXY are respected by curl, requests, urllib, # httpx, node fetch, go default transport, etc. Lowercase variants # are also set because some tools only look at one casing. "HTTPS_PROXY": proxy_url, "https_proxy": proxy_url, - "HTTP_PROXY": proxy_url, - "http_proxy": proxy_url, + "HTTP_PROXY": plain_http_url, + "http_proxy": plain_http_url, # Loopback-only NO_PROXY so localhost dev servers inside the sandbox # (test fixtures, local LLMs) don't get sent through the proxy. "NO_PROXY": "127.0.0.1,localhost,::1", diff --git a/website/docs/developer-guide/egress-internals.md b/website/docs/developer-guide/egress-internals.md index 6e274af2823..5a73dc51cd4 100644 --- a/website/docs/developer-guide/egress-internals.md +++ b/website/docs/developer-guide/egress-internals.md @@ -124,7 +124,7 @@ hermes egress start Poll loop (do-while shape): while True: if proc.poll() is not None: tail log + unlink pidfile + raise - if _port_listening("127.0.0.1", tunnel_port): break + if _port_listening(probe_host, tunnel_port): break # probe_host = configured bind host if time.time() >= deadline: break (do-while: checked AFTER first probe) time.sleep(0.1) If not listening at exit: _kill_and_wait(proc) + unlink pidfile + raise. @@ -169,13 +169,15 @@ Regression: `test_subprocess_env_strips_unrelated_secrets`, `test_subprocess_env ### Bind policy -`_default_http_listen` returns loopback (and, on Linux, the docker bridge IP as a *second list entry the rendered yaml currently discards* — see below). Never `0.0.0.0`, never `:PORT` (INADDR_ANY). +`_default_http_listen` returns a single-element list: on Linux the docker bridge gateway IP (containers reach the proxy via `host.docker.internal:host-gateway`, which resolves to the bridge gateway — a loopback bind is unreachable from inside containers there); on macOS/Windows Docker Desktop, loopback (VPNkit routes `host.docker.internal` to the host). Linux without a detectable docker0 bridge falls back to loopback with a warning. Never `0.0.0.0`, never `:PORT` (INADDR_ANY). `_detect_docker_bridge_ip` validates via `ipaddress.IPv4Address` and rejects `is_unspecified` / `is_loopback` / `is_multicast` / `is_reserved` / `is_link_local` / `is_global`. A hostile `ip` shim on PATH cannot inject `0.0.0.0`. -**v0.39 schema constraint:** the binary's `config.Proxy` struct has only a singular `http_listen` string field — there is no `http_listens` (plural) list, despite earlier comments in this module claiming otherwise. `build_proxy_config` emits only the first entry of `_default_http_listen`'s result; the second-bind path is dead code today. When the pinned `_IRON_PROXY_VERSION` is bumped to one that supports the plural form, re-enable the list-emit in `build_proxy_config` and the docker-bridge bind becomes live without further changes. +**v0.39 schema constraint and listener roles (verified live against the binary):** the binary's `config.Proxy` struct has only singular listener fields — there is no `http_listens` (plural) list. `tunnel_listen` is the CONNECT + MITM listener (what `HTTPS_PROXY` traffic hits); `http_listen` only handles absolute-form plain-HTTP forwards (a CONNECT sent to it is relayed upstream as a regular request and 400s). `build_proxy_config` therefore binds `tunnel_listen` on `tunnel_port` and `http_listen` on `tunnel_port + 1`, both on the platform bind host. The Docker backend sets `HTTPS_PROXY` to `tunnel_port` and `HTTP_PROXY` to `tunnel_port + 1`. -Regression: `test_default_bind_is_loopback_not_zero_zero` (asserts loopback bind AND that `http_listens` is NOT in the rendered yaml), `test_default_bind_uses_loopback_on_linux`, `test_detect_docker_bridge_ip_rejects_dangerous` (parametrized over 8 attack inputs). +The liveness probes (`start_proxy` poll loop, `get_status`) read the configured bind host via `_read_http_listen_from_config()` and probe THAT host — a hardcoded loopback probe would report a healthy bridge-bound daemon as dead. + +Regression: `test_default_bind_is_loopback_not_zero_zero` (asserts no INADDR_ANY AND that `http_listens` is NOT in the rendered yaml), `test_default_bind_uses_docker_bridge_on_linux`, `test_default_bind_falls_back_to_loopback_without_bridge`, `test_default_bind_is_loopback_on_macos`, `test_detect_docker_bridge_ip_rejects_dangerous` (parametrized over 8 attack inputs). ### Metrics port collision @@ -191,7 +193,7 @@ Regression: `test_default_deny_cidrs_present_when_unspecified`, `test_default_de ### Audit log fail-loud -`ensure_audit_log` raises `RuntimeError` on any `OSError`. Swallowing the failure would let the daemon create the file under the default umask, defeating the privacy promise. `cmd_setup` catches the RuntimeError and surfaces a clear error to the operator. +`ensure_audit_log` raises `RuntimeError` on any `OSError`. On the pinned v0.39 the daemon never writes this file (no `log.audit_path` field), so `cmd_setup` treats the failure as a WARNING (the file is non-load-bearing until the version bump) and qualifies the success line as "reserved". When the pin moves to a version with `log.audit_path`, revisit: the pre-create becomes load-bearing for the 0o600-from-first-byte guarantee and the wizard should fail loud again. **v0.39 schema constraint:** `log.audit_path` is NOT a field in iron-proxy v0.39's `config.Log` struct, so `build_proxy_config` accepts the `audit_log` kwarg but does NOT emit it into the rendered yaml. Per-request records on v0.39 land in `iron-proxy.log` alongside daemon-level events. The `audit.log` file is still pre-created at `0o600` with `O_NOFOLLOW` so the privacy contract holds when the pinned version is bumped to one that supports the separate stream. diff --git a/website/docs/user-guide/egress/iron-proxy.md b/website/docs/user-guide/egress/iron-proxy.md index 836b374a95f..2259805ca37 100644 --- a/website/docs/user-guide/egress/iron-proxy.md +++ b/website/docs/user-guide/egress/iron-proxy.md @@ -140,16 +140,16 @@ To override: set `proxy.upstream_deny_cidrs` to your own list. To opt out entire ### Bind policy -The proxy binds **loopback only** (`127.0.0.1:`). It does NOT bind `0.0.0.0`. This means: +The proxy never binds `0.0.0.0`. The default bind is platform-specific because iron-proxy v0.39 supports only a **single bind per daemon process**: -- A LAN peer with a leaked proxy token cannot use it — the proxy is unreachable from the network. -- Containers reach the proxy via `host.docker.internal:9090`, which Docker maps to the host gateway via `--add-host=host.docker.internal:host-gateway` on Linux. On macOS / Windows Docker Desktop, Desktop manages the gateway itself. +- **Linux:** the docker bridge gateway (`172.17.0.1:` by default). Containers reach the proxy via `host.docker.internal`, which `--add-host=host.docker.internal:host-gateway` resolves to exactly this bridge gateway IP — a loopback-only bind would be unreachable from inside sandboxes. The bridge IP is an address on the host's `docker0` interface, so it is not exposed to the LAN; it IS reachable by other containers on the default bridge network, but requests still require a minted proxy token and an allowlisted upstream. If no docker bridge is detected (docker not installed/running), the bind falls back to loopback with a warning. +- **macOS / Windows Docker Desktop:** loopback (`127.0.0.1:`). Desktop's VPNkit routes `host.docker.internal` to the host, so loopback is reachable from containers and is the least-exposed choice. -iron-proxy v0.39 only supports a single bind per daemon process — earlier drafts of this integration emitted a plural `http_listens` list with the docker bridge IP appended for direct sandbox-to-bridge connectivity, but v0.39's YAML parser rejects that field. The `host.docker.internal -> host-gateway` mapping that Docker provides is sufficient: containers resolve the hostname to the bridge IP, then connect TO the host's loopback bind through it. +A LAN peer with a leaked proxy token cannot use the proxy — neither bind is reachable from the external network. -We also pin `metrics.listen: 127.0.0.1:0` so the daemon's built-in metrics server gets an ephemeral loopback port instead of its default `:9090` — otherwise it would fight `tunnel_port: 9090` for the same socket and the daemon would refuse to start with "address already in use". +We also pin `metrics.listen: 127.0.0.1:0` so the daemon's built-in metrics server gets an ephemeral loopback port instead of its default `:9090` — otherwise it would fight `tunnel_port: 9090` for the same socket and the daemon would refuse to start with "address already in use". Note the `:0` ephemeral port is random per start and not surfaced anywhere, so metrics are effectively disabled at this pin. -If a hostile `ip` shim earlier on PATH had been able to inject a non-private IPv4 here (`0.0.0.0`, a public address, multicast, link-local, etc.) the loopback fallback still applies — we never bind anything we couldn't validate via `ipaddress.IPv4Address` + `is_*` checks. +If a hostile `ip` shim earlier on PATH had been able to inject a non-private IPv4 as the bridge address (`0.0.0.0`, a public address, multicast, link-local, etc.) the loopback fallback still applies — we never bind anything we couldn't validate via `ipaddress.IPv4Address` + `is_*` checks. ## Uncovered providers @@ -322,7 +322,7 @@ On the currently pinned binary version (**v0.39.0**) iron-proxy writes ALL outpu We still pre-create `~/.hermes/proxy/audit.log` at `0o600` with `O_NOFOLLOW` because: -1. It serves as a stable logrotate / fluent-bit / monitoring target — operators can wire downstream tooling to that path today, and when we bump the pinned version to one that supports `log.audit_path`, the records will start flowing without any operator-side reconfiguration. +1. It reserves the path for the future version bump: when the pinned version moves to one that supports `log.audit_path`, per-request records will start flowing there without operator-side reconfiguration. **Until then the file stays at 0 bytes — do not point monitoring, alerting, or forensics tooling at it yet.** Use `iron-proxy.log` for everything today. 2. The 0o600-from-first-byte guarantee defends against the upstream-fix-day where v0.40+ creates the file under its default umask if it doesn't already exist. Until that version bump lands, treat `iron-proxy.log` as the source of truth for both audiences: @@ -369,7 +369,7 @@ When the Docker backend starts a container with `proxy.enabled: true` and the da |---|---| | `-v ~/.hermes/proxy/ca.crt:/etc/ssl/certs/hermes-egress-ca.crt:ro` | Read-only mount of the CA | | `-e HTTPS_PROXY=http://host.docker.internal:9090` | Python httpx / curl / go default transport / Node fetch | -| `-e HTTP_PROXY=…` | curl + wget for plain HTTP (rare in modern stacks) | +| `-e HTTP_PROXY=http://host.docker.internal:9091` | curl + wget for plain HTTP — the plain-HTTP forward listener lives on `tunnel_port + 1` | | `-e NO_PROXY=127.0.0.1,localhost,::1` | Loopback dev servers inside the sandbox bypass the proxy | | `-e REQUESTS_CA_BUNDLE=…ca.crt` | Python `requests` | | `-e SSL_CERT_FILE=…ca.crt` | Python `ssl` module / OpenSSL — **replaces** the system store | @@ -431,7 +431,7 @@ If the nonce check fails, the code falls back to matching `argv[0]` basename aga - Agent dialing cloud metadata endpoints (`169.254.169.254`) — iron-proxy denies these by default via `upstream_deny_cidrs`, including the IPv4-mapped-v6 form `::ffff:169.254.169.254`. - DNS rebinding through an allowlisted hostname to a private IP — the deny CIDRs are checked at connect time, not at allowlist time. - Same-uid local processes reading the iron-proxy daemon's env to scrape secrets — only the env var names referenced by mappings are forwarded, not the full host env. -- A LAN peer with a leaked sandbox proxy token spending your API quota — the proxy binds loopback only, never `0.0.0.0` (containers reach it via `host.docker.internal -> host-gateway`). +- A LAN peer with a leaked sandbox proxy token spending your API quota — the proxy binds the docker bridge gateway (Linux) or loopback (Docker Desktop), never `0.0.0.0`, so it is unreachable from the external network. **What it does NOT protect against:** @@ -484,10 +484,28 @@ Look at the last 20 lines of `~/.hermes/proxy/iron-proxy.log`. Common causes: - Invalid `proxy.yaml` → run `hermes egress setup` to regenerate - CA cert / key permissions wrong → `chmod 0o600 ~/.hermes/proxy/ca.key` -### "iron-proxy did not bind 127.0.0.1:9090 within 5s" +### "iron-proxy did not bind \:9090 within 5s" The daemon started but never bound the listener. Usually means the binary is wedged or doing something expensive at startup. Check `~/.hermes/proxy/iron-proxy.log`. The orphan process is killed automatically and the pidfile cleaned up so you can just retry `hermes egress start`. +### Sandbox times out connecting to the proxy (Linux) + +The container resolves `host.docker.internal` to the docker bridge gateway and the proxy is bound there, but a host firewall (commonly `ufw` with default-deny INPUT) drops container→host traffic on `docker0`. Verify from a container: + +```bash +docker run --rm --add-host host.docker.internal:host-gateway busybox \ + nc -zv -w 3 host.docker.internal 9090 +``` + +If that times out while `hermes egress status` shows `listening`, allow the bridge subnet in your firewall, e.g. for ufw: + +```bash +sudo ufw allow in on docker0 to any port 9090 proto tcp +sudo ufw allow in on docker0 to any port 9091 proto tcp +``` + +(9091 = the plain-HTTP forward listener on `tunnel_port + 1`.) + ### Sandbox sees `HTTP 403` from the proxy The agent inside the sandbox tried to hit a host that isn't in `proxy.extra_allowed_hosts`. The 403 body explains which host. If you want to allow it, add to your config: @@ -553,7 +571,7 @@ Or watch in real-time: tail -f ~/.hermes/proxy/iron-proxy.log | jq ``` -When the pinned version moves to v0.40+ (which adds `log.audit_path`), per-request records will move to `~/.hermes/proxy/audit.log` and `iron-proxy.log` will hold only daemon-level events. The file at `audit.log` is pre-created today at `0o600` so any logrotate / monitoring tooling you wire to that path keeps working through the version bump without operator-side reconfig. +When the pinned version moves to v0.40+ (which adds `log.audit_path`), per-request records will move to `~/.hermes/proxy/audit.log` and `iron-proxy.log` will hold only daemon-level events. Until that bump, `audit.log` is an empty placeholder (pre-created at `0o600` so the future daemon inherits tight permissions) — wire your logrotate / monitoring tooling to `iron-proxy.log` today and plan to add `audit.log` after the version bump. ## Limitations (v1) @@ -563,7 +581,7 @@ When the pinned version moves to v0.40+ (which adds `log.audit_path`), per-reque - The CA is a 10-year self-signed cert on first generation. Rotation requires `openssl genrsa ...` by hand (or wait for a follow-up that adds `hermes egress rotate-ca`). - Token rotation does not auto-restart the daemon; after `--rotate-tokens` you must `hermes egress stop && hermes egress start` and then restart running sandboxes. - iron-proxy in-memory secret zeroisation is upstream-controlled. Same-uid attackers with `/proc//mem` read access can read swapped-in secrets from the daemon's memory. -- iron-proxy v0.39 only supports a **single bind per daemon** and combines daemon + per-request records into a single log stream. The integration is designed to upgrade cleanly: the moment upstream adds `proxy.http_listens` (plural) and `log.audit_path`, both wire in automatically without changing operator configs. +- iron-proxy v0.39 only supports a **single bind per daemon** (we bind the docker bridge gateway on Linux, loopback on Docker Desktop) and combines daemon + per-request records into a single log stream. When upstream adds `proxy.http_listens` (plural) and `log.audit_path`, a version bump can wire in multi-bind and the dedicated audit stream. ## See also