hermes-agent/agent
Teknium 5f66c36470
fix(redact): pass web URLs through unchanged (#34029)
* fix(redact): pass web URLs through unchanged

Magic-link checkout URLs, OAuth callbacks the agent is meant to follow,
and pre-signed share URLs were getting `?token=***` / `?code=***` /
`?signature=***` blanket-redacted by parameter NAME, which breaks any
skill that has to round-trip a URL through history (the model's tool
call arguments get sanitized before persistence — the live call fires
with the real URL, but the next turn sees `***`).

Joe Rinaldi Johnson hit this with a checkout-acceleration skill that
uses magic links in URLs.

Drops three call sites from `redact_sensitive_text`:
- `_redact_url_query_params` (was redacting `access_token`, `token`,
  `api_key`, `code`, `signature`, `key`, `auth`, etc.)
- `_redact_url_userinfo` (was redacting `https://user:pass@host`)
- `_redact_http_request_target_query_params` (was redacting access-log
  request targets like `"POST /hook?password=... HTTP/1.1"`)

The helpers themselves are kept in the module — still importable by
anything that wants to opt in explicitly.

Still redacted (unchanged):
- Vendor-prefix credential shapes (sk-, ghp_, AKIA, gAAAA, etc.)
  anywhere they appear, including inside URLs — see the
  `test_known_prefix_inside_url_still_redacted` case.
- JWTs (`eyJ...`)
- DB connection-string passwords (`postgres://admin:pw@host`) —
  these are connection strings, not web URLs the agent navigates to.
- Authorization headers, ENV assignments, JSON `apiKey`/`token` fields,
  Telegram bot tokens, private key blocks, Discord mentions, E.164
  phone numbers, and form-urlencoded bodies (request bodies, not URLs).

Tests: replaces `TestUrlQueryParamRedaction` + `TestUrlUserinfoRedaction`
with `TestWebUrlsNotRedacted`, asserting representative URLs (OAuth
callback, magic link, S3 pre-signed, websocket, userinfo, access log)
pass through unchanged. Adds positive cases proving the prefix and DB
connstr nets still fire. 74 redact tests + 10 browser-exfil + 16 PII
redaction tests all pass.

* test(codex_app_server): drop URL-query assertion from stderr-tail redaction test

The test bundled (a) sk-live-* credential-prefix redaction with (b)
URL query-param redaction. (a) is still in effect via _PREFIX_RE;
(b) was the contract we just removed in the parent commit so the
'querysecret12345' assertion stopped holding. Keep the credential-shape
assertion, drop the URL-query one.

Send-message tool's local _URL_SECRET_QUERY_RE in tools/send_message_tool.py
is independent of agent/redact.py and unchanged — its tests
(test_top_level_send_failure_redacts_query_token,
test_http_error_redacts_access_token_in_exception_text) still pass.
2026-05-28 11:32:39 -07:00
..
lsp chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) 2026-05-17 02:29:41 -07:00
secret_sources perf(cli): cut hermes startup 63% — flip head-to-head vs codex (#31968) 2026-05-25 03:06:39 -07:00
transports fix(codex): omit tools key from Codex Responses kwargs when no tools registered 2026-05-27 11:46:17 -07:00
__init__.py fix(agent): preload jiter native parser 2026-05-28 00:20:11 -07:00
account_usage.py chore: ruff auto-fix PLR6201 — tuple → set in membership tests (#23937) 2026-05-11 11:13:25 -07:00
agent_init.py feat(context-engine): host contract for external context engines 2026-05-28 01:45:30 -07:00
agent_runtime_helpers.py fix(agent): re-pad reasoning_content on cross-provider fallback to require-side providers 2026-05-28 03:21:00 -07:00
anthropic_adapter.py feat: add claude-opus-4.8 and claude-opus-4.8-fast (#34003) 2026-05-28 10:31:59 -07:00
async_utils.py fix(async): close unscheduled coroutines in all threadsafe bridges (#26584) 2026-05-15 14:00:01 -07:00
auxiliary_client.py feat(auth) normalise the way in which we check whether a user has free/paid access to nous portal so we can expose behaviour and error messages accordingly. 2026-05-28 00:19:31 -07:00
azure_identity_adapter.py feat(azure-foundry): add Microsoft Entra ID auth 2026-05-18 10:14:38 -07:00
background_review.py test: cover ci-unblocker production regressions 2026-05-27 22:14:53 -07:00
bedrock_adapter.py chore(deps): lazy-install boto3/botocore for bedrock adapter 2026-05-17 02:31:18 -07:00
browser_provider.py fix(browser): self-review pass — dead-import, log levels, future-proofing 2026-05-17 04:04:15 -07:00
browser_registry.py fix(browser): self-review pass — dead-import, log levels, future-proofing 2026-05-17 04:04:15 -07:00
chat_completion_helpers.py feat(agent): buffer retry/fallback status, surface only on terminal failure (#33816) 2026-05-28 04:53:27 -07:00
codex_responses_adapter.py fix(xai-oauth): strip service_tier and add safety-net sanitization for slash enums 2026-05-27 05:25:38 -07:00
codex_runtime.py refactor(codex): drop SDK responses.stream() helper; consume events directly (#33042) 2026-05-27 00:30:06 -07:00
context_compressor.py fix(compressor): ABC compliance — total_tokens, api_mode, logger consistency 2026-05-23 17:38:19 -07:00
context_engine.py feat(context-engine): host contract for external context engines 2026-05-28 01:45:30 -07:00
context_references.py fix(agent): fall back when rg is blocked for @folder references 2026-04-20 01:56:41 -07:00
conversation_compression.py feat(context-engine): host contract for external context engines 2026-05-28 01:45:30 -07:00
conversation_loop.py fix(agent): fallback immediately on provider content-policy blocks (#33883) 2026-05-28 07:28:24 -07:00
copilot_acp_client.py fix: guard yaml.safe_load, flock unlock, TOCTOU races, and atomic writes 2026-05-19 00:12:41 -07:00
credential_persistence.py fix: avoid persisting borrowed credential secrets (#31416) 2026-05-25 00:32:08 -07:00
credential_pool.py fix(anthropic): API-key path skips OAuth autodiscovery + prunes stale entries 2026-05-25 17:41:40 -07:00
credential_sources.py docs(auth): replace stale 'hermes login' references with 'hermes auth add' 2026-05-26 15:41:11 -07:00
curator.py fix: preserve skill packages during curator consolidation 2026-05-27 13:39:58 -07:00
curator_backup.py fix(skills): prune dependency/venv dirs from all skill scanners (#30042) 2026-05-21 14:18:02 -07:00
display.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00
error_classifier.py fix(agent): fallback immediately on provider content-policy blocks (#33883) 2026-05-28 07:28:24 -07:00
file_safety.py fix(security): block read_file on project-local .env files 2026-05-25 03:40:47 -07:00
gemini_cloudcode_adapter.py fix(agent/gemini-cloudcode): seed delta defaults for reasoning-only stream chunks 2026-05-14 08:03:56 -07:00
gemini_native_adapter.py fix(auxiliary): evict async wrappers on poisoned client (follow-up to #23482) 2026-05-11 11:13:20 -07:00
gemini_schema.py chore: remove unused imports and dead locals (ruff F401, F841) (#17010) 2026-04-28 06:46:45 -07:00
google_code_assist.py chore: remove unused imports and dead locals (ruff F401, F841) (#17010) 2026-04-28 06:46:45 -07:00
google_oauth.py docs(auth): replace stale 'hermes login' references with 'hermes auth add' 2026-05-26 15:41:11 -07:00
i18n.py feat(i18n): localize all gateway commands + web dashboard, add 8 new locales (16 total) (#22914) 2026-05-10 07:14:14 -07:00
image_gen_provider.py fix(image_gen): cache xAI ephemeral URL responses to disk (#26942) (#31759) 2026-05-24 18:10:47 -07:00
image_gen_registry.py fix(plugins): filter resolution by is_available() in web + image_gen registries 2026-05-13 22:31:28 -07:00
image_routing.py fix(agent): consult supports_vision override in auto-mode routing 2026-05-20 23:27:10 -07:00
insights.py Merge branch 'main' into feat/dashboard-skill-analytics 2026-04-20 05:25:49 -07:00
iteration_budget.py refactor(run_agent): extract OpenAI proxy, safe stdio, IterationBudget 2026-05-16 17:59:32 -07:00
jiter_preload.py fix(agent): preload jiter native parser 2026-05-28 00:20:11 -07:00
lmstudio_reasoning.py feat(agent): add lmstudio integration 2026-04-28 12:27:36 -07:00
manual_compression_feedback.py fix(compression): include system prompt + tool schemas in token estimates (#18265) 2026-04-30 23:03:54 -07:00
markdown_tables.py fix(cli): vertical fallback for markdown tables wider than terminal (#23948) 2026-05-11 16:49:13 -07:00
memory_manager.py 🐛 fix(memory): require newline after context tag 2026-05-18 10:53:08 -07:00
memory_provider.py Add Honcho runtime peer mapping 2026-05-27 10:49:33 -07:00
message_sanitization.py refactor(run_agent): extract message sanitization to agent/message_sanitization.py 2026-05-16 17:41:09 -07:00
model_metadata.py feat: add claude-opus-4.8 and claude-opus-4.8-fast (#34003) 2026-05-28 10:31:59 -07:00
models_dev.py remove Vercel AI Gateway and Vercel Sandbox (#33067) 2026-05-27 00:43:32 -07:00
moonshot_schema.py fix(moonshot): strip $ref siblings and collapse tuple items in tool schemas (#27104) 2026-05-16 13:02:19 -07:00
nous_rate_guard.py codebase: add encoding='utf-8' to all bare open() calls (PLW1514) 2026-05-08 14:27:40 -07:00
onboarding.py docs(onboarding): lead OpenClaw residue banner with migrate, warn that cleanup breaks OpenClaw (#17507) 2026-04-29 08:08:36 -07:00
plugin_llm.py feat(plugins): run any LLM call from inside a plugin via ctx.llm (#23194) 2026-05-10 07:09:28 -07:00
portal_tags.py feat(nous): unified client=hermes-client-v<version> tag on every Portal request (#24779) 2026-05-12 20:49:20 -07:00
process_bootstrap.py refactor(run_agent): extract OpenAI proxy, safe stdio, IterationBudget 2026-05-16 17:59:32 -07:00
prompt_builder.py remove Vercel AI Gateway and Vercel Sandbox (#33067) 2026-05-27 00:43:32 -07:00
prompt_caching.py fix(cache): kill long-lived prefix layout — system prompt is now byte-static within a session (#24778) 2026-05-12 20:46:04 -07:00
rate_limit_tracker.py refactor: remove dead code — 1,784 lines across 77 files (#9180) 2026-04-13 16:32:04 -07:00
redact.py fix(redact): pass web URLs through unchanged (#34029) 2026-05-28 11:32:39 -07:00
retry_utils.py feat(agent): add jittered retry backoff 2026-04-08 00:41:36 -07:00
shell_hooks.py fix: guard yaml.safe_load, flock unlock, TOCTOU races, and atomic writes 2026-05-19 00:12:41 -07:00
skill_bundles.py feat(skills): add skill bundles — alias /<name> loads multiple skills (#28373) 2026-05-18 21:38:05 -07:00
skill_commands.py fix(skills): load symlinked skill slash commands 2026-05-18 00:34:29 -07:00
skill_preprocessing.py fix: treat inline-shell timeout guard as timeout 2026-05-18 19:36:04 -07:00
skill_utils.py fix(skills): load Linux-tagged skills on Termux (android sys.platform) 2026-05-21 19:08:38 -07:00
stream_diag.py feat(agent): buffer retry/fallback status, surface only on terminal failure (#33816) 2026-05-28 04:53:27 -07:00
subdirectory_hints.py fix(subdirectory_hints): prevent loading AGENTS.md outside workspace 2026-05-25 23:17:33 -07:00
system_prompt.py fix(profiles): cross-profile soft guard on file-write tools + system-prompt hint (#31290) 2026-05-24 00:38:17 -07:00
think_scrubber.py fix(agent): stateful streaming scrubber for reasoning-block leaks (#17924) (#20184) 2026-05-05 04:33:38 -07:00
title_generator.py fix: improve telegram topic mode setup 2026-05-04 12:07:17 -07:00
tool_dispatch_helpers.py feat(security): promptware defense — shared threat patterns + memory load-time scan + tool-result delimiters (#32269) 2026-05-25 14:52:24 -07:00
tool_executor.py fix(cli): surface tool failures with specific error messages 2026-05-23 21:03:51 -07:00
tool_guardrails.py fix: add recovery hints to loop guard warnings 2026-05-19 00:12:12 -07:00
tool_result_classification.py fix: classify landed file mutations with diagnostics 2026-05-13 06:46:23 -07:00
trajectory.py Refactor Terminal and AIAgent cleanup 2026-02-21 22:31:43 -08:00
transcription_provider.py feat(stt): add register_transcription_provider() plugin hook 2026-05-25 01:41:19 -07:00
transcription_registry.py feat(stt): add register_transcription_provider() plugin hook 2026-05-25 01:41:19 -07:00
tts_provider.py feat(tts): add register_tts_provider() plugin hook (closes #30398) 2026-05-24 18:04:54 -07:00
tts_registry.py feat(tts): add register_tts_provider() plugin hook (closes #30398) 2026-05-24 18:04:54 -07:00
usage_pricing.py feat: add claude-opus-4.8 and claude-opus-4.8-fast (#34003) 2026-05-28 10:31:59 -07:00
video_gen_provider.py feat(video_gen): unified video_generate tool with pluggable provider backends (#25126) 2026-05-13 16:39:41 -07:00
video_gen_registry.py feat(video_gen): unified video_generate tool with pluggable provider backends (#25126) 2026-05-13 16:39:41 -07:00
web_search_provider.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00
web_search_registry.py chore(web): remove web_crawl tool + provider crawl plumbing (#33824) 2026-05-28 04:52:42 -07:00