Two bugs in the model fallback system:
1. Nous login leaves stale model in config (provider=nous, model=opus
from previous OpenRouter setup). Fixed by deferring the config.yaml
provider write until AFTER model selection completes, and passing the
selected model atomically via _update_config_for_provider's
default_model parameter. Previously, _update_config_for_provider was
called before model selection — if selection failed (free tier, no
models, exception), config stayed as nous+opus permanently.
2. Codex/stale providers in auxiliary fallback can't connect but block
the auto-detection chain. Added _is_connection_error() detection
(APIConnectionError, APITimeoutError, DNS failures, connection
refused) alongside the existing _is_payment_error() check in
call_llm(). When a provider endpoint is unreachable, the system now
falls back to the next available provider instead of crashing.
* fix(nix): switch nixpkgs input from nixos-24.11 to nixos-unstable
nixos-24.11 reached EOL on 2025-06-30. For a dev tool, tracking a
frozen release branch causes dependency versions to go stale.
nixos-unstable provides rolling updates and is the conventional
choice for development packages.
* docs(website): update nix flake example
---------
Co-authored-by: sk <sk@mercury>
* fix(ci): pin floating GitHub Actions tags and ascii-guard to explicit versions
Actions pinned to @main pull whatever is at that ref at execution time,
so a compromised upstream org could execute arbitrary code in CI.
- Pin DeterminateSystems/nix-installer-action to commit SHA (v22)
- Pin DeterminateSystems/magic-nix-cache-action to commit SHA (v13)
- Pin ascii-guard to 2.3.0 in docs-site-checks workflow
SHA comments include the version tag for human readability; Renovate or
Dependabot can keep these updated automatically.
* Add skill metadata extraction step in workflow
Add step to extract skill metadata for dashboard in CI workflow.
---------
Co-authored-by: Siddharth Balyan <52913345+alt-glitch@users.noreply.github.com>
Every turn now logs WHY the agent loop ended to agent.log with a
structured INFO line capturing: exit reason, model, api_calls/max,
budget usage, tool turn count, last message role, response length,
and session ID.
When the last message is a tool result and the turn was NOT
interrupted, emits WARNING level (visible in errors.log) — this is
the 'just stops' scenario users report where a tool call completes
but no continuation or final response follows.
10 tracked exit reasons: text_response, interrupted_by_user,
interrupted_during_api_call, budget_exhausted, max_iterations_reached,
all_retries_exhausted_no_response, fallback_prior_turn_content,
empty_response_exhausted, error_near_max_iterations, unknown.
Documents both debugging commands with full option tables,
examples, and usage guidance. Adds both to the top-level
commands table and as detailed sections with subsections for
log files, filtering behavior, and log rotation.
Adds a new CLI command that outputs a compact, plain-text dump of the
user's Hermes setup — version, OS, model/provider, API key presence,
toolsets, gateway status, platforms, cron jobs, skills, and any
non-default config overrides.
Designed for support context: no ANSI colors, ready to paste into
Discord/GitHub/Telegram. Secrets shown as 'set/not set' by default;
--show-keys reveals redacted prefixes (first/last 4 chars).
Files:
- hermes_cli/dump.py (new) — run_dump() implementation
- hermes_cli/main.py — parser + cmd_dump wiring
- hermes_cli/profiles.py — shell completions + subcommand set
The docs incorrectly referenced 'hermes pairing generate bluebubbles'
which doesn't exist. The existing reactive pairing flow already handles
this — when an unknown user messages the bot, it sends them a code
automatically, and the owner approves with 'hermes pairing approve'.
The old setup wizard (pre-March 2026) wrote LLM_MODEL to ~/.hermes/.env
across 12 provider flows. Commit 9302690e removed the writes but never
cleaned up existing .env files, leaving a dead variable that:
- Nothing in the codebase reads (zero os.getenv calls)
- The docs incorrectly claimed the gateway still used as fallback
- Caused user confusion when debugging model resolution issues
Changes:
- config.py: Bump _config_version 12 → 13, add migration to clear
LLM_MODEL and OPENAI_MODEL from .env (both dead since March 2026)
- environment-variables.md: Remove LLM_MODEL row, fix HERMES_MODEL
description to stop referencing it
- providers.md: Update deprecation notice from 'deprecated' to 'removed'
Parse x-ratelimit-* headers from inference API responses (Nous Portal,
OpenRouter, OpenAI-compatible) and display them in the /usage command.
- New agent/rate_limit_tracker.py: parse 12 rate limit headers (RPM/RPH/
TPM/TPH limits, remaining, reset timers), format as progress bars (CLI)
or compact one-liner (gateway)
- Hook into streaming path in run_agent.py: stream.response.headers is
available on the OpenAI SDK Stream object before chunks are consumed
- CLI /usage: appends rate limit section with progress bars + warnings
when any bucket exceeds 80%
- Gateway /usage: appends compact rate limit summary
- 24 unit tests covering parsing, formatting, edge cases
Headers captured per response:
x-ratelimit-{limit,remaining,reset}-{requests,tokens}{,-1h}
Example CLI display:
Nous Rate Limits (captured just now):
Requests/min [░░░░░░░░░░░░░░░░░░░░] 0.1% 1/800 used (799 left, resets in 59s)
Tokens/hr [░░░░░░░░░░░░░░░░░░░░] 0.0% 49/336.0M (336.0M left, resets in 52m)
Wrap is_dir() in _is_valid_subdir() and is_file() in
_load_hints_for_directory() with OSError handlers so that
inaccessible directories (e.g. /root from a non-root Daytona
host user) are silently skipped instead of crashing the agent.
The existing PermissionError PRs for prompt_builder.py (#6247,
#6321, #6355) do not cover subdirectory_hints.py, which was
identified as a separate crash path in the #6214 comments.
Ref: #6214
prune_sessions and delete_session only handled direct children when
satisfying the parent_session_id FK constraint. Multi-level chains
(A -> B -> C) caused IntegrityError because deleting B while C still
referenced it was blocked by the FK.
Fix: NULL out parent_session_id for any session whose parent is about
to be deleted. This orphans children instead of cascade-deleting them,
which also respects the prune retention window — newer child sessions
are no longer deleted just because an ancestor is old.
Reported by Aaryan2304 in PR #6463.
Updated the logic for determining the probed_url in the probe_api_models function to use the first tried URL instead of the last. This change ensures that the most relevant URL is returned when probing for models. Additionally, improved the output message in the _model_flow_custom function to provide clearer guidance based on the suggested_base_url.
The 24-hour default cooldown for 402-exhausted credentials was far too
aggressive — if a user tops up credits or the 402 was caused by an
oversized max_tokens request rather than true billing exhaustion, they
shouldn't have to wait a full day. Reduce to 1 hour (matching the
existing 429 TTL).
Inspired by PR #6493 (michalkomar).
When a model returns no content, no structured reasoning, and no tool
calls (common with open models), the agent now silently retries up to
3 times before falling through to (empty).
Silent retry (no synthetic messages) keeps the conversation history
clean, preserves prompt caching, and respects the no-synthetic-user-
injection invariant. Most empty responses from open models are
transient (provider hiccups, rate limits, sampling flukes) so a
simple retry is sufficient.
This fills the last gap in the empty-response recovery chain:
1. _last_content_with_tools fallback (prior tool turn had content)
2. Thinking-only prefill continuation (#5931 — structured reasoning)
3. Empty response silent retry (NEW — truly empty, no reasoning)
4. (empty) terminal (last resort after all retries exhausted)
Inline <think> blocks are excluded — the model chose to reason, it
just produced no visible text. That differs from truly empty.
Tests:
- Updated test_truly_empty to expect 4 API calls (1 + 3 retries)
- Added test_truly_empty_response_succeeds_on_nudge
The BlueBubbles adapter was merged but missing setup wizard support:
- Add _setup_bluebubbles() guided setup (server URL, password, allowlist,
home channel, webhook port)
- Add to _GATEWAY_PLATFORMS registry so it appears in 'hermes setup gateway'
- Add to any_messaging check and home channel missing warning
- Add to gateway status display in 'hermes setup'
- Add BLUEBUBBLES_SERVER_URL, BLUEBUBBLES_PASSWORD, BLUEBUBBLES_ALLOWED_USERS
to OPTIONAL_ENV_VARS with descriptions and categories
Previously the only way to configure BlueBubbles was manually editing .env.
Two issues resolved:
1. Add opencode.ai to _URL_TO_PROVIDER mapping so base_url routes through
models.dev lookup (which has mimo-v2-pro at 1M context) instead of
falling back to probing /models (404) and defaulting to 128K.
2. Fix _format_context_length to round cleanly: 1048576 → '1M' instead
of '1.048576M'. Applies same rounding logic to K values.
Add QEMU cross-compilation and multi-arch manifest support so Apple
Silicon (M1/M2/M3) and other ARM-based systems get native images.
- Add docker/setup-qemu-action for arm64 emulation on amd64 runners
- Smoke test stays amd64-only (load:true can't export multi-arch)
- Both push steps (main + release) now build linux/amd64,linux/arm64
- Bump timeout 30->60min for QEMU cross-compilation overhead
- Add permissions: contents: read (least-privilege hardening)
Salvaged from PR #3998 by Mibayy. Also addresses #5005 and #3913.
Co-authored-by: Mibayy <Mibayy@users.noreply.github.com>
Tests for the new behavior paths:
- Large tool outputs no longer block compaction (motivating scenario)
- Hard minimum of 3 tail messages always protected
- 1.5x soft ceiling for oversized messages
- Small conversations still compress (min 8 messages)
- Token-budget prune path in _prune_old_tool_results
- Fallback to message-count when no token budget
PR #6240 changed tail protection from protect_last_n to min(3, ...)
which increased the minimum compressible message count and shifted
tail boundaries. Three tests broke:
- test_summary_role_avoids_consecutive_user_messages: 6→8 msgs
- test_double_collision_user_head_assistant_tail: 7→8 msgs
- test_no_collision_scenarios_still_work: 6→8 msgs
All tests now exceed the new min_for_compress threshold (6) and
maintain proper role alternation in both head and tail sections.
Tail protection was effectively message-count based despite having a
token budget, because protect_last_n=20 acted as a hard floor. A single
50K-token tool output would cause all 20 recent messages to be
preserved regardless of budget, leaving little room for summarization.
Changes:
- _find_tail_cut_by_tokens: min_tail reduced from protect_last_n (20)
to 3; token budget is now the primary criterion
- Soft ceiling at 1.5x budget to avoid cutting mid-oversized-message
- _prune_old_tool_results: accepts optional protect_tail_tokens so
pruning also respects the token budget instead of a fixed count
- compress() minimum message check relaxed from protect_first_n +
protect_last_n + 1 to protect_first_n + 3 + 1
- Tool group alignment (no splitting tool_call/result) preserved
Port missing features from the hindsight-hermes external integration
package into the native plugin. Only touches plugin files — no core
changes.
Features:
- Tags on retain/recall (tags, recall_tags, recall_tags_match)
- Recall config (recall_max_tokens, recall_max_input_chars, recall_types,
recall_prompt_preamble)
- Retain controls (retain_every_n_turns, auto_retain, auto_recall,
retain_async via aretain_batch, retain_context)
- Bank config via Banks API (bank_mission, bank_retain_mission)
- Structured JSON retain with per-message timestamps
- Full session accumulation with document_id for dedup
- Custom post_setup() wizard with curses picker
- Mode-aware dep install (hindsight-client for cloud, hindsight-all for local)
- local_external mode and openai_compatible LLM provider
- OpenRouter support with auto base URL
- Auto-upgrade of hindsight-client to >=0.4.22 on session start
- Comprehensive debug logging across all operations
- 46 unit tests
- Updated README and website docs
Prevents unbounded memory growth in _assistant_threads dict.
Evicts oldest entries when exceeding _ASSISTANT_THREADS_MAX (5000),
matching the pattern used by _mentioned_threads and _seen_messages.
The test_non_internal_event_without_user_triggers_pairing test relied on
no Discord auth env vars being set, but gateway/run.py loads dotenv at
module level. In environments with DISCORD_ALLOW_ALL_USERS=True in .env,
the auth check passed instead of triggering the pairing flow.
Clear DISCORD_ALLOW_ALL_USERS, DISCORD_ALLOWED_USERS, GATEWAY_ALLOW_ALL_USERS,
and GATEWAY_ALLOWED_USERS via monkeypatch to ensure test isolation.
When a background process with notify_on_complete=True finishes, the
gateway injects a synthetic MessageEvent to notify the session. This
event was constructed without user_id, causing _is_user_authorized()
to reject it and — for DM-origin sessions — trigger the pairing flow,
sending "Hi~ I don't recognize you yet!" with a pairing code to the
chat owner.
Add an `internal` flag to MessageEvent that bypasses authorization
checks for system-generated synthetic events. Only the process watcher
sets this flag; no external/adapter code path can produce it.
Includes 4 regression tests covering the fix and the normal pairing path.
`_cleanup_task_resources` was unconditionally calling `cleanup_vm()` at
the end of every `run_conversation` (i.e. every user turn), tearing down
the docker/daytona/modal sandbox container regardless of its
`persistent_filesystem` setting. This contradicted the documented intent
of `terminal.lifetime_seconds` (idle reaper) and `container_persistent`,
and caused per-turn loss of `/workspace`, `~/.config`, agent CLI auth
state, and any other content living inside the sandbox.
The unconditional teardown was introduced in fbd3a2fd ("prevent leakage
of morph instances between tasks", 2025-11-04) to plug a Morph backend
leak, two days after `lifetime_seconds` shipped in faecbddd. It was
later refactored into `_cleanup_task_resources` in 70dd3a16 without
changing semantics. Code and docs have disagreed since.
Fix: introduce `terminal_tool.is_persistent_env(task_id)` and skip the
per-turn `cleanup_vm` when the active env is persistent. The idle reaper
(`_cleanup_inactive_envs`) still tears persistent envs down once
`terminal.lifetime_seconds` is exceeded. Non-persistent backends (Morph)
are unchanged — still torn down per turn, preserving the original
leak-prevention intent.
Combines the approaches from PR #6309 (duan78) and PR #5963 (KUSH42):
Tiered warnings (from #5963):
- Replaces boolean _context_pressure_warned with float _context_pressure_warned_at
- Fires at 85% (orange) and re-fires at 95% (red/critical)
- Adds 'compacting context...' status message before compression
Gateway dedup (from #6309):
- Class-level dict _context_pressure_last_warned survives across AIAgent
instances (gateway creates a new instance per message)
- 5-minute cooldown per session prevents warning spam
- Higher-tier warnings bypass the cooldown (85% → 95% always fires)
- Compression reset clears the dedup entry for the session
- Stale entries evicted (older than 2x cooldown) to prevent memory leak
Does NOT inject into messages — purely user-facing via _safe_print (CLI)
and status_callback (gateway). Zero prompt cache impact.
Fixes#6309. Fixes#5963.
ORIGINAL INCIDENT:
Discord forum descriptions (the topic field on ForumChannel) were invisible
to the agent. When a user set project instructions in a forum's description
(e.g. tool-evaluations), threads created in that forum had no Channel Topic
in their session context. Discovered while evaluating per-forum auto-context
injection for web-tap-terminal development threads.
ISSUE IN THE CODE:
In gateway/platforms/discord.py, all three session entry points
(_handle_message, _build_slash_event, _dispatch_thread_session) read
chat_topic via getattr(channel, 'topic', None). Discord Thread objects
don't carry a topic — only the parent ForumChannel does. So chat_topic
was always None for forum threads, and the Channel Topic line was never
injected into build_session_context_prompt output. The infrastructure to
handle this was already in place — _is_forum_parent() detects forum
channels, _format_thread_chat_name() traverses to the parent, and
build_session_context_prompt() renders Channel Topic when present. The
forum parent was being identified; its topic just wasn't being read.
HOW THIS COMMIT FIXES IT:
Adds _get_effective_topic(channel, is_thread) helper that reads
channel.topic first, then falls back to the parent forum's topic when
the channel is a thread inside a forum. All three session entry points
now call this helper instead of inlining getattr(channel, 'topic', None).
Existing tests pass unchanged.
Co-authored-by: dhabibi <9087935+dhabibi@users.noreply.github.com>
The new degradation warning reads compression_count as an int,
but the existing test's MagicMock returns a MagicMock object
for that attribute, causing '>=' comparison to fail.
Three targeted improvements to the compression system:
1. Replace hardcoded truncation limits with named class constants
(_CONTENT_MAX=6000, _CONTENT_HEAD=4000, _CONTENT_TAIL=1500,
_TOOL_ARGS_MAX=1500, _TOOL_ARGS_HEAD=1200). Previous limits
(3000/500) heavily truncated the summarizer's input — a 200-line
edit got cut to 3000 chars before the summarizer ever saw it.
2. Add '## Tools & Patterns' section to both compression prompt
templates (first-pass and iterative). Preserves working tool
invocations, preferred flags, and tool-specific discoveries
across compaction boundaries.
3. Warn users on 2nd+ compression: 'Session compressed N times —
accuracy may degrade. Consider /new to start fresh.'
Ref #499
- Bind exception in warning send handler (was using stale _ne from outer scope)
- Calculate remaining time until timeout correctly: (timeout - warning) // 60
instead of warning // 60 (which equals elapsed time, not remaining)
Introduce gateway_timeout_warning (default 900s) as a pre-timeout alert
layer. When inactivity reaches the warning threshold, a single
notification is sent to the user offering to wait or reset. If
inactivity continues to the gateway_timeout (default 1800s), the full
timeout fires as before.
This gives users a chance to intervene before work is lost on slow
API providers without disabling the safety timeout entirely.
Config: agent.gateway_timeout_warning in config.yaml, or
HERMES_AGENT_TIMEOUT_WARNING env var (0 = disable warning).
Step c in switch_model() blindly converted the first colon to a slash for
aggregator providers, even when the model name already contained a slash
(vendor/model format). This mangled variant tags like :free into /free,
causing 400 Bad Request from the API.
Fix: skip the colon→slash conversion when the model already has a slash,
since the colon is a variant tag, not a vendor separator. The module
docstring already documented this intent (line 17-18) but the
implementation didn't enforce it.
Reported via Discord. Related to PR #6088 (which identified the same bug
but placed the fix in model_normalize.py instead of model_switch.py where
the actual mangling occurs).
Local inference providers (Ollama, oMLX, llama-cpp) can take 300+ seconds
for prefill on large contexts. The 180s stale stream detector was killing
these connections while the provider was still processing.
Uses the existing is_local_endpoint() (proper URL parsing with RFC-1918,
localhost, WSL detection) instead of ad-hoc substring matching. The stale
timeout is only disabled when the user hasn't explicitly set
HERMES_STREAM_STALE_TIMEOUT — explicit user config is always honored.
Fixes#5889
Documents the per-platform tool_progress_overrides config key added in
PR #6348. Shows example YAML with Signal set to 'off' while Telegram
stays on 'verbose'. Lists all valid platform keys.
The _call_llm() and direct OpenAI fallback paths in flush_memories() both
hardcoded timeout=30.0, ignoring the user-configurable value at
auxiliary.flush_memories.timeout in config.yaml.
Remove the explicit timeout from the auxiliary _call_llm() call so that
_get_task_timeout('flush_memories') reads from config. For the direct
OpenAI fallback, import and use _get_task_timeout() instead of the
hardcoded value.
Add two regression tests verifying both code paths respect the config.
Fixes#6154
When managed_persistence is enabled, cleanup_browser() was calling
camofox_close() which destroys the server-side browser context via
DELETE /sessions/{userId}, killing login sessions across cron runs.
Add camofox_soft_cleanup() — a public wrapper that drops only the
in-memory session entry when managed persistence is on, returning True.
When persistence is off it returns False so the caller falls back to
the full camofox_close(). The inactivity reaper still handles idle
resource cleanup.
Also surface a logger.warning() when _managed_persistence_enabled()
fails to load config, replacing a silent except-and-return-False.
Salvaged from #6182 by el-analista (Eduardo Perea Fernandez).
Added public API wrapper to avoid cross-module private imports,
and test coverage for both persistence paths.
Co-authored-by: Eduardo Perea Fernandez <el-analista@users.noreply.github.com>
Fixes#4647 — Signal replies duplicated when gateway streaming is enabled.
Root cause: stream_consumer.py did not handle the case where send() returns
success=True but no message_id (Signal behavior). Every stream delta produced
a separate send() call (7+ messages instead of 2), plus the gateway sent
another full duplicate since already_sent was never set.
Changes:
- stream_consumer.py: Add elif branch for success-without-message_id — enters
fallback mode (sets already_sent, disables editing, sends only continuation)
- signal.py send(): Extract timestamp from signal-cli RPC result as message_id
so stream consumer follows normal edit→fallback path
- signal.py: Add public stop_typing() delegating to _stop_typing_indicator()
so base adapter's _keep_typing finally block can clean up typing tasks
- gateway/run.py: Per-platform tool_progress_overrides (#6164) — lets users
set e.g. signal: off while keeping telegram: all
- hermes_cli/config.py: Add tool_progress_overrides to DEFAULT_CONFIG
Refs: #4647, #6164
- Docker env tests: verify _build_init_env_args() instead of per-execute
Popen flags (env forwarding is now init-time only)
- Docker: preserve explicit forward_env bypass of blocklist from main
- Daytona tests: adapt to SDK-native timeout, _ThreadedProcessHandle,
base.py interrupt handling, HERMES_STDIN_ heredoc prefix
- Modal tests: fix _load_module to include _ThreadedProcessHandle stub,
check ensurepip in _resolve_modal_image instead of __init__
- SSH tests: mock time.sleep on base module instead of removed ssh import
- Add missing BaseEnvironment attributes to __new__()-based test fixtures
