mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-31 19:16:29 +00:00
cac9526d2a
The existing log line reports message counts, which is the least informative number available here: absorbing one tool-heavy exchange can drop hundreds of tokens while moving the count by one. There was no way to answer "is this actually helping?" from a real session. Emit one content-free JSON line per pass, in the same shape as the batch compaction telemetry: before/after tokens, the delta, the size of the absorbed exchange, the rolling summary size, duration, and running per-session totals so a whole run can be read off the last line. No transcript content rides along. Add scripts/micro_compaction_report.py to aggregate those lines into passes, outcome mix, net tokens saved, mean exchange size and durations, with an optional per-session breakdown. Measuring it immediately surfaced something worth documenting: the first pass in a session normally *costs* tokens. The summary marker carries a fixed ~400 tokens of scaffolding, paid on pass one against a single absorbed exchange. From pass two the marker is replaced rather than added, so the overhead is already paid and each exchange is close to pure saving. Break-even is typically the second or third pass. Tests cover the telemetry contract, the cumulative totals, and that first-pass/later-pass shape so nobody reads a single turn and concludes it made things worse. The estimator costs ~5 ms at 600 messages and ~20 ms at 1200, taken twice per pass, post-turn — and only once an exchange is actually in hand, so turns that no-op early pay nothing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| ci | ||
| lib | ||
| observability | ||
| tests | ||
| whatsapp-bridge | ||
| add_contributor.py | ||
| analyze_livetest.py | ||
| benchmark_browser_eval.py | ||
| build_model_catalog.py | ||
| build_skills_index.py | ||
| capture-cage-terminal.sh | ||
| check-windows-footguns.py | ||
| check_subprocess_stdin.py | ||
| contributor_audit.py | ||
| dev-sandbox.sh | ||
| discord-voice-doctor.py | ||
| docker_config_migrate.py | ||
| docker_rebootstrap_nous_session.py | ||
| generate_conformance_vectors.py | ||
| hermes-gateway | ||
| install.cmd | ||
| install.ps1 | ||
| install.sh | ||
| install_psutil_android.py | ||
| iso-certify.py | ||
| keystroke_diagnostic.py | ||
| kill_modal.sh | ||
| lint_diff.py | ||
| LIVETEST_README.md | ||
| micro_compaction_report.py | ||
| profile-tui.py | ||
| release.py | ||
| run_tests.sh | ||
| run_tests_parallel.py | ||
| sample_and_compress.py | ||
| smoke_nemo_relay_shared_metrics.py | ||
| tool_search_livetest.py | ||
| tool_search_livetest2.py | ||
| tool_search_livetest_ue.py | ||
| tool_search_livetest_ue_disc.py | ||
| tool_search_livetest_ue_hard.py | ||