mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-04-25 00:51:20 +00:00
design: compression eval harness — add three scrubbed fixtures + scrubber
Adds scripts/compression_eval/ with a design doc, README, a placeholder run_eval.py, and three checked-in scrubbed session fixtures. No working eval yet — PR is for design review before implementation. Motivation: we edit agent/context_compressor.py prompts and _template_sections by hand and ship without any automated check that compression still preserves file paths, error codes, or the active task. Factory.ai's Dec 2025 write-up (https://factory.ai/news/evaluating-compression) documents a probe-based eval scored on six dimensions. We adopt the methodology; we do not publish scores. Contents: - DESIGN.md — fixture format, probe format (recall / artifact / continuation / decision), six grading dimensions, report format, cost expectations, scrubber pipeline, open questions, and staged follow-up PR plan. - README.md — short 'what this is / when to run it' page. - run_eval.py — placeholder that prints 'not implemented, see DESIGN.md' and exits 1. - scrub_fixtures.py — reproducible pipeline that converts real sessions from ~/.hermes/sessions/*.jsonl into public-safe JSON fixtures. Applies: redact_sensitive_text, username path normalization, personal handle scrubbing, email and git-author normalization, reasoning scratchpad / <think> stripping, platform user-mention scrubbing, first-user paraphrase, system-prompt placeholder, orphan-message pruning, and tool-output size truncation for fixture readability. - fixtures/feature-impl-context-priority.json — 75 msgs / ~17k tokens. Investigate → patch → test → PR → merge. - fixtures/debug-session-feishu-id-model.json — 59 msgs / ~13k tokens. PR triage + upstream docs + decision. - fixtures/config-build-competitive-scouts.json — 61 msgs / ~23k tokens. Iterative config accumulation (11 cron jobs across 7 weekdays). PII audit: zero matches across the three fixtures for the maintainer's handle (all case variants), personal email domains, and known contributor emails. Only 'contributor@example.com' placeholder remains. Why scripts/: requires API credentials, costs ~\$1 per run, LLM-graded (non-deterministic), must not run in CI. scripts/sample_and_compress.py is the existing precedent for offline credentialed tooling.
This commit is contained in:
Teknium
parent
c6b734e24d
commit
9f5c13f874
10 changed files with 2642 additions and 0 deletions
4
.gitignore
vendored
4
.gitignore
vendored
|
|
@ -52,6 +52,10 @@ ignored/
|
||||||
.worktrees/
|
.worktrees/
|
||||||
environments/benchmarks/evals/
|
environments/benchmarks/evals/
|
||||||
|
|
||||||
|
# Compression eval run outputs (harness lives in scripts/compression_eval/)
|
||||||
|
scripts/compression_eval/results/*
|
||||||
|
!scripts/compression_eval/results/.gitkeep
|
||||||
|
|
||||||
# Web UI build output
|
# Web UI build output
|
||||||
hermes_cli/web_dist/
|
hermes_cli/web_dist/
|
||||||
|
|
||||||
|
|
|
||||||
341
scripts/compression_eval/DESIGN.md
Normal file
341
scripts/compression_eval/DESIGN.md
Normal file
|
|
@ -0,0 +1,341 @@
|
||||||
|
# Compression Eval — Design
|
||||||
|
|
||||||
|
Status: proposal. Nothing under `scripts/compression_eval/` runs in CI.
|
||||||
|
This is an offline tool authors run before merging prompt or algorithm
|
||||||
|
changes to `agent/context_compressor.py`.
|
||||||
|
|
||||||
|
## Why
|
||||||
|
|
||||||
|
We tune the compressor prompt and the `_template_sections` checklist by
|
||||||
|
hand, ship, and wait for the next real session to notice regressions.
|
||||||
|
There is no automated check that a prompt edit still preserves file
|
||||||
|
paths, error messages, or the active task across a compression.
|
||||||
|
|
||||||
|
Factory.ai's December 2025 write-up
|
||||||
|
(https://factory.ai/news/evaluating-compression) describes a
|
||||||
|
probe-based eval that scores compressed state on six dimensions. The
|
||||||
|
methodology is the valuable part — the benchmarks in the post are a
|
||||||
|
marketing piece. We adopt the methodology and discard the scoreboard.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Given a real session transcript and a bank of probe questions that
|
||||||
|
exercise what the transcript contained, answer:
|
||||||
|
|
||||||
|
1. After `ContextCompressor.compress()` runs, can the agent still
|
||||||
|
answer each probe correctly from the compressed state?
|
||||||
|
2. Which of the six dimensions (accuracy, context awareness, artifact
|
||||||
|
trail, completeness, continuity, instruction following) is the
|
||||||
|
prompt weakest on?
|
||||||
|
3. Does a prompt change improve or regress any dimension vs. the
|
||||||
|
previous run?
|
||||||
|
|
||||||
|
That is the full scope. No "compare against OpenAI and Anthropic"
|
||||||
|
benchmarking, no public scoreboard, no marketing claims.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- Not a pytest. Requires API credentials, costs money, takes minutes
|
||||||
|
per fixture, and output is LLM-graded and non-deterministic.
|
||||||
|
- Not part of `scripts/run_tests.sh`. Not invoked by CI.
|
||||||
|
- Not a replacement for the existing compressor unit tests in
|
||||||
|
`tests/agent/test_context_compressor.py` — those stay as the
|
||||||
|
structural / boundary / tool-pair-sanitization guard.
|
||||||
|
- Not a general trajectory eval. Scoped to context compaction only.
|
||||||
|
|
||||||
|
## Where it lives
|
||||||
|
|
||||||
|
```
|
||||||
|
scripts/compression_eval/
|
||||||
|
├── DESIGN.md # this file
|
||||||
|
├── README.md # how to run, cost expectations, caveats
|
||||||
|
├── run_eval.py # entry point (fire CLI, like sample_and_compress.py)
|
||||||
|
├── scrub_fixtures.py # regenerate fixtures from ~/.hermes/sessions/*.jsonl
|
||||||
|
├── fixtures/ # checked-in scrubbed session snapshots
|
||||||
|
│ ├── feature-impl-context-priority.json
|
||||||
|
│ ├── debug-session-feishu-id-model.json
|
||||||
|
│ └── config-build-competitive-scouts.json
|
||||||
|
├── probes/ # probe banks paired with fixtures
|
||||||
|
│ └── <fixture>.probes.json
|
||||||
|
├── rubric.py # grading prompt + dimension definitions
|
||||||
|
├── grader.py # judge-model call + score parsing
|
||||||
|
├── compressor_driver.py # thin wrapper over ContextCompressor
|
||||||
|
└── results/ # gitignored; timestamped output per run
|
||||||
|
└── .gitkeep
|
||||||
|
```
|
||||||
|
|
||||||
|
`scripts/` is the right home: offline tooling, no CI involvement,
|
||||||
|
precedent already set by `sample_and_compress.py`,
|
||||||
|
`contributor_audit.py`, `discord-voice-doctor.py`.
|
||||||
|
|
||||||
|
`environments/` is for Atropos RL training environments — wrong shape.
|
||||||
|
`tests/` is hermetic and credential-free — incompatible with a
|
||||||
|
probe-based eval that needs a judge model.
|
||||||
|
|
||||||
|
## Fixture format
|
||||||
|
|
||||||
|
A fixture is a single compressed-enough conversation captured from a
|
||||||
|
real session. Stored as JSON (pretty-printed, reviewable in PRs):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"name": "401-debug",
|
||||||
|
"description": "178-turn session debugging a 401 on /api/auth/login",
|
||||||
|
"model": "anthropic/claude-sonnet-4.6",
|
||||||
|
"context_length": 200000,
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "..."},
|
||||||
|
{"role": "user", "content": "..."},
|
||||||
|
{"role": "assistant", "content": "...", "tool_calls": [...]},
|
||||||
|
{"role": "tool", "tool_call_id": "...", "content": "..."}
|
||||||
|
],
|
||||||
|
"notes": "Captured 2026-04-24 from session 20260424_*.jsonl; \
|
||||||
|
PII scrubbed; secrets redacted via redact_sensitive_text."
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Sourcing fixtures
|
||||||
|
|
||||||
|
Fixtures are scrubbed snapshots of real sessions from the
|
||||||
|
maintainer's `~/.hermes/sessions/*.jsonl` store, generated
|
||||||
|
reproducibly by `scrub_fixtures.py` in this directory. Re-run the
|
||||||
|
scrubber with `python3 scripts/compression_eval/scrub_fixtures.py`
|
||||||
|
to regenerate them after a scrubber change.
|
||||||
|
|
||||||
|
Three shipped fixtures cover three different session shapes:
|
||||||
|
|
||||||
|
| Fixture | Source shape | Messages | Tokens (rough) | Tests |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| `feature-impl-context-priority` | investigate → patch → test → PR → merge | 75 | ~17k | continuation, artifact trail (2 files modified, 1 PR) |
|
||||||
|
| `debug-session-feishu-id-model` | PR triage + upstream docs + decision | 59 | ~13k | recall (PR #, error shape), decision (outcome + reason) |
|
||||||
|
| `config-build-competitive-scouts` | iterative config: 11 cron jobs across 7 weekdays | 61 | ~23k | artifact trail (which jobs, which days), iterative-merge |
|
||||||
|
|
||||||
|
The `~17k-23k` token range is below the default 50%-of-200k
|
||||||
|
compression threshold, so the eval will always **force** a
|
||||||
|
`compress()` call rather than wait for the natural trigger. That is
|
||||||
|
the intended shape — we want a controlled single-shot compression so
|
||||||
|
score deltas are attributable to the prompt change, not to whether
|
||||||
|
the threshold happened to fire at the same boundary twice.
|
||||||
|
|
||||||
|
### Scrubber pipeline
|
||||||
|
|
||||||
|
`scrub_fixtures.py` applies, per message:
|
||||||
|
|
||||||
|
1. `agent.redact.redact_sensitive_text` — API keys, tokens,
|
||||||
|
connection strings
|
||||||
|
2. Username paths: `/home/teknium` → `/home/user`
|
||||||
|
3. Personal handles: all case variants of the maintainer name → `user`
|
||||||
|
4. Email addresses → `contributor@example.com`; git
|
||||||
|
`Author: Name <addr>` header lines normalised
|
||||||
|
5. `<REASONING_SCRATCHPAD>...</REASONING_SCRATCHPAD>` and
|
||||||
|
`<think>...</think>` stripped from assistant content
|
||||||
|
6. Messaging-platform user mentions (`<@123456>`, `<@***>`) →
|
||||||
|
`<@user>`
|
||||||
|
7. First user message paraphrased to remove personal voice;
|
||||||
|
subsequent user turns kept verbatim after the redactions above
|
||||||
|
8. System prompt replaced with a generic public-safe placeholder so
|
||||||
|
we don't check in the maintainer's tuned soul/skills/memory system
|
||||||
|
block
|
||||||
|
9. Orphan empty-assistant messages (artifact of scratchpad-only
|
||||||
|
turns) and trailing tool messages with no matching assistant are
|
||||||
|
dropped
|
||||||
|
10. Tool outputs longer than 2000 chars are truncated with a size
|
||||||
|
annotation; the compressor sees that the tool was called and
|
||||||
|
returned something but not the full 16KB skill_view or 5KB
|
||||||
|
web_extract body (no signal loss for compression probes)
|
||||||
|
|
||||||
|
Before every fixture PR: grep the fixture for PII patterns. An
|
||||||
|
audit is embedded at the bottom of the scrubber as comments.
|
||||||
|
|
||||||
|
**Fixtures must stay small.** Target <150 KB per fixture, <500 KB
|
||||||
|
total for the directory. Current total: ~230 KB across three
|
||||||
|
fixtures. Larger sessions are truncated with a
|
||||||
|
`truncated_to: <index>` field in the fixture header so the cut is
|
||||||
|
reviewable.
|
||||||
|
|
||||||
|
## Probe format
|
||||||
|
|
||||||
|
One probe file per fixture, so reviewers can see the question bank
|
||||||
|
evolve alongside the fixture:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"fixture": "401-debug",
|
||||||
|
"probes": [
|
||||||
|
{
|
||||||
|
"id": "recall-error-code",
|
||||||
|
"type": "recall",
|
||||||
|
"question": "What was the original error code and endpoint?",
|
||||||
|
"expected_facts": ["401", "/api/auth/login"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "artifact-files-modified",
|
||||||
|
"type": "artifact",
|
||||||
|
"question": "Which files have been modified in this session?",
|
||||||
|
"expected_facts": ["session_store.py", "redis_client.py"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "continuation-next-step",
|
||||||
|
"type": "continuation",
|
||||||
|
"question": "What should we do next?",
|
||||||
|
"expected_facts": ["re-run the integration tests", "restart the worker"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "decision-redis-approach",
|
||||||
|
"type": "decision",
|
||||||
|
"question": "What did we decide about the Redis issue?",
|
||||||
|
"expected_facts": ["switch to redis-py 5.x", "pooled connection"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The four probe types come directly from Factory's methodology:
|
||||||
|
**recall, artifact, continuation, decision**. `expected_facts` gives
|
||||||
|
the grader concrete anchors instead of relying purely on LLM taste.
|
||||||
|
|
||||||
|
Authoring a probe bank is a one-time cost per fixture. 8-12 probes per
|
||||||
|
fixture is the target — enough to cover all four types, few enough to
|
||||||
|
grade in under a minute at reasonable cost.
|
||||||
|
|
||||||
|
## Grading
|
||||||
|
|
||||||
|
Each probe gets scored 0-5 on **six dimensions** (Factory's six):
|
||||||
|
|
||||||
|
| Dimension | What it measures |
|
||||||
|
|-----------------------|-----------------------------------------------------|
|
||||||
|
| accuracy | File paths, function names, error codes are correct |
|
||||||
|
| context_awareness | Reflects current state, not a mid-session snapshot |
|
||||||
|
| artifact_trail | Knows which files were read / modified / created |
|
||||||
|
| completeness | Addresses all parts of the probe |
|
||||||
|
| continuity | Agent can continue without re-fetching |
|
||||||
|
| instruction_following | Probe answered in the requested form |
|
||||||
|
|
||||||
|
Grading is done by a single judge-model call per probe with a
|
||||||
|
deterministic rubric prompt (see `rubric.py`). The rubric includes the
|
||||||
|
`expected_facts` list so the judge has a concrete anchor. Default
|
||||||
|
judge model: whatever the user has configured as their main model at
|
||||||
|
run time (same resolution path as `auxiliary_client.call_llm`). A
|
||||||
|
`--judge-model` flag allows overriding for consistency across runs.
|
||||||
|
|
||||||
|
Non-determinism caveat: two runs of the same fixture will produce
|
||||||
|
different scores. A single run means nothing. Report medians over
|
||||||
|
N=3 runs by default, and require an improvement of >=0.3 on any
|
||||||
|
dimension before claiming a prompt change is a win.
|
||||||
|
|
||||||
|
## Run flow
|
||||||
|
|
||||||
|
```
|
||||||
|
python scripts/compression_eval/run_eval.py [OPTIONS]
|
||||||
|
```
|
||||||
|
|
||||||
|
Options (fire-style, mirroring `sample_and_compress.py`):
|
||||||
|
|
||||||
|
| Flag | Default | Purpose |
|
||||||
|
|------------------------|------------|-------------------------------------------|
|
||||||
|
| `--fixtures` | all | Comma-separated fixture names |
|
||||||
|
| `--runs` | 3 | Runs per fixture (for median) |
|
||||||
|
| `--judge-model` | auto | Override judge model |
|
||||||
|
| `--compressor-model` | auto | Override model used *inside* the compressor |
|
||||||
|
| `--label` | timestamp | Subdirectory under `results/` |
|
||||||
|
| `--focus-topic` | none | Pass-through to `compress(focus_topic=)` |
|
||||||
|
| `--compare-to` | none | Path to a previous run for diff output |
|
||||||
|
|
||||||
|
Steps per fixture per run:
|
||||||
|
|
||||||
|
1. Load fixture JSON and probe bank.
|
||||||
|
2. Construct a `ContextCompressor` against the fixture's model.
|
||||||
|
3. Call `compressor.compress(messages)` — capture the compressed
|
||||||
|
message list.
|
||||||
|
4. For each probe: ask the judge model to role-play as the continuing
|
||||||
|
agent with only the compressed state, then grade the answer on the
|
||||||
|
six dimensions using `rubric.py`.
|
||||||
|
5. Write a per-run JSON to `results/<label>/<fixture>-run-N.json`.
|
||||||
|
6. After all runs, emit a markdown summary to
|
||||||
|
`results/<label>/report.md`.
|
||||||
|
|
||||||
|
## Report format
|
||||||
|
|
||||||
|
Pasted verbatim into PR descriptions that touch the compressor:
|
||||||
|
|
||||||
|
```
|
||||||
|
## Compression eval — label 2026-04-25_13-40-02
|
||||||
|
|
||||||
|
Main model: anthropic/claude-sonnet-4.6 Judge: same
|
||||||
|
3 runs per fixture, medians reported.
|
||||||
|
|
||||||
|
| Fixture | Accuracy | Context | Artifact | Complete | Continuity | Instruction | Overall |
|
||||||
|
|----------------|----------|---------|----------|----------|------------|-------------|---------|
|
||||||
|
| 401-debug | 4.1 | 4.0 | 2.5 | 4.3 | 3.8 | 5.0 | 3.95 |
|
||||||
|
| pr-review | 3.9 | 3.8 | 3.1 | 4.2 | 3.9 | 5.0 | 3.98 |
|
||||||
|
| feature-impl | 4.0 | 3.9 | 2.9 | 4.1 | 4.0 | 5.0 | 3.98 |
|
||||||
|
|
||||||
|
Per-probe misses (score < 3.0):
|
||||||
|
- 401-debug / artifact-files-modified: 1.7 — summary dropped redis_client.py
|
||||||
|
- pr-review / decision-auth-rewrite: 2.3 — outcome captured, reasoning dropped
|
||||||
|
```
|
||||||
|
|
||||||
|
## Cost expectations
|
||||||
|
|
||||||
|
Dominated by the judge calls. For 3 fixtures × 10 probes × 3 runs =
|
||||||
|
90 judge calls per eval run. On Claude Sonnet 4.6 that is roughly
|
||||||
|
$0.50-$1.50 per full eval depending on probe length. The compressor
|
||||||
|
itself makes 1 call per fixture × 3 runs = 9 additional calls.
|
||||||
|
|
||||||
|
**This is not a check to run after every commit.** It is a
|
||||||
|
before-merge check for PRs that touch:
|
||||||
|
|
||||||
|
- `agent/context_compressor.py` — any change to `_template_sections`,
|
||||||
|
`_generate_summary`, or `compress()`.
|
||||||
|
- `agent/auxiliary_client.py` — when changing how compression tasks
|
||||||
|
are routed.
|
||||||
|
- `agent/prompt_builder.py` — when the compression-note phrasing
|
||||||
|
changes.
|
||||||
|
|
||||||
|
## Open questions (to resolve before implementing)
|
||||||
|
|
||||||
|
1. **Fixture scrubbing: manual or scripted?** A scripted scrub that
|
||||||
|
also replaces project names / hostnames would lower the cost of
|
||||||
|
contributing a new fixture. Risk: over-aggressive replacement
|
||||||
|
destroys the signal the probe depends on. Propose: start manual,
|
||||||
|
add scripted helpers once we have 3 fixtures and know the common
|
||||||
|
PII shapes.
|
||||||
|
|
||||||
|
2. **Judge model selection.** Factory uses GPT-5.2. We can't pin one
|
||||||
|
— user's main model changes. Options: (a) grade with main model
|
||||||
|
(cheap, inconsistent across users), (b) require a specific judge
|
||||||
|
model (e.g. `claude-sonnet-4.6`), inconsistent for users without
|
||||||
|
access. Propose (a) with a `--judge-model` override, and make the
|
||||||
|
model name prominent in the report so comparisons across machines
|
||||||
|
are legible.
|
||||||
|
|
||||||
|
3. **Noise floor.** Before landing prompt changes, run the current
|
||||||
|
prompt N=10 times to measure per-dimension stddev. That tells us
|
||||||
|
the minimum delta to call a change significant. Suspect 0.2-0.3 on
|
||||||
|
a 0-5 scale. Decision deferred until after the first fixture is
|
||||||
|
landed.
|
||||||
|
|
||||||
|
4. **Iterative-merge coverage.** The real Factory-vs-Anthropic
|
||||||
|
difference is incremental merge vs. regenerate. A fixture that
|
||||||
|
only compresses once doesn't exercise our iterative path. Add a
|
||||||
|
fourth fixture that forces two compressions (manually chained),
|
||||||
|
with probes that test whether information from the first
|
||||||
|
compression survives the second. Deferred to a follow-up PR.
|
||||||
|
|
||||||
|
## Implementation order
|
||||||
|
|
||||||
|
This PR: design doc + scaffolding + **three checked-in fixtures** +
|
||||||
|
scrubber script. `run_eval.py` is still a placeholder that prints a
|
||||||
|
pointer to DESIGN.md.
|
||||||
|
|
||||||
|
Follow-ups, each a separate PR:
|
||||||
|
|
||||||
|
1. Probe banks for the three fixtures (~8-12 probes each), plus
|
||||||
|
`rubric.py` + `grader.py` + `compressor_driver.py`. Enough to
|
||||||
|
produce a full report.
|
||||||
|
2. Wire results output, `--compare-to` diff mode, and the report
|
||||||
|
markdown template.
|
||||||
|
3. Iterative-merge fixture (two chained compressions) + follow-ups
|
||||||
|
from the open questions.
|
||||||
|
|
||||||
|
Each follow-up is independently useful.
|
||||||
59
scripts/compression_eval/README.md
Normal file
59
scripts/compression_eval/README.md
Normal file
|
|
@ -0,0 +1,59 @@
|
||||||
|
# compression_eval
|
||||||
|
|
||||||
|
Offline eval harness for `agent/context_compressor.py`. Runs a real
|
||||||
|
conversation transcript through the compressor, then probes the
|
||||||
|
compressed state with targeted questions graded on six dimensions.
|
||||||
|
|
||||||
|
**Status:** design only. See `DESIGN.md` for the full proposal and
|
||||||
|
open questions. `run_eval.py` is a placeholder.
|
||||||
|
|
||||||
|
## When to run
|
||||||
|
|
||||||
|
Before merging changes to:
|
||||||
|
|
||||||
|
- `agent/context_compressor.py`
|
||||||
|
- `agent/auxiliary_client.py` routing for compression tasks
|
||||||
|
- `agent/prompt_builder.py` compression-note phrasing
|
||||||
|
|
||||||
|
## Not for CI
|
||||||
|
|
||||||
|
This harness makes real model calls, costs ~$1 per run on a mainstream
|
||||||
|
model, takes minutes, and is LLM-graded (non-deterministic). It lives
|
||||||
|
in `scripts/` and is invoked by hand. `tests/` and
|
||||||
|
`scripts/run_tests.sh` do not touch it.
|
||||||
|
|
||||||
|
## Usage (once implemented)
|
||||||
|
|
||||||
|
```
|
||||||
|
python scripts/compression_eval/run_eval.py
|
||||||
|
python scripts/compression_eval/run_eval.py --fixtures=401-debug
|
||||||
|
python scripts/compression_eval/run_eval.py --runs=5 --label=my-prompt-v2
|
||||||
|
python scripts/compression_eval/run_eval.py --compare-to=results/2026-04-24_baseline
|
||||||
|
```
|
||||||
|
|
||||||
|
Results land in `results/<label>/report.md` and are intended to be
|
||||||
|
pasted verbatim into PR descriptions.
|
||||||
|
|
||||||
|
## Fixtures
|
||||||
|
|
||||||
|
Three scrubbed session snapshots live under `fixtures/`:
|
||||||
|
|
||||||
|
- `feature-impl-context-priority.json` — 75 msgs, investigate →
|
||||||
|
patch → test → PR → merge
|
||||||
|
- `debug-session-feishu-id-model.json` — 59 msgs, PR triage +
|
||||||
|
upstream docs + decision
|
||||||
|
- `config-build-competitive-scouts.json` — 61 msgs, iterative
|
||||||
|
config accumulation (11 cron jobs)
|
||||||
|
|
||||||
|
Regenerate them from the maintainer's `~/.hermes/sessions/*.jsonl`
|
||||||
|
with `python3 scripts/compression_eval/scrub_fixtures.py`. The
|
||||||
|
scrubber pipeline and PII-audit checklist are documented in
|
||||||
|
`DESIGN.md` under **Scrubber pipeline**.
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- `agent/context_compressor.py` — the thing under test
|
||||||
|
- `tests/agent/test_context_compressor.py` — structural unit tests
|
||||||
|
that do run in CI
|
||||||
|
- `scripts/sample_and_compress.py` — the closest existing script in
|
||||||
|
shape (offline, credential-requiring, not in CI)
|
||||||
|
|
@ -0,0 +1,536 @@
|
||||||
|
{
|
||||||
|
"name": "config-build-competitive-scouts",
|
||||||
|
"description": "~60-turn iterative config/build session: user wants a set of weekly cron jobs that scan competing AI coding agents (openclaw, nanoclaw, ironclaw, codex, opencode, claude-code, kilo-code, gemini-cli, cline, aider, roo) for merged PRs or web updates worth porting to hermes-agent. User adds one target per turn; agent creates each cron job and re-states the accumulated schedule. Exercises artifact trail (which jobs are configured, which days) and iterative state accumulation — the canonical case for iterative-merge summarization.",
|
||||||
|
"model": "anthropic/claude-sonnet-4.6",
|
||||||
|
"context_length": 200000,
|
||||||
|
"source": "~/.hermes/sessions/20260328_160817_77bd258b.jsonl",
|
||||||
|
"truncated_to": null,
|
||||||
|
"scrubbed_at": "2026-04-24T07:40:21Z",
|
||||||
|
"scrubbing_passes": [
|
||||||
|
"redact_sensitive_text (agent.redact)",
|
||||||
|
"username paths replaced with /home/user",
|
||||||
|
"personal handles (all case variants of the maintainer name) replaced with 'user'",
|
||||||
|
"email addresses replaced with contributor@example.com",
|
||||||
|
"git 'Author: Name <addr>' header lines normalised",
|
||||||
|
"reasoning scratchpad blocks stripped from assistant content",
|
||||||
|
"think tag blocks stripped from assistant content",
|
||||||
|
"messaging-platform user mentions replaced with <@user>",
|
||||||
|
"first user message paraphrased to remove personal voice",
|
||||||
|
"subsequent user messages kept verbatim (after above redactions)",
|
||||||
|
"system prompt replaced with generic public-safe placeholder",
|
||||||
|
"orphan empty-assistant messages and trailing tool messages dropped",
|
||||||
|
"tool outputs longer than 2000 chars truncated with a note"
|
||||||
|
],
|
||||||
|
"messages": [
|
||||||
|
{
|
||||||
|
"role": "system",
|
||||||
|
"content": "You are a helpful AI coding assistant with access to tools (terminal, file editing, search, web, etc.). You operate in a conversational loop: the user gives you a task, you call tools to accomplish it, and you report back concisely."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "Set up a cron job for the agent every Sunday to scan all PRs merged into openclaw that week, decide which are worth adding to hermes-agent, and open PRs porting those features."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "\n\nBefore I set this up, I want to make sure I have the right repo. Let me find \"openclaw\" on GitHub.",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01XB5ckQ41b53VF6PpqgeoWD",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"openclaw github repository\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01TxqehehXyL5Y8SJWh8f4JC",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"openclaw AI agent github\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://github.com/openclaw\",\n \"title\": \"OpenClaw - GitHub\",\n \"description\": \"Your personal, open source AI assistant. openclaw has 23 repositories available. Follow their code on GitHub.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/openclaw/openclaw\",\n \"title\": \"OpenClaw — Personal AI Assistant - GitHub\",\n \"description\": \"Your own personal AI assistant. Any OS. Any Platform. The lobster way. - GitHub - openclaw/openclaw: Your own personal AI assistant.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/orgs/openclaw/repositories\",\n \"title\": \"openclaw repositories - GitHub\",\n \"description\": \"Your personal, open source AI assistant. openclaw has 23 repositories available. Follow their code on GitHub.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/Gen-Verse/OpenClaw-RL\",\n \"title\": \"OpenClaw-RL: Train any agent simply by talking - GitHub\",\n \"description\": \"OpenClaw-RL is a fully asynchronous reinforcement learning framework that turns everyday conversations into training signals for personalized AI ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://www.getopenclaw.ai/en/blog/openclaw-github\",\n \"title\": \"OpenClaw GitHub: Source Code, Repositories & How to Contribute\",\n \"description\": \"Connect OpenClaw to GitHub for AI-powered development workflows. Review PRs, manage issues, and automate code tasks with natural language. Guide ...\",\n \"category\": null\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01XB5ckQ41b53VF6PpqgeoWD"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://github.com/openclaw/openclaw\",\n \"title\": \"OpenClaw — Personal AI Assistant - GitHub\",\n \"description\": \"Agent to Agent (sessions_* tools). Use these to coordinate work across sessions without jumping between chat surfaces. sessions_list — discover ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/nearai/ironclaw\",\n \"title\": \"IronClaw is OpenClaw inspired implementation in Rust ... - GitHub\",\n \"description\": \"IronClaw is built on a simple principle: your AI assistant should work for you, not against you. In a world where AI systems are ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/SamurAIGPT/Best-AI-Agents/blob/main/README.md\",\n \"title\": \"awesome-openclaw/README.md at main - GitHub\",\n \"description\": \"OpenClaw (formerly Moltbot, originally Clawdbot) is a free and open-source autonomous AI agent created by Peter Steinberger. It runs locally, connects to ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/snarktank/antfarm\",\n \"title\": \"GitHub - snarktank/antfarm: Build your agent team in OpenClaw with ...\",\n \"description\": \"Build your agent team in OpenClaw with one command. You don't need to hire a dev team. You need to define one. Antfarm gives you a team of specialized AI agents ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://www.youtube.com/watch?v=yBbLs3VAayc\",\n \"title\": \"OpenClaw + GitHub: Building a Real AI Teammate - YouTube\",\n \"description\": \"Learn how to turn OpenClaw into your very own autonomous AI developer agent! In this tutorial, we build directly on our previous Google ...\",\n \"category\": null\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01TxqehehXyL5Y8SJWh8f4JC"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Found it - github.com/openclaw/openclaw, an open-source AI assistant. Let me look at the repo structure and recent PRs to craft a solid prompt for the cron job.",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01MPkkocnPW2AXSzd13FoC4p",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_extract",
|
||||||
|
"arguments": "{\"urls\": [\"https://github.com/openclaw/openclaw\"]}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"results\": [\n {\n \"url\": \"https://github.com/openclaw/openclaw\",\n \"title\": \"GitHub - openclaw/openclaw: Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞 · GitHub\",\n \"content\": \"# 🦞 OpenClaw — Personal AI Assistant\\n\\n**OpenClaw** is a personal AI assistant designed to run on your own devices, acting as a local-first control plane (Gateway) that connects to various messaging channels and hardware nodes. It is built for **Molty**, a \\\"space lobster\\\" AI assistant.\\n\\n- **Website:** [openclaw.ai](https://openclaw.ai/)\\n- **License:** MIT\\n- **GitHub Stats:** 339k Stars | 66.8k Forks | 1,364 Contributors\\n\\n---\\n\\n## 🚀 Quick Start & Installation\\n\\nOpenClaw requires **Node 24 (recommended)** or Node 22.16+.\\n\\n### Recommended CLI Setup\\nThe `onboard` command guides you through setting up the gateway, workspace, channels, and skills.\\n```bash\\nnpm install -g openclaw@latest\\n# or: pnpm add -g openclaw@latest\\n\\nopenclaw onboard --install-daemon\\n```\\n\\n### Basic Commands\\n```bash\\n# Start the gateway\\nopenclaw gateway --port 18789 --verbose\\n\\n# Send a direct message via CLI\\nopenclaw message send --to +1234567890 --message \\\"Hello from OpenClaw\\\"\\n\\n# Interact with the assistant\\nopenclaw agent --message \\\"Ship checklist\\\" --thinking high\\n```\\n\\n---\\n\\n## 🛠 Key Features & Subsystems\\n\\n### 1. Multi-Channel Inbox\\nOpenClaw supports an extensive list of messaging platforms:\\n* **Mainstream:** WhatsApp, Telegram, Slack, Discord, Google Chat, Microsoft Teams.\\n* **Privacy/Niche:** Signal, Matrix, iMessage (via BlueBubbles), Signal, IRC, Nostr.\\n* **Regional/Specialized:** WeChat (via Tencent iLink plugin), Feishu, LINE, Zalo, Mattermost, Nextcloud Talk, Synology Chat, Tlon, Twitch.\\n\\n### 2. Hardware Nodes & Companion Apps\\n* \n\n[... tool output truncated for fixture — original was 4283 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01MPkkocnPW2AXSzd13FoC4p"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Good, I have a clear picture of OpenClaw. It's a Node.js AI assistant with a lot of similar feature areas (multi-platform messaging, browser control, agent routing, etc.). Let me set up the cron job now.",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01NVc5kbrRYuXwoYsydWkbQ4",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "cronjob",
|
||||||
|
"arguments": "{\"action\": \"create\", \"name\": \"openclaw-pr-scout\", \"schedule\": \"0 17 * * 0\", \"deliver\": \"origin\", \"skill\": \"hermes-agent-dev\", \"prompt\": \"Weekly OpenClaw PR Scout — scan merged PRs from openclaw/openclaw and port worthy features to hermes-agent.\\n\\nSTEP 1: GATHER MERGED PRs FROM THE PAST WEEK\\n- Use `gh pr list --repo openclaw/openclaw --state merged --search \\\"merged:>=$(date -d '7 days ago' +%Y-%m-%d)\\\" --limit 50 --json number,title,body,mergedAt,url,files,additions,deletions` to get all PRs merged in the last 7 days.\\n- If the date command fails, calculate the date manually (today minus 7 days) and use that.\\n\\nSTEP 2: ANALYZE EACH PR\\nFor each merged PR:\\n- Read the PR title, description, and changed files list.\\n- For promising PRs, read the full diff: `gh pr diff <number> --repo openclaw/openclaw`\\n- Categorize relevance to hermes-agent. Focus on:\\n * New tool implementations or tool improvements\\n * Gateway/messaging platform features (Telegram, Discord, Slack, WhatsApp, Signal, etc.)\\n * Agent loop improvements (context management, prompt caching, compression)\\n * CLI UX improvements\\n * Security/safety features\\n * Configuration system improvements\\n * Cron/scheduling features\\n * Browser automation improvements\\n * Multi-agent coordination\\n * Bug fixes that might apply to similar patterns in hermes-agent\\n- Skip PRs that are: purely cosmetic/docs, Node.js/TypeScript-specific infra, dependency bumps, or clearly irrelevant to a Python codebase.\\n\\nSTEP 3: SCORE AND SELECT\\nRate each relevant PR on:\\n- Impact: How useful would this be for hermes-agent users?\\n- Feasibility: How hard to implement in Python/hermes-agent's architecture?\\n- Novelty: Does hermes-agent already have this feature?\\nOnly proceed with PRs scoring high on all three.\\n\\nSTEP 4: IMPLEMENT WORTHY PRs\\nFor each selected PR:\\n- The hermes-agent repo is at ~/.hermes/hermes-agent. Work in the main repo directory.\\n- Create a feature branch: `git checkout -b openclaw-port/<short-description> origin/main`\\n- Implement the feature adapted to hermes-agent's Python architecture. Follow AGENTS.md conventions (tool registration pattern, config system, slash commands, etc.).\\n- The hermes-agent-dev skill is loaded — follow its workflow for making changes, running tests, and creating PRs.\\n- Run the relevant test subset to verify the implementation.\\n- Commit with a clear message referencing the source: \\\"Port from openclaw/openclaw#<number>: <description>\\\"\\n- Push and open a PR with `gh pr create` — in the PR body:\\n * Link to the source OpenClaw PR\\n * Explain what was ported and how it was adapted\\n * Note any architectural differences in the implementation\\n * Mark as draft if tests need more work\\n\\nSTEP 5: SUMMARY REPORT\\nAfter all analysis is complete, provide a summary report:\\n- Total PRs merged in openclaw/openclaw this week\\n- How many were evaluated as potentially relevant\\n- How many PRs were opened in hermes-agent (with links)\\n- Brief notes on any interesting trends or features that weren't ported but are worth watching\\n\\nIf no PRs are worth porting this week, just deliver the summary report explaining what was reviewed and why nothing qualified.\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"job_id\": \"e6bfff604af4\",\n \"name\": \"openclaw-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"schedule\": \"0 17 * * 0\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-29T17:00:00-07:00\",\n \"job\": {\n \"job_id\": \"e6bfff604af4\",\n \"name\": \"openclaw-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"prompt_preview\": \"Weekly OpenClaw PR Scout \\u2014 scan merged PRs from openclaw/openclaw and port worthy features to hermes...\",\n \"model\": null,\n \"provider\": null,\n \"base_url\": null,\n \"schedule\": \"0 17 * * 0\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-29T17:00:00-07:00\",\n \"last_run_at\": null,\n \"last_status\": null,\n \"enabled\": true,\n \"state\": \"scheduled\",\n \"paused_at\": null,\n \"paused_reason\": null\n },\n \"message\": \"Cron job 'openclaw-pr-scout' created.\"\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01NVc5kbrRYuXwoYsydWkbQ4"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "All set. Here's what's running:\n\nopenclaw-pr-scout (e6bfff604af4)\n→ Every Sunday at 10 AM Pacific (5 PM UTC)\n→ Delivers results back to this Telegram chat\n→ hermes-agent-dev skill loaded for proper repo workflow\n\nWhat it does each week:\n① Pulls all merged PRs from openclaw/openclaw for the past 7 days via gh CLI\n② Reads diffs and evaluates each for relevance to hermes-agent (new tools, gateway features, agent loop improvements, CLI UX, security, etc.)\n③ Scores on impact, feasibility, and novelty — skips docs/deps/TS-specific stuff\n④ For worthy PRs, creates feature branches, implements the Python adaptation, runs tests, and opens draft PRs in hermes-agent referencing the source\n⑤ Sends you a summary report either way — what was reviewed, what was ported, what's worth watching\n\nFirst run is tomorrow (Sunday March 29). Want me to do a dry run now to see what this week's PRs look like?"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "Nah its fine. Can you make another one that scans nanoclaw and another that scans ironclaw etc"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "\n\n",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01WNuuzg2HZkpBGFdZLtJpSU",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"nanoclaw github AI assistant\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01CNr4vkdFCdEQj4KmMPqzac",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"ironclaw github nearai AI assistant\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://github.com/qwibitai/nanoclaw\",\n \"title\": \"GitHub - qwibitai/nanoclaw\",\n \"description\": \"An AI assistant that runs agents securely in their own containers. Lightweight, built to be easily understood and completely customized for your ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/hustcc/nano-claw\",\n \"title\": \"hustcc/nano-claw - GitHub\",\n \"description\": \"nano-claw is a TypeScript + Node.js implementation of nanobot, an ultra-lightweight personal AI assistant inspired by OpenClaw.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://www.reddit.com/r/selfhosted/comments/1r3p7s7/i_built_nanoclaw_a_lightweight_selfhosted_ai/\",\n \"title\": \"I built NanoClaw - a lightweight self-hosted AI assistant with ... - Reddit\",\n \"description\": \"I built NanoClaw - a lightweight self-hosted AI assistant with container isolation, WhatsApp integration, scheduled jobs, memory · Core is ~500 ...\",\n \"category\": null\n },\n {\n \"url\": \"https://www.youtube.com/watch?v=AQ5uiLyr8bQ\",\n \"title\": \"The Lightweight, Secure AI Assistant OpenClaw Should Have Been\",\n \"description\": \"... github.com/gavrielc/nanoclaw Cisco article - https ... NanoClaw: The Lightweight, Secure AI Assistant OpenClaw Should Have Been.\",\n \"category\": null\n },\n {\n \"url\": \"https://virtuslab.com/blog/ai/nano-claw-your-personal-ai-butler/\",\n \"title\": \"GitHub All-Stars #14: NanoClaw - VirtusLab\",\n \"description\": \"We're looking at NanoClaw: a lightweight, container-isolated personal AI assistant that connects to popular messaging apps and runs on ...\",\n \"category\": null\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01WNuuzg2HZkpBGFdZLtJpSU"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://github.com/nearai/ironclaw\",\n \"title\": \"IronClaw is OpenClaw inspired implementation in Rust ... - GitHub\",\n \"description\": \"IronClaw defaults to NEAR AI but supports many LLM providers out of the box. Built-in providers include Anthropic, OpenAI, GitHub Copilot, ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/nearai/ironclaw/releases\",\n \"title\": \"Releases · nearai/ironclaw - GitHub\",\n \"description\": \"IronClaw is OpenClaw inspired implementation in Rust focused on privacy and security - Releases · nearai/ironclaw.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/nearai/ironclaw/issues\",\n \"title\": \"Issues · nearai/ironclaw - GitHub\",\n \"description\": \"IronClaw is OpenClaw inspired implementation in Rust focused on privacy and security - nearai/ironclaw.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/nearai/ironclaw/actions\",\n \"title\": \"nearai/ironclaw · GitHub - Workflow runs\",\n \"description\": \"IronClaw is OpenClaw inspired implementation in Rust focused on privacy and security - Workflow runs · nearai/ironclaw.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://www.mintlify.com/logicminds/ironclaw/index\",\n \"title\": \"Your secure personal AI assistant - IronClaw - Mintlify\",\n \"description\": \"With defense-in-depth security, local-first data storage, and self-expanding capabilities, IronClaw is the AI assistant you can actually trust.\",\n \"category\": null\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01CNr4vkdFCdEQj4KmMPqzac"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Got them. nanoclaw (qwibitai/nanoclaw) is a lightweight container-isolated assistant, ironclaw (nearai/ironclaw) is a Rust rewrite focused on privacy/security. Creating both now.",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01N5UJKWPPgQn1zEbH7nKmdr",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "cronjob",
|
||||||
|
"arguments": "{\"action\": \"create\", \"name\": \"nanoclaw-pr-scout\", \"schedule\": \"0 17 * * 0\", \"deliver\": \"origin\", \"skill\": \"hermes-agent-dev\", \"prompt\": \"Weekly NanoClaw PR Scout — scan merged PRs from qwibitai/nanoclaw and port worthy features to hermes-agent.\\n\\nSTEP 1: GATHER MERGED PRs FROM THE PAST WEEK\\n- Use `gh pr list --repo qwibitai/nanoclaw --state merged --search \\\"merged:>=$(date -d '7 days ago' +%Y-%m-%d)\\\" --limit 50 --json number,title,body,mergedAt,url,files,additions,deletions` to get all PRs merged in the last 7 days.\\n- If the date command fails, calculate the date manually (today minus 7 days) and use that.\\n\\nSTEP 2: ANALYZE EACH PR\\nNanoClaw is a lightweight, container-isolated AI assistant (TypeScript/Node.js). It has WhatsApp integration, scheduled jobs, memory, and focuses on security via container sandboxing.\\n\\nFor each merged PR:\\n- Read the PR title, description, and changed files list.\\n- For promising PRs, read the full diff: `gh pr diff <number> --repo qwibitai/nanoclaw`\\n- Categorize relevance to hermes-agent. Focus on:\\n * Container isolation / sandboxing patterns (hermes-agent has Docker/Singularity environments)\\n * New tool implementations or tool improvements\\n * Gateway/messaging platform features (Telegram, Discord, Slack, WhatsApp, Signal, etc.)\\n * Agent loop improvements (context management, prompt caching, compression)\\n * CLI UX improvements\\n * Security/safety features — NanoClaw emphasizes this\\n * Scheduling/cron features\\n * Memory system improvements\\n * Bug fixes that might apply to similar patterns in hermes-agent\\n- Skip PRs that are: purely cosmetic/docs, TypeScript/Node.js-specific infra, dependency bumps, or clearly irrelevant to a Python codebase.\\n\\nSTEP 3: SCORE AND SELECT\\nRate each relevant PR on:\\n- Impact: How useful would this be for hermes-agent users?\\n- Feasibility: How hard to implement in Python/hermes-agent's architecture?\\n- Novelty: Does hermes-agent already have this feature?\\nOnly proceed with PRs scoring high on all three.\\n\\nSTEP 4: IMPLEMENT WORTHY PRs\\nFor each selected PR:\\n- The hermes-agent repo is at ~/.hermes/hermes-agent. Work in the main repo directory.\\n- Create a feature branch: `git checkout -b nanoclaw-port/<short-description> origin/main`\\n- Implement the feature adapted to hermes-agent's Python architecture. Follow AGENTS.md conventions (tool registration pattern, config system, slash commands, etc.).\\n- The hermes-agent-dev skill is loaded — follow its workflow for making changes, running tests, and creating PRs.\\n- Run the relevant test subset to verify the implementation.\\n- Commit with a clear message referencing the source: \\\"Port from qwibitai/nanoclaw#<number>: <description>\\\"\\n- Push and open a PR with `gh pr create` — in the PR body:\\n * Link to the source NanoClaw PR\\n * Explain what was ported and how it was adapted\\n * Note any architectural differences in the implementation\\n * Mark as draft if tests need more work\\n\\nSTEP 5: SUMMARY REPORT\\nAfter all analysis is complete, provide a summary report:\\n- Total PRs merged in qwibitai/nanoclaw this week\\n- How many were evaluated as potentially relevant\\n- How many PRs were opened in hermes-agent (with links)\\n- Brief notes on any interesting trends or features that weren't ported but are worth watching\\n\\nIf no PRs are worth porting this week, just deliver the summary report explaining what was reviewed and why nothing qualified.\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01QW3UQvamZyJwMk4kF1PXXd",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "cronjob",
|
||||||
|
"arguments": "{\"action\": \"create\", \"name\": \"ironclaw-pr-scout\", \"schedule\": \"0 17 * * 0\", \"deliver\": \"origin\", \"skill\": \"hermes-agent-dev\", \"prompt\": \"Weekly IronClaw PR Scout — scan merged PRs from nearai/ironclaw and port worthy features to hermes-agent.\\n\\nSTEP 1: GATHER MERGED PRs FROM THE PAST WEEK\\n- Use `gh pr list --repo nearai/ironclaw --state merged --search \\\"merged:>=$(date -d '7 days ago' +%Y-%m-%d)\\\" --limit 50 --json number,title,body,mergedAt,url,files,additions,deletions` to get all PRs merged in the last 7 days.\\n- If the date command fails, calculate the date manually (today minus 7 days) and use that.\\n\\nSTEP 2: ANALYZE EACH PR\\nIronClaw is a Rust implementation of an OpenClaw-inspired AI assistant, focused on privacy and security. It supports NEAR AI, Anthropic, OpenAI, GitHub Copilot and other providers. Being Rust-native, it may have interesting approaches to performance, concurrency, and memory safety.\\n\\nFor each merged PR:\\n- Read the PR title, description, and changed files list.\\n- For promising PRs, read the full diff: `gh pr diff <number> --repo nearai/ironclaw`\\n- Categorize relevance to hermes-agent. Focus on:\\n * Privacy/security patterns — IronClaw's core differentiator\\n * Provider integration improvements (multi-provider routing, fallback logic)\\n * New tool implementations or tool improvements\\n * Gateway/messaging platform features\\n * Agent loop improvements (context management, streaming, concurrency patterns)\\n * CLI UX improvements\\n * Sandboxing / isolation approaches\\n * Memory/session management\\n * Performance optimizations that could translate to Python (algorithmic improvements, caching strategies)\\n * Bug fixes that might apply to similar patterns in hermes-agent\\n- Skip PRs that are: purely cosmetic/docs, Rust-specific infra (cargo, build system), dependency bumps, or low-level Rust optimizations with no conceptual equivalent in Python.\\n\\nSTEP 3: SCORE AND SELECT\\nRate each relevant PR on:\\n- Impact: How useful would this be for hermes-agent users?\\n- Feasibility: How hard to implement in Python/hermes-agent's architecture?\\n- Novelty: Does hermes-agent already have this feature?\\nOnly proceed with PRs scoring high on all three.\\n\\nSTEP 4: IMPLEMENT WORTHY PRs\\nFor each selected PR:\\n- The hermes-agent repo is at ~/.hermes/hermes-agent. Work in the main repo directory.\\n- Create a feature branch: `git checkout -b ironclaw-port/<short-description> origin/main`\\n- Implement the feature adapted to hermes-agent's Python architecture. Follow AGENTS.md conventions (tool registration pattern, config system, slash commands, etc.).\\n- The hermes-agent-dev skill is loaded — follow its workflow for making changes, running tests, and creating PRs.\\n- Run the relevant test subset to verify the implementation.\\n- Commit with a clear message referencing the source: \\\"Port from nearai/ironclaw#<number>: <description>\\\"\\n- Push and open a PR with `gh pr create` — in the PR body:\\n * Link to the source IronClaw PR\\n * Explain what was ported and how it was adapted\\n * Note any architectural differences in the implementation\\n * Mark as draft if tests need more work\\n\\nSTEP 5: SUMMARY REPORT\\nAfter all analysis is complete, provide a summary report:\\n- Total PRs merged in nearai/ironclaw this week\\n- How many were evaluated as potentially relevant\\n- How many PRs were opened in hermes-agent (with links)\\n- Brief notes on any interesting trends or features that weren't ported but are worth watching\\n\\nIf no PRs are worth porting this week, just deliver the summary report explaining what was reviewed and why nothing qualified.\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"job_id\": \"34e4741dc587\",\n \"name\": \"nanoclaw-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"schedule\": \"0 17 * * 0\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-29T17:00:00-07:00\",\n \"job\": {\n \"job_id\": \"34e4741dc587\",\n \"name\": \"nanoclaw-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"prompt_preview\": \"Weekly NanoClaw PR Scout \\u2014 scan merged PRs from qwibitai/nanoclaw and port worthy features to hermes...\",\n \"model\": null,\n \"provider\": null,\n \"base_url\": null,\n \"schedule\": \"0 17 * * 0\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-29T17:00:00-07:00\",\n \"last_run_at\": null,\n \"last_status\": null,\n \"enabled\": true,\n \"state\": \"scheduled\",\n \"paused_at\": null,\n \"paused_reason\": null\n },\n \"message\": \"Cron job 'nanoclaw-pr-scout' created.\"\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01N5UJKWPPgQn1zEbH7nKmdr"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"job_id\": \"379479d3c79a\",\n \"name\": \"ironclaw-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"schedule\": \"0 17 * * 0\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-29T17:00:00-07:00\",\n \"job\": {\n \"job_id\": \"379479d3c79a\",\n \"name\": \"ironclaw-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"prompt_preview\": \"Weekly IronClaw PR Scout \\u2014 scan merged PRs from nearai/ironclaw and port worthy features to hermes-a...\",\n \"model\": null,\n \"provider\": null,\n \"base_url\": null,\n \"schedule\": \"0 17 * * 0\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-29T17:00:00-07:00\",\n \"last_run_at\": null,\n \"last_status\": null,\n \"enabled\": true,\n \"state\": \"scheduled\",\n \"paused_at\": null,\n \"paused_reason\": null\n },\n \"message\": \"Cron job 'ironclaw-pr-scout' created.\"\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01QW3UQvamZyJwMk4kF1PXXd"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Done. All three scouts are live, all running Sundays at 10 AM Pacific:\n\n① openclaw-pr-scout — openclaw/openclaw (Node.js, 339k stars)\n② nanoclaw-pr-scout — qwibitai/nanoclaw (lightweight, container-isolated)\n③ ironclaw-pr-scout — nearai/ironclaw (Rust, privacy/security focused)\n\nEach one has hermes-agent-dev loaded, reports back here, and knows to adapt features to Python/hermes-agent conventions before opening PRs. First runs all fire tomorrow.\n\nYou mentioned \"etc\" — are there other OpenClaw-adjacent projects you want covered? I can spin up more of these in seconds."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "Then on tuesdays a job for doing this with openai codex repo"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "\n\n",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01T8ugTLsuB9qY4rR1fEJ9Zt",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"openai codex github repository CLI agent 2025 2026\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://github.com/openai/codex\",\n \"title\": \"openai/codex: Lightweight coding agent that runs in your terminal\",\n \"description\": \"npm i -g @openai/codex or brew install --cask codex. Codex CLI is a coding agent from OpenAI that runs locally on your computer. Codex CLI splash.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://community.openai.com/t/how-to-set-up-git-and-github-cli-gh-for-codex-web-cloud/1371830\",\n \"title\": \"How to set up git and GitHub CLI (gh) for Codex Web (Cloud)\",\n \"description\": \"You need to add a GitHub token as a secret in your environment and change YOUR/REPO to your org/repo. Then Codex should be able to use both git ...\",\n \"category\": null\n },\n {\n \"url\": \"https://www.ai.cc/blogs/how-to-use-openai-codex-ai-coding-guide/\",\n \"title\": \"How to Use OpenAI Codex: Complete AI Coding Agent Guide (2026)\",\n \"description\": \"Introduced in a research preview in May 2025, Codex allows users to delegate coding tasks such as writing features, fixing bugs, answering ...\",\n \"category\": null\n },\n {\n \"url\": \"https://openai.com/index/introducing-codex/\",\n \"title\": \"Introducing Codex - OpenAI\",\n \"description\": \"Introducing Codex: a cloud-based software engineering agent that can work on many tasks in parallel, powered by codex-1.\",\n \"category\": null\n },\n {\n \"url\": \"https://github.com/openai/codex/discussions/6109\",\n \"title\": \"How can I create custom agents (e.g., Review, Test, Refactor) using ...\",\n \"description\": \"I have been experimenting with the OpenAI Codex CLI, and I would like to build a setup similar to Claude Code, where I can define multiple specialized agents — ...\",\n \"category\": \"github\"\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01T8ugTLsuB9qY4rR1fEJ9Zt"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_018ymLeBDrf9PGC1wf2Xt2Hv",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "cronjob",
|
||||||
|
"arguments": "{\"action\": \"create\", \"name\": \"codex-pr-scout\", \"schedule\": \"0 17 * * 2\", \"deliver\": \"origin\", \"skill\": \"hermes-agent-dev\", \"prompt\": \"Weekly OpenAI Codex PR Scout — scan merged PRs from openai/codex and port worthy features to hermes-agent.\\n\\nSTEP 1: GATHER MERGED PRs FROM THE PAST WEEK\\n- Use `gh pr list --repo openai/codex --state merged --search \\\"merged:>=$(date -d '7 days ago' +%Y-%m-%d)\\\" --limit 50 --json number,title,body,mergedAt,url,files,additions,deletions` to get all PRs merged in the last 7 days.\\n- If the date command fails, calculate the date manually (today minus 7 days) and use that.\\n\\nSTEP 2: ANALYZE EACH PR\\nOpenAI Codex is a lightweight coding agent CLI (TypeScript/Node.js) from OpenAI. It runs locally in the terminal, focuses on code generation/editing, has sandboxing, and supports multiple approval policies. Key areas of overlap with hermes-agent: terminal tool execution, file editing, sandboxing, agent loop design, CLI UX.\\n\\nFor each merged PR:\\n- Read the PR title, description, and changed files list.\\n- For promising PRs, read the full diff: `gh pr diff <number> --repo openai/codex`\\n- Categorize relevance to hermes-agent. Focus on:\\n * Terminal execution patterns (sandboxing, approval policies, command safety)\\n * File editing / patching approaches\\n * Agent loop improvements (context management, tool calling patterns, iteration control)\\n * CLI UX innovations (spinners, output formatting, interactive modes)\\n * Model routing / provider management\\n * Code generation and editing strategies\\n * Security/safety features (dangerous command detection, sandboxing)\\n * Configuration system patterns\\n * Multi-file editing workflows\\n * Streaming / response handling\\n * Bug fixes that might apply to similar patterns in hermes-agent\\n- Skip PRs that are: purely cosmetic/docs, TypeScript/Node.js-specific infra, dependency bumps, npm/build tooling, or clearly irrelevant to a Python codebase.\\n\\nSTEP 3: SCORE AND SELECT\\nRate each relevant PR on:\\n- Impact: How useful would this be for hermes-agent users?\\n- Feasibility: How hard to implement in Python/hermes-agent's architecture?\\n- Novelty: Does hermes-agent already have this feature?\\nOnly proceed with PRs scoring high on all three.\\n\\nSTEP 4: IMPLEMENT WORTHY PRs\\nFor each selected PR:\\n- The hermes-agent repo is at ~/.hermes/hermes-agent. Work in the main repo directory.\\n- Create a feature branch: `git checkout -b codex-port/<short-description> origin/main`\\n- Implement the feature adapted to hermes-agent's Python architecture. Follow AGENTS.md conventions (tool registration pattern, config system, slash commands, etc.).\\n- The hermes-agent-dev skill is loaded — follow its workflow for making changes, running tests, and creating PRs.\\n- Run the relevant test subset to verify the implementation.\\n- Commit with a clear message referencing the source: \\\"Port from openai/codex#<number>: <description>\\\"\\n- Push and open a PR with `gh pr create` — in the PR body:\\n * Link to the source Codex PR\\n * Explain what was ported and how it was adapted\\n * Note any architectural differences in the implementation\\n * Mark as draft if tests need more work\\n\\nSTEP 5: SUMMARY REPORT\\nAfter all analysis is complete, provide a summary report:\\n- Total PRs merged in openai/codex this week\\n- How many were evaluated as potentially relevant\\n- How many PRs were opened in hermes-agent (with links)\\n- Brief notes on any interesting trends or features that weren't ported but are worth watching\\n\\nIf no PRs are worth porting this week, just deliver the summary report explaining what was reviewed and why nothing qualified.\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"job_id\": \"4f0c15608223\",\n \"name\": \"codex-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"schedule\": \"0 17 * * 2\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-31T17:00:00-07:00\",\n \"job\": {\n \"job_id\": \"4f0c15608223\",\n \"name\": \"codex-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"prompt_preview\": \"Weekly OpenAI Codex PR Scout \\u2014 scan merged PRs from openai/codex and port worthy features to hermes-...\",\n \"model\": null,\n \"provider\": null,\n \"base_url\": null,\n \"schedule\": \"0 17 * * 2\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-31T17:00:00-07:00\",\n \"last_run_at\": null,\n \"last_status\": null,\n \"enabled\": true,\n \"state\": \"scheduled\",\n \"paused_at\": null,\n \"paused_reason\": null\n },\n \"message\": \"Cron job 'codex-pr-scout' created.\"\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_018ymLeBDrf9PGC1wf2Xt2Hv"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Done. codex-pr-scout is live — Tuesdays at 10 AM Pacific, scanning openai/codex for terminal execution patterns, sandboxing, file editing workflows, CLI UX, and anything else worth pulling into hermes-agent.\n\nFull roster now:\n\n⚙ Sundays — openclaw, nanoclaw, ironclaw\n⚙ Tuesdays — openai codex\n\nAll reporting back here with summaries and draft PRs."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "Thursdays, opencode next"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "\n\n",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_018HbV71r6ucSedapKxynQSr",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"opencode github CLI coding agent terminal\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://opencode.ai/\",\n \"title\": \"OpenCode | The open source AI coding agent\",\n \"description\": \"What is OpenCode? OpenCode is an open source agent that helps you write code in your terminal, IDE, or desktop. [*]. LSP enabled Automatically loads the ...\",\n \"category\": null\n },\n {\n \"url\": \"https://opencode.ai/docs/\",\n \"title\": \"Intro | AI coding agent built for the terminal - OpenCode\",\n \"description\": \"OpenCode is an open source AI coding agent. It's available as a terminal-based interface, desktop app, or IDE extension.\",\n \"category\": null\n },\n {\n \"url\": \"https://opencode.ai/docs/cli/\",\n \"title\": \"CLI | OpenCode\",\n \"description\": \"OpenCode CLI options and commands ... Install the GitHub agent in your repository. Terminal window. opencode github install.\",\n \"category\": null\n },\n {\n \"url\": \"https://github.com/anomalyco/opencode\",\n \"title\": \"anomalyco/opencode: The open source coding agent. - GitHub\",\n \"description\": \"OpenCode includes two built-in agents you can switch between with the Tab key. ... Also included is a general subagent for complex searches and multistep tasks.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://johnmaeda.medium.com/get-started-with-agent-skills-in-github-copilot-cli-and-opencode-cli-in-2026-on-macos-part-1-of-2-6627dfc25530\",\n \"title\": \"Get Started with Agent Skills in GitHub Copilot CLI and OpenCode ...\",\n \"description\": \"So I'll describe two routes you can take that are the best options to take when using GitHub Copilot CLI and OpenCode CLI in early 2026. Note: ...\",\n \"category\": null\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_018HbV71r6ucSedapKxynQSr"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_0164xxb1SbFiC4xvGeNjtBxS",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "cronjob",
|
||||||
|
"arguments": "{\"action\": \"create\", \"name\": \"opencode-pr-scout\", \"schedule\": \"0 17 * * 4\", \"deliver\": \"origin\", \"skill\": \"hermes-agent-dev\", \"prompt\": \"Weekly OpenCode PR Scout — scan merged PRs from anomalyco/opencode and port worthy features to hermes-agent.\\n\\nSTEP 1: GATHER MERGED PRs FROM THE PAST WEEK\\n- Use `gh pr list --repo anomalyco/opencode --state merged --search \\\"merged:>=$(date -d '7 days ago' +%Y-%m-%d)\\\" --limit 50 --json number,title,body,mergedAt,url,files,additions,deletions` to get all PRs merged in the last 7 days.\\n- If the date command fails, calculate the date manually (today minus 7 days) and use that.\\n\\nSTEP 2: ANALYZE EACH PR\\nOpenCode is an open-source AI coding agent (Go-based) from Anomaly. It runs in the terminal, as a desktop app, or IDE extension. Key features: LSP integration, built-in dual agents (switchable with Tab), subagent for complex searches, GitHub agent integration. Areas of overlap with hermes-agent: terminal-based coding workflows, file editing, agent loop design, subagent delegation, CLI/TUI UX.\\n\\nFor each merged PR:\\n- Read the PR title, description, and changed files list.\\n- For promising PRs, read the full diff: `gh pr diff <number> --repo anomalyco/opencode`\\n- Categorize relevance to hermes-agent. Focus on:\\n * LSP integration patterns (code intelligence, diagnostics)\\n * Multi-agent / subagent delegation approaches\\n * Terminal UI innovations (TUI layout, keybindings, output formatting)\\n * File editing / patching strategies\\n * Agent loop improvements (context management, tool calling, iteration control)\\n * IDE extension patterns (if applicable to hermes-agent's ACP adapter)\\n * Provider routing / model switching\\n * Session management\\n * GitHub integration features\\n * Streaming / response handling\\n * Bug fixes that might apply to similar patterns in hermes-agent\\n- Skip PRs that are: purely cosmetic/docs, Go-specific infra (go.mod, build tooling), dependency bumps, or clearly irrelevant to a Python codebase.\\n\\nSTEP 3: SCORE AND SELECT\\nRate each relevant PR on:\\n- Impact: How useful would this be for hermes-agent users?\\n- Feasibility: How hard to implement in Python/hermes-agent's architecture?\\n- Novelty: Does hermes-agent already have this feature?\\nOnly proceed with PRs scoring high on all three.\\n\\nSTEP 4: IMPLEMENT WORTHY PRs\\nFor each selected PR:\\n- The hermes-agent repo is at ~/.hermes/hermes-agent. Work in the main repo directory.\\n- Create a feature branch: `git checkout -b opencode-port/<short-description> origin/main`\\n- Implement the feature adapted to hermes-agent's Python architecture. Follow AGENTS.md conventions (tool registration pattern, config system, slash commands, etc.).\\n- The hermes-agent-dev skill is loaded — follow its workflow for making changes, running tests, and creating PRs.\\n- Run the relevant test subset to verify the implementation.\\n- Commit with a clear message referencing the source: \\\"Port from anomalyco/opencode#<number>: <description>\\\"\\n- Push and open a PR with `gh pr create` — in the PR body:\\n * Link to the source OpenCode PR\\n * Explain what was ported and how it was adapted\\n * Note any architectural differences in the implementation\\n * Mark as draft if tests need more work\\n\\nSTEP 5: SUMMARY REPORT\\nAfter all analysis is complete, provide a summary report:\\n- Total PRs merged in anomalyco/opencode this week\\n- How many were evaluated as potentially relevant\\n- How many PRs were opened in hermes-agent (with links)\\n- Brief notes on any interesting trends or features that weren't ported but are worth watching\\n\\nIf no PRs are worth porting this week, just deliver the summary report explaining what was reviewed and why nothing qualified.\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"job_id\": \"704b50e44ddd\",\n \"name\": \"opencode-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"schedule\": \"0 17 * * 4\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-04-02T17:00:00-07:00\",\n \"job\": {\n \"job_id\": \"704b50e44ddd\",\n \"name\": \"opencode-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"prompt_preview\": \"Weekly OpenCode PR Scout \\u2014 scan merged PRs from anomalyco/opencode and port worthy features to herme...\",\n \"model\": null,\n \"provider\": null,\n \"base_url\": null,\n \"schedule\": \"0 17 * * 4\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-04-02T17:00:00-07:00\",\n \"last_run_at\": null,\n \"last_status\": null,\n \"enabled\": true,\n \"state\": \"scheduled\",\n \"paused_at\": null,\n \"paused_reason\": null\n },\n \"message\": \"Cron job 'opencode-pr-scout' created.\"\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_0164xxb1SbFiC4xvGeNjtBxS"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Live. anomalyco/opencode, Thursdays at 10 AM Pacific. That Go codebase has some interesting angles — LSP integration, dual-agent switching, and the TUI work could translate well.\n\nFull schedule:\n\n⚙ Sun — openclaw, nanoclaw, ironclaw\n⚙ Tue — openai codex\n⚙ Thu — opencode\n\nYou've got eyes on five competing agent projects now, all funneling into hermes-agent PRs."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "Then scan for claude code updates as well on fridays - they dont have an opensource repo, so we will need that one to scan the web for updates and implement prs without direct code references"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "\n\n",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01MJjZ1qu45pewCmpS516YcD",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "cronjob",
|
||||||
|
"arguments": "{\"action\": \"create\", \"name\": \"claude-code-scout\", \"schedule\": \"0 17 * * 5\", \"deliver\": \"origin\", \"skill\": \"hermes-agent-dev\", \"prompt\": \"Weekly Claude Code Feature Scout — scan the web for Claude Code updates and implement worthy features in hermes-agent.\\n\\nClaude Code is Anthropic's closed-source CLI coding agent. There is no public repo to scan PRs from, so this job relies on web research to discover new features, changes, and capabilities.\\n\\nSTEP 1: WEB RESEARCH\\nSearch broadly for Claude Code updates from the past week. Run multiple searches:\\n- \\\"Claude Code update\\\" (filter to past week)\\n- \\\"Claude Code new feature\\\"\\n- \\\"Claude Code changelog\\\"\\n- \\\"Anthropic Claude Code release\\\"\\n- \\\"Claude Code CLI\\\" site:anthropic.com\\n- \\\"Claude Code\\\" site:docs.anthropic.com\\n- \\\"Claude Code\\\" site:x.com (Twitter/X announcements)\\n- \\\"Claude Code\\\" site:reddit.com (community discoveries)\\n\\nAlso check these URLs directly for recent changes:\\n- https://docs.anthropic.com/en/docs/claude-code (official docs — look for new sections or changes)\\n- https://www.anthropic.com/blog (recent blog posts mentioning Claude Code)\\n- https://github.com/anthropics/claude-code/blob/main/CHANGELOG.md (they may publish a changelog even without open-sourcing)\\n\\nCollect all findings: new features, UX changes, capability additions, workflow improvements, integration patterns, CLI flags, configuration options, hook systems, etc.\\n\\nSTEP 2: ANALYZE AND CATEGORIZE\\nFor each discovered update/feature, evaluate relevance to hermes-agent:\\n- Terminal execution patterns and sandboxing\\n- File editing approaches (diff display, multi-file editing)\\n- Context management (how they handle large codebases, file inclusion)\\n- Permission/approval models for dangerous operations\\n- CLI UX patterns (output formatting, progress display, interactive modes)\\n- Hook/plugin systems\\n- Git workflow integration\\n- Model routing and provider handling\\n- Session/conversation management\\n- MCP (Model Context Protocol) integration — Claude Code is a major MCP consumer\\n- Subagent/delegation patterns\\n- Cost tracking and token management\\n- IDE integration approaches (VS Code, JetBrains)\\n- Configuration system design\\n\\nSkip anything that's: Anthropic-model-specific (prompt caching details unique to Claude), marketing fluff with no technical substance, or features hermes-agent already has with equivalent or better implementation.\\n\\nSTEP 3: SCORE AND SELECT\\nRate each feature on:\\n- Impact: How useful would this be for hermes-agent users?\\n- Feasibility: Can this be implemented without access to their source code?\\n- Novelty: Does hermes-agent already have this or something comparable?\\n- Clarity: Is the feature well-enough described to implement from documentation/descriptions alone?\\nOnly proceed with features scoring high across all four.\\n\\nSTEP 4: IMPLEMENT WORTHY FEATURES\\nFor each selected feature:\\n- The hermes-agent repo is at ~/.hermes/hermes-agent. Work in the main repo directory.\\n- Create a feature branch: `git checkout -b claude-code-inspired/<short-description> origin/main`\\n- Implement the feature adapted to hermes-agent's Python architecture. Follow AGENTS.md conventions (tool registration pattern, config system, slash commands, etc.).\\n- The hermes-agent-dev skill is loaded — follow its workflow for making changes, running tests, and creating PRs.\\n- Since there's no source code to reference, write clean implementations based on the documented behavior. Note in comments where the inspiration came from.\\n- Run the relevant test subset to verify the implementation.\\n- Commit with a clear message: \\\"Inspired by Claude Code: <description>\\\"\\n- Push and open a PR with `gh pr create` — in the PR body:\\n * Link to the source material (blog post, docs page, tweet) where the feature was discovered\\n * Explain the feature as Claude Code implements it\\n * Describe how hermes-agent's implementation differs or adapts the concept\\n * Mark as draft if tests need more work\\n\\nSTEP 5: SUMMARY REPORT\\nAfter all research and analysis is complete, provide a summary report:\\n- What sources were checked and what updates were found\\n- Which features were evaluated as potentially relevant\\n- How many PRs were opened in hermes-agent (with links)\\n- Notable trends in Claude Code's development direction\\n- Features that weren't implemented but are worth watching for future consideration\\n\\nIf no features are worth implementing this week, deliver the summary report explaining what was found and why nothing qualified. Even a \\\"quiet week\\\" report is valuable for tracking Claude Code's trajectory.\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"job_id\": \"3d5a7b9a0844\",\n \"name\": \"claude-code-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"schedule\": \"0 17 * * 5\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-04-03T17:00:00-07:00\",\n \"job\": {\n \"job_id\": \"3d5a7b9a0844\",\n \"name\": \"claude-code-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"prompt_preview\": \"Weekly Claude Code Feature Scout \\u2014 scan the web for Claude Code updates and implement worthy feature...\",\n \"model\": null,\n \"provider\": null,\n \"base_url\": null,\n \"schedule\": \"0 17 * * 5\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-04-03T17:00:00-07:00\",\n \"last_run_at\": null,\n \"last_status\": null,\n \"enabled\": true,\n \"state\": \"scheduled\",\n \"paused_at\": null,\n \"paused_reason\": null\n },\n \"message\": \"Cron job 'claude-code-scout' created.\"\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01MJjZ1qu45pewCmpS516YcD"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Done. This one's a different beast — instead of scanning PRs it does broad web sweeps across Anthropic's blog, docs, X, Reddit, and changelogs, then reverse-engineers features worth porting from descriptions alone.\n\nFull weekly schedule:\n\n⚙ Sun — openclaw, nanoclaw, ironclaw (PR scan)\n⚙ Tue — openai codex (PR scan)\n⚙ Thu — opencode (PR scan)\n⚙ Fri — claude code (web intel scan)\n\nYou've basically got a competitive intelligence pipeline running against every major agent project, all feeding PRs into hermes-agent. Saturdays, Mondays, and Wednesdays are still open if you want to add more targets."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "Is kilo code open source?"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "\n\n",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_018PRB7R7JNQbCDLBjcR2NxY",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"Kilo Code open source github AI coding agent\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://github.com/kilo-org/kilocode\",\n \"title\": \"GitHub - Kilo-Org/kilocode\",\n \"description\": \"Kilo is the all-in-one agentic engineering platform. Build, ship, and iterate faster with the most popular open source coding agent. #1 coding agent on ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://kilo.ai/\",\n \"title\": \"Kilo - Kilo: The Open Source AI Coding Agent for VS Code ...\",\n \"description\": \"Build, ship, and iterate faster with Kilo, the most popular open source AI coding agent. Secure, local-first, 500+ models. Start faster today.\",\n \"category\": null\n },\n {\n \"url\": \"https://github.com/kilo-org/kilo\",\n \"title\": \"GitHub - Kilo-Org/kilo: Kilo is the all-in-one agentic engineering ...\",\n \"description\": \"Kilo is the all-in-one agentic engineering platform. Build, ship, and iterate faster with the most popular open source coding agent.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/Kilo-Org\",\n \"title\": \"Kilo Code - GitHub\",\n \"description\": \"Kilo is the all-in-one agentic engineering platform. Build, ship, and iterate faster with the most popular open source coding agent. #1 coding agent on ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://kilo.ai/code\",\n \"title\": \"Kilo: The Open Source AI Coding Agent for VS Code, JetBrains, and ...\",\n \"description\": \"Build, ship, and iterate faster with Kilo, the most popular open source AI coding agent. Secure, local-first, 500+ models.\",\n \"category\": null\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_018PRB7R7JNQbCDLBjcR2NxY"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Yep — fully open source at Kilo-Org/kilocode on GitHub. VS Code / JetBrains IDE extension, 500+ model support, claims to be the #1 coding agent on the marketplace. Want me to add it to the rotation?"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "Ok add kilo code for mondays"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01YH1BfSikNrQBMVTSZXu5zq",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "cronjob",
|
||||||
|
"arguments": "{\"action\": \"create\", \"name\": \"kilocode-pr-scout\", \"schedule\": \"0 17 * * 1\", \"deliver\": \"origin\", \"skill\": \"hermes-agent-dev\", \"prompt\": \"Weekly Kilo Code PR Scout — scan merged PRs from Kilo-Org/kilocode and port worthy features to hermes-agent.\\n\\nSTEP 1: GATHER MERGED PRs FROM THE PAST WEEK\\n- Use `gh pr list --repo Kilo-Org/kilocode --state merged --search \\\"merged:>=$(date -d '7 days ago' +%Y-%m-%d)\\\" --limit 50 --json number,title,body,mergedAt,url,files,additions,deletions` to get all PRs merged in the last 7 days.\\n- If the date command fails, calculate the date manually (today minus 7 days) and use that.\\n\\nSTEP 2: ANALYZE EACH PR\\nKilo Code is an open-source AI coding agent that runs as a VS Code / JetBrains IDE extension. It supports 500+ models, is local-first, and focuses on agentic engineering workflows. Key areas of overlap with hermes-agent: multi-model support, file editing, code generation, tool use patterns, IDE integration (hermes-agent has an ACP adapter), and agent loop design.\\n\\nFor each merged PR:\\n- Read the PR title, description, and changed files list.\\n- For promising PRs, read the full diff: `gh pr diff <number> --repo Kilo-Org/kilocode`\\n- Categorize relevance to hermes-agent. Focus on:\\n * IDE integration patterns (relevant to hermes-agent's ACP adapter for VS Code/Zed/JetBrains)\\n * Multi-model provider routing and switching (500+ model support)\\n * File editing / diff display / multi-file workflows\\n * Agent loop improvements (context management, tool calling, iteration control)\\n * Code intelligence features (diagnostics, LSP usage, codebase understanding)\\n * Approval/permission models for code changes\\n * Session and conversation management\\n * MCP (Model Context Protocol) integration\\n * Streaming and response handling\\n * Cost tracking and token management\\n * Configuration and customization patterns\\n * Security/sandboxing approaches\\n * Bug fixes that might apply to similar patterns in hermes-agent\\n- Skip PRs that are: purely cosmetic/docs, TypeScript/VS Code extension-specific infra, dependency bumps, marketplace publishing, or clearly irrelevant to a Python codebase.\\n\\nSTEP 3: SCORE AND SELECT\\nRate each relevant PR on:\\n- Impact: How useful would this be for hermes-agent users?\\n- Feasibility: How hard to implement in Python/hermes-agent's architecture?\\n- Novelty: Does hermes-agent already have this feature?\\nOnly proceed with PRs scoring high on all three.\\n\\nSTEP 4: IMPLEMENT WORTHY PRs\\nFor each selected PR:\\n- The hermes-agent repo is at ~/.hermes/hermes-agent. Work in the main repo directory.\\n- Create a feature branch: `git checkout -b kilocode-port/<short-description> origin/main`\\n- Implement the feature adapted to hermes-agent's Python architecture. Follow AGENTS.md conventions (tool registration pattern, config system, slash commands, etc.).\\n- The hermes-agent-dev skill is loaded — follow its workflow for making changes, running tests, and creating PRs.\\n- Run the relevant test subset to verify the implementation.\\n- Commit with a clear message referencing the source: \\\"Port from Kilo-Org/kilocode#<number>: <description>\\\"\\n- Push and open a PR with `gh pr create` — in the PR body:\\n * Link to the source Kilo Code PR\\n * Explain what was ported and how it was adapted\\n * Note any architectural differences in the implementation\\n * Mark as draft if tests need more work\\n\\nSTEP 5: SUMMARY REPORT\\nAfter all analysis is complete, provide a summary report:\\n- Total PRs merged in Kilo-Org/kilocode this week\\n- How many were evaluated as potentially relevant\\n- How many PRs were opened in hermes-agent (with links)\\n- Brief notes on any interesting trends or features that weren't ported but are worth watching\\n\\nIf no PRs are worth porting this week, just deliver the summary report explaining what was reviewed and why nothing qualified.\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"job_id\": \"00bf3ec29397\",\n \"name\": \"kilocode-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"schedule\": \"0 17 * * 1\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-30T17:00:00-07:00\",\n \"job\": {\n \"job_id\": \"00bf3ec29397\",\n \"name\": \"kilocode-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"prompt_preview\": \"Weekly Kilo Code PR Scout \\u2014 scan merged PRs from Kilo-Org/kilocode and port worthy features to herme...\",\n \"model\": null,\n \"provider\": null,\n \"base_url\": null,\n \"schedule\": \"0 17 * * 1\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-30T17:00:00-07:00\",\n \"last_run_at\": null,\n \"last_status\": null,\n \"enabled\": true,\n \"state\": \"scheduled\",\n \"paused_at\": null,\n \"paused_reason\": null\n },\n \"message\": \"Cron job 'kilocode-pr-scout' created.\"\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01YH1BfSikNrQBMVTSZXu5zq"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Done. Updated schedule:\n\n⚙ Sun — openclaw, nanoclaw, ironclaw\n⚙ Mon — kilo code\n⚙ Tue — openai codex\n⚙ Thu — opencode\n⚙ Fri — claude code (web scan)\n\nWednesday and Saturday still open. You're covering the full spectrum now — IDE agents (kilo), terminal agents (codex, opencode), platform agents (openclaw, nanoclaw), systems-level (ironclaw), and closed-source (claude code)."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "We are only missing saturday? What other major agents are out there? Maybe check gemini cli? See whats popular on openrouter apps"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "\n\nActually Wednesday is open too. Let me dig around.",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01KmKwt78g3uPrSym3NDknHg",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"Gemini CLI google open source github coding agent\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01KQuJru9zYQX4PytPezMStA",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"openrouter apps popular AI coding agents 2026\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_017GwNVRXF2e5nVVR8JfkQeA",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"top open source AI coding agents terminal CLI 2026\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://github.com/google-gemini/gemini-cli\",\n \"title\": \"google-gemini/gemini-cli: An open-source AI agent that ... - GitHub\",\n \"description\": \"Gemini CLI is an open-source AI agent that brings the power of Gemini directly into your terminal. It provides lightweight access to Gemini, giving you the ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://geminicli.com/\",\n \"title\": \"Gemini CLI: Build, debug & deploy with AI\",\n \"description\": \"Build debug & deploy with AI. Query and edit large codebases, generate apps from images or PDFs, and automate complex workflows—all from your terminal with ...\",\n \"category\": null\n },\n {\n \"url\": \"https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemini-cli-open-source-ai-agent/\",\n \"title\": \"Google announces Gemini CLI: your open-source AI agent\",\n \"description\": \"This open-source AI agent brings Gemini directly into your terminal for coding, problem-solving, and task management. You can access Gemini 2.5 ...\",\n \"category\": null\n },\n {\n \"url\": \"https://www.reddit.com/r/googlecloud/comments/1lk55a4/gemini_cli_your_opensource_ai_agent/\",\n \"title\": \"Gemini CLI: your open-source AI agent : r/googlecloud - Reddit\",\n \"description\": \"We are excited to announce the release of Gemini CLI , a command-line AI workflow tool that connects to your tools, understands your code ...\",\n \"category\": null\n },\n {\n \"url\": \"https://developers.google.com/gemini-code-assist/docs/gemini-cli\",\n \"title\": \"Gemini CLI | Gemini Code Assist - Google for Developers\",\n \"description\": \"The Gemini command line interface (CLI) is an open source AI agent that provides access to Gemini directly in your terminal.\",\n \"category\": null\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01KmKwt78g3uPrSym3NDknHg"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://openrouter.ai/apps\",\n \"title\": \"App & Agent Rankings - OpenRouter\",\n \"description\": \"Top Coding Agents · 1. Favicon for https://kilocode.ai/. Kilo Code. AI coding agent for VS Code · 2. Favicon for https://claude.ai/apple-touch-icon. Claude Code.\",\n \"category\": null\n },\n {\n \"url\": \"https://www.teamday.ai/blog/top-ai-models-openrouter-2026\",\n \"title\": \"Best AI Models for Chat & Agents: OpenRouter Ranked (March 2026)\",\n \"description\": \"Ranked: best OpenRouter models in March 2026. GPT-5.4, Claude Sonnet 4.6, Gemini 3.1, DeepSeek V3.2, and top free models compared by quality ...\",\n \"category\": null\n },\n {\n \"url\": \"https://openrouter.ai/collections/programming\",\n \"title\": \"Best AI Models for Coding - OpenRouter\",\n \"description\": \"This collection features top coding models from Anthropic, Google, xAI, OpenAI and more, all accessible through a single API. From agentic coding workflows to ...\",\n \"category\": null\n },\n {\n \"url\": \"https://www.reddit.com/r/LocalLLaMA/comments/1r6g14s/4_of_the_top_5_most_used_models_on_openrouter/\",\n \"title\": \"4 of the top 5 most used models on OpenRouter this week are Open ...\",\n \"description\": \"If it's not open source people will just use the model directly from the source provider. I'm suprised seeing gemini on this list.\",\n \"category\": null\n },\n {\n \"url\": \"https://www.robylon.ai/blog/leading-ai-coding-agents-of-2026\",\n \"title\": \"12 Best AI Coding Agents in 2026 - #5 Will Surprise Most Devs\",\n \"description\": \"The 12 Best AI Coding Agents in 2026 include Cursor, GitHub Copilot, Windsurf, and Bolt.new, Lovable AI, v0 by Vercel, Zencoder, ...\",\n \"category\": null\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01KQuJru9zYQX4PytPezMStA"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://www.faros.ai/blog/best-ai-coding-agents-2026\",\n \"title\": \"Best AI Coding Agents for 2026: Real-World Developer Reviews\",\n \"description\": \"A developer-focused look at the best AI coding agents in 2026, comparing Claude Code, Cursor, Codex, Copilot, Cline, and more—with guidance ...\",\n \"category\": null\n },\n {\n \"url\": \"https://www.tembo.io/blog/coding-cli-tools-comparison\",\n \"title\": \"The 2026 Guide to Coding CLI Tools: 15 AI Agents Compared\",\n \"description\": \"A comprehensive comparison of 15 coding CLI tools—from Claude Code and Codex to Aider and Goose. We break down features, pricing, ...\",\n \"category\": null\n },\n {\n \"url\": \"https://www.reddit.com/r/AI_Agents/comments/1r4jtx8/what_are_the_best_cli_ai_agents_right_now_trying/\",\n \"title\": \"What are the best CLI AI agents right now? Trying to replace Cursor ...\",\n \"description\": \"The best tool so far has been Cursor CLI agents because they reliably work through tasks, resolve build and test problems, and keep iterating ...\",\n \"category\": null\n },\n {\n \"url\": \"https://blog.logrocket.com/tested-5-ai-cli-tools/\",\n \"title\": \"I tested 5 AI CLI tools: Here's how they stack up - LogRocket Blog\",\n \"description\": \"The Gemini CLI is a free, open-source AI agent that brings Gemini's capabilities right into your terminal. Let's quickly see how to set it up.\",\n \"category\": null\n },\n {\n \"url\": \"https://www.mightybot.ai/blog/coding-ai-agents-for-accelerating-engineering-workflows\",\n \"title\": \"Mighty Blog - 9 Best AI Coding Agents in 2026, Ranked - MightyBot\",\n \"description\": \"The Top Coding AI Agents in 2026 · 1. Claude Code (Anthropic) — Best Overall AI Coding Agent · 2. Codex (OpenAI) — Best Cloud-Based Coding Agent.\",\n \"category\": null\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_017GwNVRXF2e5nVVR8JfkQeA"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01SujzQrcXJuZJbXMcfXh5KK",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_extract",
|
||||||
|
"arguments": "{\"urls\": [\"https://openrouter.ai/apps\",\"https://www.tembo.io/blog/coding-cli-tools-comparison\"]}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"results\": [\n {\n \"url\": \"https://openrouter.ai/apps\",\n \"title\": \"App & Agent Rankings | OpenRouter\",\n \"content\": \"# OpenRouter App & Agent Rankings Summary\\n\\nThis report summarizes the top-performing AI applications and agents integrated with OpenRouter, ranked by token usage and growth metrics.\\n\\n## 🏆 Top 4 Most Popular (All-Time)\\nThese are the largest public apps opting into usage tracking:\\n1. **OpenClaw**: \\\"The AI that actually does things\\\" — **16T tokens**\\n2. **Kilo Code**: AI coding agent for VS Code — **5.28T tokens**\\n3. **Claude Code**: \\\"The AI for problem solvers\\\" — **1.99T tokens**\\n4. **BLACKBOXAI**: AI agent for builders — **1.29T tokens**\\n\\n---\\n\\n## 📈 Trending (Fastest Growing This Week)\\n* **Cinder Paper Replication**: +1964% (8.53B tokens)\\n* **Ampere**: +547% (11.3B tokens)\\n* **Cline**: +132% (489B tokens)\\n* **Scripty**: +100% (9.23B tokens)\\n* **Hermes Agent**: +97% (95.2B tokens)\\n\\n---\\n\\n## 📂 Category Rankings (by Token Usage)\\n\\n### 💻 Coding Agents\\n1. **Kilo Code**: VS Code extension (259B tokens)\\n2. **Claude Code**: Problem-solving agent (97.6B tokens)\\n3. **Cline**: Autonomous IDE agent (81B tokens)\\n4. **Hermes Agent**: CLI/Personal agent (15.8B tokens)\\n5. **Roo Code**: Multi-agent dev team for editors (14.2B tokens)\\n\\n### 🚀 Productivity & Tools\\n1. **OpenClaw**: Action-oriented AI (803B tokens)\\n2. **Hermes Agent**: (15.8B tokens)\\n3. **Open WebUI**: Extensible, self-hosted interface (4.41B tokens)\\n4. **Gobii**: Always-on web browsing agents (3.64B tokens)\\n5. **BuddyPro AI**: Knowledge-to-digital-twin builder (2.18B tokens)\\n\\n### 🎨 Creative & Media\\n1. **Descript**: AI Video & Podcast Editor (40.6B tokens)\\n2. **VidMuse**: Music video/shorts generator (3.63B tokens)\\n3. **Novel\n\n[... tool output truncated for fixture — original was 8951 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01SujzQrcXJuZJbXMcfXh5KK"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Here's what the landscape looks like. Gemini CLI is indeed open source (google-gemini/gemini-cli).\n\nOpenRouter coding agent rankings by token usage:\n\n① Kilo Code — 259B tokens (covered, Mon)\n② Claude Code — 97.6B tokens (covered, Fri)\n③ Cline — 81B tokens (open source, NOT covered)\n④ Hermes Agent — 15.8B tokens (that's us, trending +97%)\n⑤ Roo Code — 14.2B tokens (open source, NOT covered)\n\nPlus from the broader CLI comparison, notable open source agents we're missing:\n\n◆ Gemini CLI (google-gemini/gemini-cli) — 1M context window, generous free tier\n◆ Cline — #3 on OpenRouter, VS Code native, human-in-the-loop, browser automation\n◆ Roo Code — multi-agent dev team for editors\n◆ Aider — 39K stars, the open source pioneer, \"watch mode\" for IDE integration\n◆ Goose (Block) — Apache 2.0, heavy MCP integration\n◆ Crush (Charmbracelet) — broadest platform support, mid-session model switching\n\nYou've got Wednesday and Saturday open. Cline at 81B tokens is the obvious gap — it's the #3 coding agent on OpenRouter and we're not watching it. Gemini CLI is the other big one you mentioned.\n\nWant me to slot Gemini CLI on one day and Cline on the other? Or a different combo?"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "Make gemini cline aider and roo the final day we have open"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "\n\nLet me grab the exact repos for all four first.",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01GxYRwEUZJYRSUrq5w2Bxf1",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"Cline github repository open source VS Code agent\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01DbZNVCCC37o5Es1ApLj7UT",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"Aider github repository paul-gauthier coding agent\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_012UYs8u6UytVBLaXSXAARD4",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"Roo Code github repository open source multi-agent\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://github.com/cline/cline\",\n \"title\": \"GitHub - cline/cline: Autonomous coding agent right in your IDE ...\",\n \"description\": \"Autonomous coding agent right in your IDE, capable of creating/editing files, executing commands, using the browser, and more with your permission every ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://cline.bot/\",\n \"title\": \"Cline - AI Coding, Open Source and Uncompromised\",\n \"description\": \"Open-source AI coding agent with Plan/Act modes, MCP integration, and terminal-first workflows. Trusted by 5M+ developers worldwide.\",\n \"category\": null\n },\n {\n \"url\": \"https://github.com/cline\",\n \"title\": \"Cline - GitHub\",\n \"description\": \"Autonomous coding agent right in your IDE, capable of creating/editing files, executing commands, using the browser, and more with your permission every step ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/orgs/cline/repositories?type=all\",\n \"title\": \"cline repositories - GitHub\",\n \"description\": \"Autonomous coding agent right in your IDE, capable of creating/editing files, executing commands, using the browser, and more with your permission every ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/cline/cline/discussions/1007\",\n \"title\": \"Multi-Agent Software Development with Cline #1007 - GitHub\",\n \"description\": \"Implement a multi-agent system within Cline, inspired by the ChatDev paper, where specialized AI agents collaborate to simulate a software development company.\",\n \"category\": \"github\"\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01GxYRwEUZJYRSUrq5w2Bxf1"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://github.com/aider-ai/aider\",\n \"title\": \"GitHub - Aider-AI/aider: aider is AI pair programming in your terminal\",\n \"description\": \"AI Pair Programming in Your Terminal. Aider lets you pair program with LLMs to start a new project or build on your existing codebase. aider screencast.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/lloydchang/Aider-AI-aider-fka-paul-gauthier-aider\",\n \"title\": \"aider is AI pair programming in your terminal · GitHub\",\n \"description\": \"Aider lets you pair program with LLMs, to edit code in your local git repository. Start a new project or work with an existing git repo.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/paul-gauthier/aider/issues/507\",\n \"title\": \"Aider Researcher · Issue #507 - GitHub\",\n \"description\": \"If Aider thinks that a task is non-trivial, it can run Google search, get all of the sites and rank them in terms of usability, how modern they are, and ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/paul-gauthier/aider/issues/669\",\n \"title\": \"Multi Agent (Supervisor/Reviewer agent) · Issue #669 · Aider-AI/aider\",\n \"description\": \"Aider is currently the top scoring agent on SWE Bench. So there's some reason to trust that it's current strategy is solid.\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://codesandbox.io/s/paul-gauthier-aider-lgsxg8\",\n \"title\": \"paul-gauthier/aider - Codesandbox\",\n \"description\": \"Aider is a command line tool that lets you pair program with GPT-3.5/GPT-4, to edit code stored in your local git repository.\",\n \"category\": null\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01DbZNVCCC37o5Es1ApLj7UT"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://github.com/RooCodeInc/Roo-Code\",\n \"title\": \"Roo Code gives you a whole dev team of AI agents in your ... - GitHub\",\n \"description\": \"What Can Roo Code Do For YOU? · Generate Code from natural language descriptions and specs · Adapt with Modes: Code, Architect, Ask, Debug, and Custom Modes ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/RooCodeInc\",\n \"title\": \"Roo Code - GitHub\",\n \"description\": \"Build databases, automations, apps & agents with AI — no code. Open source platform available on cloud and self-hosted. GDPR, HIPAA, SOC 2 compliant. Best ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://github.com/RooCodeInc/Roo-Code/issues/11929\",\n \"title\": \"GNAP: git-native task board for Roo Code's multi-agent team ...\",\n \"description\": \"GNAP proposes a fundamentally different coordination model: multiple independent Roo Code instances (possibly on different machines) ...\",\n \"category\": \"github\"\n },\n {\n \"url\": \"https://roocode.com/\",\n \"title\": \"Roo Code – The AI dev team that gets things done\",\n \"description\": \"Roo Code is open-source and fully customizable, letting you integrate any AI model you choose (e.g, OpenAI, Anthropic, local LLMs, etc.). It's built for multi- ...\",\n \"category\": null\n },\n {\n \"url\": \"https://www.reddit.com/r/RooCode/comments/1ioyp4j/proposal_roocode_community_github_repository_for/\",\n \"title\": \"Roo-Code Community GitHub Repository for Sharing Setups ...\",\n \"description\": \"A centralized place where we can share our setups and customizations for Roo-Code. A GitHub repository could be the perfect solution for this!\",\n \"category\": null\n }\n ]\n }\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_012UYs8u6UytVBLaXSXAARD4"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Got all four repos. We've got Wednesday and Saturday open — I'll split them 2 and 2 to balance the load.",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01BeTjeQqS9Hxe8Y5qfXxKa4",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "cronjob",
|
||||||
|
"arguments": "{\"action\": \"create\", \"name\": \"gemini-cli-pr-scout\", \"schedule\": \"0 17 * * 3\", \"deliver\": \"origin\", \"skill\": \"hermes-agent-dev\", \"prompt\": \"Weekly Gemini CLI PR Scout — scan merged PRs from google-gemini/gemini-cli and port worthy features to hermes-agent.\\n\\nSTEP 1: GATHER MERGED PRs FROM THE PAST WEEK\\n- Use `gh pr list --repo google-gemini/gemini-cli --state merged --search \\\"merged:>=$(date -d '7 days ago' +%Y-%m-%d)\\\" --limit 50 --json number,title,body,mergedAt,url,files,additions,deletions` to get all PRs merged in the last 7 days.\\n- If the date command fails, calculate the date manually (today minus 7 days) and use that.\\n\\nSTEP 2: ANALYZE EACH PR\\nGemini CLI is Google's open-source AI agent for the terminal (TypeScript/Node.js). Key differentiators: 1M token context window, Google Search grounding for fact-checking, generous free tier (60 RPM / 1000 RPD), built-in multimodal capabilities. Areas of overlap with hermes-agent: terminal agent workflow, tool execution, web search integration, context management, CLI UX.\\n\\nFor each merged PR:\\n- Read the PR title, description, and changed files list.\\n- For promising PRs, read the full diff: `gh pr diff <number> --repo google-gemini/gemini-cli`\\n- Categorize relevance to hermes-agent. Focus on:\\n * Large context management strategies (how they handle 1M token windows)\\n * Search grounding / web integration patterns\\n * Multimodal input handling (images, PDFs in terminal context)\\n * Tool execution and sandboxing\\n * Agent loop design (iteration, context compression, token budgets)\\n * CLI UX innovations (output formatting, progress, interactive modes)\\n * Configuration and extensibility patterns\\n * MCP integration\\n * Session and conversation management\\n * Streaming and response handling\\n * Bug fixes that might apply to similar patterns in hermes-agent\\n- Skip PRs that are: purely cosmetic/docs, TypeScript/Node.js-specific infra, dependency bumps, Google Cloud-specific integrations, or clearly irrelevant to a Python codebase.\\n\\nSTEP 3: SCORE AND SELECT\\nRate each relevant PR on:\\n- Impact: How useful would this be for hermes-agent users?\\n- Feasibility: How hard to implement in Python/hermes-agent's architecture?\\n- Novelty: Does hermes-agent already have this feature?\\nOnly proceed with PRs scoring high on all three.\\n\\nSTEP 4: IMPLEMENT WORTHY PRs\\nFor each selected PR:\\n- The hermes-agent repo is at ~/.hermes/hermes-agent. Work in the main repo directory.\\n- Create a feature branch: `git checkout -b gemini-cli-port/<short-description> origin/main`\\n- Implement the feature adapted to hermes-agent's Python architecture. Follow AGENTS.md conventions.\\n- The hermes-agent-dev skill is loaded — follow its workflow for making changes, running tests, and creating PRs.\\n- Run the relevant test subset to verify the implementation.\\n- Commit with a clear message: \\\"Port from google-gemini/gemini-cli#<number>: <description>\\\"\\n- Push and open a PR with `gh pr create` — link the source PR, explain the adaptation, mark as draft if needed.\\n\\nSTEP 5: SUMMARY REPORT\\nProvide a summary: total PRs merged, how many relevant, PRs opened in hermes-agent (with links), notable trends. If nothing worth porting, explain what was reviewed and why.\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01895kW4hoYBgXHAxQti6koD",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "cronjob",
|
||||||
|
"arguments": "{\"action\": \"create\", \"name\": \"cline-pr-scout\", \"schedule\": \"0 17 * * 3\", \"deliver\": \"origin\", \"skill\": \"hermes-agent-dev\", \"prompt\": \"Weekly Cline PR Scout — scan merged PRs from cline/cline and port worthy features to hermes-agent.\\n\\nSTEP 1: GATHER MERGED PRs FROM THE PAST WEEK\\n- Use `gh pr list --repo cline/cline --state merged --search \\\"merged:>=$(date -d '7 days ago' +%Y-%m-%d)\\\" --limit 50 --json number,title,body,mergedAt,url,files,additions,deletions` to get all PRs merged in the last 7 days.\\n- If the date command fails, calculate the date manually (today minus 7 days) and use that.\\n\\nSTEP 2: ANALYZE EACH PR\\nCline is the #3 coding agent on OpenRouter (81B tokens). It's an autonomous VS Code extension with a human-in-the-loop philosophy — every file change requires explicit user approval. Key features: Plan/Act modes, MCP integration, browser automation (launching browsers and capturing screenshots), terminal-first workflows. 5M+ developers.\\n\\nFor each merged PR:\\n- Read the PR title, description, and changed files list.\\n- For promising PRs, read the full diff: `gh pr diff <number> --repo cline/cline`\\n- Categorize relevance to hermes-agent. Focus on:\\n * Plan/Act mode patterns (planning before executing — hermes-agent has similar concepts)\\n * Browser automation approaches (relevant to hermes-agent's browser_tool)\\n * Human-in-the-loop approval patterns (relevant to hermes-agent's approval.py)\\n * MCP integration patterns (hermes-agent has mcp_tool.py)\\n * File editing strategies and diff display\\n * Context management and token budgets\\n * Tool execution patterns\\n * Multi-step task orchestration\\n * Provider routing and model support\\n * Screenshot/vision integration\\n * Configuration and customization\\n * Bug fixes that might apply to similar patterns in hermes-agent\\n- Skip PRs that are: purely cosmetic/docs, VS Code extension API-specific, TypeScript infra, dependency bumps, or marketplace publishing.\\n\\nSTEP 3: SCORE AND SELECT\\nRate each relevant PR on:\\n- Impact: How useful would this be for hermes-agent users?\\n- Feasibility: How hard to implement in Python/hermes-agent's architecture?\\n- Novelty: Does hermes-agent already have this feature?\\nOnly proceed with PRs scoring high on all three.\\n\\nSTEP 4: IMPLEMENT WORTHY PRs\\nFor each selected PR:\\n- The hermes-agent repo is at ~/.hermes/hermes-agent. Work in the main repo directory.\\n- Create a feature branch: `git checkout -b cline-port/<short-description> origin/main`\\n- Implement the feature adapted to hermes-agent's Python architecture. Follow AGENTS.md conventions.\\n- The hermes-agent-dev skill is loaded — follow its workflow.\\n- Run the relevant test subset to verify.\\n- Commit: \\\"Port from cline/cline#<number>: <description>\\\"\\n- Push and open a PR — link source, explain adaptation, draft if needed.\\n\\nSTEP 5: SUMMARY REPORT\\nProvide a summary: total PRs merged, how many relevant, PRs opened (with links), notable trends. If nothing worth porting, explain what was reviewed and why.\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01CApeiwZyxpRiyA24aNfVSJ",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "cronjob",
|
||||||
|
"arguments": "{\"action\": \"create\", \"name\": \"aider-pr-scout\", \"schedule\": \"0 17 * * 6\", \"deliver\": \"origin\", \"skill\": \"hermes-agent-dev\", \"prompt\": \"Weekly Aider PR Scout — scan merged PRs from Aider-AI/aider and port worthy features to hermes-agent.\\n\\nSTEP 1: GATHER MERGED PRs FROM THE PAST WEEK\\n- Use `gh pr list --repo Aider-AI/aider --state merged --search \\\"merged:>=$(date -d '7 days ago' +%Y-%m-%d)\\\" --limit 50 --json number,title,body,mergedAt,url,files,additions,deletions` to get all PRs merged in the last 7 days.\\n- If the date command fails, calculate the date manually (today minus 7 days) and use that.\\n\\nSTEP 2: ANALYZE EACH PR\\nAider is the open-source pioneer of AI pair programming in the terminal (Python). 39K+ GitHub stars, 15B tokens processed weekly. Key features: automatic git commits with sensible messages, \\\"watch mode\\\" for IDE integration, support for virtually every LLM (Claude, GPT, DeepSeek, local via Ollama), repo-map for codebase understanding, and battle-tested reliability. Being Python-based like hermes-agent, code patterns translate more directly.\\n\\nFor each merged PR:\\n- Read the PR title, description, and changed files list.\\n- For promising PRs, read the full diff: `gh pr diff <number> --repo Aider-AI/aider`\\n- Categorize relevance to hermes-agent. Focus on:\\n * Repo-map / codebase indexing strategies (understanding large codebases)\\n * Git integration patterns (auto-commit, diff handling)\\n * \\\"Watch mode\\\" and IDE integration approaches\\n * Model support and provider routing (Aider supports many models)\\n * Edit format strategies (whole file, diff, udiff, etc.)\\n * Context management and token optimization\\n * Agent loop and retry logic\\n * CLI UX patterns\\n * Streaming and response handling\\n * Linting and code quality integration\\n * Testing integration\\n * Bug fixes — especially valuable since Aider is also Python\\n- Skip PRs that are: purely cosmetic/docs, CI/CD, dependency bumps, or benchmark-specific.\\n\\nSTEP 3: SCORE AND SELECT\\nRate each relevant PR on:\\n- Impact: How useful would this be for hermes-agent users?\\n- Feasibility: How hard to implement in hermes-agent's architecture? (Easier than usual since both are Python)\\n- Novelty: Does hermes-agent already have this feature?\\nOnly proceed with PRs scoring high on all three.\\n\\nSTEP 4: IMPLEMENT WORTHY PRs\\nFor each selected PR:\\n- The hermes-agent repo is at ~/.hermes/hermes-agent. Work in the main repo directory.\\n- Create a feature branch: `git checkout -b aider-port/<short-description> origin/main`\\n- Implement the feature adapted to hermes-agent's architecture. Follow AGENTS.md conventions.\\n- The hermes-agent-dev skill is loaded — follow its workflow.\\n- Since both projects are Python, some patterns may translate almost directly — but always adapt to hermes-agent's conventions (tool registry, config system, etc.).\\n- Run the relevant test subset to verify.\\n- Commit: \\\"Port from Aider-AI/aider#<number>: <description>\\\"\\n- Push and open a PR — link source, explain adaptation, draft if needed.\\n\\nSTEP 5: SUMMARY REPORT\\nProvide a summary: total PRs merged, how many relevant, PRs opened (with links), notable trends. If nothing worth porting, explain what was reviewed and why.\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_019omdXZNGsb4EqvxaFRzKSK",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "cronjob",
|
||||||
|
"arguments": "{\"action\": \"create\", \"name\": \"roocode-pr-scout\", \"schedule\": \"0 17 * * 6\", \"deliver\": \"origin\", \"skill\": \"hermes-agent-dev\", \"prompt\": \"Weekly Roo Code PR Scout — scan merged PRs from RooCodeInc/Roo-Code and port worthy features to hermes-agent.\\n\\nSTEP 1: GATHER MERGED PRs FROM THE PAST WEEK\\n- Use `gh pr list --repo RooCodeInc/Roo-Code --state merged --search \\\"merged:>=$(date -d '7 days ago' +%Y-%m-%d)\\\" --limit 50 --json number,title,body,mergedAt,url,files,additions,deletions` to get all PRs merged in the last 7 days.\\n- If the date command fails, calculate the date manually (today minus 7 days) and use that.\\n\\nSTEP 2: ANALYZE EACH PR\\nRoo Code is an open-source multi-agent dev team for VS Code/editors (#5 coding agent on OpenRouter, 14.2B tokens). Key features: multiple agent modes (Code, Architect, Ask, Debug, Custom), multi-agent coordination, custom mode creation, and extensive model support. Forked from Cline originally but diverged significantly.\\n\\nFor each merged PR:\\n- Read the PR title, description, and changed files list.\\n- For promising PRs, read the full diff: `gh pr diff <number> --repo RooCodeInc/Roo-Code`\\n- Categorize relevance to hermes-agent. Focus on:\\n * Multi-agent mode patterns (Code/Architect/Ask/Debug — hermes-agent has subagent delegation)\\n * Custom mode/persona creation (relevant to hermes-agent's skin system and agent customization)\\n * Agent coordination and handoff patterns\\n * File editing and multi-file workflows\\n * Context management strategies\\n * MCP integration\\n * Tool execution patterns\\n * Provider routing and model switching\\n * Session management\\n * Memory and knowledge persistence\\n * Browser automation\\n * Configuration and extensibility\\n * Bug fixes that might apply to similar patterns in hermes-agent\\n- Skip PRs that are: purely cosmetic/docs, VS Code extension API-specific, TypeScript infra, dependency bumps, or marketplace publishing.\\n\\nSTEP 3: SCORE AND SELECT\\nRate each relevant PR on:\\n- Impact: How useful would this be for hermes-agent users?\\n- Feasibility: How hard to implement in Python/hermes-agent's architecture?\\n- Novelty: Does hermes-agent already have this feature?\\nOnly proceed with PRs scoring high on all three.\\n\\nSTEP 4: IMPLEMENT WORTHY PRs\\nFor each selected PR:\\n- The hermes-agent repo is at ~/.hermes/hermes-agent. Work in the main repo directory.\\n- Create a feature branch: `git checkout -b roocode-port/<short-description> origin/main`\\n- Implement the feature adapted to hermes-agent's Python architecture. Follow AGENTS.md conventions.\\n- The hermes-agent-dev skill is loaded — follow its workflow.\\n- Run the relevant test subset to verify.\\n- Commit: \\\"Port from RooCodeInc/Roo-Code#<number>: <description>\\\"\\n- Push and open a PR — link source, explain adaptation, draft if needed.\\n\\nSTEP 5: SUMMARY REPORT\\nProvide a summary: total PRs merged, how many relevant, PRs opened (with links), notable trends. If nothing worth porting, explain what was reviewed and why.\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"job_id\": \"16de93694a6c\",\n \"name\": \"gemini-cli-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"schedule\": \"0 17 * * 3\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-04-01T17:00:00-07:00\",\n \"job\": {\n \"job_id\": \"16de93694a6c\",\n \"name\": \"gemini-cli-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"prompt_preview\": \"Weekly Gemini CLI PR Scout \\u2014 scan merged PRs from google-gemini/gemini-cli and port worthy features ...\",\n \"model\": null,\n \"provider\": null,\n \"base_url\": null,\n \"schedule\": \"0 17 * * 3\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-04-01T17:00:00-07:00\",\n \"last_run_at\": null,\n \"last_status\": null,\n \"enabled\": true,\n \"state\": \"scheduled\",\n \"paused_at\": null,\n \"paused_reason\": null\n },\n \"message\": \"Cron job 'gemini-cli-pr-scout' created.\"\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01BeTjeQqS9Hxe8Y5qfXxKa4"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"job_id\": \"92f1dabc9169\",\n \"name\": \"cline-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"schedule\": \"0 17 * * 3\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-04-01T17:00:00-07:00\",\n \"job\": {\n \"job_id\": \"92f1dabc9169\",\n \"name\": \"cline-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"prompt_preview\": \"Weekly Cline PR Scout \\u2014 scan merged PRs from cline/cline and port worthy features to hermes-agent.\\n\\n...\",\n \"model\": null,\n \"provider\": null,\n \"base_url\": null,\n \"schedule\": \"0 17 * * 3\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-04-01T17:00:00-07:00\",\n \"last_run_at\": null,\n \"last_status\": null,\n \"enabled\": true,\n \"state\": \"scheduled\",\n \"paused_at\": null,\n \"paused_reason\": null\n },\n \"message\": \"Cron job 'cline-pr-scout' created.\"\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01895kW4hoYBgXHAxQti6koD"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"job_id\": \"a477e19d9ded\",\n \"name\": \"aider-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"schedule\": \"0 17 * * 6\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-28T17:00:00-07:00\",\n \"job\": {\n \"job_id\": \"a477e19d9ded\",\n \"name\": \"aider-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"prompt_preview\": \"Weekly Aider PR Scout \\u2014 scan merged PRs from Aider-AI/aider and port worthy features to hermes-agent...\",\n \"model\": null,\n \"provider\": null,\n \"base_url\": null,\n \"schedule\": \"0 17 * * 6\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-28T17:00:00-07:00\",\n \"last_run_at\": null,\n \"last_status\": null,\n \"enabled\": true,\n \"state\": \"scheduled\",\n \"paused_at\": null,\n \"paused_reason\": null\n },\n \"message\": \"Cron job 'aider-pr-scout' created.\"\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01CApeiwZyxpRiyA24aNfVSJ"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"job_id\": \"b9eac48dc7a1\",\n \"name\": \"roocode-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"schedule\": \"0 17 * * 6\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-28T17:00:00-07:00\",\n \"job\": {\n \"job_id\": \"b9eac48dc7a1\",\n \"name\": \"roocode-pr-scout\",\n \"skill\": \"hermes-agent-dev\",\n \"skills\": [\n \"hermes-agent-dev\"\n ],\n \"prompt_preview\": \"Weekly Roo Code PR Scout \\u2014 scan merged PRs from RooCodeInc/Roo-Code and port worthy features to herm...\",\n \"model\": null,\n \"provider\": null,\n \"base_url\": null,\n \"schedule\": \"0 17 * * 6\",\n \"repeat\": \"forever\",\n \"deliver\": \"origin\",\n \"next_run_at\": \"2026-03-28T17:00:00-07:00\",\n \"last_run_at\": null,\n \"last_status\": null,\n \"enabled\": true,\n \"state\": \"scheduled\",\n \"paused_at\": null,\n \"paused_reason\": null\n },\n \"message\": \"Cron job 'roocode-pr-scout' created.\"\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_019omdXZNGsb4EqvxaFRzKSK"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Full week is loaded. Split the four across the two open days:\n\nComplete schedule, all at 10 AM Pacific:\n\n⚙ Sun — openclaw, nanoclaw, ironclaw\n⚙ Mon — kilo code\n⚙ Tue — openai codex\n⚙ Wed — gemini cli, cline\n⚙ Thu — opencode\n⚙ Fri — claude code (web scan)\n⚙ Sat — aider, roo code\n\n10 scouts across 7 days, covering every major agent project in the space. Aider is especially interesting since it's also Python — ports from that one should be the cleanest. Cline at 81B tokens on OpenRouter is the one I'd watch closest for high-impact features.\n\nAider and Roo Code first runs are tonight actually (Saturday), so you'll get reports from those two within a few hours."
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,626 @@
|
||||||
|
{
|
||||||
|
"name": "debug-session-feishu-id-model",
|
||||||
|
"description": "~60-turn debug/triage PR-review session: a third-party bug report says the gateway's Feishu adapter misuses the open_id / union_id / user_id identity model (open_id is app-scoped, not the bot's canonical ID). An open community PR (#8388) tries to fix it. Agent reviews the PR against current main, fetches upstream Feishu/Lark identity docs, and produces a decision. Exercises long tool-heavy context with PR diffs, upstream docs, and a clear decision at the end — the classic 'can the summary still name the PR number, the root cause, and the decision?' scenario.",
|
||||||
|
"model": "anthropic/claude-sonnet-4.6",
|
||||||
|
"context_length": 200000,
|
||||||
|
"source": "~/.hermes/sessions/20260412_233741_3f2119a8.jsonl",
|
||||||
|
"truncated_to": 58,
|
||||||
|
"scrubbed_at": "2026-04-24T07:40:21Z",
|
||||||
|
"scrubbing_passes": [
|
||||||
|
"redact_sensitive_text (agent.redact)",
|
||||||
|
"username paths replaced with /home/user",
|
||||||
|
"personal handles (all case variants of the maintainer name) replaced with 'user'",
|
||||||
|
"email addresses replaced with contributor@example.com",
|
||||||
|
"git 'Author: Name <addr>' header lines normalised",
|
||||||
|
"reasoning scratchpad blocks stripped from assistant content",
|
||||||
|
"think tag blocks stripped from assistant content",
|
||||||
|
"messaging-platform user mentions replaced with <@user>",
|
||||||
|
"first user message paraphrased to remove personal voice",
|
||||||
|
"subsequent user messages kept verbatim (after above redactions)",
|
||||||
|
"system prompt replaced with generic public-safe placeholder",
|
||||||
|
"orphan empty-assistant messages and trailing tool messages dropped",
|
||||||
|
"tool outputs longer than 2000 chars truncated with a note"
|
||||||
|
],
|
||||||
|
"messages": [
|
||||||
|
{
|
||||||
|
"role": "system",
|
||||||
|
"content": "You are a helpful AI coding assistant with access to tools (terminal, file editing, search, web, etc.). You operate in a conversational loop: the user gives you a task, you call tools to accomplish it, and you report back concisely."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "A community user reports the Feishu/Lark adapter gets the identity model wrong — open_id is app-scoped, not the bot's canonical ID. There's an open PR #8388 trying to fix it. Use the hermes-agent-dev skill and the pr-triage-salvage skill to review it."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "\n\nInteresting. Let me pull up that PR and take a look at what they're proposing.",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01DhZUXFbRmZecEXrKGbG3p1",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "skill_view",
|
||||||
|
"arguments": "{\"name\": \"pr-triage-salvage\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01NxU3C2z6gbGvXHiG3umKmQ",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd ~/repos/hermes-agent && git fetch origin pull/8388/head:pr-8388 2>&1 | tail -5\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"success\": true, \"name\": \"pr-triage-salvage\", \"description\": \"Triage external PRs — review, cherry-pick/salvage valid fixes onto main, close broken/redundant ones. Handles stale PRs, mixed-quality changes, and contributor attribution.\", \"tags\": [\"github\", \"pr\", \"triage\", \"salvage\", \"cherry-pick\", \"code-review\"], \"related_skills\": [], \"content\": \"---\\nname: pr-triage-salvage\\ndescription: Triage external PRs — review, cherry-pick/salvage valid fixes onto main, close broken/redundant ones. Handles stale PRs, mixed-quality changes, and contributor attribution.\\ntags: [github, pr, triage, salvage, cherry-pick, code-review]\\ntriggers:\\n - triaging PRs\\n - salvaging a PR\\n - reviewing community contributions\\n - cherry-picking external fixes\\n - mega-PR with many bundled features\\n - large community PR review\\n---\\n\\n# PR Triage & Salvage Workflow\\n\\nSystematic process for reviewing external PRs, salvaging valid fixes, closing broken/redundant ones.\\n\\n## Workflow\\n\\n### 1. Assess the PR\\n\\n```bash\\ngh pr view <NUMBER>\\ngh pr diff <NUMBER>\\n# Check staleness\\ngit fetch origin pull/<NUMBER>/head:pr-<NUMBER>\\ngit rev-list --count $(git merge-base origin/main pr-<NUMBER>)..origin/main\\n```\\n\\n**Batch triage (30+ PRs):** Bucket by similarity, flag dupes. Pick most complete.\\n\\n**Mega-PR decomp:** Cherry-pick if commits grouped, reimplement if interleaved.\\n\\n**Conflicting PRs:** Build combined fix from scratch on main; credit all. **Partial fix on main:** Before salvaging, check\n\n[Truncated: tool response was 102,838 chars. Full output could not be saved to sandbox.]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01DhZUXFbRmZecEXrKGbG3p1"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"/usr/bin/bash: line 3: cd: /home/user/repos/hermes-agent: No such file or directory\", \"exit_code\": 1, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01NxU3C2z6gbGvXHiG3umKmQ"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01W6fYofRYbMWf294dDumDt5",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"find /home -maxdepth 4 -name \\\"hermes-agent\\\" -type d 2>/dev/null; find / -maxdepth 4 -name \\\"hermes-agent\\\" -type d 2>/dev/null | head -5\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"/home/user/.hermes/hermes-agent\\n/home/user/.hermes/hermes-agent\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01W6fYofRYbMWf294dDumDt5"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01FYdZ4867K7kL78kcQfgez3",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git fetch origin pull/8388/head:pr-8388 2>&1 | tail -5\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"From github.com:NousResearch/hermes-agent\\n * [new ref] refs/pull/8388/head -> pr-8388\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01FYdZ4867K7kL78kcQfgez3"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01K3yrYegnpxP8nDf8gvsSzP",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && gh pr view 8388 2>&1\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"title:\\tfix(feishu): correct Feishu identity model and enable multi-account routing\\nstate:\\tOPEN\\nauthor:\\thchen13\\nlabels:\\t\\nassignees:\\t\\nreviewers:\\t\\nprojects:\\t\\nmilestone:\\t\\nnumber:\\t8388\\nurl:\\thttps://github.com/NousResearch/hermes-agent/pull/8388\\nadditions:\\t4647\\ndeletions:\\t239\\nauto-merge:\\tdisabled\\n--\\n## What this PR fixes\\n\\nThis PR corrects Hermes' Feishu identity model.\\n\\nThe core bug is that Hermes previously treated Feishu IDs with the wrong mental model, especially around bot identity versus human identity. In practice this caused Hermes to behave as if Feishu were effectively a single-bot platform and led to incorrect routing assumptions in group mentions, home-channel delivery, and cross-account usage.\\n\\nThe most important correction is this:\\n\\n- `open_id` is **not** the canonical identity of a Feishu bot.\\n- A Feishu bot/app is identified by its configured `app_id`.\\n- For human users, `user_id` is the stable tenant-scoped identifier.\\n- `open_id` is an app-scoped alias for a human user, so the same person can have different `open_id` values under different bot namespaces.\\n\\nThat identity-model mismatch is the root bug. This PR fixes that model first, then builds the routing and tooling changes on top of it.\\n\\n## What follows from that fix\\n\\nOnce Hermes uses the correct Feishu identity model, a few capabilities naturally become possible or necessary:\\n\\n- multi-account Feishu support under one Hermes agent\\n- account-aware session routing\\n- account-aware outbound delivery\\n- correct group-mention handling against the receiving bot account\\n- agent-side Feishu identity/chat inspection via `feishu_id`\\n\\nSo multi-account and ID-aware delivery are not the primary story by themselves; they are the downstream capability expansio\n\n[... tool output truncated for fixture — original was 4163 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01K3yrYegnpxP8nDf8gvsSzP"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "4647 additions for a \"bug fix\" - that's a feature PR wearing a bugfix hat. Let me dig into the actual diff.",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01C9hZc9tuYcb8juESP7WkU2",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 --stat 2>&1\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"cron/scheduler.py | 140 +-\\n gateway/config.py | 276 +++-\\n gateway/delivery.py | 31 +-\\n gateway/platforms/base.py | 4 +-\\n gateway/platforms/feishu.py | 275 +++-\\n gateway/platforms/webhook.py | 37 +-\\n gateway/run.py | 619 +++++++--\\n gateway/session.py | 16 +-\\n gateway/session_context.py | 10 +\\n model_tools.py | 1 +\\n tests/cron/test_scheduler.py | 158 +++\\n .../gateway/test_command_bypass_active_session.py | 12 +\\n tests/gateway/test_config.py | 125 +-\\n tests/gateway/test_feishu.py | 241 +++-\\n tests/gateway/test_feishu_ws_runtime.py | 86 ++\\n tests/gateway/test_runner_fatal_adapter.py | 32 +-\\n tests/gateway/test_session.py | 12 +\\n tests/gateway/test_session_env.py | 43 +-\\n tests/gateway/test_sethome_multi_account.py | 268 ++++\\n tests/gateway/test_shared_thread_sender_prefix.py | 92 ++\\n tests/gateway/test_unauthorized_dm_behavior.py | 137 +-\\n tests/tools/test_feishu_id_tool.py | 606 +++++++++\\n tests/tools/test_send_message_tool.py | 113 ++\\n tools/cronjob_tools.py | 4 +\\n tools/feishu_id_tool.py | 1340 ++++++++++++++++++++\\n tools/send_message_tool.py | 73 +-\\n toolsets.py | 2 +\\n website/docs/user-guide/messaging/feishu.md | 133 +-\\n 28 files changed, 4647 insertions(+), 239 deletions(-)\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01C9hZc9tuYcb8juESP7WkU2"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_013LzgRSQhAqppkoTuFj3LoC",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- gateway/platforms/feishu.py 2>&1 | head -200\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01LgEXZxn6fvcRVnCLNmT1kP",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- gateway/run.py 2>&1 | head -300\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_012AJZhndXjKnzGL55KMHTe1",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_search",
|
||||||
|
"arguments": "{\"query\": \"Feishu Lark open_id vs user_id app-scoped identity model API documentation\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"diff --git a/gateway/platforms/feishu.py b/gateway/platforms/feishu.py\\nindex 7fce74de..8789cd68 100644\\n--- a/gateway/platforms/feishu.py\\n+++ b/gateway/platforms/feishu.py\\n@@ -71,7 +71,9 @@ try:\\n UpdateMessageRequest,\\n UpdateMessageRequestBody,\\n )\\n+ from lark_oapi.core import AccessTokenType, HttpMethod\\n from lark_oapi.core.const import FEISHU_DOMAIN, LARK_DOMAIN\\n+ from lark_oapi.core.model.base_request import BaseRequest\\n from lark_oapi.event.callback.model.p2_card_action_trigger import P2CardActionTriggerResponse\\n from lark_oapi.event.dispatcher_handler import EventDispatcherHandler\\n from lark_oapi.ws import Client as FeishuWSClient\\n@@ -85,6 +87,9 @@ except ImportError:\\n FeishuWSClient = None # type: ignore[assignment]\\n FEISHU_DOMAIN = None # type: ignore[assignment]\\n LARK_DOMAIN = None # type: ignore[assignment]\\n+ AccessTokenType = None # type: ignore[assignment]\\n+ HttpMethod = None # type: ignore[assignment]\\n+ BaseRequest = None # type: ignore[assignment]\\n \\n FEISHU_WEBSOCKET_AVAILABLE = websockets is not None\\n FEISHU_WEBHOOK_AVAILABLE = aiohttp is not None\\n@@ -274,6 +279,7 @@ class FeishuAdapterSettings:\\n encrypt_key: str\\n verification_token: str\\n group_policy: str\\n+ require_mention: bool\\n allowed_group_users: frozenset[str]\\n bot_open_id: str\\n bot_user_id: str\\n@@ -301,6 +307,7 @@ class FeishuGroupRule:\\n \\\"\\\"\\\"Per-group policy rule for controlling which users may interact with the bot.\\\"\\\"\\\"\\n \\n policy: str # \\\"open\\\" | \\\"allowlist\\\" | \\\"blacklist\\\" | \\\"admin_only\\\" | \\\"disabled\\\"\\n+ require_mention: Optional[bool] = None\\n allowlist: set[str] = field(default_factory=set)\\n blacklist: set[str] = field(default\n\n[... tool output truncated for fixture — original was 9031 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_013LzgRSQhAqppkoTuFj3LoC"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"diff --git a/gateway/run.py b/gateway/run.py\\nindex 31fe724f..74bc1a1e 100644\\n--- a/gateway/run.py\\n+++ b/gateway/run.py\\n@@ -241,6 +241,7 @@ if not _configured_cwd or _configured_cwd in (\\\".\\\", \\\"auto\\\", \\\"cwd\\\"):\\n from gateway.config import (\\n Platform,\\n GatewayConfig,\\n+ HomeChannel,\\n load_gateway_config,\\n )\\n from gateway.session import (\\n@@ -534,6 +535,7 @@ class GatewayRunner:\\n def __init__(self, config: Optional[GatewayConfig] = None):\\n self.config = config or load_gateway_config()\\n self.adapters: Dict[Platform, BasePlatformAdapter] = {}\\n+ self._adapters_by_binding: Dict[Any, BasePlatformAdapter] = {}\\n \\n # Load ephemeral config from config.yaml / env vars.\\n # Both are injected at API-call time only and never persisted.\\n@@ -591,8 +593,8 @@ class GatewayRunner:\\n self._pending_approvals: Dict[str, Dict[str, Any]] = {}\\n \\n # Track platforms that failed to connect for background reconnection.\\n- # Key: Platform enum, Value: {\\\"config\\\": platform_config, \\\"attempts\\\": int, \\\"next_retry\\\": float}\\n- self._failed_platforms: Dict[Platform, Dict[str, Any]] = {}\\n+ # Key: (platform, account_id), Value: {\\\"config\\\": platform_config, \\\"attempts\\\": int, \\\"next_retry\\\": float}\\n+ self._failed_platforms: Dict[Any, Dict[str, Any]] = {}\\n \\n # Track pending /update prompt responses per session.\\n # Key: session_key, Value: True when a prompt is waiting for user input.\\n@@ -633,6 +635,142 @@ class GatewayRunner:\\n # Track background tasks to prevent garbage collection mid-execution\\n self._background_tasks: set = set()\\n \\n+ def _adapter_binding_key(\\n+ self,\\n+ platform: Platform,\\n+ account_id: O\n\n[... tool output truncated for fixture — original was 14156 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01LgEXZxn6fvcRVnCLNmT1kP"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"success\": true,\n \"data\": {\n \"web\": [\n {\n \"url\": \"https://open.feishu.cn/document/home/user-identity-introduction/introduction\",\n \"title\": \"User identification concepts - Developer Guides - Documentation\",\n \"description\": \"User identity within a tenant. The user_id of the same Feishu user in tenant A and tenant B is different. user_id is also known as employee_id, ...\",\n \"category\": null\n },\n {\n \"url\": \"https://open.feishu.cn/document/server-docs/contact-v3/user/batch_get_id\",\n \"title\": \"Obtain user ID via email or mobile number - Documentation\",\n \"description\": \"Optional values are: open_id ：Identifies a user to an app. The same user has different Open IDs in different apps. How to get Open ID.\",\n \"category\": null\n },\n {\n \"url\": \"https://open.larkoffice.com/document/server-docs/contact-v3/department/search\",\n \"title\": \"Search for departments - Documentation - Feishu Open Platform\",\n \"description\": \"User ID is usually used to communicate user data between different apps. How to get User ID. Default value: open_id. When the value is user_id , ...\",\n \"category\": null\n },\n {\n \"url\": \"https://open.feishu.cn/document/uAjLw4CM/ukTMukTMukTM/reference/contact-v3/user/create\",\n \"title\": \"Create a user - 服务端 API - Documentation - Feishu Open Platform\",\n \"description\": \"User ID is usually used to communicate user data between different apps. How to get User ID. Default value: open_id. When the value is user_id , ...\",\n \"category\": null\n },\n {\n \"url\": \"https://open.feishu.cn/document/uAjLw4CM/ukTMukTMukTM/reference/contact-v3/department/children\",\n \"title\": \"Obtain the list of sub-departments - Server API \n\n[... tool output truncated for fixture — original was 2037 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_012AJZhndXjKnzGL55KMHTe1"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01HcmboQ91UbC3TgT6nwk6ek",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_extract",
|
||||||
|
"arguments": "{\"urls\": [\"https://open.feishu.cn/document/home/user-identity-introduction/introduction\"]}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01QUVxXbPkFjDSN8tLe45JDE",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- tools/feishu_id_tool.py 2>&1 | head -100\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01KVSbFkSywrEDjRjT8bMz7p",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- gateway/config.py 2>&1 | head -200\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"results\": [\n {\n \"url\": \"https://open.feishu.cn/document/home/user-identity-introduction/introduction\",\n \"title\": \"Feishu Open Platform\",\n \"content\": \"Feedback\\n\\nOnCall\\n\\nCollapse\\n\\nExpand\",\n \"error\": null\n }\n ]\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01HcmboQ91UbC3TgT6nwk6ek"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"diff --git a/tools/feishu_id_tool.py b/tools/feishu_id_tool.py\\nnew file mode 100644\\nindex 00000000..a73dcbe7\\n--- /dev/null\\n+++ b/tools/feishu_id_tool.py\\n@@ -0,0 +1,1340 @@\\n+\\\"\\\"\\\"Feishu identity/chat lookup tool with official API first, local fallback.\\n+\\n+Primary resolution uses the official Feishu APIs for:\\n+- user identity lookup by open_id/user_id/union_id\\n+- chat lookup by chat_id\\n+- chat search by name/query\\n+- chat member enumeration\\n+\\n+When official API credentials are unavailable, or an authoritative lookup does\\n+not succeed, the tool falls back to Hermes' locally observed session metadata\\n+from ``~/.hermes/sessions/sessions.json``.\\n+\\\"\\\"\\\"\\n+\\n+from __future__ import annotations\\n+\\n+import json\\n+import logging\\n+import os\\n+import re\\n+from pathlib import Path\\n+from typing import Any, Dict, List, Optional, Tuple\\n+\\n+from hermes_constants import get_hermes_home\\n+\\n+logger = logging.getLogger(__name__)\\n+\\n+_SESSION_KEY_ACCOUNT_RE = re.compile(r\\\":feishu\\\\[([^\\\\]]+)\\\\]:\\\")\\n+\\n+_ID_PREFIX_MAP = {\\n+ \\\"ou_\\\": \\\"open_id\\\",\\n+ \\\"on_\\\": \\\"union_id\\\",\\n+ \\\"u_\\\": \\\"user_id\\\",\\n+}\\n+\\n+FEISHU_ID_SCHEMA = {\\n+ \\\"name\\\": \\\"feishu_id\\\",\\n+ \\\"description\\\": (\\n+ \\\"Inspect Feishu identities, chats, and session routes for Hermes. \\\"\\n+ \\\"Uses official Feishu APIs when credentials are configured, and falls back \\\"\\n+ \\\"to Hermes' local observed session metadata when needed. Supports chat_id/name \\\"\\n+ \\\"lookup, open_id/user_id/union_id lookup, group member lookup, and session-key resolution.\\\"\\n+ ),\\n+ \\\"parameters\\\": {\\n+ \\\"type\\\": \\\"object\\\",\\n+ \\\"properties\\\": {\\n+ \\\"action\\\": {\\n+ \\\"type\\\": \\\"string\\\",\\n+ \\\"enum\\\": [\\n+ \n\n[... tool output truncated for fixture — original was 3795 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01QUVxXbPkFjDSN8tLe45JDE"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"diff --git a/gateway/config.py b/gateway/config.py\\nindex 7d616592..a263ce01 100644\\n--- a/gateway/config.py\\n+++ b/gateway/config.py\\n@@ -79,13 +79,17 @@ class HomeChannel:\\n platform: Platform\\n chat_id: str\\n name: str # Human-readable name for display\\n+ account_id: Optional[str] = None\\n \\n def to_dict(self) -> Dict[str, Any]:\\n- return {\\n+ payload = {\\n \\\"platform\\\": self.platform.value,\\n \\\"chat_id\\\": self.chat_id,\\n \\\"name\\\": self.name,\\n }\\n+ if self.account_id:\\n+ payload[\\\"account_id\\\"] = self.account_id\\n+ return payload\\n \\n @classmethod\\n def from_dict(cls, data: Dict[str, Any]) -> \\\"HomeChannel\\\":\\n@@ -93,6 +97,7 @@ class HomeChannel:\\n platform=Platform(data[\\\"platform\\\"]),\\n chat_id=str(data[\\\"chat_id\\\"]),\\n name=data.get(\\\"name\\\", \\\"Home\\\"),\\n+ account_id=str(data[\\\"account_id\\\"]).strip() if data.get(\\\"account_id\\\") else None,\\n )\\n \\n \\n@@ -175,6 +180,16 @@ class PlatformConfig:\\n home_channel = None\\n if \\\"home_channel\\\" in data:\\n home_channel = HomeChannel.from_dict(data[\\\"home_channel\\\"])\\n+ extra = data.get(\\\"extra\\\", {})\\n+ if not isinstance(extra, dict):\\n+ extra = {}\\n+ # Preserve unknown top-level keys inside ``extra`` so platform-specific\\n+ # schemas can evolve without forcing a core dataclass change first.\\n+ extra = dict(extra)\\n+ for key, value in data.items():\\n+ if key in {\\\"enabled\\\", \\\"token\\\", \\\"api_key\\\", \\\"home_channel\\\", \\\"reply_to_mode\\\", \\\"extra\\\"}:\\n+ continue\\n+ extra[key] = value\\n \\n return cls(\\n \n\n[... tool output truncated for fixture — original was 8772 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01KVSbFkSywrEDjRjT8bMz7p"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_013pi1eiBL8aos8si1TesgJQ",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "web_extract",
|
||||||
|
"arguments": "{\"urls\": [\"https://open.larkoffice.com/document/home/user-identity-introduction/introduction\"]}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01YRkjL2FzF8kkd8mCFKhvLp",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && grep -n \\\"open_id\\\\|bot_open_id\\\\|bot_user_id\\\" gateway/platforms/feishu.py | head -40\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\n \"results\": [\n {\n \"url\": \"https://open.larkoffice.com/document/home/user-identity-introduction/introduction\",\n \"title\": \"Feishu Open Platform\",\n \"content\": \"Feedback\\n\\nOnCall\\n\\nCollapse\\n\\nExpand\",\n \"error\": null\n }\n ]\n}",
|
||||||
|
"tool_call_id": "toolu_bdrk_013pi1eiBL8aos8si1TesgJQ"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"230: \\\"open_id\\\",\\n278: bot_open_id: str\\n279: bot_user_id: str\\n539: str(element.get(\\\"open_id\\\", \\\"\\\")).strip()\\n1122: bot_open_id=os.getenv(\\\"FEISHU_BOT_OPEN_ID\\\", \\\"\\\").strip(),\\n1123: bot_user_id=os.getenv(\\\"FEISHU_BOT_USER_ID\\\", \\\"\\\").strip(),\\n1177: self._bot_open_id = settings.bot_open_id\\n1178: self._bot_user_id = settings.bot_user_id\\n1952: open_id = str(getattr(operator, \\\"open_id\\\", \\\"\\\") or \\\"\\\")\\n1953: if not chat_id or not open_id:\\n1954: logger.debug(\\\"[Feishu] Card action missing chat_id or operator open_id, dropping\\\")\\n1987: sender_id = SimpleNamespace(open_id=open_id, user_id=None, union_id=None)\\n1989: user_name = sender_profile.get(\\\"user_name\\\") or open_id\\n2013: sender_id = SimpleNamespace(open_id=open_id, user_id=None, union_id=None)\\n2033: logger.info(\\\"[Feishu] Routing card action %r from %s in %s as synthetic command\\\", action_tag, open_id, chat_id)\\n2903: open_id = getattr(sender_id, \\\"open_id\\\", None) or None\\n2906: primary_id = open_id or user_id\\n2917: ID-type detection mirrors openclaw: ou_ → open_id, on_ → union_id, else user_id.\\n2934: id_type = \\\"open_id\\\"\\n3011: sender_open_id = getattr(sender_id, \\\"open_id\\\", None)\\n3013: sender_ids = {sender_open_id, sender_user_id} - {None}\\n3064: mention_open_id = getattr(mention_id, \\\"open_id\\\", None)\\n3068: if self._bot_open_id and mention_open_id == self._bot_open_id:\\n3070: if self._bot_user_id and mention_user_id == self._bot_user_id:\\n3080: if self._bot_open_id and self._bot_open_id in mentioned_ids:\\n3082: if self._bot_user_id and self._bot_user_id in mentioned_ids:\\n\n\n[... tool output truncated for fixture — original was 2530 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01YRkjL2FzF8kkd8mCFKhvLp"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01XPRM4pSLsPSJ6TrC9FjfjA",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 --stat -- tools/feishu_id_tool.py gateway/run.py gateway/config.py cron/scheduler.py tools/send_message_tool.py tools/cronjob_tools.py toolsets.py model_tools.py 2>&1\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"cron/scheduler.py | 140 ++++-\\n gateway/config.py | 276 ++++++++-\\n gateway/run.py | 619 ++++++++++++++++----\\n model_tools.py | 1 +\\n tools/cronjob_tools.py | 4 +\\n tools/feishu_id_tool.py | 1340 ++++++++++++++++++++++++++++++++++++++++++++\\n tools/send_message_tool.py | 73 ++-\\n toolsets.py | 2 +\\n 8 files changed, 2314 insertions(+), 141 deletions(-)\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01XPRM4pSLsPSJ6TrC9FjfjA"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01CW7jkdTW628KrHJQG9j9Dz",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- cron/scheduler.py 2>&1 | head -150\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01XkuWrYyTcNSvCpYAQ1aLxu",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- tools/send_message_tool.py 2>&1 | head -150\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"diff --git a/cron/scheduler.py b/cron/scheduler.py\\nindex e6db77c0..7f9b9e66 100644\\n--- a/cron/scheduler.py\\n+++ b/cron/scheduler.py\\n@@ -84,26 +84,25 @@ def _resolve_delivery_target(job: dict) -> Optional[dict]:\\n \\n if deliver == \\\"origin\\\":\\n if origin:\\n- return {\\n+ target = {\\n \\\"platform\\\": origin[\\\"platform\\\"],\\n \\\"chat_id\\\": str(origin[\\\"chat_id\\\"]),\\n \\\"thread_id\\\": origin.get(\\\"thread_id\\\"),\\n }\\n+ if origin.get(\\\"account_id\\\"):\\n+ target[\\\"account_id\\\"] = origin.get(\\\"account_id\\\")\\n+ return target\\n # Origin missing (e.g. job created via API/script) — try each\\n # platform's home channel as a fallback instead of silently dropping.\\n- for platform_name in (\\\"matrix\\\", \\\"telegram\\\", \\\"discord\\\", \\\"slack\\\", \\\"bluebubbles\\\"):\\n- chat_id = os.getenv(f\\\"{platform_name.upper()}_HOME_CHANNEL\\\", \\\"\\\")\\n- if chat_id:\\n+ for platform_name in (\\\"matrix\\\", \\\"telegram\\\", \\\"discord\\\", \\\"slack\\\", \\\"bluebubbles\\\", \\\"feishu\\\"):\\n+ target = _resolve_home_delivery_target(platform_name, origin=None)\\n+ if target:\\n logger.info(\\n \\\"Job '%s' has deliver=origin but no origin; falling back to %s home channel\\\",\\n job.get(\\\"name\\\", job.get(\\\"id\\\", \\\"?\\\")),\\n platform_name,\\n )\\n- return {\\n- \\\"platform\\\": platform_name,\\n- \\\"chat_id\\\": chat_id,\\n- \\\"thread_id\\\": None,\\n- }\\n+ return target\\n return None\\n \\n if \\\":\\\" in deliver:\\n@@ -131,31 +130,113 @@ def _resolve_\n\n[... tool output truncated for fixture — original was 5780 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01CW7jkdTW628KrHJQG9j9Dz"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"diff --git a/tools/send_message_tool.py b/tools/send_message_tool.py\\nindex a2b3e984..275cd764 100644\\n--- a/tools/send_message_tool.py\\n+++ b/tools/send_message_tool.py\\n@@ -99,6 +99,45 @@ def _handle_list():\\n return json.dumps(_error(f\\\"Failed to load channel directory: {e}\\\"))\\n \\n \\n+def _get_config_callable(config, name):\\n+ \\\"\\\"\\\"Return a real config helper method, ignoring MagicMock auto-attributes.\\\"\\\"\\\"\\n+ try:\\n+ instance_attr = vars(config).get(name)\\n+ except Exception:\\n+ instance_attr = None\\n+ if callable(instance_attr):\\n+ return instance_attr\\n+\\n+ class_attr = getattr(type(config), name, None)\\n+ if callable(class_attr):\\n+ return getattr(config, name)\\n+ return None\\n+\\n+\\n+def _resolve_platform_config(config, platform, account_id=None):\\n+ \\\"\\\"\\\"Support both GatewayConfig and lightweight test doubles.\\\"\\\"\\\"\\n+ getter = _get_config_callable(config, \\\"get_platform_config\\\")\\n+ if getter:\\n+ try:\\n+ return getter(platform, account_id=account_id)\\n+ except TypeError:\\n+ return getter(platform)\\n+\\n+ platforms = getattr(config, \\\"platforms\\\", {}) or {}\\n+ return platforms.get(platform)\\n+\\n+\\n+def _resolve_home_channel(config, platform, account_id=None):\\n+ \\\"\\\"\\\"Support both GatewayConfig and lightweight test doubles.\\\"\\\"\\\"\\n+ getter = _get_config_callable(config, \\\"get_home_channel\\\")\\n+ if getter:\\n+ try:\\n+ return getter(platform, account_id=account_id)\\n+ except TypeError:\\n+ return getter(platform)\\n+ return None\\n+\\n+\\n def _handle_send(args):\\n \\\"\\\"\\\"Send a message to a platform target.\\\"\\\"\\\"\\n target = args.get(\\\"target\\\", \\\"\\\")\\n@@ -168,9 +207,12 @@ def _handle_send(args)\n\n[... tool output truncated for fixture — original was 5807 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01XkuWrYyTcNSvCpYAQ1aLxu"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_015UcVYQRoZkvXo2X3WVE383",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- gateway/run.py 2>&1 | wc -l\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_019zsHzqWjkBuyBKHSdRvG7R",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- gateway/run.py 2>&1 | grep \\\"^+\\\" | grep -c \\\"account_id\\\\|binding\\\\|_ensure_adapter\\\\|_register_adapter\\\\|_unregister\\\\|_get_adapter\\\\|_all_adapter\\\\|_iter_enabled\\\\|_binding_label\\\\|_adapters_by_binding\\\"\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01HPmygaWCphhGAbJLgDgz3v",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- gateway/run.py 2>&1 | grep -c \\\"^[+-]\\\" \"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"1162\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_015UcVYQRoZkvXo2X3WVE383"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"154\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_019zsHzqWjkBuyBKHSdRvG7R"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"621\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01HPmygaWCphhGAbJLgDgz3v"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_017ie77GPPRhyGna7raJCzS4",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- gateway/run.py 2>&1 | grep \\\"^+\\\" | grep -v \\\"^+++\\\" | grep -c \\\"feishu_id\\\\|feishu\\\\|sethome.*feishu\\\\|FEISHU\\\"\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01XaiiozutypWDXfZDsuRbx5",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- gateway/run.py 2>&1 | tail -500 | head -300\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"9\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_017ie77GPPRhyGna7raJCzS4"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"- adapter = self.adapters.get(source.platform)\\n+ adapter = self._get_adapter_for_source(source)\\n if adapter:\\n adapter._pending_messages[_quick_key] = event\\n return None\\n@@ -3167,7 +3397,7 @@ class GatewayRunner:\\n and platform_name not in policy.notify_exclude_platforms\\n )\\n if should_notify:\\n- adapter = self.adapters.get(source.platform)\\n+ adapter = self._get_adapter_for_source(source)\\n if adapter:\\n if reset_reason == \\\"suspended\\\":\\n reason_text = \\\"previous session was stopped or interrupted\\\"\\n@@ -3491,9 +3721,8 @@ class GatewayRunner:\\n # Skip for webhooks - they deliver directly to configured targets (github_comment, etc.)\\n if not history and source.platform and source.platform != Platform.LOCAL and source.platform != Platform.WEBHOOK:\\n platform_name = source.platform.value\\n- env_key = f\\\"{platform_name.upper()}_HOME_CHANNEL\\\"\\n- if not os.getenv(env_key):\\n- adapter = self.adapters.get(source.platform)\\n+ if not self._has_home_channel_for_source(source):\\n+ adapter = self._get_adapter_for_source(source)\\n if adapter:\\n await adapter.send(\\n source.chat_id,\\n@@ -3537,6 +3766,132 @@ class GatewayRunner:\\n if message_text is None:\\n return\\n \\n+ if event.media_urls:\\n+ image_paths = []\\n+ for i, path in enumerate(event.media_urls):\\n+ # Check media_types if available; otherwise infer from me\n\n[... tool output truncated for fixture — original was 16254 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01XaiiozutypWDXfZDsuRbx5"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01THagoH9sx4am3t1y9fkQcS",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && grep -n \\\"_enrich_message_with_vision\\\\|_enrich_message_with_transcription\\\\|reply_to_text\\\\|preprocess_context_references_async\\\\|_TEXT_EXTENSIONS\\\\|MessageType.DOCUMENT\\\" gateway/run.py | head -20\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01LbDoTuApRArrf6MXfQ8eBQ",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- gateway/run.py 2>&1 | grep \\\"^+\\\" | grep -c \\\"vision\\\\|transcri\\\\|reply_to_text\\\\|context_references\\\\|DOCUMENT\\\\|_TEXT_EXTENSION\\\\|_stt_\\\\|voice message\\\"\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"2974: message_text = await self._enrich_message_with_vision(\\n2980: message_text = await self._enrich_message_with_transcription(\\n3013: if event.media_urls and event.message_type == MessageType.DOCUMENT:\\n3016: _TEXT_EXTENSIONS = {\\\".txt\\\", \\\".md\\\", \\\".csv\\\", \\\".log\\\", \\\".json\\\", \\\".xml\\\", \\\".yaml\\\", \\\".yml\\\", \\\".toml\\\", \\\".ini\\\", \\\".cfg\\\"}\\n3023: if _ext in _TEXT_EXTENSIONS:\\n3054: if getattr(event, \\\"reply_to_text\\\", None) and event.reply_to_message_id:\\n3055: reply_snippet = event.reply_to_text[:500]\\n3066: from agent.context_references import preprocess_context_references_async\\n3074: _ctx_result = await preprocess_context_references_async(\\n6978: async def _enrich_message_with_vision(\\n7046: async def _enrich_message_with_transcription(\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01THagoH9sx4am3t1y9fkQcS"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"21\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01LbDoTuApRArrf6MXfQ8eBQ"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01Wdw4Lho816SVHt3BJQkMjL",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git log --oneline origin/main -- gateway/run.py | head -5\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01BCB9orVkTcaHTtYwjcYMJW",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git show origin/main:gateway/run.py | grep -c \\\"_enrich_message_with_vision\\\\|_enrich_message_with_transcription\\\\|reply_to_text.*reply_to_message_id\\\\|preprocess_context_references_async\\\\|MessageType.DOCUMENT\\\"\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"964ef681 fix(gateway): improve /restart response with fallback instructions\\n276d20e6 fix(gateway): /restart uses service restart under systemd instead of detached subprocess\\n8a64f3e3 feat(gateway): notify /restart requester when gateway comes back online\\na0cd2c53 fix(gateway): verbose tool progress no longer truncates args when tool_preview_length is 0 (#8735)\\n3cd6cbee feat: add /debug slash command for all platforms\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01Wdw4Lho816SVHt3BJQkMjL"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"8\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01BCB9orVkTcaHTtYwjcYMJW"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01AGkGk8zcyZ8zUGxHGBGYL5",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git show origin/main:gateway/run.py | grep -n \\\"_enrich_message_with_vision\\\\|_enrich_message_with_transcription\\\\|reply_to_text.*reply_to_message_id\\\\|preprocess_context_references_async\\\\|MessageType.DOCUMENT\\\" | head -20\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"2974: message_text = await self._enrich_message_with_vision(\\n2980: message_text = await self._enrich_message_with_transcription(\\n3013: if event.media_urls and event.message_type == MessageType.DOCUMENT:\\n3054: if getattr(event, \\\"reply_to_text\\\", None) and event.reply_to_message_id:\\n3066: from agent.context_references import preprocess_context_references_async\\n3074: _ctx_result = await preprocess_context_references_async(\\n6978: async def _enrich_message_with_vision(\\n7046: async def _enrich_message_with_transcription(\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01AGkGk8zcyZ8zUGxHGBGYL5"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_012CzR8xizYCPkv2MLJpRixn",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git log --oneline origin/main..pr-8388 2>&1\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"05b87f47 fix(gateway): remove stale shared-thread sender check\\nbc62efd6 feat(feishu): support multi-account routing and id-aware delivery\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_012CzR8xizYCPkv2MLJpRixn"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_019sat1paLRz2Y4vkrrkF6vu",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- gateway/session_context.py 2>&1\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01RnHYKvDVUjGw2scWn7J7uN",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- gateway/delivery.py 2>&1\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"diff --git a/gateway/session_context.py b/gateway/session_context.py\\nindex b9fdcdfa..56d977c1 100644\\n--- a/gateway/session_context.py\\n+++ b/gateway/session_context.py\\n@@ -43,19 +43,23 @@ from contextvars import ContextVar\\n # ---------------------------------------------------------------------------\\n \\n _SESSION_PLATFORM: ContextVar[str] = ContextVar(\\\"HERMES_SESSION_PLATFORM\\\", default=\\\"\\\")\\n+_SESSION_ACCOUNT_ID: ContextVar[str] = ContextVar(\\\"HERMES_SESSION_ACCOUNT_ID\\\", default=\\\"\\\")\\n _SESSION_CHAT_ID: ContextVar[str] = ContextVar(\\\"HERMES_SESSION_CHAT_ID\\\", default=\\\"\\\")\\n _SESSION_CHAT_NAME: ContextVar[str] = ContextVar(\\\"HERMES_SESSION_CHAT_NAME\\\", default=\\\"\\\")\\n _SESSION_THREAD_ID: ContextVar[str] = ContextVar(\\\"HERMES_SESSION_THREAD_ID\\\", default=\\\"\\\")\\n _SESSION_USER_ID: ContextVar[str] = ContextVar(\\\"HERMES_SESSION_USER_ID\\\", default=\\\"\\\")\\n+_SESSION_USER_ID_ALT: ContextVar[str] = ContextVar(\\\"HERMES_SESSION_USER_ID_ALT\\\", default=\\\"\\\")\\n _SESSION_USER_NAME: ContextVar[str] = ContextVar(\\\"HERMES_SESSION_USER_NAME\\\", default=\\\"\\\")\\n _SESSION_KEY: ContextVar[str] = ContextVar(\\\"HERMES_SESSION_KEY\\\", default=\\\"\\\")\\n \\n _VAR_MAP = {\\n \\\"HERMES_SESSION_PLATFORM\\\": _SESSION_PLATFORM,\\n+ \\\"HERMES_SESSION_ACCOUNT_ID\\\": _SESSION_ACCOUNT_ID,\\n \\\"HERMES_SESSION_CHAT_ID\\\": _SESSION_CHAT_ID,\\n \\\"HERMES_SESSION_CHAT_NAME\\\": _SESSION_CHAT_NAME,\\n \\\"HERMES_SESSION_THREAD_ID\\\": _SESSION_THREAD_ID,\\n \\\"HERMES_SESSION_USER_ID\\\": _SESSION_USER_ID,\\n+ \\\"HERMES_SESSION_USER_ID_ALT\\\": _SESSION_USER_ID_ALT,\\n \\\"HERMES_SESSION_USER_NAME\\\": _SESSION_USER_NAME,\\n \\\"HERMES_SESSION_KEY\\\": _SESSION_KEY,\\n }\\n@@ -63,10 +67,12 @@ _VAR_MAP = {\\n \\n def set_session_vars(\\n platform: str = \\\"\\\",\\n+ account_id: str = \\\"\\\",\\n chat_\n\n[... tool output truncated for fixture — original was 2894 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_019sat1paLRz2Y4vkrrkF6vu"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"diff --git a/gateway/delivery.py b/gateway/delivery.py\\nindex d7fa6afd..e357b93e 100644\\n--- a/gateway/delivery.py\\n+++ b/gateway/delivery.py\\n@@ -37,6 +37,7 @@ class DeliveryTarget:\\n - \\\"telegram:123456\\\" → specific Telegram chat\\n \\\"\\\"\\\"\\n platform: Platform\\n+ account_id: Optional[str] = None\\n chat_id: Optional[str] = None # None means use home channel\\n thread_id: Optional[str] = None\\n is_origin: bool = False\\n@@ -59,6 +60,7 @@ class DeliveryTarget:\\n if origin:\\n return cls(\\n platform=origin.platform,\\n+ account_id=origin.account_id,\\n chat_id=origin.chat_id,\\n thread_id=origin.thread_id,\\n is_origin=True,\\n@@ -122,6 +124,7 @@ class DeliveryRouter:\\n \\\"\\\"\\\"\\n self.config = config\\n self.adapters = adapters or {}\\n+ self.account_adapters: Dict[tuple[Platform, Optional[str]], Any] = {}\\n self.output_dir = get_hermes_home() / \\\"cron\\\" / \\\"output\\\"\\n \\n async def deliver(\\n@@ -228,14 +231,29 @@ class DeliveryRouter:\\n metadata: Optional[Dict[str, Any]]\\n ) -> Dict[str, Any]:\\n \\\"\\\"\\\"Deliver content to a messaging platform.\\\"\\\"\\\"\\n- adapter = self.adapters.get(target.platform)\\n- \\n+ if not target.chat_id:\\n+ home = self.config.get_home_channel(target.platform, account_id=target.account_id)\\n+ if home:\\n+ target = DeliveryTarget(\\n+ platform=target.platform,\\n+ account_id=getattr(home, \\\"account_id\\\", None) or target.account_id,\\n+ chat_id=home.chat_id,\\n+ thread_id=target.thread_id,\\n+ \n\n[... tool output truncated for fixture — original was 2946 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01RnHYKvDVUjGw2scWn7J7uN"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01MuFViB9E4hgh1Z8qRdQ3Kt",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- tests/tools/test_feishu_id_tool.py 2>&1 | head -100\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01UUFRbWLBdD2nKzLTgij43u",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && wc -l tools/feishu_id_tool.py 2>/dev/null; git show pr-8388:tools/feishu_id_tool.py 2>/dev/null | wc -l\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"diff --git a/tests/tools/test_feishu_id_tool.py b/tests/tools/test_feishu_id_tool.py\\nnew file mode 100644\\nindex 00000000..c1e90daf\\n--- /dev/null\\n+++ b/tests/tools/test_feishu_id_tool.py\\n@@ -0,0 +1,606 @@\\n+import json\\n+from datetime import datetime\\n+from pathlib import Path\\n+from types import SimpleNamespace\\n+from unittest.mock import patch\\n+\\n+from gateway.config import Platform\\n+from gateway.session import SessionEntry, SessionSource\\n+from tools.feishu_id_tool import feishu_id_tool\\n+\\n+\\n+class _FakeResponse:\\n+ def __init__(self, *, success: bool, data=None, code=0, msg=\\\"ok\\\"):\\n+ self._success = success\\n+ self.data = data\\n+ self.code = code\\n+ self.msg = msg\\n+\\n+ def success(self):\\n+ return self._success\\n+\\n+\\n+def _make_fake_client(\\n+ *,\\n+ user=None,\\n+ chat=None,\\n+ members=None,\\n+ batch_users=None,\\n+ search_chats=None,\\n+):\\n+ def _user_get(_request):\\n+ if user is None:\\n+ return _FakeResponse(success=False, code=404, msg=\\\"user not found\\\")\\n+ return _FakeResponse(success=True, data=SimpleNamespace(user=user))\\n+\\n+ def _user_batch(_request):\\n+ if batch_users is None:\\n+ return _FakeResponse(success=False, code=404, msg=\\\"batch not found\\\")\\n+ return _FakeResponse(success=True, data=SimpleNamespace(items=batch_users))\\n+\\n+ def _chat_get(_request):\\n+ if chat is None:\\n+ return _FakeResponse(success=False, code=404, msg=\\\"chat not found\\\")\\n+ return _FakeResponse(success=True, data=chat)\\n+\\n+ def _chat_search(_request):\\n+ if search_chats is None:\\n+ return _FakeResponse(success=False, code=404, msg=\\\"search not found\\\")\\n+ return _FakeResponse(success=True\n\n[... tool output truncated for fixture — original was 3324 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01MuFViB9E4hgh1Z8qRdQ3Kt"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"1340\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01UUFRbWLBdD2nKzLTgij43u"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01DMDwTRn33PPs873SPDLihR",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 2>&1 | grep -c \\\"_get_config_callable\\\\|_resolve_platform_config\\\\|_resolve_home_channel\\\"\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01THAT7RkuypsXBnDyFxztjZ",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 2>&1 | grep \\\"^+.*def _get_config_callable\\\\|^+.*def _resolve_platform_config\\\\|^+.*def _resolve_home_channel\\\"\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"24\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01DMDwTRn33PPs873SPDLihR"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"+def _get_config_callable(config, name):\\n+def _resolve_platform_config(config, platform, account_id=None):\\n+def _resolve_home_channel(config, platform, account_id=None):\\n+def _get_config_callable(config, name):\\n+def _resolve_platform_config(config, account_id: Optional[str] = None):\\n+def _get_config_callable(config, name):\\n+def _resolve_platform_config(config, platform, account_id=None):\\n+def _resolve_home_channel(config, platform, account_id=None):\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_01THAT7RkuypsXBnDyFxztjZ"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"tool_calls": [
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_012p43DPonvSLugt3FaYoJNv",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git show 05b87f47 --stat 2>&1\"}"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "toolu_bdrk_01VkHMceN4tHnGDcWV4tDUt9",
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "terminal",
|
||||||
|
"arguments": "{\"command\": \"cd /home/user/.hermes/hermes-agent && git diff origin/main...pr-8388 -- tests/gateway/test_shared_thread_sender_prefix.py 2>&1 | head -50\"}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"commit 05b87f47911ae28bdad14d2281ad454b8587aa2a\\nAuthor: contributor <contributor@example.com>\\nDate: Mon Apr 13 13:18:48 2026 +0800\\n\\n fix(gateway): remove stale shared-thread sender check\\n\\n gateway/run.py | 2 -\\n tests/gateway/test_shared_thread_sender_prefix.py | 92 +++++++++++++++++++++++\\n 2 files changed, 92 insertions(+), 2 deletions(-)\", \"exit_code\": 0, \"error\": null}",
|
||||||
|
"tool_call_id": "toolu_bdrk_012p43DPonvSLugt3FaYoJNv"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"content": "{\"output\": \"diff --git a/tests/gateway/test_shared_thread_sender_prefix.py b/tests/gateway/test_shared_thread_sender_prefix.py\\nnew file mode 100644\\nindex 00000000..777db5c5\\n--- /dev/null\\n+++ b/tests/gateway/test_shared_thread_sender_prefix.py\\n@@ -0,0 +1,92 @@\\n+import importlib\\n+import sys\\n+import types\\n+from datetime import datetime\\n+from types import SimpleNamespace\\n+from unittest.mock import AsyncMock, MagicMock\\n+\\n+import pytest\\n+\\n+from gateway.config import GatewayConfig, Platform, PlatformConfig\\n+from gateway.platforms.base import MessageEvent\\n+from gateway.session import SessionEntry, SessionSource\\n+\\n+\\contributor@example.com\\n+async def test_handle_message_with_agent_shared_thread_prefixes_sender_once(monkeypatch):\\n+ fake_dotenv = types.ModuleType(\\\"dotenv\\\")\\n+ fake_dotenv.load_dotenv = lambda *args, **kwargs: None\\n+ monkeypatch.setitem(sys.modules, \\\"dotenv\\\", fake_dotenv)\\n+\\n+ gateway_run = importlib.import_module(\\\"gateway.run\\\")\\n+ GatewayRunner = gateway_run.GatewayRunner\\n+\\n+ runner = object.__new__(GatewayRunner)\\n+ runner.config = GatewayConfig(\\n+ platforms={Platform.TELEGRAM: PlatformConfig(enabled=True, token=\\\"fake-token\\\")}\\n+ )\\n+ adapter = MagicMock()\\n+ adapter.send = AsyncMock()\\n+ adapter.stop_typing = AsyncMock()\\n+ runner.adapters = {Platform.TELEGRAM: adapter}\\n+ runner._voice_mode = {}\\n+ runner.hooks = SimpleNamespace(emit=AsyncMock(), loaded_hooks=False)\\n+ runner.session_store = MagicMock()\\n+ runner.session_store.get_or_create_session.return_value = SessionEntry(\\n+ session_key=\\\"agent:main:telegram:group:-1001:17585\\\",\\n+ session_id=\\\"sess-1\\\",\\n+ created_at=datetime.now(),\\n+ updated_at=datetime.now(),\\n+ platform=Platform.T\n\n[... tool output truncated for fixture — original was 2006 chars]",
|
||||||
|
"tool_call_id": "toolu_bdrk_01VkHMceN4tHnGDcWV4tDUt9"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "Here's my review:\n\nThe core claim about Feishu's identity model is technically correct. Feishu's own docs confirm: \"open_id identifies a user to an app. The same user has different Open IDs in different apps.\" So yes, `open_id` is app-scoped, not a canonical identity.\n\nBut the rest of this PR is a bait-and-switch.\n\n**What it claims to be:** A Feishu identity model bugfix.\n\n**What it actually is:** A 4647-line feature PR that adds:\n\n→ Full multi-account Feishu support (account-aware routing, binding keys, adapter registry overhaul in gateway/run.py)\n→ A brand new 1340-line `feishu_id` tool (user/chat/member lookup via Feishu API)\n→ Thread-local websocket proxy refactoring for concurrent bot connections\n→ Account-aware delivery, session context, cron scheduler changes\n→ New session context variables (`HERMES_SESSION_ACCOUNT_ID`, `USER_ID_ALT`)\n→ A separate unrelated fix (\"remove stale shared-thread sender check\") bundled in\n\n**The \"bug\" doesn't actually affect anyone.** Current single-bot Feishu deployments work fine. `open_id` is technically an app-scoped alias, but when you only have one app, it's de facto unique and stable. The identity model is imprecise in naming but functionally correct for the existing use case.\n\n**Code quality concerns:**\n→ `_get_config_callable`, `_resolve_platform_config`, and `_resolve_home_channel` are copy-pasted 3 separate times across `cron/scheduler.py`, `tools/send_message_tool.py`, and `tools/feishu_id_tool.py` instead of living in a shared module\n→ `_ensure_adapter_registry()` has defensive `not hasattr` checks that suggest the multi-account plumbing was bolted on rather than designed in\n→ 619 lines of changes to gateway/run.py, most of which is adapter-binding indirection for a feature nobody requested\n\n**Recommendation:** Close it. The identity model observation is valid as documentation/knowledge, but this PR uses it to justify smuggling in a massive multi-account feature. If multi-account Feishu is ever needed, it should be a planned feature with proper design review, not a 4600-line \"bugfix.\" The actual identity correction (if wanted) would be maybe 50 lines of renaming/comments."
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
File diff suppressed because one or more lines are too long
0
scripts/compression_eval/probes/.gitkeep
Normal file
0
scripts/compression_eval/probes/.gitkeep
Normal file
0
scripts/compression_eval/results/.gitkeep
Normal file
0
scripts/compression_eval/results/.gitkeep
Normal file
28
scripts/compression_eval/run_eval.py
Executable file
28
scripts/compression_eval/run_eval.py
Executable file
|
|
@ -0,0 +1,28 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Compression eval — entry point (placeholder).
|
||||||
|
|
||||||
|
The implementation is tracked in DESIGN.md. This script currently only
|
||||||
|
prints a pointer to the design doc so nobody mistakes an unimplemented
|
||||||
|
harness for a broken one.
|
||||||
|
|
||||||
|
See scripts/compression_eval/DESIGN.md for the full proposal.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
_DESIGN = Path(__file__).parent / "DESIGN.md"
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
print("compression_eval: not implemented yet")
|
||||||
|
print(f"See {_DESIGN} for the proposed design and open questions.")
|
||||||
|
print()
|
||||||
|
print("Implementation is landing in follow-up PRs, one fixture at a time.")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
370
scripts/compression_eval/scrub_fixtures.py
Executable file
370
scripts/compression_eval/scrub_fixtures.py
Executable file
|
|
@ -0,0 +1,370 @@
|
||||||
|
"""One-shot fixture scrubber for scripts/compression_eval/fixtures/.
|
||||||
|
|
||||||
|
Source: ~/.hermes/sessions/<file>.jsonl
|
||||||
|
Output: .worktrees/.../scripts/compression_eval/fixtures/<name>.json
|
||||||
|
|
||||||
|
Scrubbing passes:
|
||||||
|
1. agent.redact.redact_sensitive_text — API keys, tokens, connection strings
|
||||||
|
2. Username paths — /home/teknium/ → /home/user/, ~/.hermes/ preserved as-is
|
||||||
|
(that path is universal)
|
||||||
|
3. Personal handles — "Teknium"/"teknium"/"teknium1" → "user"
|
||||||
|
4. Reasoning scratchpads — strip <REASONING_SCRATCHPAD>...</REASONING_SCRATCHPAD>
|
||||||
|
blocks and <think>...</think> tags (personality leakage risk)
|
||||||
|
5. session_meta line — drop entirely, we only need the messages
|
||||||
|
6. User message personality — lightly paraphrase the first user message to keep
|
||||||
|
task intent while removing "vibe"; subsequent user turns kept verbatim
|
||||||
|
since they're short instructions
|
||||||
|
|
||||||
|
The fixture format matches DESIGN.md:
|
||||||
|
{
|
||||||
|
"name": "...",
|
||||||
|
"description": "...",
|
||||||
|
"model": "...", # best guess from original session
|
||||||
|
"context_length": 200000,
|
||||||
|
"messages": [...], # OpenAI-format, only role/content/tool_calls/tool_call_id/tool_name
|
||||||
|
"notes": "Scrubbed from ~/.hermes/sessions/... on 2026-04-24"
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Dict, List
|
||||||
|
|
||||||
|
# Resolve the hermes-agent checkout relative to this script so agent.redact
|
||||||
|
# imports cleanly whether we run from a worktree or a main clone.
|
||||||
|
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
sys.path.insert(0, str(_REPO_ROOT))
|
||||||
|
from agent.redact import redact_sensitive_text # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
|
SESSION_DIR = Path.home() / ".hermes" / "sessions"
|
||||||
|
# Resolve FIXTURES_DIR relative to this script so the scrubber runs the
|
||||||
|
# same way inside a worktree, a main checkout, or from a contributor's
|
||||||
|
# clone at a different path.
|
||||||
|
FIXTURES_DIR = Path(__file__).resolve().parent / "fixtures"
|
||||||
|
|
||||||
|
# (source_file, output_name, description, user_first_paraphrase, model_guess, context_length, truncate_at)
|
||||||
|
# truncate_at: keep messages[:truncate_at] (None = keep all). Applied BEFORE
|
||||||
|
# orphan-empty-assistant cleanup.
|
||||||
|
SPECS = [
|
||||||
|
(
|
||||||
|
"20260321_060441_fef7be92.jsonl",
|
||||||
|
"feature-impl-context-priority",
|
||||||
|
"~75-turn feature-impl: user asks how multiple project-context files "
|
||||||
|
"(.hermes.md / AGENTS.md / CLAUDE.md / .cursorrules) are handled when "
|
||||||
|
"all are present; agent investigates the codebase, designs a priority "
|
||||||
|
"order, patches the loader + tests, live-tests with a scenario "
|
||||||
|
"directory, commits to a feature branch, opens a PR, and merges after "
|
||||||
|
"approval. Exercises investigate → decide → implement → verify → "
|
||||||
|
"ship flow with clear artifact trail (2 files modified, 1 PR).",
|
||||||
|
(
|
||||||
|
"If .hermes.md, AGENTS.md, CLAUDE.md, and .cursorrules all exist in "
|
||||||
|
"the same directory, does the agent load all of them or pick one? "
|
||||||
|
"Use the hermes-agent-dev skill to check."
|
||||||
|
),
|
||||||
|
"anthropic/claude-sonnet-4.6",
|
||||||
|
200000,
|
||||||
|
74, # cut at "Merged and pulled. Main is current." — drops trailing unrelated cron-delivery messages
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"20260412_233741_3f2119a8.jsonl",
|
||||||
|
"debug-session-feishu-id-model",
|
||||||
|
"~60-turn debug/triage PR-review session: a third-party bug report "
|
||||||
|
"says the gateway's Feishu adapter misuses the open_id / union_id / "
|
||||||
|
"user_id identity model (open_id is app-scoped, not the bot's "
|
||||||
|
"canonical ID). An open community PR (#8388) tries to fix it. Agent "
|
||||||
|
"reviews the PR against current main, fetches upstream Feishu/Lark "
|
||||||
|
"identity docs, and produces a decision. Exercises long tool-heavy "
|
||||||
|
"context with PR diffs, upstream docs, and a clear decision at the "
|
||||||
|
"end — the classic 'can the summary still name the PR number, the "
|
||||||
|
"root cause, and the decision?' scenario.",
|
||||||
|
(
|
||||||
|
"A community user reports the Feishu/Lark adapter gets the identity "
|
||||||
|
"model wrong — open_id is app-scoped, not the bot's canonical ID. "
|
||||||
|
"There's an open PR #8388 trying to fix it. Use the hermes-agent-dev "
|
||||||
|
"skill and the pr-triage-salvage skill to review it."
|
||||||
|
),
|
||||||
|
"anthropic/claude-sonnet-4.6",
|
||||||
|
200000,
|
||||||
|
58, # end at "Here's my review: ..." — clean decision point before the "close it, implement cleaner" pivot
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"20260328_160817_77bd258b.jsonl",
|
||||||
|
"config-build-competitive-scouts",
|
||||||
|
"~60-turn iterative config/build session: user wants a set of weekly "
|
||||||
|
"cron jobs that scan competing AI coding agents (openclaw, nanoclaw, "
|
||||||
|
"ironclaw, codex, opencode, claude-code, kilo-code, gemini-cli, "
|
||||||
|
"cline, aider, roo) for merged PRs or web updates worth porting to "
|
||||||
|
"hermes-agent. User adds one target per turn; agent creates each cron "
|
||||||
|
"job and re-states the accumulated schedule. Exercises artifact trail "
|
||||||
|
"(which jobs are configured, which days) and iterative state "
|
||||||
|
"accumulation — the canonical case for iterative-merge summarization.",
|
||||||
|
(
|
||||||
|
"Set up a cron job for the agent every Sunday to scan all PRs "
|
||||||
|
"merged into openclaw that week, decide which are worth adding to "
|
||||||
|
"hermes-agent, and open PRs porting those features."
|
||||||
|
),
|
||||||
|
"anthropic/claude-sonnet-4.6",
|
||||||
|
200000,
|
||||||
|
None,
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
# Tool outputs beyond this size in chars are replaced with a short
|
||||||
|
# placeholder — a 16KB skill_view dump or 5KB web_extract result
|
||||||
|
# doesn't contribute to the compression eval signal but bloats the
|
||||||
|
# fixture size and PR diff readability. The compressor sees the
|
||||||
|
# placeholder and still knows the tool was called and returned
|
||||||
|
# something useful.
|
||||||
|
_TOOL_OUTPUT_MAX = 2000
|
||||||
|
|
||||||
|
|
||||||
|
def _maybe_truncate_tool_output(text: str, tool_name: str) -> str:
|
||||||
|
if not text or len(text) <= _TOOL_OUTPUT_MAX:
|
||||||
|
return text
|
||||||
|
keep = _TOOL_OUTPUT_MAX - 200
|
||||||
|
head = text[:keep]
|
||||||
|
return (
|
||||||
|
head
|
||||||
|
+ f"\n\n[... tool output truncated for fixture — original was {len(text)} chars"
|
||||||
|
+ (f" from {tool_name}" if tool_name else "")
|
||||||
|
+ "]"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
_PATH_RE = re.compile(r"/home/teknium\b")
|
||||||
|
_USER_RE = re.compile(r"\bteknium1\b|\bTeknium\b|\bteknium\b", re.IGNORECASE)
|
||||||
|
# Only strip scratchpads in ASSISTANT content, not tool results (might be legit)
|
||||||
|
_SCRATCH_RE = re.compile(
|
||||||
|
r"<REASONING_SCRATCHPAD>.*?</REASONING_SCRATCHPAD>\s*", re.DOTALL
|
||||||
|
)
|
||||||
|
_THINK_RE = re.compile(r"<think>.*?</think>\s*", re.DOTALL)
|
||||||
|
# Discord/Telegram user mention leakage in messaging-platform sessions
|
||||||
|
_USER_MENTION_RE = re.compile(r"<@\*{3}>|<@\d+>")
|
||||||
|
# Contributor emails (from git show output etc) — anything@domain.tld
|
||||||
|
# Keep noreply@github-style placeholders obvious; real personal emails get
|
||||||
|
# replaced with a contributor placeholder.
|
||||||
|
_EMAIL_RE = re.compile(r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b")
|
||||||
|
# "Author: Name <email>" git-show headers — rewrite the whole line
|
||||||
|
_GIT_AUTHOR_RE = re.compile(r"Author:\s*[^<\n]+<[^>]+>")
|
||||||
|
|
||||||
|
|
||||||
|
def _scrub_text(text: str, *, drop_scratchpads: bool = False) -> str:
|
||||||
|
"""Apply the pipeline to a raw text string.
|
||||||
|
|
||||||
|
drop_scratchpads only affects assistant messages — tool outputs that
|
||||||
|
happen to contain similar markers are left alone.
|
||||||
|
"""
|
||||||
|
if not text:
|
||||||
|
return text
|
||||||
|
if drop_scratchpads:
|
||||||
|
text = _SCRATCH_RE.sub("", text)
|
||||||
|
text = _THINK_RE.sub("", text)
|
||||||
|
text = _PATH_RE.sub("/home/user", text)
|
||||||
|
text = _USER_RE.sub("user", text)
|
||||||
|
text = _USER_MENTION_RE.sub("<@user>", text)
|
||||||
|
# Rewrite git "Author: Name <email>" lines before generic email replace
|
||||||
|
text = _GIT_AUTHOR_RE.sub("Author: contributor <contributor@example.com>", text)
|
||||||
|
text = _EMAIL_RE.sub("contributor@example.com", text)
|
||||||
|
text = redact_sensitive_text(text)
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def _content_to_str(content: Any) -> str:
|
||||||
|
if content is None:
|
||||||
|
return ""
|
||||||
|
if isinstance(content, str):
|
||||||
|
return content
|
||||||
|
if isinstance(content, list):
|
||||||
|
parts = []
|
||||||
|
for p in content:
|
||||||
|
if isinstance(p, dict) and "text" in p:
|
||||||
|
parts.append(p["text"])
|
||||||
|
elif isinstance(p, str):
|
||||||
|
parts.append(p)
|
||||||
|
return "\n".join(parts)
|
||||||
|
return str(content)
|
||||||
|
|
||||||
|
|
||||||
|
def _scrub_tool_calls(tool_calls: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
|
||||||
|
out = []
|
||||||
|
for tc in tool_calls or []:
|
||||||
|
if not isinstance(tc, dict):
|
||||||
|
continue
|
||||||
|
fn = tc.get("function", {}) or {}
|
||||||
|
args = fn.get("arguments", "")
|
||||||
|
if isinstance(args, str):
|
||||||
|
args = _scrub_text(args)
|
||||||
|
new_tc = {
|
||||||
|
"id": tc.get("id", ""),
|
||||||
|
"type": tc.get("type", "function"),
|
||||||
|
"function": {
|
||||||
|
"name": fn.get("name", ""),
|
||||||
|
"arguments": args,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
out.append(new_tc)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _scrub_message(m: Dict[str, Any], *, first_user_paraphrase: str | None, user_turn_idx: List[int]) -> Dict[str, Any] | None:
|
||||||
|
role = m.get("role")
|
||||||
|
if role in (None, "session_meta"):
|
||||||
|
return None
|
||||||
|
|
||||||
|
content = _content_to_str(m.get("content"))
|
||||||
|
|
||||||
|
if role == "assistant":
|
||||||
|
content = _scrub_text(content, drop_scratchpads=True)
|
||||||
|
elif role == "user":
|
||||||
|
# Use paraphrase for the very first user turn only
|
||||||
|
user_turn_idx[0] += 1
|
||||||
|
if user_turn_idx[0] == 1 and first_user_paraphrase is not None:
|
||||||
|
content = first_user_paraphrase
|
||||||
|
else:
|
||||||
|
content = _scrub_text(content)
|
||||||
|
else:
|
||||||
|
content = _scrub_text(content)
|
||||||
|
# Truncate large tool outputs
|
||||||
|
if role == "tool":
|
||||||
|
tn = m.get("tool_name") or m.get("name") or ""
|
||||||
|
content = _maybe_truncate_tool_output(content, tn)
|
||||||
|
|
||||||
|
new_msg: Dict[str, Any] = {"role": role, "content": content}
|
||||||
|
|
||||||
|
if role == "assistant":
|
||||||
|
tcs = m.get("tool_calls") or []
|
||||||
|
if tcs:
|
||||||
|
new_msg["tool_calls"] = _scrub_tool_calls(tcs)
|
||||||
|
if role == "tool":
|
||||||
|
if m.get("tool_call_id"):
|
||||||
|
new_msg["tool_call_id"] = m["tool_call_id"]
|
||||||
|
if m.get("tool_name") or m.get("name"):
|
||||||
|
new_msg["tool_name"] = m.get("tool_name") or m.get("name")
|
||||||
|
|
||||||
|
return new_msg
|
||||||
|
|
||||||
|
|
||||||
|
def build_fixture(
|
||||||
|
source_file: str,
|
||||||
|
output_name: str,
|
||||||
|
description: str,
|
||||||
|
first_user_paraphrase: str,
|
||||||
|
model_guess: str,
|
||||||
|
context_length: int,
|
||||||
|
truncate_at: int | None = None,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
src = SESSION_DIR / source_file
|
||||||
|
raw_msgs: List[Dict[str, Any]] = []
|
||||||
|
with src.open() as fh:
|
||||||
|
for line in fh:
|
||||||
|
try:
|
||||||
|
raw_msgs.append(json.loads(line))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# Skip session_meta lines up front so truncate_at counts real messages
|
||||||
|
raw_msgs = [m for m in raw_msgs if m.get("role") != "session_meta"]
|
||||||
|
if truncate_at is not None:
|
||||||
|
raw_msgs = raw_msgs[:truncate_at]
|
||||||
|
|
||||||
|
user_turn_counter = [0]
|
||||||
|
scrubbed: List[Dict[str, Any]] = []
|
||||||
|
for m in raw_msgs:
|
||||||
|
new = _scrub_message(
|
||||||
|
m,
|
||||||
|
first_user_paraphrase=first_user_paraphrase,
|
||||||
|
user_turn_idx=user_turn_counter,
|
||||||
|
)
|
||||||
|
if new is not None:
|
||||||
|
scrubbed.append(new)
|
||||||
|
|
||||||
|
# Drop empty-content assistant messages that have no tool_calls
|
||||||
|
# (artifact of scratchpad-only turns post-scrub)
|
||||||
|
pruned: List[Dict[str, Any]] = []
|
||||||
|
for m in scrubbed:
|
||||||
|
if (
|
||||||
|
m["role"] == "assistant"
|
||||||
|
and not (m.get("content") or "").strip()
|
||||||
|
and not m.get("tool_calls")
|
||||||
|
):
|
||||||
|
continue
|
||||||
|
pruned.append(m)
|
||||||
|
# Trim trailing orphan tool messages (no matching assistant)
|
||||||
|
while pruned and pruned[-1]["role"] == "tool":
|
||||||
|
pruned.pop()
|
||||||
|
scrubbed = pruned
|
||||||
|
|
||||||
|
# Inject a synthetic public-safe system message so the compressor has
|
||||||
|
# a head to anchor on. The real system prompts embed personality and
|
||||||
|
# platform-specific content we don't want checked in.
|
||||||
|
system_msg = {
|
||||||
|
"role": "system",
|
||||||
|
"content": (
|
||||||
|
"You are a helpful AI coding assistant with access to tools "
|
||||||
|
"(terminal, file editing, search, web, etc.). You operate in a "
|
||||||
|
"conversational loop: the user gives you a task, you call tools "
|
||||||
|
"to accomplish it, and you report back concisely."
|
||||||
|
),
|
||||||
|
}
|
||||||
|
if scrubbed and scrubbed[0].get("role") == "system":
|
||||||
|
scrubbed[0] = system_msg
|
||||||
|
else:
|
||||||
|
scrubbed.insert(0, system_msg)
|
||||||
|
|
||||||
|
fixture = {
|
||||||
|
"name": output_name,
|
||||||
|
"description": description,
|
||||||
|
"model": model_guess,
|
||||||
|
"context_length": context_length,
|
||||||
|
"source": f"~/.hermes/sessions/{source_file}",
|
||||||
|
"truncated_to": truncate_at,
|
||||||
|
"scrubbed_at": datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||||
|
"scrubbing_passes": [
|
||||||
|
"redact_sensitive_text (agent.redact)",
|
||||||
|
"username paths replaced with /home/user",
|
||||||
|
"personal handles (all case variants of the maintainer name) replaced with 'user'",
|
||||||
|
"email addresses replaced with contributor@example.com",
|
||||||
|
"git 'Author: Name <addr>' header lines normalised",
|
||||||
|
"reasoning scratchpad blocks stripped from assistant content",
|
||||||
|
"think tag blocks stripped from assistant content",
|
||||||
|
"messaging-platform user mentions replaced with <@user>",
|
||||||
|
"first user message paraphrased to remove personal voice",
|
||||||
|
"subsequent user messages kept verbatim (after above redactions)",
|
||||||
|
"system prompt replaced with generic public-safe placeholder",
|
||||||
|
"orphan empty-assistant messages and trailing tool messages dropped",
|
||||||
|
f"tool outputs longer than {_TOOL_OUTPUT_MAX} chars truncated with a note",
|
||||||
|
],
|
||||||
|
"messages": scrubbed,
|
||||||
|
}
|
||||||
|
return fixture
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
FIXTURES_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
for spec in SPECS:
|
||||||
|
source_file, output_name, description, paraphrase, model, ctx, truncate = spec
|
||||||
|
fixture = build_fixture(
|
||||||
|
source_file=source_file,
|
||||||
|
output_name=output_name,
|
||||||
|
description=description,
|
||||||
|
first_user_paraphrase=paraphrase,
|
||||||
|
model_guess=model,
|
||||||
|
context_length=ctx,
|
||||||
|
truncate_at=truncate,
|
||||||
|
)
|
||||||
|
out_path = FIXTURES_DIR / f"{output_name}.json"
|
||||||
|
with out_path.open("w") as fh:
|
||||||
|
json.dump(fixture, fh, indent=2, ensure_ascii=False)
|
||||||
|
size_kb = out_path.stat().st_size / 1024
|
||||||
|
print(f" {output_name}.json {size_kb:.1f} KB {len(fixture['messages'])} msgs")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
Loading…
Add table
Add a link
Reference in a new issue