feat(agent): track per-model token usage for mid-session model switches

The `sessions` table records only the initial (model, billing_provider)
for a session, so when a user switches models mid-session (via `/model`
or programmatically) every token — including the switched model's — is
attributed to the first model. Insights/billing reports then hide the
cost of the new model entirely (e.g. a session that started on deepseek
and switched to opus shows $0 for opus).

Add a `session_model_usage` table keyed (session_id, model,
billing_provider) that accumulates each per-API-call delta under the
model active at the time of the call. `update_token_counts()` is the
single chokepoint every per-call delta flows through (CLI, gateway,
cron, delegated, codex), so recording there captures accurate
attribution on every platform. Only the incremental path records — the
gateway's `absolute=True` summary overwrite is skipped to avoid
double-counting cumulative totals that can't be split per model. When a
call omits the model, it falls back to the session's recorded model,
matching the existing COALESCE-from-session summary behaviour.

Insights `_compute_model_breakdown` now aggregates tokens and cost from
`session_model_usage`, so a switched session splits correctly across
models, with a defensive fallback to the per-session aggregate for any
session lacking usage rows. A v17 migration backfills one usage row per
existing token-bearing session from its aggregate totals (idempotent via
INSERT OR IGNORE), validated lossless against a 1.3 GB production DB.

Tests: per-model recording, mid-session split, model fallback, absolute
no-double-count, v17 backfill, and an insights-level switch breakdown.

Fixes #51607.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Thomas Connally 2026-06-23 21:23:43 -05:00 committed by Teknium
parent 022c4991fc
commit cb7f6bbb2e
4 changed files with 438 additions and 20 deletions

View file

@ -320,6 +320,35 @@ class TestInsightsPopulated:
claude = next(m for m in models if "claude-sonnet" in m["model"])
assert claude["sessions"] == 2
def test_model_breakdown_splits_mid_session_switch(self, db):
"""A session that switches models mid-flight is split across both
models in the breakdown, not dumped on the initial model (#51607).
"""
now = time.time()
db.create_session(session_id="sw", source="cli",
model="deepseek/deepseek-v4-pro")
# 40k tokens on deepseek, then switch and 50k on opus.
db.update_token_counts("sw", input_tokens=40000, output_tokens=8000,
model="deepseek/deepseek-v4-pro",
billing_provider="deepseek", api_call_count=2)
db.update_session_model("sw", "anthropic/claude-opus-4.8")
db.update_token_counts("sw", input_tokens=50000, output_tokens=4000,
model="anthropic/claude-opus-4.8",
billing_provider="openrouter", api_call_count=3)
db._conn.commit()
report = InsightsEngine(db).generate(days=30)
models = {m["model"]: m for m in report["models"]}
assert "deepseek-v4-pro" in models
assert "claude-opus-4.8" in models
# Tokens attributed to the model that actually incurred them.
assert models["deepseek-v4-pro"]["input_tokens"] == 40000
assert models["claude-opus-4.8"]["input_tokens"] == 50000
assert models["claude-opus-4.8"]["api_calls"] == 3
# The summary row's single model would have hidden one of these.
assert models["deepseek-v4-pro"]["total_tokens"] == 48000
assert models["claude-opus-4.8"]["total_tokens"] == 54000
def test_platform_breakdown(self, populated_db):
engine = InsightsEngine(populated_db)
report = engine.generate(days=30)