DeepSeek's own API (api.deepseek.com) reports context-cache hits as
top-level usage.prompt_cache_hit_tokens / prompt_cache_miss_tokens
(prompt_tokens = hit + miss), not the OpenAI nested
prompt_tokens_details.cached_tokens shape. Neither normalize_usage()
nor the chat_completions transport's extract_cache_stats() read those
fields, so direct DeepSeek sessions always showed 0 cache-hit tokens:
invisible in accounting, mis-billed at the full input rate, and 0%
cache display.
Both layers now fall back to prompt_cache_hit_tokens when the nested
shape is absent; the nested value wins when both are present (proxies).
Fixes#61871.
Widens the deepseek-v4-flash addition to the whole stale-snapshot class:
- deepseek-v4-pro: $1.74/$3.48 → $0.435/$0.87, cache-read $0.003625
(DeepSeek's 2026-07 price cut; every pro session was over-reporting 4x)
- deepseek-chat / deepseek-reasoner: deprecated 2026-07-24, now alias
v4-flash non-thinking/thinking modes — repriced to match flash
(reasoner was $0.55/$2.19 with no cache rate)
- cache_read added to every row; pricing_version unified at
deepseek-pricing-2026-07
- invariant tests: aliases price identically to flash; every deepseek
row carries cache_read < input
DeepSeek's /models endpoint returns no pricing, so direct-provider routes fall back to the _OFFICIAL_DOCS_PRICING snapshot. The table included deepseek-v4-pro but not the newer deepseek-v4-flash, so flash sessions reported $0.00 with cost_source "none". Add the flash entry (values from DeepSeek's official pricing page, mirroring the v4-pro entry; DeepSeek bills no separate cache-write cost) plus two regression tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Refresh _OFFICIAL_DOCS_PRICING fireworks entries against current
docs.fireworks.ai/serverless/pricing: qwen3p6-plus is gone (replaced
by qwen3p7-plus); add glm-5p2/5p1, kimi-k2p7-code, deepseek-v4-flash,
minimax-m3/m2p7, gpt-oss-120b/20b, and the routers/*-fast tiers with
their distinct higher rates.
- Picker pricing via get_pricing_for_provider('fireworks'): pure dict
transform over the shared models.dev in-memory/disk cache (1h TTL) +
_pricing_cache memoization — no new network call on the picker path.
- Wire pricing display into the generic api-key-provider setup flow so
Fireworks model pickers show $/M columns like OpenRouter/Nous do.
- Invariant tests: plugin fallback_models all priced, fast tiers price
higher than standard, every row carries cache_read < input.
Fireworks-hosted sessions previously showed estimated_cost_usd = 0
because (a) _OFFICIAL_DOCS_PRICING had no Fireworks entries and (b)
resolve_billing_route() had no branch for provider="fireworks",
falling through to billing_mode="unknown".
Adds entries for the three Fireworks models hermes operators are
most likely to route through (Kimi K2.6, DeepSeek V4 Pro, Qwen3.6-Plus)
and a routing branch that triggers on either explicit
provider="fireworks" or api.fireworks.ai base_url match. Mirrors the
recently-merged MiniMax addition pattern; pricing snapshot sourced
from https://docs.fireworks.ai/serverless/pricing and the per-model
pages on fireworks.ai.
Tests cover: (a) full Fireworks model id resolves to the snapshot
entry, (b) base_url alone is sufficient to route, (c) end-to-end
estimate returns "estimated" status with the expected dollar amount.
A follow-up upstream issue is open proposing a dynamic pricing
source (e.g. litellm's pricing JSON) as a permanent fix to the
PR-per-model treadmill that this snapshot keeps adding to.
Bedrock Claude routes through the AnthropicBedrock SDK and injects
cache_control, so cached tokens are always reported — but the pricing
table had no cache cost fields for any Bedrock model, so /usage showed
"cost unknown" on every cached session. Also, cross-region inference
profiles (us./global./eu. prefixes) never matched the bare pricing keys.
- Add cache_read/cache_write rates to the four Bedrock Claude rows
(read 0.1x input, write 1.25x input per the Bedrock pricing page).
- Normalize the cross-region prefix in the Bedrock pricing lookup,
mirroring is_anthropic_bedrock_model's prefix list.
Closes#50295.
Rebased onto current main and re-ported across the restructured
surfaces: model flows now thread confirm_provider/base_url/api_key
through hermes_cli/model_setup_flows.py, the Discord picker lives in
plugins/platforms/discord/adapter.py, and the web dashboard picker
applies chat-mode switches via config.set so the expensive-model
confirmation can ride the response.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* remove Vercel AI Gateway provider and Vercel Sandbox terminal backend
Both Vercel-hosted integrations are removed end-to-end. Users on the AI
Gateway should switch to OpenRouter or one of the other aggregators
(Nous Portal, Kilo Code). Users on the Vercel Sandbox backend should
switch to Docker, Modal, Daytona, or SSH.
What's removed:
- `plugins/model-providers/ai-gateway/` provider plugin
- `hermes_cli/vercel_auth.py` Vercel-Sandbox auth helper
- `tools/environments/vercel_sandbox.py` terminal backend
- `ai-gateway` provider wiring across auth, doctor, setup, models,
config, status, providers, main, web_server, model_normalize, dump
- `vercel_sandbox` backend wiring across terminal_tool, file_tools,
code_execution_tool, file_operations, approval, skills_tool,
environments/local, credential_files, lazy_deps, prompt_builder,
cli, gateway/run
- `AI_GATEWAY_BASE_URL` constant, `_AI_GATEWAY_HEADERS` auxiliary-client
header set, run_agent base-URL header/reasoning special-cases
- `[vercel]` pyproject extra and `vercel`/`vercel-workers` from uv.lock
- env vars: `AI_GATEWAY_API_KEY`, `AI_GATEWAY_BASE_URL`, `VERCEL_TOKEN`,
`VERCEL_PROJECT_ID`, `VERCEL_TEAM_ID`, `VERCEL_OIDC_TOKEN`,
`TERMINAL_VERCEL_RUNTIME`
- Tests: deletes test_ai_gateway_models.py and
test_vercel_sandbox_environment.py; scrubs references across 23
surviving test files (no entire tests deleted unless they were
dedicated to AI Gateway / Sandbox)
- Docs: provider tables, env-var reference, setup guides, security
notes, tool config, terminal-backend tables — English plus zh-Hans
i18n parity
- `hermes-agent` skill: provider table entry and remote-backend list
What stays (intentional):
- `popular-web-designs/templates/vercel.md` — CSS design reference,
unrelated to Vercel-the-AI-product
- `x-vercel-id` in `stream_diag.py` headers — generic Vercel CDN
response header, useful diag signal on any Vercel-hosted endpoint
- `vercel-labs/agent-browser` URL in browser config — lightpanda
browser project, different OSS effort
- `userStories.json` historical contributor entry mentioning Vercel
Sandbox — archive, not active docs
Validation:
- 1153 tests in the 22 targeted files pass (`scripts/run_tests.sh`)
- Full repo `py_compile` clean
- Live import of every touched module + invariant check (no
`ai-gateway` in `PROVIDER_REGISTRY`, no `_AI_GATEWAY_HEADERS`, no
`vercel_sandbox` in `_REMOTE_TERMINAL_BACKENDS`)
* test: convert profile-count check from change-detector to invariant
The hardcoded "== 34" assertion broke when ai-gateway was removed.
Per AGENTS.md change-detector-test guidance, assert the relationship
(registry count >= number of plugin dirs) instead of a literal count.
Counts shift when providers are added/removed; that's expected.
deepseek-v4-pro has been routable since v0.12 but was missing from
the _OFFICIAL_DOCS_PRICING table. Sessions using this model showed
as "unknown cost" in hermes insights instead of a dollar estimate.
Add pricing entry using published list prices:
- input: \$1.74/M tokens
- output: \$3.48/M tokens
- cache_read: \$0.0145/M tokens
Uses standard list rates (not the 75% promo) so estimates remain
accurate after promo expires 2026-05-31.
Closes#24218
Port from cline/cline#10266.
When OpenAI-compatible proxies (OpenRouter, Vercel AI Gateway, Cline)
route Claude models, they sometimes surface the Anthropic-native cache
counters (`cache_read_input_tokens`, `cache_creation_input_tokens`) at
the top level of the `usage` object instead of nesting them inside
`prompt_tokens_details`. Our chat-completions branch of
`normalize_usage()` only read the nested `prompt_tokens_details` fields,
so those responses:
- reported `cache_write_tokens = 0` even when the model actually did a
prompt-cache write,
- reported only some of the cache-read tokens when the proxy exposed them
top-level only,
- overstated `input_tokens` by the missed cache-write amount, which in
turn made cost estimation and the status-bar cache-hit percentage wrong
for Claude traffic going through these gateways.
Now the chat-completions branch tries the OpenAI-standard
`prompt_tokens_details` first and falls back to the top-level
Anthropic-shape fields only if the nested values are absent/zero. The
Anthropic and Codex Responses branches are unchanged.
Regression guards added for three shapes: top-level write + nested read,
top-level-only, and both-present (nested wins).
* perf: cache base_url.lower() via property, consolidate triple load_config(), hoist set constant
run_agent.py:
- Add base_url property that auto-caches _base_url_lower on every
assignment, eliminating 12+ redundant .lower() calls per API cycle
across __init__, _build_api_kwargs, _supports_reasoning_extra_body,
and the main conversation loop
- Consolidate three separate load_config() disk reads in __init__
(memory, skills, compression) into a single call, reusing the
result dict for all three config sections
model_tools.py:
- Hoist _READ_SEARCH_TOOLS set to module level (was rebuilt inside
handle_function_call on every tool invocation)
* Use endpoint metadata for custom model context and pricing
---------
Co-authored-by: kshitij <82637225+kshitijk4poor@users.noreply.github.com>
