Commit graph

8 commits

Author SHA1 Message Date
Teknium
39975613b1
test: prune wave 2 + speed fixes — 28,106 → 19,757 test functions, suite wall 315s → 294s
Second, deeper pass over tools/gateway/hermes_cli plus first pass over
the trees wave 1 missed (acp, acp_adapter, skills, computer_use, docker,
dashboard, conformance, monitoring, secret_sources, hermes_state,
providers). Same rubric as wave 1 (AGENTS.md test policy); security,
alternation/caching invariants, issue-number regressions, and E2E kept.

Real test-quality fixes found and rooted out along the way:
- tests/tools/test_command_guards.py made real auxiliary-LLM HTTPS calls
  (DEFAULT_CONFIG smart-approval leaked in) — pinned approval
  mode=manual via autouse fixture: 17.4s → 0.4s.
- test_model_switch_custom_providers.py / test_user_providers_model_switch.py
  silently probed live provider catalogs (~2s/test) — stubbed
  cached_provider_model_ids/provider_model_ids/fetch_api_models.
- test_telegram_noise_filter.py: 15-platform copy-paste matrix over
  shared gateway.run logic → 3 representative platforms (55s → 3.9s).
- test_gateway_shutdown.py: stop()'s 5s interrupt-deadline loop spun on
  MagicMock agents — interrupt.side_effect now clears _running_agents
  (22s → 1.0s).
- test_gateway_inactivity_timeout.py poll-harness timings shrunk 3-5x
  (24s → 1.1s); test_mcp_stability.py backoff/SIGTERM-grace sleeps
  patched (15.4s → 2.5s); test_async_delegation.py negative-drain wait
  5s → 0.5s.
- test_telegram_init_deadline.py: loop-block margin restored to 1.0s
  with rationale comment — the watchdog-dump assertion needs the loop
  blocked well past deadline+grace under parallel load (flaked once in
  the 40-worker verification run at a 0.2s margin).

Verification: full hermetic suite via scripts/run_tests.sh —
2,438 files, 21,718 tests passed, 0 failed, 293.9s wall.
Suite totals vs original baseline: 46,820 → 19,757 test functions
(−57.8%), wall 583.5s → 293.9s (−50%), subprocess CPU 13,564s → 11,623s.
2026-07-29 13:39:40 -07:00
Teknium
6b81590c55
test: prune low-value tests suite-wide (wave 1) — 46,820 → 28,106 test functions
Systematic prune per AGENTS.md test policy, one pass over every major
test tree (gateway, hermes_cli, tools, agent, run_agent, plugins, cli,
cron, tui_gateway, honcho/openviking, root-level):

- DELETE: source-reading tests (read_text/getsource on prod files),
  change-detector tests (exact catalog counts, model-name snapshots,
  config version literals), mock-echo tests (assert a mock returns what
  it was told), assertion-free/trivial tests, near-duplicate
  parametrizations (boundaries + one representative kept), async/sync
  twin duplicates, cosmetic within-file variations.
- KEEP (mandatory): security/redaction/approval guards, message-role
  alternation invariants, prompt-caching/deterministic-call-id
  invariants, issue-number regression tests (deduped), E2E tests.
- 6 test files deleted outright (script-style/no-assert or fully
  redundant); conftest.py, fakes/, fixtures/ untouched.
- tests/acp/conftest.py added: autouse fixture stubs the live
  models.dev/GitHub/Copilot/Anthropic inventory fetches that ACP server
  tests performed on every session create — test_server.py 147s → 3.4s,
  and the tests are now genuinely hermetic.
- Sleep-based slowness shrunk where safe (codex_ttfb_watchdog,
  compression_concurrent_fork, etc.); no wall-clock assertion tightened.

Verification: full hermetic suite via scripts/run_tests.sh —
2439 files, 31,130 tests passed, 0 failed, 0 flaky retries, 315s wall
(baseline: 583s wall, 13,564s subprocess CPU).
2026-07-29 13:10:23 -07:00
teknium1
6179da5496 fix(dashboard): one gateway liveness ladder for status + channels
The sidebar strip and the Channels page could contradict each other on
the same page load — "Gateway running" next to "The gateway is not
running." /api/status and /api/messaging/platforms each open-coded their
own liveness ladder: status probed GATEWAY_HEALTH_URL and scoped its
PID/state reads to the requested profile, messaging did neither and used
the uncached raw PID probe.

Three deployments hit the split: a cross-container gateway (no local PID,
only the health probe can see it), a profile-scoped dashboard (messaging
borrowed a DIFFERENT profile's runtime state, reporting a false
"connected" that hides a real outage — #71211), and a launch-service
managed gateway with no PID file.

Adds resolve_gateway_liveness() in gateway/status.py as the single ladder
(cached PID -> HTTP health probe -> runtime-status PID with
expected_home) and routes both endpoints, /api/messaging/platforms/{id}/test,
and the kanban dispatcher-presence probe through it. Probe callables are
injectable so the existing monkeypatch seams keep working, and
GatewayLiveness.probe_error distinguishes "down" from "couldn't tell" so
the kanban warning keeps failing OPEN instead of crying wolf.

Closes #71211.
2026-07-26 08:08:38 -07:00
Teknium
fe2d847aca test: valid-shape Telegram token in port-binding guard fixture
The #62803 branch predates PR #64636's Telegram token-shape validation
on the messaging platform PUT endpoint; align the new guard test's
fixture with the validated format.
2026-07-16 07:17:55 -07:00
PRATHAMESH75
e984a61306 fix(dashboard): reject port-binding channels on secondary multiplexed profiles
The Channels API (PUT /api/messaging/platforms/{id}) accepted and persisted
enabling a port-binding platform on a secondary profile while
gateway.multiplex_profiles is on — a config the gateway only rejects on its
next start, aborting startup with MultiplexConfigError for every multiplexed
profile.

Validate before any .env/config.yaml write and return 409 for the enable
attempt. Disabling and clearing env stay allowed so an already-invalid
profile can be repaired. The port-binding platform set moves to
gateway/config.py (PORT_BINDING_PLATFORM_VALUES) as the single source of
truth shared by gateway startup validation and the dashboard, so the two
policies cannot drift. Platform config mutations now get a names-only audit
log line.

Fixes #62791
2026-07-16 07:17:55 -07:00
Shannon Sands
4a69a6620e test(dashboard): use valid Telegram tokens in profile tests 2026-07-16 00:21:01 -07:00
Shannon Sands
674e8b098a Fix dashboard gateway profile scoping 2026-06-17 05:40:57 -07:00
Teknium
88dbf95105
fix(dashboard): profile-scope Channels endpoints and seed per-profile .env (#44792)
Two halves of the same community report (dashboard Profile Builder):

1. A fresh dashboard/CLI-created profile got no .env file unless cloned,
   so it silently inherited API keys and messaging tokens from the shell
   environment / root install. create_profile() now seeds a placeholder
   .env (0600) for non-clone profiles, matching the SOUL.md seeding.

2. The Channels endpoints (/api/messaging/platforms GET/PUT/test) were
   not profile-scoped: they read/wrote the dashboard process's own .env
   via load_env()/save_env_value() regardless of the global profile
   switcher. They now accept the standard optional profile param (body
   beats query on the PUT, matching other scoped writes) and run inside
   _profile_scope(). When scoped, the payload no longer falls back to
   os.environ or load_gateway_config()'s env-override layer — both carry
   the ROOT install's credentials and would misreport them as the
   profile's. /api/messaging/platforms added to PROFILE_SCOPED_PREFIXES
   so the sidebar switcher scopes the Channels page automatically.
2026-06-12 02:09:28 -07:00