Commit graph

4 commits

Author SHA1 Message Date
kshitijk4poor
c1647dadba fix(tests): resolve 53 CI test failures across 8 root causes
1. Telegram xdist mock pollution (37 tests): Add tests/gateway/conftest.py
   with a shared _ensure_telegram_mock() that runs at collection time.
   Under pytest-xdist, test_telegram_caption_merge.py (bare top-level
   import, no mock) would trigger the ImportError fallback in
   gateway/platforms/telegram.py, caching ChatType=None and Update=Any
   for the entire worker — cascading into 37 downstream failures.

2. VIRTUAL_ENV env var leak (4 tests): TestDetectVenvDir tests monkeypatched
   sys.prefix but didn't clear VIRTUAL_ENV. After commit 50c35dca added a
   VIRTUAL_ENV check to _detect_venv_dir(), CI's real venv leaked through.

3. Copilot base_url missing (1 test): _resolve_runtime_from_pool_entry()
   set api_mode for copilot but didn't add the base_url fallback — unlike
   openrouter, anthropic, and codex which all have one. Production bug.

4. Stale vision model assertion (1 test): _PROVIDER_VISION_MODELS added
   zai -> glm-5v-turbo but the test still expected the main model glm-5.1.

5. Reasoning item id intentionally stripped (1 test): Production code at
   run_agent.py:3738 deliberately excludes 'id' from reasoning items
   (store=False causes API 404). Test was asserting the old behavior.

6. context_length warning not reaching custom_providers (1 test): The test
   didn't pass base_url to AIAgent, so self.base_url was empty and the
   custom_providers URL comparison at line 1302 never matched.

7. Matrix room ID URL-encoding (1 test): Production code now URL-encodes
   room IDs (!room:example.com -> %21room%3Aexample.com) but the test
   assertion wasn't updated.

8. Google Workspace calendar tests (2 tests): Tests assert on +agenda CLI
   args that don't exist in the production calendar_list() function. They
   only 'passed' before because _gws_binary() returned None, the Python
   SDK fallback ran, googleapiclient import failed, SystemExit was raised,
   and post-exit assertions were never reached. Skip when gws not installed.

Remaining 4 failures (test_run_progress_topics.py) are pre-existing flaky
tests that fail inconsistently under xdist — confirmed on clean main.
2026-04-16 07:24:16 +05:30
Dylan Socolobsky
c6dba918b3
fix(tests): fix several failing/flaky tests on main (#6777)
* fix(tests): mock is_safe_url in tests that use example.com

Tests using example.com URLs were failing because is_safe_url does a real DNS lookup which fails in environments where example.com doesn't resolve, causing the request to be blocked before reaching the already-mocked HTTP client. This should fix around 17 failing tests.

These tests test logic, caching, etc. so mocking this method should not modify them in any way. TestMattermostSendUrlAsFile was already doing this so we follow the same pattern.

* fix(test): use case-insensitive lookup for model context length check

DEFAULT_CONTEXT_LENGTHS uses inconsistent casing (MiniMax keys are lowercase, Qwen keys are mixed-case) so the test was broken in some cases since it couldn't find the model.

* fix(test): patch is_linux in systemd gateway restart test

The test only patched is_macos to False but didn't patch is_linux to True. On macOS hosts, is_linux() returns False and the systemd restart code path is skipped entirely, making the assertion fail.

* fix(test): use non-blocklisted env var in docker forward_env tests

GITHUB_TOKEN is in api_key_env_vars and thus in _HERMES_PROVIDER_ENV_BLOCKLIST so the env var is silently dropped, we replace it with a non-blocked one like DATABASE_URL so the tests actually work.

* fix(test): fully isolate _has_any_provider_configured from host env

_has_any_provider_configured() checks all env vars from PROVIDER_REGISTRY (not just the 5 the tests were clearing) and also calls get_auth_status() which detects gh auth token for Copilot. On machines with any of these set, the function returns True before reaching the code path under test.

Clear all registry vars and mock get_auth_status so host credentials don't interfere.

* fix(test): correct path to hermes_base_env.py in tool parser tests

Path(__file__).parent.parent resolved to tests/, not the project root.
The file lives at environments/hermes_base_env.py so we need one more parent level.

* fix(test): accept optional HTML fields in Matrix send payload

_send_matrix sometimes adds format and formatted_body when the markdown library is installed. The test was doing an exact dict equality check which broke. Check required fields instead.

* fix(test): add config.yaml to codex vision requirements test

The test only wrote auth.json but not config.yaml, so _read_main_provider() returned empty and vision auto-detect never tried the codex provider. Add a config.yaml pointing at openai-codex so the fallback path actually resolves the client.

* fix(test): clear OPENROUTER_API_KEY in _isolate_hermes_home

run_agent.py calls load_hermes_dotenv() at import time, which injects API keys from ~/.hermes/.env into os.environ before any test fixture runs. This caused test_agent_loop_tool_calling to make real API calls instead of skipping, which ends up making some tests fail.

* fix(test): add get_rate_limit_state to agent mock in usage report tests

_show_usage now calls agent.get_rate_limit_state() for rate limit
  display. The SimpleNamespace mock was missing this method.

* fix(test): update expected Camofox config version from 12 to 13

* fix(test): mock _get_enabled_platforms in nous managed defaults test

Importing gateway.run leaks DISCORD_BOT_TOKEN into os.environ, which makes _get_enabled_platforms() return ["cli", "discord"] instead of just ["cli"]. tools_command loops per platform, so apply_nous_managed_defaults
  runs twice: the first call sets config values, the second sees them as
  already configured and returns an empty set, causing the assertion to
  fail.
2026-04-09 13:17:06 -07:00
WAXLYY
c1818b7e9e fix(tools): redact query secrets in send_message errors 2026-04-06 16:49:52 -07:00
Teknium
57481c8ac5
fix(tools): implement send_message routing for Matrix, Mattermost, HomeAssistant, DingTalk (#3796)
* fix(tools): implement send_message routing for Matrix, Mattermost, HomeAssistant, DingTalk

Matrix, Mattermost, HomeAssistant, and DingTalk were present in
platform_map but fell through to the "not yet implemented" else branch,
causing send_message tool calls to silently fail on these platforms.

Add four async sender functions:
- _send_mattermost: POST /api/v4/posts via Mattermost REST API
- _send_matrix: PUT /_matrix/client/v3/rooms/.../send via Matrix CS API
- _send_homeassistant: POST /api/services/notify/notify via HA REST API
- _send_dingtalk: POST to session webhook URL

Add routing in _send_to_platform() and 17 unit tests covering success,
HTTP errors, missing config, env var fallback, and Matrix txn_id uniqueness.

* fix: pass platform tokens explicitly to Mattermost/Matrix/HA senders

The original PR passed pconfig.extra to sender functions, but tokens
live at pconfig.token (not in extra). This caused the senders to always
fall through to env var lookup instead of using the gateway-resolved
token.

Changes:
- Mattermost/Matrix/HA: accept token as first arg, matching the
  Telegram/Discord/Slack sender pattern
- DingTalk: add DINGTALK_WEBHOOK_URL env var fallback + docstring
  explaining the session-webhook vs robot-webhook difference
- Tests updated for new signatures + new DingTalk env var test

---------

Co-authored-by: sprmn24 <oncuevtv@gmail.com>
2026-03-29 15:17:46 -07:00