mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-04-25 00:51:20 +00:00
feat(webhook): direct delivery mode for zero-LLM push notifications (#12473)
External services can now push plain-text notifications to a user's chat via the webhook adapter without invoking the agent. Set deliver_only=true on a route and the rendered prompt template becomes the literal message body — dispatched directly to the configured target (Telegram, Discord, Slack, GitHub PR comment, etc.). Reuses all existing webhook infrastructure: HMAC-SHA256 signature validation, per-route rate limiting, idempotency cache, body-size limits, template rendering with dot-notation, home-channel fallback. No new HTTP server, no new auth scheme, no new port. Use cases: Supabase/Firebase webhooks → user notifications, monitoring alert forwarding, inter-agent pings, background job completion alerts. Changes: - gateway/platforms/webhook.py: new _direct_deliver() helper + early dispatch branch in _handle_webhook when deliver_only=true. Startup validation rejects deliver_only with deliver=log. - hermes_cli/main.py + hermes_cli/webhook.go: --deliver-only flag on subscribe; list/show output marks direct-delivery routes. - website/docs/user-guide/messaging/webhooks.md: new Direct Delivery Mode section with config example, CLI example, response codes. - skills/devops/webhook-subscriptions/SKILL.md: document --deliver-only with use cases (bumped to v1.1.0). - tests/gateway/test_webhook_deliver_only.py: 14 new tests covering agent bypass, template rendering, status codes, HMAC still enforced, idempotency still applies, rate limit still applies, startup validation, and direct-deliver dispatch. Validation: 78 webhook tests pass (64 existing + 14 new). E2E verified with real aiohttp server + real urllib POST — agent not invoked, target adapter.send() called with rendered template, duplicate delivery_id suppressed. Closes the gap identified in PR #12117 (thanks to @H1an1 / Antenna team) without adding a second HTTP ingress server.
This commit is contained in:
parent
66ee081dc1
commit
206a449b29
6 changed files with 699 additions and 4 deletions
|
|
@ -13,6 +13,10 @@ Each route defines:
|
||||||
- skills: optional list of skills to load for the agent
|
- skills: optional list of skills to load for the agent
|
||||||
- deliver: where to send the response (github_comment, telegram, etc.)
|
- deliver: where to send the response (github_comment, telegram, etc.)
|
||||||
- deliver_extra: additional delivery config (repo, pr_number, chat_id)
|
- deliver_extra: additional delivery config (repo, pr_number, chat_id)
|
||||||
|
- deliver_only: if true, skip the agent — the rendered prompt IS the
|
||||||
|
message that gets delivered. Use for external push notifications
|
||||||
|
(Supabase, monitoring alerts, inter-agent pings) where zero LLM cost
|
||||||
|
and sub-second delivery matter more than agent reasoning.
|
||||||
|
|
||||||
Security:
|
Security:
|
||||||
- HMAC secret is required per route (validated at startup)
|
- HMAC secret is required per route (validated at startup)
|
||||||
|
|
@ -122,6 +126,19 @@ class WebhookAdapter(BasePlatformAdapter):
|
||||||
f"For testing without auth, set secret to '{_INSECURE_NO_AUTH}'."
|
f"For testing without auth, set secret to '{_INSECURE_NO_AUTH}'."
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# deliver_only routes bypass the agent — the POST body becomes a
|
||||||
|
# direct push notification via the configured delivery target.
|
||||||
|
# Validate up-front so misconfiguration surfaces at startup rather
|
||||||
|
# than on the first webhook POST.
|
||||||
|
if route.get("deliver_only"):
|
||||||
|
deliver = route.get("deliver", "log")
|
||||||
|
if not deliver or deliver == "log":
|
||||||
|
raise ValueError(
|
||||||
|
f"[webhook] Route '{name}' has deliver_only=true but "
|
||||||
|
f"deliver is '{deliver}'. Direct delivery requires a "
|
||||||
|
f"real target (telegram, discord, slack, github_comment, etc.)."
|
||||||
|
)
|
||||||
|
|
||||||
app = web.Application()
|
app = web.Application()
|
||||||
app.router.add_get("/health", self._handle_health)
|
app.router.add_get("/health", self._handle_health)
|
||||||
app.router.add_post("/webhooks/{route_name}", self._handle_webhook)
|
app.router.add_post("/webhooks/{route_name}", self._handle_webhook)
|
||||||
|
|
@ -419,6 +436,64 @@ class WebhookAdapter(BasePlatformAdapter):
|
||||||
)
|
)
|
||||||
self._seen_deliveries[delivery_id] = now
|
self._seen_deliveries[delivery_id] = now
|
||||||
|
|
||||||
|
# ── Direct delivery mode (deliver_only) ─────────────────
|
||||||
|
# Skip the agent entirely — the rendered prompt IS the message we
|
||||||
|
# deliver. Use case: external services (Supabase, monitoring,
|
||||||
|
# cron jobs, other agents) that need to push a plain notification
|
||||||
|
# to a user's chat with zero LLM cost. Reuses the same HMAC auth,
|
||||||
|
# rate limiting, idempotency, and template rendering as agent mode.
|
||||||
|
if route_config.get("deliver_only"):
|
||||||
|
delivery = {
|
||||||
|
"deliver": route_config.get("deliver", "log"),
|
||||||
|
"deliver_extra": self._render_delivery_extra(
|
||||||
|
route_config.get("deliver_extra", {}), payload
|
||||||
|
),
|
||||||
|
"payload": payload,
|
||||||
|
}
|
||||||
|
logger.info(
|
||||||
|
"[webhook] direct-deliver event=%s route=%s target=%s msg_len=%d delivery=%s",
|
||||||
|
event_type,
|
||||||
|
route_name,
|
||||||
|
delivery["deliver"],
|
||||||
|
len(prompt),
|
||||||
|
delivery_id,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
result = await self._direct_deliver(prompt, delivery)
|
||||||
|
except Exception:
|
||||||
|
logger.exception(
|
||||||
|
"[webhook] direct-deliver failed route=%s delivery=%s",
|
||||||
|
route_name,
|
||||||
|
delivery_id,
|
||||||
|
)
|
||||||
|
return web.json_response(
|
||||||
|
{"status": "error", "error": "Delivery failed", "delivery_id": delivery_id},
|
||||||
|
status=502,
|
||||||
|
)
|
||||||
|
|
||||||
|
if result.success:
|
||||||
|
return web.json_response(
|
||||||
|
{
|
||||||
|
"status": "delivered",
|
||||||
|
"route": route_name,
|
||||||
|
"target": delivery["deliver"],
|
||||||
|
"delivery_id": delivery_id,
|
||||||
|
},
|
||||||
|
status=200,
|
||||||
|
)
|
||||||
|
# Delivery attempted but target rejected it — surface as 502
|
||||||
|
# with a generic error (don't leak adapter-level detail).
|
||||||
|
logger.warning(
|
||||||
|
"[webhook] direct-deliver target rejected route=%s target=%s error=%s",
|
||||||
|
route_name,
|
||||||
|
delivery["deliver"],
|
||||||
|
result.error,
|
||||||
|
)
|
||||||
|
return web.json_response(
|
||||||
|
{"status": "error", "error": "Delivery failed", "delivery_id": delivery_id},
|
||||||
|
status=502,
|
||||||
|
)
|
||||||
|
|
||||||
# Use delivery_id in session key so concurrent webhooks on the
|
# Use delivery_id in session key so concurrent webhooks on the
|
||||||
# same route get independent agent runs (not queued/interrupted).
|
# same route get independent agent runs (not queued/interrupted).
|
||||||
session_chat_id = f"webhook:{route_name}:{delivery_id}"
|
session_chat_id = f"webhook:{route_name}:{delivery_id}"
|
||||||
|
|
@ -572,6 +647,34 @@ class WebhookAdapter(BasePlatformAdapter):
|
||||||
# Response delivery
|
# Response delivery
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
async def _direct_deliver(
|
||||||
|
self, content: str, delivery: dict
|
||||||
|
) -> SendResult:
|
||||||
|
"""Deliver *content* directly without invoking the agent.
|
||||||
|
|
||||||
|
Used by ``deliver_only`` routes: the rendered template becomes the
|
||||||
|
literal message body, and we dispatch to the same delivery helpers
|
||||||
|
that the agent-mode ``send()`` flow uses. All target types that
|
||||||
|
work in agent mode work here — Telegram, Discord, Slack, GitHub
|
||||||
|
PR comments, etc.
|
||||||
|
"""
|
||||||
|
deliver_type = delivery.get("deliver", "log")
|
||||||
|
|
||||||
|
if deliver_type == "log":
|
||||||
|
# Shouldn't reach here — startup validation rejects deliver_only
|
||||||
|
# with deliver=log — but guard defensively.
|
||||||
|
logger.info("[webhook] direct-deliver log-only: %s", content[:200])
|
||||||
|
return SendResult(success=True)
|
||||||
|
|
||||||
|
if deliver_type == "github_comment":
|
||||||
|
return await self._deliver_github_comment(content, delivery)
|
||||||
|
|
||||||
|
# Fall through to the cross-platform dispatcher, which validates the
|
||||||
|
# target name and routes via the gateway runner.
|
||||||
|
return await self._deliver_cross_platform(
|
||||||
|
deliver_type, content, delivery
|
||||||
|
)
|
||||||
|
|
||||||
async def _deliver_github_comment(
|
async def _deliver_github_comment(
|
||||||
self, content: str, delivery: dict
|
self, content: str, delivery: dict
|
||||||
) -> SendResult:
|
) -> SendResult:
|
||||||
|
|
|
||||||
|
|
@ -7002,6 +7002,13 @@ For more help on a command:
|
||||||
wh_sub.add_argument(
|
wh_sub.add_argument(
|
||||||
"--secret", default="", help="HMAC secret (auto-generated if omitted)"
|
"--secret", default="", help="HMAC secret (auto-generated if omitted)"
|
||||||
)
|
)
|
||||||
|
wh_sub.add_argument(
|
||||||
|
"--deliver-only",
|
||||||
|
action="store_true",
|
||||||
|
help="Skip the agent — deliver the rendered prompt directly as the "
|
||||||
|
"message. Zero LLM cost. Requires --deliver to be a real target "
|
||||||
|
"(not 'log').",
|
||||||
|
)
|
||||||
|
|
||||||
webhook_subparsers.add_parser(
|
webhook_subparsers.add_parser(
|
||||||
"list", aliases=["ls"], help="List all dynamic subscriptions"
|
"list", aliases=["ls"], help="List all dynamic subscriptions"
|
||||||
|
|
|
||||||
|
|
@ -155,6 +155,15 @@ def _cmd_subscribe(args):
|
||||||
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
"created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if getattr(args, "deliver_only", False):
|
||||||
|
if route["deliver"] == "log":
|
||||||
|
print(
|
||||||
|
"Error: --deliver-only requires --deliver to be a real target "
|
||||||
|
"(telegram, discord, slack, github_comment, etc.) — not 'log'."
|
||||||
|
)
|
||||||
|
return
|
||||||
|
route["deliver_only"] = True
|
||||||
|
|
||||||
if args.deliver_chat_id:
|
if args.deliver_chat_id:
|
||||||
route["deliver_extra"] = {"chat_id": args.deliver_chat_id}
|
route["deliver_extra"] = {"chat_id": args.deliver_chat_id}
|
||||||
|
|
||||||
|
|
@ -172,9 +181,12 @@ def _cmd_subscribe(args):
|
||||||
else:
|
else:
|
||||||
print(" Events: (all)")
|
print(" Events: (all)")
|
||||||
print(f" Deliver: {route['deliver']}")
|
print(f" Deliver: {route['deliver']}")
|
||||||
|
if route.get("deliver_only"):
|
||||||
|
print(" Mode: direct delivery (no agent, zero LLM cost)")
|
||||||
if route.get("prompt"):
|
if route.get("prompt"):
|
||||||
prompt_preview = route["prompt"][:80] + ("..." if len(route["prompt"]) > 80 else "")
|
prompt_preview = route["prompt"][:80] + ("..." if len(route["prompt"]) > 80 else "")
|
||||||
print(f" Prompt: {prompt_preview}")
|
label = "Message" if route.get("deliver_only") else "Prompt"
|
||||||
|
print(f" {label}: {prompt_preview}")
|
||||||
print(f"\n Configure your service to POST to the URL above.")
|
print(f"\n Configure your service to POST to the URL above.")
|
||||||
print(f" Use the secret for HMAC-SHA256 signature validation.")
|
print(f" Use the secret for HMAC-SHA256 signature validation.")
|
||||||
print(f" The gateway must be running to receive events (hermes gateway run).\n")
|
print(f" The gateway must be running to receive events (hermes gateway run).\n")
|
||||||
|
|
@ -192,6 +204,8 @@ def _cmd_list(args):
|
||||||
for name, route in subs.items():
|
for name, route in subs.items():
|
||||||
events = ", ".join(route.get("events", [])) or "(all)"
|
events = ", ".join(route.get("events", [])) or "(all)"
|
||||||
deliver = route.get("deliver", "log")
|
deliver = route.get("deliver", "log")
|
||||||
|
if route.get("deliver_only"):
|
||||||
|
deliver = f"{deliver} (direct — no agent)"
|
||||||
desc = route.get("description", "")
|
desc = route.get("description", "")
|
||||||
print(f" ◆ {name}")
|
print(f" ◆ {name}")
|
||||||
if desc:
|
if desc:
|
||||||
|
|
|
||||||
|
|
@ -1,10 +1,10 @@
|
||||||
---
|
---
|
||||||
name: webhook-subscriptions
|
name: webhook-subscriptions
|
||||||
description: Create and manage webhook subscriptions for event-driven agent activation. Use when the user wants external services to trigger agent runs automatically.
|
description: Create and manage webhook subscriptions for event-driven agent activation, or for direct push notifications (zero LLM cost). Use when the user wants external services to trigger agent runs OR push notifications to chats.
|
||||||
version: 1.0.0
|
version: 1.1.0
|
||||||
metadata:
|
metadata:
|
||||||
hermes:
|
hermes:
|
||||||
tags: [webhook, events, automation, integrations]
|
tags: [webhook, events, automation, integrations, notifications, push]
|
||||||
---
|
---
|
||||||
|
|
||||||
# Webhook Subscriptions
|
# Webhook Subscriptions
|
||||||
|
|
@ -154,6 +154,29 @@ hermes webhook subscribe alerts \
|
||||||
--deliver origin
|
--deliver origin
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Direct delivery (no agent, zero LLM cost)
|
||||||
|
|
||||||
|
For use cases where you just want to push a notification through to a user's chat — no reasoning, no agent loop — add `--deliver-only`. The rendered `--prompt` template becomes the literal message body and is dispatched directly to the target adapter.
|
||||||
|
|
||||||
|
Use this for:
|
||||||
|
- External service push notifications (Supabase/Firebase webhooks → Telegram)
|
||||||
|
- Monitoring alerts that should forward verbatim
|
||||||
|
- Inter-agent pings where one agent is telling another agent's user something
|
||||||
|
- Any webhook where an LLM round trip would be wasted effort
|
||||||
|
|
||||||
|
```bash
|
||||||
|
hermes webhook subscribe antenna-matches \
|
||||||
|
--deliver telegram \
|
||||||
|
--deliver-chat-id "123456789" \
|
||||||
|
--deliver-only \
|
||||||
|
--prompt "🎉 New match: {match.user_name} matched with you!" \
|
||||||
|
--description "Antenna match notifications"
|
||||||
|
```
|
||||||
|
|
||||||
|
The POST returns `200 OK` on successful delivery, `502` on target failure — so upstream services can retry intelligently. HMAC auth, rate limits, and idempotency still apply.
|
||||||
|
|
||||||
|
Requires `--deliver` to be a real target (telegram, discord, slack, github_comment, etc.) — `--deliver log` is rejected because log-only direct delivery is pointless.
|
||||||
|
|
||||||
## Security
|
## Security
|
||||||
|
|
||||||
- Each subscription gets an auto-generated HMAC-SHA256 secret (or provide your own with `--secret`)
|
- Each subscription gets an auto-generated HMAC-SHA256 secret (or provide your own with `--secret`)
|
||||||
|
|
|
||||||
473
tests/gateway/test_webhook_deliver_only.py
Normal file
473
tests/gateway/test_webhook_deliver_only.py
Normal file
|
|
@ -0,0 +1,473 @@
|
||||||
|
"""Tests for the webhook adapter's ``deliver_only`` route mode.
|
||||||
|
|
||||||
|
``deliver_only`` lets external services (Supabase webhooks, monitoring
|
||||||
|
alerts, background jobs, other agents) push plain-text notifications to
|
||||||
|
a user's chat via the webhook adapter WITHOUT invoking the agent. The
|
||||||
|
rendered prompt template becomes the literal message body.
|
||||||
|
|
||||||
|
Covers:
|
||||||
|
- Agent is NOT invoked (``handle_message`` never called)
|
||||||
|
- Rendered content is delivered to the target platform adapter
|
||||||
|
- HTTP returns 200 OK on success, 502 on delivery failure
|
||||||
|
- Startup validation rejects ``deliver_only`` without a real delivery target
|
||||||
|
- HMAC auth, rate limiting, and idempotency still apply
|
||||||
|
"""
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import hashlib
|
||||||
|
import hmac
|
||||||
|
import json
|
||||||
|
from unittest.mock import AsyncMock, MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from aiohttp import web
|
||||||
|
from aiohttp.test_utils import TestClient, TestServer
|
||||||
|
|
||||||
|
from gateway.config import Platform, PlatformConfig
|
||||||
|
from gateway.platforms.base import MessageEvent, SendResult
|
||||||
|
from gateway.platforms.webhook import WebhookAdapter, _INSECURE_NO_AUTH
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Helpers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _make_adapter(routes, **extra_kw) -> WebhookAdapter:
|
||||||
|
extra = {"host": "0.0.0.0", "port": 0, "routes": routes}
|
||||||
|
extra.update(extra_kw)
|
||||||
|
config = PlatformConfig(enabled=True, extra=extra)
|
||||||
|
return WebhookAdapter(config)
|
||||||
|
|
||||||
|
|
||||||
|
def _create_app(adapter: WebhookAdapter) -> web.Application:
|
||||||
|
app = web.Application()
|
||||||
|
app.router.add_get("/health", adapter._handle_health)
|
||||||
|
app.router.add_post("/webhooks/{route_name}", adapter._handle_webhook)
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
def _wire_mock_target(adapter: WebhookAdapter, platform_name: str = "telegram"):
|
||||||
|
"""Attach a gateway_runner with a mocked target adapter."""
|
||||||
|
mock_target = AsyncMock()
|
||||||
|
mock_target.send = AsyncMock(return_value=SendResult(success=True))
|
||||||
|
|
||||||
|
mock_runner = MagicMock()
|
||||||
|
mock_runner.adapters = {Platform(platform_name): mock_target}
|
||||||
|
mock_runner.config.get_home_channel.return_value = None
|
||||||
|
|
||||||
|
adapter.gateway_runner = mock_runner
|
||||||
|
return mock_target
|
||||||
|
|
||||||
|
|
||||||
|
# ===================================================================
|
||||||
|
# Core behaviour: agent bypass
|
||||||
|
# ===================================================================
|
||||||
|
|
||||||
|
class TestDeliverOnlyBypassesAgent:
|
||||||
|
"""The whole point of the feature — handle_message must not be called."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_post_delivers_directly_without_agent(self):
|
||||||
|
routes = {
|
||||||
|
"match-alert": {
|
||||||
|
"secret": _INSECURE_NO_AUTH,
|
||||||
|
"deliver": "telegram",
|
||||||
|
"deliver_only": True,
|
||||||
|
"deliver_extra": {"chat_id": "12345"},
|
||||||
|
"prompt": "{payload.user} matched with {payload.other}!",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes)
|
||||||
|
mock_target = _wire_mock_target(adapter)
|
||||||
|
|
||||||
|
# Guard: handle_message must NOT be called in deliver_only mode
|
||||||
|
handle_message_calls: list[MessageEvent] = []
|
||||||
|
|
||||||
|
async def _capture(event):
|
||||||
|
handle_message_calls.append(event)
|
||||||
|
|
||||||
|
adapter.handle_message = _capture
|
||||||
|
|
||||||
|
app = _create_app(adapter)
|
||||||
|
body = json.dumps(
|
||||||
|
{"payload": {"user": "alice", "other": "bob"}}
|
||||||
|
).encode()
|
||||||
|
|
||||||
|
async with TestClient(TestServer(app)) as cli:
|
||||||
|
resp = await cli.post(
|
||||||
|
"/webhooks/match-alert",
|
||||||
|
data=body,
|
||||||
|
headers={
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"X-GitHub-Delivery": "delivery-1",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert resp.status == 200
|
||||||
|
data = await resp.json()
|
||||||
|
assert data["status"] == "delivered"
|
||||||
|
assert data["route"] == "match-alert"
|
||||||
|
assert data["target"] == "telegram"
|
||||||
|
|
||||||
|
# Let any background tasks settle before asserting no agent call
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
|
||||||
|
# Agent was NOT invoked
|
||||||
|
assert handle_message_calls == []
|
||||||
|
|
||||||
|
# Target adapter.send() WAS called with the rendered template
|
||||||
|
mock_target.send.assert_awaited_once()
|
||||||
|
call_args = mock_target.send.await_args
|
||||||
|
chat_id_arg, content_arg = call_args.args[0], call_args.args[1]
|
||||||
|
assert chat_id_arg == "12345"
|
||||||
|
assert content_arg == "alice matched with bob!"
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_template_rendering_works(self):
|
||||||
|
"""Dot-notation template variables resolve in deliver_only mode."""
|
||||||
|
routes = {
|
||||||
|
"alert": {
|
||||||
|
"secret": _INSECURE_NO_AUTH,
|
||||||
|
"deliver": "telegram",
|
||||||
|
"deliver_only": True,
|
||||||
|
"deliver_extra": {"chat_id": "chat-1"},
|
||||||
|
"prompt": "Build {build.number} status: {build.status}",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes)
|
||||||
|
mock_target = _wire_mock_target(adapter)
|
||||||
|
app = _create_app(adapter)
|
||||||
|
|
||||||
|
async with TestClient(TestServer(app)) as cli:
|
||||||
|
resp = await cli.post(
|
||||||
|
"/webhooks/alert",
|
||||||
|
json={"build": {"number": 77, "status": "FAILED"}},
|
||||||
|
headers={"X-GitHub-Delivery": "d-render-1"},
|
||||||
|
)
|
||||||
|
assert resp.status == 200
|
||||||
|
|
||||||
|
mock_target.send.assert_awaited_once()
|
||||||
|
content_arg = mock_target.send.await_args.args[1]
|
||||||
|
assert content_arg == "Build 77 status: FAILED"
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_thread_id_passed_through(self):
|
||||||
|
"""deliver_extra.thread_id flows through to the target adapter."""
|
||||||
|
routes = {
|
||||||
|
"r": {
|
||||||
|
"secret": _INSECURE_NO_AUTH,
|
||||||
|
"deliver": "telegram",
|
||||||
|
"deliver_only": True,
|
||||||
|
"deliver_extra": {"chat_id": "c-1", "thread_id": "topic-42"},
|
||||||
|
"prompt": "hi",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes)
|
||||||
|
mock_target = _wire_mock_target(adapter)
|
||||||
|
|
||||||
|
app = _create_app(adapter)
|
||||||
|
async with TestClient(TestServer(app)) as cli:
|
||||||
|
resp = await cli.post(
|
||||||
|
"/webhooks/r",
|
||||||
|
json={},
|
||||||
|
headers={"X-GitHub-Delivery": "d-thread-1"},
|
||||||
|
)
|
||||||
|
assert resp.status == 200
|
||||||
|
|
||||||
|
assert mock_target.send.await_args.kwargs["metadata"] == {
|
||||||
|
"thread_id": "topic-42"
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ===================================================================
|
||||||
|
# HTTP status codes
|
||||||
|
# ===================================================================
|
||||||
|
|
||||||
|
class TestDeliverOnlyStatusCodes:
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_delivery_failure_returns_502(self):
|
||||||
|
"""If the target adapter returns SendResult(success=False), 502."""
|
||||||
|
routes = {
|
||||||
|
"r": {
|
||||||
|
"secret": _INSECURE_NO_AUTH,
|
||||||
|
"deliver": "telegram",
|
||||||
|
"deliver_only": True,
|
||||||
|
"deliver_extra": {"chat_id": "c-1"},
|
||||||
|
"prompt": "hi",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes)
|
||||||
|
mock_target = _wire_mock_target(adapter)
|
||||||
|
mock_target.send = AsyncMock(
|
||||||
|
return_value=SendResult(success=False, error="rate limited by tg")
|
||||||
|
)
|
||||||
|
|
||||||
|
app = _create_app(adapter)
|
||||||
|
async with TestClient(TestServer(app)) as cli:
|
||||||
|
resp = await cli.post(
|
||||||
|
"/webhooks/r",
|
||||||
|
json={},
|
||||||
|
headers={"X-GitHub-Delivery": "d-fail-1"},
|
||||||
|
)
|
||||||
|
assert resp.status == 502
|
||||||
|
data = await resp.json()
|
||||||
|
# Generic error — no adapter-level detail leaks
|
||||||
|
assert data["error"] == "Delivery failed"
|
||||||
|
assert "rate limited" not in json.dumps(data)
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_delivery_exception_returns_502(self):
|
||||||
|
"""If adapter.send() raises, we return 502 (not 500)."""
|
||||||
|
routes = {
|
||||||
|
"r": {
|
||||||
|
"secret": _INSECURE_NO_AUTH,
|
||||||
|
"deliver": "telegram",
|
||||||
|
"deliver_only": True,
|
||||||
|
"deliver_extra": {"chat_id": "c-1"},
|
||||||
|
"prompt": "hi",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes)
|
||||||
|
mock_target = _wire_mock_target(adapter)
|
||||||
|
mock_target.send = AsyncMock(side_effect=RuntimeError("tg exploded"))
|
||||||
|
|
||||||
|
app = _create_app(adapter)
|
||||||
|
async with TestClient(TestServer(app)) as cli:
|
||||||
|
resp = await cli.post(
|
||||||
|
"/webhooks/r",
|
||||||
|
json={},
|
||||||
|
headers={"X-GitHub-Delivery": "d-exc-1"},
|
||||||
|
)
|
||||||
|
assert resp.status == 502
|
||||||
|
data = await resp.json()
|
||||||
|
assert data["error"] == "Delivery failed"
|
||||||
|
# Exception message must not leak
|
||||||
|
assert "exploded" not in json.dumps(data)
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_target_platform_not_connected_returns_502(self):
|
||||||
|
"""deliver_only to a platform the gateway doesn't have → 502."""
|
||||||
|
routes = {
|
||||||
|
"r": {
|
||||||
|
"secret": _INSECURE_NO_AUTH,
|
||||||
|
"deliver": "discord", # not configured in mock runner
|
||||||
|
"deliver_only": True,
|
||||||
|
"deliver_extra": {"chat_id": "c-1"},
|
||||||
|
"prompt": "hi",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes)
|
||||||
|
_wire_mock_target(adapter, platform_name="telegram") # only TG wired
|
||||||
|
|
||||||
|
app = _create_app(adapter)
|
||||||
|
async with TestClient(TestServer(app)) as cli:
|
||||||
|
resp = await cli.post(
|
||||||
|
"/webhooks/r",
|
||||||
|
json={},
|
||||||
|
headers={"X-GitHub-Delivery": "d-no-platform-1"},
|
||||||
|
)
|
||||||
|
assert resp.status == 502
|
||||||
|
|
||||||
|
|
||||||
|
# ===================================================================
|
||||||
|
# Startup validation
|
||||||
|
# ===================================================================
|
||||||
|
|
||||||
|
class TestDeliverOnlyStartupValidation:
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_deliver_only_with_log_deliver_rejected(self):
|
||||||
|
"""deliver_only=true + deliver=log is nonsense — reject at connect()."""
|
||||||
|
routes = {
|
||||||
|
"bad": {
|
||||||
|
"secret": _INSECURE_NO_AUTH,
|
||||||
|
"deliver": "log",
|
||||||
|
"deliver_only": True,
|
||||||
|
"prompt": "hi",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes)
|
||||||
|
with pytest.raises(ValueError, match="deliver_only=true but deliver is 'log'"):
|
||||||
|
await adapter.connect()
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_deliver_only_with_missing_deliver_rejected(self):
|
||||||
|
"""deliver_only=true with no deliver field defaults to 'log' → reject."""
|
||||||
|
routes = {
|
||||||
|
"bad": {
|
||||||
|
"secret": _INSECURE_NO_AUTH,
|
||||||
|
# no deliver field
|
||||||
|
"deliver_only": True,
|
||||||
|
"prompt": "hi",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes)
|
||||||
|
with pytest.raises(ValueError, match="deliver_only=true"):
|
||||||
|
await adapter.connect()
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_deliver_only_with_real_target_accepted(self):
|
||||||
|
"""Sanity check — a valid deliver_only config passes validation."""
|
||||||
|
routes = {
|
||||||
|
"good": {
|
||||||
|
"secret": _INSECURE_NO_AUTH,
|
||||||
|
"deliver": "telegram",
|
||||||
|
"deliver_only": True,
|
||||||
|
"deliver_extra": {"chat_id": "c-1"},
|
||||||
|
"prompt": "hi",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes)
|
||||||
|
# connect() does more than validation (binds a socket) — we just
|
||||||
|
# want to verify the validation doesn't raise. Call it and tear
|
||||||
|
# down immediately.
|
||||||
|
try:
|
||||||
|
started = await adapter.connect()
|
||||||
|
if started:
|
||||||
|
await adapter.disconnect()
|
||||||
|
except ValueError:
|
||||||
|
pytest.fail("valid deliver_only config should not raise ValueError")
|
||||||
|
|
||||||
|
|
||||||
|
# ===================================================================
|
||||||
|
# Security + reliability invariants still hold
|
||||||
|
# ===================================================================
|
||||||
|
|
||||||
|
class TestDeliverOnlySecurityInvariants:
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_hmac_still_enforced(self):
|
||||||
|
"""deliver_only does NOT bypass HMAC validation."""
|
||||||
|
secret = "real-secret-123"
|
||||||
|
routes = {
|
||||||
|
"r": {
|
||||||
|
"secret": secret,
|
||||||
|
"deliver": "telegram",
|
||||||
|
"deliver_only": True,
|
||||||
|
"deliver_extra": {"chat_id": "c-1"},
|
||||||
|
"prompt": "hi",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes)
|
||||||
|
mock_target = _wire_mock_target(adapter)
|
||||||
|
|
||||||
|
app = _create_app(adapter)
|
||||||
|
async with TestClient(TestServer(app)) as cli:
|
||||||
|
# No signature header → reject
|
||||||
|
resp = await cli.post(
|
||||||
|
"/webhooks/r",
|
||||||
|
json={},
|
||||||
|
headers={"X-GitHub-Delivery": "d-noauth-1"},
|
||||||
|
)
|
||||||
|
assert resp.status == 401
|
||||||
|
|
||||||
|
# Target never called
|
||||||
|
mock_target.send.assert_not_awaited()
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_idempotency_still_applies(self):
|
||||||
|
"""Same delivery_id posted twice → second is suppressed."""
|
||||||
|
routes = {
|
||||||
|
"r": {
|
||||||
|
"secret": _INSECURE_NO_AUTH,
|
||||||
|
"deliver": "telegram",
|
||||||
|
"deliver_only": True,
|
||||||
|
"deliver_extra": {"chat_id": "c-1"},
|
||||||
|
"prompt": "hi",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes)
|
||||||
|
mock_target = _wire_mock_target(adapter)
|
||||||
|
|
||||||
|
app = _create_app(adapter)
|
||||||
|
async with TestClient(TestServer(app)) as cli:
|
||||||
|
r1 = await cli.post(
|
||||||
|
"/webhooks/r",
|
||||||
|
json={},
|
||||||
|
headers={"X-GitHub-Delivery": "dup-1"},
|
||||||
|
)
|
||||||
|
assert r1.status == 200
|
||||||
|
|
||||||
|
r2 = await cli.post(
|
||||||
|
"/webhooks/r",
|
||||||
|
json={},
|
||||||
|
headers={"X-GitHub-Delivery": "dup-1"},
|
||||||
|
)
|
||||||
|
# Existing webhook adapter treats duplicates as 200 + status=duplicate
|
||||||
|
assert r2.status == 200
|
||||||
|
data = await r2.json()
|
||||||
|
assert data["status"] == "duplicate"
|
||||||
|
|
||||||
|
# Target was called exactly once
|
||||||
|
assert mock_target.send.await_count == 1
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_rate_limit_still_applies(self):
|
||||||
|
"""Route-level rate limit caps deliver_only POSTs too."""
|
||||||
|
routes = {
|
||||||
|
"r": {
|
||||||
|
"secret": _INSECURE_NO_AUTH,
|
||||||
|
"deliver": "telegram",
|
||||||
|
"deliver_only": True,
|
||||||
|
"deliver_extra": {"chat_id": "c-1"},
|
||||||
|
"prompt": "hi",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
adapter = _make_adapter(routes, rate_limit=2)
|
||||||
|
_wire_mock_target(adapter)
|
||||||
|
|
||||||
|
app = _create_app(adapter)
|
||||||
|
async with TestClient(TestServer(app)) as cli:
|
||||||
|
for i in range(2):
|
||||||
|
r = await cli.post(
|
||||||
|
"/webhooks/r",
|
||||||
|
json={},
|
||||||
|
headers={"X-GitHub-Delivery": f"rl-{i}"},
|
||||||
|
)
|
||||||
|
assert r.status == 200
|
||||||
|
|
||||||
|
# Third within the window → 429
|
||||||
|
r3 = await cli.post(
|
||||||
|
"/webhooks/r",
|
||||||
|
json={},
|
||||||
|
headers={"X-GitHub-Delivery": "rl-3"},
|
||||||
|
)
|
||||||
|
assert r3.status == 429
|
||||||
|
|
||||||
|
|
||||||
|
# ===================================================================
|
||||||
|
# Unit: _direct_deliver dispatch
|
||||||
|
# ===================================================================
|
||||||
|
|
||||||
|
class TestDirectDeliverUnit:
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_dispatches_to_cross_platform_for_messaging_targets(self):
|
||||||
|
adapter = _make_adapter({})
|
||||||
|
mock_target = _wire_mock_target(adapter, "telegram")
|
||||||
|
|
||||||
|
result = await adapter._direct_deliver(
|
||||||
|
"hello",
|
||||||
|
{"deliver": "telegram", "deliver_extra": {"chat_id": "c-1"}},
|
||||||
|
)
|
||||||
|
assert result.success is True
|
||||||
|
mock_target.send.assert_awaited_once_with(
|
||||||
|
"c-1", "hello", metadata=None
|
||||||
|
)
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_dispatches_to_github_comment(self):
|
||||||
|
adapter = _make_adapter({})
|
||||||
|
with patch.object(
|
||||||
|
adapter, "_deliver_github_comment",
|
||||||
|
new=AsyncMock(return_value=SendResult(success=True)),
|
||||||
|
) as mock_gh:
|
||||||
|
result = await adapter._direct_deliver(
|
||||||
|
"review body",
|
||||||
|
{
|
||||||
|
"deliver": "github_comment",
|
||||||
|
"deliver_extra": {"repo": "org/r", "pr_number": "1"},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert result.success is True
|
||||||
|
mock_gh.assert_awaited_once()
|
||||||
|
|
@ -72,6 +72,7 @@ Routes define how different webhook sources are handled. Each route is a named e
|
||||||
| `skills` | No | List of skill names to load for the agent run. |
|
| `skills` | No | List of skill names to load for the agent run. |
|
||||||
| `deliver` | No | Where to send the response: `github_comment`, `telegram`, `discord`, `slack`, `signal`, `sms`, `whatsapp`, `matrix`, `mattermost`, `homeassistant`, `email`, `dingtalk`, `feishu`, `wecom`, `weixin`, `bluebubbles`, `qqbot`, or `log` (default). |
|
| `deliver` | No | Where to send the response: `github_comment`, `telegram`, `discord`, `slack`, `signal`, `sms`, `whatsapp`, `matrix`, `mattermost`, `homeassistant`, `email`, `dingtalk`, `feishu`, `wecom`, `weixin`, `bluebubbles`, `qqbot`, or `log` (default). |
|
||||||
| `deliver_extra` | No | Additional delivery config — keys depend on `deliver` type (e.g. `repo`, `pr_number`, `chat_id`). Values support the same `{dot.notation}` templates as `prompt`. |
|
| `deliver_extra` | No | Additional delivery config — keys depend on `deliver` type (e.g. `repo`, `pr_number`, `chat_id`). Values support the same `{dot.notation}` templates as `prompt`. |
|
||||||
|
| `deliver_only` | No | If `true`, skip the agent entirely — the rendered `prompt` template becomes the literal message that gets delivered. Zero LLM cost, sub-second delivery. See [Direct Delivery Mode](#direct-delivery-mode) for use cases. Requires `deliver` to be a real target (not `log`). |
|
||||||
|
|
||||||
### Full example
|
### Full example
|
||||||
|
|
||||||
|
|
@ -240,6 +241,80 @@ For cross-platform delivery, the target platform must also be enabled and connec
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Direct Delivery Mode {#direct-delivery-mode}
|
||||||
|
|
||||||
|
By default, every webhook POST triggers an agent run — the payload becomes a prompt, the agent processes it, and the agent's response is delivered. This costs LLM tokens on every event.
|
||||||
|
|
||||||
|
For use cases where you just want to **push a plain notification** — no reasoning, no agent loop, just deliver the message — set `deliver_only: true` on the route. The rendered `prompt` template becomes the literal message body, and the adapter dispatches it directly to the configured delivery target.
|
||||||
|
|
||||||
|
### When to use direct delivery
|
||||||
|
|
||||||
|
- **External service push** — Supabase/Firebase webhook fires on a database change → notify a user in Telegram instantly
|
||||||
|
- **Monitoring alerts** — Datadog/Grafana alert webhook → push to a Discord channel
|
||||||
|
- **Inter-agent pings** — Agent A notifies Agent B's user that a long-running task finished
|
||||||
|
- **Background job completion** — Cron job finishes → post result to Slack
|
||||||
|
|
||||||
|
Benefits:
|
||||||
|
|
||||||
|
- **Zero LLM tokens** — the agent is never invoked
|
||||||
|
- **Sub-second delivery** — a single adapter call, no reasoning loop
|
||||||
|
- **Same security as agent mode** — HMAC auth, rate limits, idempotency, and body-size limits all still apply
|
||||||
|
- **Synchronous response** — the POST returns `200 OK` once delivery succeeds, or `502` if the target rejects it, so your upstream service can retry intelligently
|
||||||
|
|
||||||
|
### Example: Telegram push from Supabase
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
platforms:
|
||||||
|
webhook:
|
||||||
|
enabled: true
|
||||||
|
extra:
|
||||||
|
port: 8644
|
||||||
|
secret: "global-secret"
|
||||||
|
routes:
|
||||||
|
antenna-matches:
|
||||||
|
secret: "antenna-webhook-secret"
|
||||||
|
deliver: "telegram"
|
||||||
|
deliver_only: true
|
||||||
|
prompt: "🎉 New match: {match.user_name} matched with you!"
|
||||||
|
deliver_extra:
|
||||||
|
chat_id: "{match.telegram_chat_id}"
|
||||||
|
```
|
||||||
|
|
||||||
|
Your Supabase edge function signs the payload with HMAC-SHA256 and POSTs to `https://your-server:8644/webhooks/antenna-matches`. The webhook adapter validates the signature, renders the template from the payload, delivers to Telegram, and returns `200 OK`.
|
||||||
|
|
||||||
|
### Example: Dynamic subscription via CLI
|
||||||
|
|
||||||
|
```bash
|
||||||
|
hermes webhook subscribe antenna-matches \
|
||||||
|
--deliver telegram \
|
||||||
|
--deliver-chat-id "123456789" \
|
||||||
|
--deliver-only \
|
||||||
|
--prompt "🎉 New match: {match.user_name} matched with you!" \
|
||||||
|
--description "Antenna match notifications"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Response codes
|
||||||
|
|
||||||
|
| Status | Meaning |
|
||||||
|
|--------|---------|
|
||||||
|
| `200 OK` | Delivered successfully. Body: `{"status": "delivered", "route": "...", "target": "...", "delivery_id": "..."}` |
|
||||||
|
| `200 OK` (status=duplicate) | Duplicate `X-GitHub-Delivery` ID within the idempotency TTL (1 hour). Not re-delivered. |
|
||||||
|
| `401 Unauthorized` | HMAC signature invalid or missing. |
|
||||||
|
| `400 Bad Request` | Malformed JSON body. |
|
||||||
|
| `404 Not Found` | Unknown route name. |
|
||||||
|
| `413 Payload Too Large` | Body exceeded `max_body_bytes`. |
|
||||||
|
| `429 Too Many Requests` | Route rate limit exceeded. |
|
||||||
|
| `502 Bad Gateway` | Target adapter rejected the message or raised. The error is logged server-side; the response body is a generic `Delivery failed` to avoid leaking adapter internals. |
|
||||||
|
|
||||||
|
### Configuration gotchas
|
||||||
|
|
||||||
|
- `deliver_only: true` requires `deliver` to be a real target. `deliver: log` (or omitting `deliver`) is rejected at startup — the adapter refuses to start if it finds a misconfigured route.
|
||||||
|
- The `skills` field is ignored in direct delivery mode (no agent runs, so there's nothing to inject skills into).
|
||||||
|
- Template rendering uses the same `{dot.notation}` syntax as agent mode, including the `{__raw__}` token.
|
||||||
|
- Idempotency uses the same `X-GitHub-Delivery` / `X-Request-ID` header — retries with the same ID return `status=duplicate` and do NOT re-deliver.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Dynamic Subscriptions (CLI) {#dynamic-subscriptions}
|
## Dynamic Subscriptions (CLI) {#dynamic-subscriptions}
|
||||||
|
|
||||||
In addition to static routes in `config.yaml`, you can create webhook subscriptions dynamically using the `hermes webhook` CLI command. This is especially useful when the agent itself needs to set up event-driven triggers.
|
In addition to static routes in `config.yaml`, you can create webhook subscriptions dynamically using the `hermes webhook` CLI command. This is especially useful when the agent itself needs to set up event-driven triggers.
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue