mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-30 19:09:28 +00:00
feat(gateway): iMessage-style message reactions — storage, RPC, agent tool, model context
Reactions live in the existing messages.display_metadata JSON column (no new table), with iOS Tapback semantics enforced DB-side: one reaction per author per message, re-tap retracts, different emoji replaces. The desktop catches up to the reaction contract five platform adapters already ship. - SessionDB: set/get_message_reaction, latest_message_row_id (role + offset + require_text so invisible tool-call-only rows are never targeted), take_unseen_reactions (announce-exactly-once), get_message_role - message.react RPC: accepts row_id or newest_role for live messages that haven't learned their durable id yet - react_to_message tool: desktop-gated (check_fn), defaults to the user's latest visible message, messages_back for retroactive reactions - Model context rides run_message only (beside the speech-interrupted note): the persisted prompt stays clean, so no [The user reacted …] scaffolding in transcripts, and no cached prefix ever changes - Resume projection forwards row_id + reactions; _row_id is stripped from outgoing API copies next to display_metadata
This commit is contained in:
parent
71a86101ab
commit
7d92056c49
9 changed files with 820 additions and 8 deletions
146
.plans/message-reactions.md
Normal file
146
.plans/message-reactions.md
Normal file
|
|
@ -0,0 +1,146 @@
|
|||
# Message reactions (desktop tapbacks)
|
||||
|
||||
Two-way emoji reactions on individual messages in the desktop transcript: the
|
||||
user reacts to any message, the agent reacts to a user message, and both sides
|
||||
read the other's reactions as conversational signal.
|
||||
|
||||
## What already exists
|
||||
|
||||
Hermes already models reactions on the **platform** side — the desktop is the
|
||||
only surface without them.
|
||||
|
||||
| Surface | Reaction support | Where |
|
||||
|---|---|---|
|
||||
| Agent → platform message | `send_message(action="react"/"unreact")` | `tools/send_message_tool.py:266` `_handle_react()` |
|
||||
| Photon / iMessage | tapbacks in + out, routed only for messages we sent | `plugins/platforms/photon/adapter.py:1240-1283` |
|
||||
| Telegram | `setMessageReaction`, config-gated | `plugins/platforms/telegram/adapter.py:9669+` |
|
||||
| Slack / Matrix / Feishu / Discord | inbound reaction events → hooks | `gateway/run.py:4688` `_handle_reaction_event()` → `HookRegistry.emit("reaction:added")` |
|
||||
| Adapter contract | `add_reaction()` / `remove_reaction()` coroutines, `set_reaction_handler()` | `gateway/platforms/base.py:3330` |
|
||||
| Core "affection" detector | regex on user text → `vibe`, drives CLI pet / TUI heart / desktop hearts | `agent/reactions.py`, `agent/turn_context.py:592-604` |
|
||||
|
||||
Two things follow from that table:
|
||||
|
||||
1. **The agent-facing verb already exists.** `send_message(action="react")` is
|
||||
the established shape. A desktop reaction should extend that tool, not add a
|
||||
new core tool — every new tool ships on every API call (AGENTS.md footprint
|
||||
ladder).
|
||||
2. **The inbound convention already exists.** Photon turns a tapback into a
|
||||
normal message event with `reply_to_message_id` + `reply_to_is_own_message`,
|
||||
and the gateway prefixes `[Replying to your previous message: "…"]`
|
||||
(`gateway/run.py:13125-13132`). Desktop reactions should read the same way to
|
||||
the model.
|
||||
|
||||
Nothing exists on the desktop side: `grep -ri reaction` across `apps/desktop`
|
||||
finds only the pet-overlay hearts.
|
||||
|
||||
## Prior art
|
||||
|
||||
**iOS Tapback** ([Apple](https://support.apple.com/guide/iphone/react-with-tapbacks-iph018d3c336/ios)):
|
||||
double-tap or touch-and-hold a message → floating pill above the bubble with
|
||||
heart / thumbs-up / thumbs-down / haha / ‼️ / ❓, swipe left for suggested emoji
|
||||
and stickers, or tap the emoji button for the full keyboard. **One tapback per
|
||||
message per person** — tapping the same one again removes it, tapping a
|
||||
different one replaces it. Multiple people's tapbacks stack on the badge.
|
||||
|
||||
**Platform data models** converge on the same shape:
|
||||
|
||||
| Platform | Model | Add / remove |
|
||||
|---|---|---|
|
||||
| Slack | `{name, count, users[]}` | [`reactions.add`](https://docs.slack.dev/reference/methods/reactions.add) / `reactions.remove`, emits `reaction_added` |
|
||||
| Discord | `{emoji, count, me}` on the message object | `PUT`/`DELETE .../reactions/{emoji}/@me` |
|
||||
| Telegram | `reaction: [{type:"emoji", emoji:"👍"}]` — replaces the whole set | `setMessageReaction`, `is_big` for the big animation |
|
||||
|
||||
Telegram's "set the whole array" is the closest match to iOS semantics and the
|
||||
simplest thing to persist.
|
||||
|
||||
**assistant-ui has no reaction primitive.** `@assistant-ui/react` 0.14.24 (MIT,
|
||||
vendored at `apps/desktop/node_modules`): zero hits for "reaction" in `core/src`,
|
||||
`react/src`, `dist/`, or the 2.2 MB `llms-full.txt` docs dump. What exists is a
|
||||
hard-coded binary `FeedbackAdapter` (`"positive" | "negative"`,
|
||||
`core/src/adapters/feedback.ts`) that throws when unconfigured and only writes
|
||||
back onto assistant messages. Not usable for emoji, not usable on user messages.
|
||||
|
||||
**But `metadata.custom` is the supported extension channel** and this repo
|
||||
already uses it: `ThreadUserMessage`/`ThreadAssistantMessage`/`ThreadSystemMessage`
|
||||
all carry `metadata.custom: Record<string, unknown>` (`core/src/types/message.ts:319-366`),
|
||||
and `chat-runtime.ts:397` already ships `custom: { attachmentRefs }` through it.
|
||||
|
||||
**Emoji picker survey** (npm week of 2026-07-22, sizes measured from the
|
||||
published ESM entry):
|
||||
|
||||
| Library | License | Weekly DL | gzip | Headless | Latest |
|
||||
|---|---|---|---|---|---|
|
||||
| **frimousse** | MIT | 573k | **8.5 kB** | ✅ fully unstyled, composable parts | 0.3.0 · 2025-07-15 |
|
||||
| emoji-picker-react | MIT | 1.31M | 87 kB | ❌ own CSS-in-JS (flairup) | 4.19.1 · 2026-04-27 |
|
||||
| emoji-mart | MIT | 2.22M | ~120 kB w/ data | ❌ Preact + shadow styling | 5.6.0 · **2024-04-25**, 217 open issues |
|
||||
| emoji-picker-element | Apache-2.0 | 183k | — | ❌ Web Component / Shadow DOM | 1.29.1 · 2026-03-01 |
|
||||
|
||||
No picker is currently a dependency (only `emoji-regex`, transitive). Already
|
||||
paid for and reusable: `radix-ui` (Popover), `motion`, `@tanstack/react-virtual`,
|
||||
Tailwind v4.
|
||||
|
||||
## Recommendation
|
||||
|
||||
**Hand-roll the tapback pill; add frimousse only behind the "+".** Six fixed
|
||||
emoji in a pill is ~40 lines of JSX against existing tokens — pulling 87 kB of
|
||||
`emoji-picker-react` to render six buttons, plus a CSS engine that fights
|
||||
`DESIGN.md`, is backwards. frimousse is headless, dependency-free, 10× smaller,
|
||||
and exposes `emojibaseUrl` so the data can be bundled as a Vite asset instead of
|
||||
hitting jsDelivr (Electron must work offline).
|
||||
|
||||
### Data model
|
||||
|
||||
One reaction per author per message, Telegram-style whole-set replacement:
|
||||
|
||||
```ts
|
||||
type MessageReaction = { emoji: string; author: 'user' | 'agent'; at: number }
|
||||
```
|
||||
|
||||
Persisted in the existing `messages.display_metadata` JSON column
|
||||
(`hermes_state_common.py:215`) — no new table. It already survives insert,
|
||||
compaction, and every read projection, and
|
||||
`set_latest_matching_message_display_kind()` (`hermes_state.py:5292`) is the
|
||||
precedent for stamping metadata onto an already-persisted row.
|
||||
|
||||
### Model context
|
||||
|
||||
Reactions must reach the model **without breaking prompt caching**. The
|
||||
`api_messages` build loop strips `display_metadata` from every outgoing copy
|
||||
(`agent/conversation_loop.py:1443-1446`) precisely so display state never
|
||||
becomes a provider field. Two candidate paths:
|
||||
|
||||
| Path | Cache impact | Notes |
|
||||
|---|---|---|
|
||||
| Rewrite the reacted-to message's content to carry the annotation | **Breaks the cached prefix** — mutates past context | Rejected. AGENTS.md: prompt caching is sacred. |
|
||||
| Deliver the reaction as the *next* turn's leading annotation, mirroring photon | Prefix untouched; only the new turn carries it | Matches `[Replying to your previous message: "…"]` (`gateway/run.py:13125`), which the agent already understands |
|
||||
|
||||
The second is the same trick the platform adapters already use, so the model
|
||||
sees a familiar shape and no existing conversation is rewritten.
|
||||
|
||||
### Attach points
|
||||
|
||||
| Concern | File | Lines |
|
||||
|---|---|---|
|
||||
| Assistant hover bar | `apps/desktop/src/components/assistant-ui/thread/assistant-message.tsx` | 134–175 |
|
||||
| User hover cluster | `apps/desktop/src/components/assistant-ui/thread/user-message.tsx` | 296–336 |
|
||||
| Callback threading (ref caveat 79–99) | `apps/desktop/src/components/assistant-ui/thread/index.tsx` | 109–133 |
|
||||
| `metadata.custom` → runtime | `apps/desktop/src/lib/chat-runtime.ts` | 384–432 |
|
||||
| RPC client ↔ server pattern | `sidebar/session-actions-menu.tsx:62-89` ↔ `tui_gateway/server.py:8322` | — |
|
||||
| Persistence | `hermes_state_common.py:192-216`, `hermes_state.py:5292-5324` | — |
|
||||
| Prompt injection / strip | `agent/conversation_loop.py` | 1430–1529 |
|
||||
|
||||
### Known gaps to solve first
|
||||
|
||||
- **No durable message id crosses the gateway RPC path.** `_history_to_messages()`
|
||||
(`tui_gateway/server.py:6545`) builds `{"role", "text"}` and drops the id. The
|
||||
REST path carries `messages.id` incidentally via `SELECT *` but TS
|
||||
`SessionMessage` (`types/hermes.ts:513-533`) doesn't declare it. Renderer ids
|
||||
are ephemeral and change shape between rehydrated (`<ts>-<i>-<role>`), live
|
||||
(`assistant-<ms>`), and optimistic (`user-<ms>-<rand>`) messages. A reaction
|
||||
needs a stable key — this is the first thing to fix.
|
||||
- **WeakMap identity cache** in `apps/desktop/src/app/chat/runtime-repository.ts:26-66`
|
||||
keys normalized `ThreadMessage` by `ChatMessage` identity. A reaction change
|
||||
must produce a **new** `ChatMessage` object or the UI renders stale.
|
||||
- **Rewind rewrites rows** (`replace_messages`), so anything keyed by row id
|
||||
needs cascade handling — an argument for keeping reactions in
|
||||
`display_metadata` on the row itself rather than a side table.
|
||||
|
|
@ -1445,6 +1445,12 @@ def run_conversation(
|
|||
api_msg.pop("display_kind", None)
|
||||
api_msg.pop("display_metadata", None)
|
||||
|
||||
# Durable row identity stamped by _rows_to_conversation so the
|
||||
# desktop can address a specific persisted message (reactions).
|
||||
# Bookkeeping, never a provider field — only the chat-completions
|
||||
# transport strips underscore keys, so drop it centrally here.
|
||||
api_msg.pop("_row_id", None)
|
||||
|
||||
# Inject ephemeral context into the current turn's user message.
|
||||
# Sources: memory manager prefetch + plugin pre_llm_call hooks
|
||||
# with target="user_message" (the default). Both are
|
||||
|
|
|
|||
221
hermes_state.py
221
hermes_state.py
|
|
@ -5647,6 +5647,214 @@ class SessionDB(SessionSearchMixin, SessionSchemaMixin, SessionPortabilityMixin)
|
|||
|
||||
return bool(self._execute_write(_do))
|
||||
|
||||
#: Key under which message reactions live inside ``display_metadata``.
|
||||
#: Reactions share the existing per-message JSON column rather than a side
|
||||
#: table so they survive rewind/compaction row rewrites with the row itself.
|
||||
REACTIONS_METADATA_KEY = "reactions"
|
||||
|
||||
def set_message_reaction(
|
||||
self,
|
||||
session_id: str,
|
||||
message_row_id: int,
|
||||
emoji: Optional[str],
|
||||
*,
|
||||
author: str = "user",
|
||||
) -> Optional[List[Dict[str, Any]]]:
|
||||
"""Set (or with ``emoji=None`` clear) *author*'s reaction on one message.
|
||||
|
||||
iOS Tapback semantics: one reaction per author per message. Re-sending
|
||||
the same emoji clears it, a different emoji replaces it. Returns the
|
||||
message's full reaction list after the write, or ``None`` when the row
|
||||
doesn't exist or isn't part of *session_id*.
|
||||
"""
|
||||
if not session_id or message_row_id is None:
|
||||
return None
|
||||
|
||||
def _do(conn):
|
||||
row = conn.execute(
|
||||
"SELECT display_metadata FROM messages WHERE id = ? AND session_id = ?",
|
||||
(message_row_id, session_id),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
|
||||
meta = self._decode_display_metadata(row[0]) or {}
|
||||
existing = meta.get(self.REACTIONS_METADATA_KEY)
|
||||
reactions = [
|
||||
r
|
||||
for r in (existing if isinstance(existing, list) else [])
|
||||
if isinstance(r, dict) and r.get("author") != author
|
||||
]
|
||||
previous = next(
|
||||
(
|
||||
r
|
||||
for r in (existing if isinstance(existing, list) else [])
|
||||
if isinstance(r, dict) and r.get("author") == author
|
||||
),
|
||||
None,
|
||||
)
|
||||
# Tapping the live reaction again retracts it.
|
||||
toggling_off = (
|
||||
emoji is not None and previous is not None and previous.get("emoji") == emoji
|
||||
)
|
||||
if emoji and not toggling_off:
|
||||
reactions.append(
|
||||
{"emoji": _scrub_surrogates(emoji), "author": author, "at": time.time()}
|
||||
)
|
||||
|
||||
if reactions:
|
||||
meta[self.REACTIONS_METADATA_KEY] = reactions
|
||||
else:
|
||||
meta.pop(self.REACTIONS_METADATA_KEY, None)
|
||||
|
||||
conn.execute(
|
||||
"UPDATE messages SET display_metadata = ? WHERE id = ?",
|
||||
(self._encode_display_metadata(meta) if meta else None, message_row_id),
|
||||
)
|
||||
return reactions
|
||||
|
||||
return self._execute_write(_do)
|
||||
|
||||
def get_message_reactions(
|
||||
self, session_id: str, message_row_id: int
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Return the reaction list persisted on one message row (never ``None``)."""
|
||||
if not session_id or message_row_id is None:
|
||||
return []
|
||||
|
||||
with self._lock:
|
||||
row = self._conn.execute(
|
||||
"SELECT display_metadata FROM messages WHERE id = ? AND session_id = ?",
|
||||
(message_row_id, session_id),
|
||||
).fetchone()
|
||||
|
||||
if row is None:
|
||||
return []
|
||||
|
||||
meta = self._decode_display_metadata(row[0]) or {}
|
||||
reactions = meta.get(self.REACTIONS_METADATA_KEY)
|
||||
|
||||
return [r for r in reactions if isinstance(r, dict)] if isinstance(reactions, list) else []
|
||||
|
||||
def take_unseen_reactions(
|
||||
self, session_id: str, *, author: str = "user"
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Return *author*'s not-yet-surfaced reactions and mark them seen.
|
||||
|
||||
Powers the cache-safe model-context path: reactions are announced on the
|
||||
NEXT user turn (never by rewriting the message that was reacted to), and
|
||||
the ``seen`` stamp guarantees each one is announced exactly once.
|
||||
"""
|
||||
if not session_id:
|
||||
return []
|
||||
|
||||
def _do(conn):
|
||||
rows = conn.execute(
|
||||
"SELECT id, role, content, display_metadata FROM messages "
|
||||
"WHERE session_id = ? AND active = 1 AND display_metadata IS NOT NULL "
|
||||
"ORDER BY id",
|
||||
(session_id,),
|
||||
).fetchall()
|
||||
|
||||
pending = []
|
||||
for row in rows:
|
||||
meta = self._decode_display_metadata(row["display_metadata"])
|
||||
if not meta:
|
||||
continue
|
||||
reactions = meta.get(self.REACTIONS_METADATA_KEY)
|
||||
if not isinstance(reactions, list):
|
||||
continue
|
||||
|
||||
changed = False
|
||||
for reaction in reactions:
|
||||
if (
|
||||
not isinstance(reaction, dict)
|
||||
or reaction.get("author") != author
|
||||
or reaction.get("seen")
|
||||
):
|
||||
continue
|
||||
reaction["seen"] = True
|
||||
changed = True
|
||||
content = self._decode_content(row["content"])
|
||||
pending.append(
|
||||
{
|
||||
"row_id": row["id"],
|
||||
"role": row["role"],
|
||||
"emoji": reaction.get("emoji") or "",
|
||||
"text": content if isinstance(content, str) else "",
|
||||
}
|
||||
)
|
||||
|
||||
if changed:
|
||||
conn.execute(
|
||||
"UPDATE messages SET display_metadata = ? WHERE id = ?",
|
||||
(self._encode_display_metadata(meta), row["id"]),
|
||||
)
|
||||
|
||||
return pending
|
||||
|
||||
return self._execute_write(_do) or []
|
||||
|
||||
def latest_message_row_id(
|
||||
self, session_id: str, *, role: str = "user", offset: int = 0, require_text: bool = True
|
||||
) -> Optional[int]:
|
||||
"""Row id of the most recent active message with *role*, or ``None``.
|
||||
|
||||
Two callers, same need — "the message I mean, without an id": the agent
|
||||
defaulting to the turn that triggered it, and the desktop reacting to a
|
||||
live message that hasn't round-tripped through a resume yet.
|
||||
``offset`` steps to earlier turns (1 = the one before the latest) so a
|
||||
reaction can land retroactively — "two messages ago" is how the caller
|
||||
thinks about it.
|
||||
|
||||
``require_text`` (default) skips rows with no plain-text content —
|
||||
tool-call-only assistant turns and attachment stubs don't render as
|
||||
bubbles, so "the latest message" as a HUMAN means it must never
|
||||
resolve to one (a reaction landing on an invisible row looks dropped,
|
||||
and its annotation quotes an empty string).
|
||||
"""
|
||||
if not session_id or role not in {"user", "assistant"} or offset < 0:
|
||||
return None
|
||||
|
||||
text_filter = (
|
||||
"AND content IS NOT NULL AND TRIM(content) != '' " if require_text else ""
|
||||
)
|
||||
|
||||
with self._lock:
|
||||
row = self._conn.execute(
|
||||
"SELECT id FROM messages WHERE session_id = ? AND role = ? "
|
||||
f"AND active = 1 {text_filter}ORDER BY id DESC LIMIT 1 OFFSET ?",
|
||||
(session_id, role, int(offset)),
|
||||
).fetchone()
|
||||
|
||||
return row[0] if row else None
|
||||
|
||||
def latest_user_message_row_id(self, session_id: str) -> Optional[int]:
|
||||
"""Row id of the most recent active user message, or ``None``.
|
||||
|
||||
The agent's default reaction target: "the message that triggered me",
|
||||
so the model never has to thread row ids through a tool call (mirrors
|
||||
the photon adapter's ``_record_last_inbound``).
|
||||
"""
|
||||
return self.latest_message_row_id(session_id, role="user")
|
||||
|
||||
def get_message_role(self, session_id: str, row_id: int) -> Optional[str]:
|
||||
"""Role of the active message at *row_id* in *session_id*, or ``None``.
|
||||
|
||||
Lets a reaction event carry the target's role so a renderer can match
|
||||
a live message that doesn't know its durable row id yet.
|
||||
"""
|
||||
if not session_id:
|
||||
return None
|
||||
|
||||
with self._lock:
|
||||
row = self._conn.execute(
|
||||
"SELECT role FROM messages WHERE id = ? AND session_id = ? AND active = 1",
|
||||
(int(row_id), session_id),
|
||||
).fetchone()
|
||||
|
||||
return row[0] if row else None
|
||||
|
||||
def _insert_message_rows(self, conn, session_id: str, messages: List[Dict[str, Any]]) -> tuple[int, int]:
|
||||
"""Insert *messages* as fresh active rows for *session_id*.
|
||||
|
||||
|
|
@ -6154,10 +6362,7 @@ class SessionDB(SessionSearchMixin, SessionSchemaMixin, SessionPortabilityMixin)
|
|||
with self._lock:
|
||||
placeholders = ",".join("?" for _ in session_ids)
|
||||
rows = self._conn.execute(
|
||||
"SELECT role, content, tool_call_id, tool_calls, tool_name, effect_disposition, "
|
||||
"finish_reason, reasoning, reasoning_content, reasoning_details, "
|
||||
"codex_reasoning_items, codex_message_items, platform_message_id, observed, timestamp, "
|
||||
"api_content, display_kind, display_metadata "
|
||||
f"SELECT {self._CONVERSATION_ROW_COLUMNS} "
|
||||
f"FROM messages WHERE session_id IN ({placeholders})"
|
||||
# Order by AUTOINCREMENT id (true insertion order), NOT timestamp:
|
||||
# append_message stamps rows with time.time(), which is not
|
||||
|
|
@ -6182,7 +6387,7 @@ class SessionDB(SessionSearchMixin, SessionSchemaMixin, SessionPortabilityMixin)
|
|||
# get_messages_as_conversation and get_resume_conversations so a single
|
||||
# SELECT can feed both the model-fed and display views.
|
||||
_CONVERSATION_ROW_COLUMNS = (
|
||||
"role, content, tool_call_id, tool_calls, tool_name, effect_disposition, "
|
||||
"id, role, content, tool_call_id, tool_calls, tool_name, effect_disposition, "
|
||||
"finish_reason, reasoning, reasoning_content, reasoning_details, "
|
||||
"codex_reasoning_items, codex_message_items, platform_message_id, observed, timestamp, "
|
||||
"api_content, display_kind, display_metadata"
|
||||
|
|
@ -6209,6 +6414,12 @@ class SessionDB(SessionSearchMixin, SessionSchemaMixin, SessionPortabilityMixin)
|
|||
if row["role"] in {"user", "assistant"} and isinstance(content, str):
|
||||
content = sanitize_context(content).strip()
|
||||
msg = {"role": row["role"], "content": content}
|
||||
# Durable per-message identity for surfaces that need to address a
|
||||
# specific row later (desktop reactions). Underscore-prefixed so
|
||||
# every transport's convert_messages() strips it before the wire —
|
||||
# the established escape hatch for agent-internal bookkeeping.
|
||||
if row["id"] is not None:
|
||||
msg["_row_id"] = row["id"]
|
||||
# api_content is the byte-fidelity sidecar: the exact string sent
|
||||
# to the API when it differed from the clean content. Returned
|
||||
# VERBATIM — no sanitize_context, no strip — because the replay
|
||||
|
|
|
|||
169
tests/test_message_reactions.py
Normal file
169
tests/test_message_reactions.py
Normal file
|
|
@ -0,0 +1,169 @@
|
|||
"""Message reactions: persistence, tapback semantics, and cache safety.
|
||||
|
||||
Behavior contracts, not snapshots — these assert how reactions must RELATE to
|
||||
the transcript (one per author, announced once, never mutating history), so
|
||||
they survive refactors of where reactions are stored.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from hermes_state import SessionDB
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def db(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
|
||||
return SessionDB(db_path=tmp_path / "state.db")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def session(db):
|
||||
key = db.create_session("react-test", "test")
|
||||
db.append_message(key, "user", "how do i center a div")
|
||||
db.append_message(key, "assistant", "use flexbox")
|
||||
rows = [m["_row_id"] for m in db.get_messages_as_conversation(key)]
|
||||
|
||||
return key, rows
|
||||
|
||||
|
||||
def test_conversation_rows_carry_durable_row_id(session, db):
|
||||
"""Every projected message exposes its messages.id — reactions key off it."""
|
||||
key, rows = session
|
||||
|
||||
assert all(isinstance(r, int) for r in rows)
|
||||
assert rows == sorted(rows), "row ids must follow insertion order"
|
||||
assert len(set(rows)) == len(rows), "row ids must be unique"
|
||||
|
||||
|
||||
def test_one_reaction_per_author(session, db):
|
||||
"""A second emoji from the same author REPLACES the first (iOS Tapback)."""
|
||||
key, rows = session
|
||||
|
||||
db.set_message_reaction(key, rows[0], "\u2764\ufe0f", author="user")
|
||||
reactions = db.set_message_reaction(key, rows[0], "\U0001f602", author="user")
|
||||
|
||||
assert [r["emoji"] for r in reactions] == ["\U0001f602"]
|
||||
|
||||
|
||||
def test_repeating_an_emoji_retracts_it(session, db):
|
||||
"""Tapping the live reaction again clears it."""
|
||||
key, rows = session
|
||||
|
||||
db.set_message_reaction(key, rows[0], "\U0001f44d", author="user")
|
||||
reactions = db.set_message_reaction(key, rows[0], "\U0001f44d", author="user")
|
||||
|
||||
assert reactions == []
|
||||
assert db.get_message_reactions(key, rows[0]) == []
|
||||
|
||||
|
||||
def test_authors_are_independent(session, db):
|
||||
"""User and agent each hold their own slot on the same message."""
|
||||
key, rows = session
|
||||
|
||||
db.set_message_reaction(key, rows[0], "\u2764\ufe0f", author="user")
|
||||
reactions = db.set_message_reaction(key, rows[0], "\U0001f525", author="agent")
|
||||
|
||||
assert {r["author"] for r in reactions} == {"user", "agent"}
|
||||
|
||||
remaining = db.set_message_reaction(key, rows[0], None, author="user")
|
||||
assert [r["author"] for r in remaining] == ["agent"]
|
||||
|
||||
|
||||
def test_rejects_rows_outside_the_session(session, db):
|
||||
"""A row id from another conversation is never writable."""
|
||||
key, rows = session
|
||||
other = db.create_session("other", "test")
|
||||
db.append_message(other, "user", "elsewhere")
|
||||
|
||||
assert db.set_message_reaction(key, 9999, "\u2764\ufe0f") is None
|
||||
assert db.set_message_reaction("no-such-session", rows[0], "\u2764\ufe0f") is None
|
||||
|
||||
|
||||
def test_clearing_every_reaction_leaves_no_metadata(session, db):
|
||||
"""An empty reaction set removes the key instead of persisting `[]`."""
|
||||
key, rows = session
|
||||
|
||||
db.set_message_reaction(key, rows[0], "\u2764\ufe0f", author="user")
|
||||
db.set_message_reaction(key, rows[0], None, author="user")
|
||||
|
||||
message = db.get_messages_as_conversation(key)[0]
|
||||
assert "display_metadata" not in message
|
||||
|
||||
|
||||
def test_reactions_survive_reload(session, db):
|
||||
"""Reactions are durable, not in-memory display state."""
|
||||
key, rows = session
|
||||
db.set_message_reaction(key, rows[1], "\U0001f525", author="agent")
|
||||
|
||||
reopened = SessionDB(db_path=db.db_path)
|
||||
assert [r["emoji"] for r in reopened.get_message_reactions(key, rows[1])] == ["\U0001f525"]
|
||||
|
||||
|
||||
def test_unseen_reactions_are_taken_exactly_once(session, db):
|
||||
"""The model is told about a reaction on ONE turn, never twice."""
|
||||
key, rows = session
|
||||
db.set_message_reaction(key, rows[1], "\u2764\ufe0f", author="user")
|
||||
|
||||
first = db.take_unseen_reactions(key, author="user")
|
||||
assert [e["emoji"] for e in first] == ["\u2764\ufe0f"]
|
||||
assert first[0]["row_id"] == rows[1]
|
||||
assert first[0]["text"] == "use flexbox"
|
||||
|
||||
assert db.take_unseen_reactions(key, author="user") == []
|
||||
|
||||
|
||||
def test_a_new_reaction_becomes_unseen_again(session, db):
|
||||
"""Replacing a seen reaction re-arms the announcement."""
|
||||
key, rows = session
|
||||
db.set_message_reaction(key, rows[1], "\u2764\ufe0f", author="user")
|
||||
db.take_unseen_reactions(key, author="user")
|
||||
|
||||
db.set_message_reaction(key, rows[1], "\U0001f525", author="user")
|
||||
|
||||
assert [e["emoji"] for e in db.take_unseen_reactions(key, author="user")] == ["\U0001f525"]
|
||||
|
||||
|
||||
def test_take_unseen_filters_by_author(session, db):
|
||||
"""The agent's own reactions are never fed back to it as user input."""
|
||||
key, rows = session
|
||||
db.set_message_reaction(key, rows[0], "\U0001f60a", author="agent")
|
||||
|
||||
assert db.take_unseen_reactions(key, author="user") == []
|
||||
|
||||
|
||||
def test_reacting_never_mutates_message_content(session, db):
|
||||
"""CACHE SAFETY: reacting must not rewrite any already-sent message.
|
||||
|
||||
Rewriting a past message would invalidate the provider's cached prefix for
|
||||
the whole conversation — the reason reactions ride display_metadata and are
|
||||
announced on the NEXT turn instead.
|
||||
"""
|
||||
key, rows = session
|
||||
before = [m["content"] for m in db.get_messages_as_conversation(key)]
|
||||
|
||||
db.set_message_reaction(key, rows[0], "\u2764\ufe0f", author="user")
|
||||
db.set_message_reaction(key, rows[1], "\U0001f525", author="agent")
|
||||
db.take_unseen_reactions(key, author="user")
|
||||
|
||||
after = [m["content"] for m in db.get_messages_as_conversation(key)]
|
||||
assert before == after
|
||||
|
||||
|
||||
def test_latest_user_message_is_the_agents_default_target(session, db):
|
||||
"""The agent reacts to "the message that triggered me" without an id."""
|
||||
key, rows = session
|
||||
assert db.latest_user_message_row_id(key) == rows[0]
|
||||
|
||||
db.append_message(key, "user", "thanks!")
|
||||
newest = db.get_messages_as_conversation(key)[-1]["_row_id"]
|
||||
|
||||
assert db.latest_user_message_row_id(key) == newest
|
||||
|
||||
|
||||
def test_row_id_never_reaches_the_provider(session, db):
|
||||
"""_row_id is underscore-prefixed so transports strip it before the wire."""
|
||||
key, _rows = session
|
||||
|
||||
for message in db.get_messages_as_conversation(key):
|
||||
assert "_row_id" in message
|
||||
assert all(not k.startswith("_") or k == "_row_id" for k in message)
|
||||
154
tools/react_to_message_tool.py
Normal file
154
tools/react_to_message_tool.py
Normal file
|
|
@ -0,0 +1,154 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Let the agent react to a message with an emoji in the Hermes desktop app.
|
||||
|
||||
The conversational counterpart to the user's tapback: the same reaction store,
|
||||
the same one-per-author semantics, just written with ``author="agent"``.
|
||||
|
||||
Gated on ``HERMES_DESKTOP`` (like the other GUI affordances) so it costs nothing
|
||||
on every other surface — the platform adapters already expose reactions through
|
||||
``send_message(action="react")``, and this is the desktop's equivalent.
|
||||
|
||||
Defaults to the message that triggered this turn (the photon precedent: the
|
||||
model shouldn't have to thread row ids through tool calls), and emits
|
||||
``message.reaction`` so the renderer paints it without waiting for a resume.
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
from gateway.session_context import get_session_env
|
||||
from tools import desktop_ui
|
||||
from tools.registry import registry, tool_error
|
||||
from utils import env_var_enabled
|
||||
|
||||
|
||||
def _open_session_db():
|
||||
"""Open the SessionDB for the profile owning this turn, or ``None``."""
|
||||
try:
|
||||
from hermes_state import SessionDB
|
||||
|
||||
return SessionDB()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def react_to_message_tool(emoji: str, message_row_id=None, messages_back=None) -> str:
|
||||
"""Attach (or with an empty ``emoji`` retract) the agent's reaction."""
|
||||
emoji = (emoji or "").strip()
|
||||
session_key = get_session_env("HERMES_SESSION_KEY", "") or get_session_env(
|
||||
"HERMES_SESSION_ID", ""
|
||||
)
|
||||
|
||||
if not session_key:
|
||||
return tool_error("No active session — reactions need a persisted conversation.")
|
||||
|
||||
db = _open_session_db()
|
||||
if db is None:
|
||||
return tool_error("Session storage is unavailable.")
|
||||
|
||||
row_id = message_row_id
|
||||
target_role = "user"
|
||||
if row_id is None:
|
||||
# Default target: the latest user message. `messages_back` steps to
|
||||
# earlier user turns (1 = the one before, etc.) for retroactive
|
||||
# reactions — quoting text would be ambiguous, ids aren't visible to
|
||||
# the model, but "two messages ago" is how a person thinks about it.
|
||||
back = max(0, int(messages_back or 0))
|
||||
row_id = db.latest_message_row_id(session_key, role="user", offset=back)
|
||||
if row_id is None:
|
||||
return tool_error(
|
||||
f"No user message found {back} back." if back else "No user message to react to yet."
|
||||
)
|
||||
else:
|
||||
row = db.get_message_role(session_key, int(row_id))
|
||||
target_role = row or "user"
|
||||
|
||||
try:
|
||||
reactions = db.set_message_reaction(
|
||||
session_key, int(row_id), emoji or None, author="agent"
|
||||
)
|
||||
except Exception as exc:
|
||||
return tool_error(f"Failed to set the reaction: {exc}")
|
||||
|
||||
if reactions is None:
|
||||
return tool_error(f"Message {row_id} is not part of this conversation.")
|
||||
|
||||
# Paint it live. A missing bridge (non-desktop surface) is not an error —
|
||||
# the reaction is persisted either way and shows on the next load.
|
||||
# `role` lets the renderer match a live message that doesn't know its
|
||||
# durable row id yet (it only learns rowId on resume).
|
||||
try:
|
||||
desktop_ui.emit(
|
||||
"message.reaction",
|
||||
{"row_id": int(row_id), "reactions": reactions, "role": target_role},
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return json.dumps(
|
||||
{"success": True, "row_id": int(row_id), "reactions": reactions}, ensure_ascii=False
|
||||
)
|
||||
|
||||
|
||||
def check_react_requirements() -> bool:
|
||||
"""Desktop GUI only — HERMES_DESKTOP is set on the gateway the app spawns."""
|
||||
return env_var_enabled("HERMES_DESKTOP")
|
||||
|
||||
|
||||
REACT_TO_MESSAGE_SCHEMA = {
|
||||
"name": "react_to_message",
|
||||
"description": (
|
||||
"React to a message with a single emoji, the way you'd tapback in iMessage. "
|
||||
"Reach for it when a reaction is what a person would do: something funny gets "
|
||||
"a 😂, warmth gets a ❤️, a plan you're on board with gets a 👍 — then just "
|
||||
"carry on with whatever the message actually needs. If a reaction says it "
|
||||
"all, it can BE the reply (skip the redundant 'sounds good!' turn). Use it "
|
||||
"like a person would: occasionally, when felt — not on every message, and "
|
||||
"never as a status signal. NEVER narrate or explain a reaction ('I reacted "
|
||||
"with...', 'Reacting now') — the emoji appearing on the bubble is the whole "
|
||||
"point, and commentary kills it. Defaults to the user's most recent message. "
|
||||
"One reaction per message: a different emoji replaces yours, an empty string "
|
||||
"retracts it."
|
||||
),
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"emoji": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"The emoji to react with (e.g. '❤️', '😂', '👍'). Pass an empty "
|
||||
"string to remove your reaction."
|
||||
),
|
||||
},
|
||||
"message_row_id": {
|
||||
"type": "integer",
|
||||
"description": (
|
||||
"Optional. The specific message to react to. Omit to react to the "
|
||||
"user's latest message, which is almost always what you want."
|
||||
),
|
||||
},
|
||||
"messages_back": {
|
||||
"type": "integer",
|
||||
"description": (
|
||||
"Optional. React to an EARLIER user message: 1 = the one before "
|
||||
"the latest, 2 = two before, and so on. For when something lands "
|
||||
"late — the joke you only got after answering."
|
||||
),
|
||||
},
|
||||
},
|
||||
"required": ["emoji"],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
registry.register(
|
||||
name="react_to_message",
|
||||
toolset="terminal",
|
||||
schema=REACT_TO_MESSAGE_SCHEMA,
|
||||
handler=lambda args, **kw: react_to_message_tool(
|
||||
emoji=args.get("emoji", ""),
|
||||
message_row_id=args.get("message_row_id"),
|
||||
messages_back=args.get("messages_back"),
|
||||
),
|
||||
check_fn=check_react_requirements,
|
||||
emoji="💛",
|
||||
)
|
||||
|
|
@ -34,9 +34,10 @@ _HERMES_CORE_TOOLS = [
|
|||
# Terminal + process management
|
||||
"terminal", "process",
|
||||
# Desktop GUI affordances: read the embedded terminal pane, close an agent's
|
||||
# read-only terminal tab, open a URL/file in the preview pane, and focus a
|
||||
# pane (all gated on HERMES_DESKTOP via check_fn — hidden outside the GUI).
|
||||
"read_terminal", "close_terminal", "open_preview", "focus_pane",
|
||||
# read-only terminal tab, open a URL/file in the preview pane, focus a
|
||||
# pane, and react to a message with an emoji (all gated on HERMES_DESKTOP
|
||||
# via check_fn — hidden outside the GUI).
|
||||
"read_terminal", "close_terminal", "open_preview", "focus_pane", "react_to_message",
|
||||
# File manipulation
|
||||
"read_file", "write_file", "patch", "search_files",
|
||||
# Vision + image generation
|
||||
|
|
|
|||
|
|
@ -6,11 +6,55 @@ are rebound onto server.py's globals at install time — see method_ctx.py.
|
|||
|
||||
from .method_ctx import HandlerRegistry
|
||||
|
||||
import types
|
||||
|
||||
_registry = HandlerRegistry()
|
||||
method = _registry.method
|
||||
_profile_scoped = _registry.profile_scoped
|
||||
|
||||
|
||||
def _pending_reaction_notes(session: dict) -> str:
|
||||
"""Note block describing reactions the user added since the last turn, or "".
|
||||
|
||||
Applied to the MODEL INPUT only (``run_message``, beside the
|
||||
speech-interrupted note) — never to the text that gets persisted. Prefixing
|
||||
the persisted prompt bakes scaffolding into the transcript, which every
|
||||
surface then renders as a garbled user message on reload. Each reaction is
|
||||
announced once — the row is stamped ``seen`` on read.
|
||||
"""
|
||||
session_key = str(session.get("session_key") or "")
|
||||
if not session_key:
|
||||
return ""
|
||||
|
||||
try:
|
||||
with _session_db(session) as db:
|
||||
if db is None:
|
||||
return ""
|
||||
pending = db.take_unseen_reactions(session_key, author="user")
|
||||
except Exception:
|
||||
logger.debug("Failed to read pending reactions", exc_info=True)
|
||||
return ""
|
||||
|
||||
if not pending:
|
||||
return ""
|
||||
|
||||
notes = []
|
||||
for entry in pending:
|
||||
snippet = (entry.get("text") or "").strip().replace("\n", " ")
|
||||
if len(snippet) > 120:
|
||||
snippet = snippet[:120] + "…"
|
||||
emoji = entry.get("emoji") or ""
|
||||
whose = "their own" if entry.get("role") == "user" else "your"
|
||||
if snippet:
|
||||
notes.append(f'[The user reacted {emoji} to {whose} message: "{snippet}"]')
|
||||
else:
|
||||
# A row with no plain text (attachment-only, or a tool-call-only
|
||||
# assistant turn) — an empty quote reads worse than no quote.
|
||||
notes.append(f"[The user reacted {emoji} to {whose} earlier message]")
|
||||
|
||||
return "\n".join(notes)
|
||||
|
||||
|
||||
@method("prompt.submit")
|
||||
def _(rid, params: dict) -> dict:
|
||||
from hermes_cli.input_sanitize import sanitize_user_prompt_text
|
||||
|
|
@ -833,3 +877,13 @@ def _(rid, params: dict) -> dict:
|
|||
def register(server) -> None:
|
||||
"""Bind this module's handlers onto ``server``'s globals and registry."""
|
||||
_registry.install(server)
|
||||
# Module-level helpers aren't @method handlers, so install() doesn't see
|
||||
# them — but server.py's run path calls this one (run_message enrichment,
|
||||
# beside the speech-interrupted note). Rebind and publish it the same way.
|
||||
server._pending_reaction_notes = types.FunctionType(
|
||||
_pending_reaction_notes.__code__,
|
||||
vars(server),
|
||||
_pending_reaction_notes.__name__,
|
||||
_pending_reaction_notes.__defaults__,
|
||||
_pending_reaction_notes.__closure__,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -919,6 +919,59 @@ def _(rid, params: dict) -> dict:
|
|||
return _err(rid, 5007, str(e))
|
||||
|
||||
|
||||
@method("message.react")
|
||||
def _(rid, params: dict) -> dict:
|
||||
"""Set or clear one author's emoji reaction on a persisted message.
|
||||
|
||||
iOS Tapback semantics, enforced in the DB layer: one reaction per author
|
||||
per message, re-sending the same emoji retracts it. ``emoji: null`` clears
|
||||
unconditionally. ``row_id`` is the durable ``messages.id`` forwarded by
|
||||
``_history_to_messages`` — the renderer's own message ids are ephemeral.
|
||||
"""
|
||||
session, err = _sess_nowait(params, rid)
|
||||
if err:
|
||||
return err
|
||||
|
||||
# A live message hasn't round-tripped through a resume, so the desktop has
|
||||
# no durable row id for it yet. It can instead name the ROLE whose newest
|
||||
# row it means — which is the message the user just reacted to.
|
||||
newest_role = str(params.get("newest_role") or "").strip()
|
||||
row_id = params.get("row_id")
|
||||
if row_id is None and newest_role not in {"user", "assistant"}:
|
||||
return _err(rid, 4023, "row_id or newest_role required")
|
||||
|
||||
emoji = params.get("emoji")
|
||||
if emoji is not None:
|
||||
emoji = str(emoji).strip()
|
||||
if not emoji:
|
||||
return _err(rid, 4024, "emoji must be a non-empty string or null")
|
||||
|
||||
author = str(params.get("author") or "user").strip()
|
||||
if author not in {"user", "agent"}:
|
||||
return _err(rid, 4025, "author must be 'user' or 'agent'")
|
||||
|
||||
with _session_db(session) as db:
|
||||
if db is None:
|
||||
return _db_unavailable_error(rid, code=5007)
|
||||
try:
|
||||
if row_id is None:
|
||||
row_id = db.latest_message_row_id(
|
||||
session["session_key"], role=newest_role
|
||||
)
|
||||
if row_id is None:
|
||||
return _err(rid, 4040, "no message to react to yet")
|
||||
reactions = db.set_message_reaction(
|
||||
session["session_key"], int(row_id), emoji, author=author
|
||||
)
|
||||
except Exception as e:
|
||||
return _err(rid, 5007, str(e))
|
||||
|
||||
if reactions is None:
|
||||
return _err(rid, 4040, "message not found in this session")
|
||||
|
||||
return _ok(rid, {"row_id": int(row_id), "reactions": reactions})
|
||||
|
||||
|
||||
@method("llm.oneshot")
|
||||
def _(rid, params: dict) -> dict:
|
||||
"""Run a single stateless LLM request outside any conversation.
|
||||
|
|
|
|||
|
|
@ -6655,6 +6655,13 @@ def _history_to_messages(history: list[dict]) -> list[dict]:
|
|||
if not content_text.strip() and not has_reasoning:
|
||||
continue
|
||||
msg = {"role": role, "text": content_text}
|
||||
# Durable row identity, stamped by _rows_to_conversation. The renderer's
|
||||
# own message ids are ephemeral (timestamp+index derived, and a
|
||||
# different shape for live vs rehydrated vs optimistic rows), so
|
||||
# anything that addresses a specific persisted message later — message
|
||||
# reactions — needs this instead.
|
||||
if m.get("_row_id") is not None:
|
||||
msg["row_id"] = m["_row_id"]
|
||||
if role == "user":
|
||||
invocation = _skill_scaffold_projection(content_text)
|
||||
if invocation:
|
||||
|
|
@ -9132,6 +9139,17 @@ def _run_prompt_submit(
|
|||
elif isinstance(run_message, list):
|
||||
run_message = [{"type": "text", "text": SPEECH_INTERRUPTED_NOTE}, *run_message]
|
||||
|
||||
# Reactions the user added since the last turn ride the MODEL INPUT
|
||||
# only (same enrichment channel as the speech-interrupted note);
|
||||
# persist_user_message below stays the clean prompt, so no
|
||||
# scaffolding reaches the transcript. Cache-safe: annotating the
|
||||
# NEW turn never rewrites an already-sent message.
|
||||
if reaction_notes := _pending_reaction_notes(session):
|
||||
if isinstance(run_message, str):
|
||||
run_message = f"{reaction_notes}\n\n{run_message}"
|
||||
elif isinstance(run_message, list):
|
||||
run_message = [{"type": "text", "text": reaction_notes}, *run_message]
|
||||
|
||||
def _stream(delta):
|
||||
with session["history_lock"]:
|
||||
_append_inflight_delta(session, delta)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue