diff --git a/.plans/message-reactions.md b/.plans/message-reactions.md new file mode 100644 index 00000000000..377d29b1763 --- /dev/null +++ b/.plans/message-reactions.md @@ -0,0 +1,146 @@ +# Message reactions (desktop tapbacks) + +Two-way emoji reactions on individual messages in the desktop transcript: the +user reacts to any message, the agent reacts to a user message, and both sides +read the other's reactions as conversational signal. + +## What already exists + +Hermes already models reactions on the **platform** side — the desktop is the +only surface without them. + +| Surface | Reaction support | Where | +|---|---|---| +| Agent → platform message | `send_message(action="react"/"unreact")` | `tools/send_message_tool.py:266` `_handle_react()` | +| Photon / iMessage | tapbacks in + out, routed only for messages we sent | `plugins/platforms/photon/adapter.py:1240-1283` | +| Telegram | `setMessageReaction`, config-gated | `plugins/platforms/telegram/adapter.py:9669+` | +| Slack / Matrix / Feishu / Discord | inbound reaction events → hooks | `gateway/run.py:4688` `_handle_reaction_event()` → `HookRegistry.emit("reaction:added")` | +| Adapter contract | `add_reaction()` / `remove_reaction()` coroutines, `set_reaction_handler()` | `gateway/platforms/base.py:3330` | +| Core "affection" detector | regex on user text → `vibe`, drives CLI pet / TUI heart / desktop hearts | `agent/reactions.py`, `agent/turn_context.py:592-604` | + +Two things follow from that table: + +1. **The agent-facing verb already exists.** `send_message(action="react")` is + the established shape. A desktop reaction should extend that tool, not add a + new core tool — every new tool ships on every API call (AGENTS.md footprint + ladder). +2. **The inbound convention already exists.** Photon turns a tapback into a + normal message event with `reply_to_message_id` + `reply_to_is_own_message`, + and the gateway prefixes `[Replying to your previous message: "…"]` + (`gateway/run.py:13125-13132`). Desktop reactions should read the same way to + the model. + +Nothing exists on the desktop side: `grep -ri reaction` across `apps/desktop` +finds only the pet-overlay hearts. + +## Prior art + +**iOS Tapback** ([Apple](https://support.apple.com/guide/iphone/react-with-tapbacks-iph018d3c336/ios)): +double-tap or touch-and-hold a message → floating pill above the bubble with +heart / thumbs-up / thumbs-down / haha / ‼️ / ❓, swipe left for suggested emoji +and stickers, or tap the emoji button for the full keyboard. **One tapback per +message per person** — tapping the same one again removes it, tapping a +different one replaces it. Multiple people's tapbacks stack on the badge. + +**Platform data models** converge on the same shape: + +| Platform | Model | Add / remove | +|---|---|---| +| Slack | `{name, count, users[]}` | [`reactions.add`](https://docs.slack.dev/reference/methods/reactions.add) / `reactions.remove`, emits `reaction_added` | +| Discord | `{emoji, count, me}` on the message object | `PUT`/`DELETE .../reactions/{emoji}/@me` | +| Telegram | `reaction: [{type:"emoji", emoji:"👍"}]` — replaces the whole set | `setMessageReaction`, `is_big` for the big animation | + +Telegram's "set the whole array" is the closest match to iOS semantics and the +simplest thing to persist. + +**assistant-ui has no reaction primitive.** `@assistant-ui/react` 0.14.24 (MIT, +vendored at `apps/desktop/node_modules`): zero hits for "reaction" in `core/src`, +`react/src`, `dist/`, or the 2.2 MB `llms-full.txt` docs dump. What exists is a +hard-coded binary `FeedbackAdapter` (`"positive" | "negative"`, +`core/src/adapters/feedback.ts`) that throws when unconfigured and only writes +back onto assistant messages. Not usable for emoji, not usable on user messages. + +**But `metadata.custom` is the supported extension channel** and this repo +already uses it: `ThreadUserMessage`/`ThreadAssistantMessage`/`ThreadSystemMessage` +all carry `metadata.custom: Record` (`core/src/types/message.ts:319-366`), +and `chat-runtime.ts:397` already ships `custom: { attachmentRefs }` through it. + +**Emoji picker survey** (npm week of 2026-07-22, sizes measured from the +published ESM entry): + +| Library | License | Weekly DL | gzip | Headless | Latest | +|---|---|---|---|---|---| +| **frimousse** | MIT | 573k | **8.5 kB** | ✅ fully unstyled, composable parts | 0.3.0 · 2025-07-15 | +| emoji-picker-react | MIT | 1.31M | 87 kB | ❌ own CSS-in-JS (flairup) | 4.19.1 · 2026-04-27 | +| emoji-mart | MIT | 2.22M | ~120 kB w/ data | ❌ Preact + shadow styling | 5.6.0 · **2024-04-25**, 217 open issues | +| emoji-picker-element | Apache-2.0 | 183k | — | ❌ Web Component / Shadow DOM | 1.29.1 · 2026-03-01 | + +No picker is currently a dependency (only `emoji-regex`, transitive). Already +paid for and reusable: `radix-ui` (Popover), `motion`, `@tanstack/react-virtual`, +Tailwind v4. + +## Recommendation + +**Hand-roll the tapback pill; add frimousse only behind the "+".** Six fixed +emoji in a pill is ~40 lines of JSX against existing tokens — pulling 87 kB of +`emoji-picker-react` to render six buttons, plus a CSS engine that fights +`DESIGN.md`, is backwards. frimousse is headless, dependency-free, 10× smaller, +and exposes `emojibaseUrl` so the data can be bundled as a Vite asset instead of +hitting jsDelivr (Electron must work offline). + +### Data model + +One reaction per author per message, Telegram-style whole-set replacement: + +```ts +type MessageReaction = { emoji: string; author: 'user' | 'agent'; at: number } +``` + +Persisted in the existing `messages.display_metadata` JSON column +(`hermes_state_common.py:215`) — no new table. It already survives insert, +compaction, and every read projection, and +`set_latest_matching_message_display_kind()` (`hermes_state.py:5292`) is the +precedent for stamping metadata onto an already-persisted row. + +### Model context + +Reactions must reach the model **without breaking prompt caching**. The +`api_messages` build loop strips `display_metadata` from every outgoing copy +(`agent/conversation_loop.py:1443-1446`) precisely so display state never +becomes a provider field. Two candidate paths: + +| Path | Cache impact | Notes | +|---|---|---| +| Rewrite the reacted-to message's content to carry the annotation | **Breaks the cached prefix** — mutates past context | Rejected. AGENTS.md: prompt caching is sacred. | +| Deliver the reaction as the *next* turn's leading annotation, mirroring photon | Prefix untouched; only the new turn carries it | Matches `[Replying to your previous message: "…"]` (`gateway/run.py:13125`), which the agent already understands | + +The second is the same trick the platform adapters already use, so the model +sees a familiar shape and no existing conversation is rewritten. + +### Attach points + +| Concern | File | Lines | +|---|---|---| +| Assistant hover bar | `apps/desktop/src/components/assistant-ui/thread/assistant-message.tsx` | 134–175 | +| User hover cluster | `apps/desktop/src/components/assistant-ui/thread/user-message.tsx` | 296–336 | +| Callback threading (ref caveat 79–99) | `apps/desktop/src/components/assistant-ui/thread/index.tsx` | 109–133 | +| `metadata.custom` → runtime | `apps/desktop/src/lib/chat-runtime.ts` | 384–432 | +| RPC client ↔ server pattern | `sidebar/session-actions-menu.tsx:62-89` ↔ `tui_gateway/server.py:8322` | — | +| Persistence | `hermes_state_common.py:192-216`, `hermes_state.py:5292-5324` | — | +| Prompt injection / strip | `agent/conversation_loop.py` | 1430–1529 | + +### Known gaps to solve first + +- **No durable message id crosses the gateway RPC path.** `_history_to_messages()` + (`tui_gateway/server.py:6545`) builds `{"role", "text"}` and drops the id. The + REST path carries `messages.id` incidentally via `SELECT *` but TS + `SessionMessage` (`types/hermes.ts:513-533`) doesn't declare it. Renderer ids + are ephemeral and change shape between rehydrated (`--`), live + (`assistant-`), and optimistic (`user--`) messages. A reaction + needs a stable key — this is the first thing to fix. +- **WeakMap identity cache** in `apps/desktop/src/app/chat/runtime-repository.ts:26-66` + keys normalized `ThreadMessage` by `ChatMessage` identity. A reaction change + must produce a **new** `ChatMessage` object or the UI renders stale. +- **Rewind rewrites rows** (`replace_messages`), so anything keyed by row id + needs cascade handling — an argument for keeping reactions in + `display_metadata` on the row itself rather than a side table. diff --git a/agent/conversation_loop.py b/agent/conversation_loop.py index 5d96b8e6512..3ca96898cf9 100644 --- a/agent/conversation_loop.py +++ b/agent/conversation_loop.py @@ -1445,6 +1445,12 @@ def run_conversation( api_msg.pop("display_kind", None) api_msg.pop("display_metadata", None) + # Durable row identity stamped by _rows_to_conversation so the + # desktop can address a specific persisted message (reactions). + # Bookkeeping, never a provider field — only the chat-completions + # transport strips underscore keys, so drop it centrally here. + api_msg.pop("_row_id", None) + # Inject ephemeral context into the current turn's user message. # Sources: memory manager prefetch + plugin pre_llm_call hooks # with target="user_message" (the default). Both are diff --git a/hermes_state.py b/hermes_state.py index c23487ca30e..1f89a834d1e 100644 --- a/hermes_state.py +++ b/hermes_state.py @@ -5647,6 +5647,214 @@ class SessionDB(SessionSearchMixin, SessionSchemaMixin, SessionPortabilityMixin) return bool(self._execute_write(_do)) + #: Key under which message reactions live inside ``display_metadata``. + #: Reactions share the existing per-message JSON column rather than a side + #: table so they survive rewind/compaction row rewrites with the row itself. + REACTIONS_METADATA_KEY = "reactions" + + def set_message_reaction( + self, + session_id: str, + message_row_id: int, + emoji: Optional[str], + *, + author: str = "user", + ) -> Optional[List[Dict[str, Any]]]: + """Set (or with ``emoji=None`` clear) *author*'s reaction on one message. + + iOS Tapback semantics: one reaction per author per message. Re-sending + the same emoji clears it, a different emoji replaces it. Returns the + message's full reaction list after the write, or ``None`` when the row + doesn't exist or isn't part of *session_id*. + """ + if not session_id or message_row_id is None: + return None + + def _do(conn): + row = conn.execute( + "SELECT display_metadata FROM messages WHERE id = ? AND session_id = ?", + (message_row_id, session_id), + ).fetchone() + if row is None: + return None + + meta = self._decode_display_metadata(row[0]) or {} + existing = meta.get(self.REACTIONS_METADATA_KEY) + reactions = [ + r + for r in (existing if isinstance(existing, list) else []) + if isinstance(r, dict) and r.get("author") != author + ] + previous = next( + ( + r + for r in (existing if isinstance(existing, list) else []) + if isinstance(r, dict) and r.get("author") == author + ), + None, + ) + # Tapping the live reaction again retracts it. + toggling_off = ( + emoji is not None and previous is not None and previous.get("emoji") == emoji + ) + if emoji and not toggling_off: + reactions.append( + {"emoji": _scrub_surrogates(emoji), "author": author, "at": time.time()} + ) + + if reactions: + meta[self.REACTIONS_METADATA_KEY] = reactions + else: + meta.pop(self.REACTIONS_METADATA_KEY, None) + + conn.execute( + "UPDATE messages SET display_metadata = ? WHERE id = ?", + (self._encode_display_metadata(meta) if meta else None, message_row_id), + ) + return reactions + + return self._execute_write(_do) + + def get_message_reactions( + self, session_id: str, message_row_id: int + ) -> List[Dict[str, Any]]: + """Return the reaction list persisted on one message row (never ``None``).""" + if not session_id or message_row_id is None: + return [] + + with self._lock: + row = self._conn.execute( + "SELECT display_metadata FROM messages WHERE id = ? AND session_id = ?", + (message_row_id, session_id), + ).fetchone() + + if row is None: + return [] + + meta = self._decode_display_metadata(row[0]) or {} + reactions = meta.get(self.REACTIONS_METADATA_KEY) + + return [r for r in reactions if isinstance(r, dict)] if isinstance(reactions, list) else [] + + def take_unseen_reactions( + self, session_id: str, *, author: str = "user" + ) -> List[Dict[str, Any]]: + """Return *author*'s not-yet-surfaced reactions and mark them seen. + + Powers the cache-safe model-context path: reactions are announced on the + NEXT user turn (never by rewriting the message that was reacted to), and + the ``seen`` stamp guarantees each one is announced exactly once. + """ + if not session_id: + return [] + + def _do(conn): + rows = conn.execute( + "SELECT id, role, content, display_metadata FROM messages " + "WHERE session_id = ? AND active = 1 AND display_metadata IS NOT NULL " + "ORDER BY id", + (session_id,), + ).fetchall() + + pending = [] + for row in rows: + meta = self._decode_display_metadata(row["display_metadata"]) + if not meta: + continue + reactions = meta.get(self.REACTIONS_METADATA_KEY) + if not isinstance(reactions, list): + continue + + changed = False + for reaction in reactions: + if ( + not isinstance(reaction, dict) + or reaction.get("author") != author + or reaction.get("seen") + ): + continue + reaction["seen"] = True + changed = True + content = self._decode_content(row["content"]) + pending.append( + { + "row_id": row["id"], + "role": row["role"], + "emoji": reaction.get("emoji") or "", + "text": content if isinstance(content, str) else "", + } + ) + + if changed: + conn.execute( + "UPDATE messages SET display_metadata = ? WHERE id = ?", + (self._encode_display_metadata(meta), row["id"]), + ) + + return pending + + return self._execute_write(_do) or [] + + def latest_message_row_id( + self, session_id: str, *, role: str = "user", offset: int = 0, require_text: bool = True + ) -> Optional[int]: + """Row id of the most recent active message with *role*, or ``None``. + + Two callers, same need — "the message I mean, without an id": the agent + defaulting to the turn that triggered it, and the desktop reacting to a + live message that hasn't round-tripped through a resume yet. + ``offset`` steps to earlier turns (1 = the one before the latest) so a + reaction can land retroactively — "two messages ago" is how the caller + thinks about it. + + ``require_text`` (default) skips rows with no plain-text content — + tool-call-only assistant turns and attachment stubs don't render as + bubbles, so "the latest message" as a HUMAN means it must never + resolve to one (a reaction landing on an invisible row looks dropped, + and its annotation quotes an empty string). + """ + if not session_id or role not in {"user", "assistant"} or offset < 0: + return None + + text_filter = ( + "AND content IS NOT NULL AND TRIM(content) != '' " if require_text else "" + ) + + with self._lock: + row = self._conn.execute( + "SELECT id FROM messages WHERE session_id = ? AND role = ? " + f"AND active = 1 {text_filter}ORDER BY id DESC LIMIT 1 OFFSET ?", + (session_id, role, int(offset)), + ).fetchone() + + return row[0] if row else None + + def latest_user_message_row_id(self, session_id: str) -> Optional[int]: + """Row id of the most recent active user message, or ``None``. + + The agent's default reaction target: "the message that triggered me", + so the model never has to thread row ids through a tool call (mirrors + the photon adapter's ``_record_last_inbound``). + """ + return self.latest_message_row_id(session_id, role="user") + + def get_message_role(self, session_id: str, row_id: int) -> Optional[str]: + """Role of the active message at *row_id* in *session_id*, or ``None``. + + Lets a reaction event carry the target's role so a renderer can match + a live message that doesn't know its durable row id yet. + """ + if not session_id: + return None + + with self._lock: + row = self._conn.execute( + "SELECT role FROM messages WHERE id = ? AND session_id = ? AND active = 1", + (int(row_id), session_id), + ).fetchone() + + return row[0] if row else None + def _insert_message_rows(self, conn, session_id: str, messages: List[Dict[str, Any]]) -> tuple[int, int]: """Insert *messages* as fresh active rows for *session_id*. @@ -6154,10 +6362,7 @@ class SessionDB(SessionSearchMixin, SessionSchemaMixin, SessionPortabilityMixin) with self._lock: placeholders = ",".join("?" for _ in session_ids) rows = self._conn.execute( - "SELECT role, content, tool_call_id, tool_calls, tool_name, effect_disposition, " - "finish_reason, reasoning, reasoning_content, reasoning_details, " - "codex_reasoning_items, codex_message_items, platform_message_id, observed, timestamp, " - "api_content, display_kind, display_metadata " + f"SELECT {self._CONVERSATION_ROW_COLUMNS} " f"FROM messages WHERE session_id IN ({placeholders})" # Order by AUTOINCREMENT id (true insertion order), NOT timestamp: # append_message stamps rows with time.time(), which is not @@ -6182,7 +6387,7 @@ class SessionDB(SessionSearchMixin, SessionSchemaMixin, SessionPortabilityMixin) # get_messages_as_conversation and get_resume_conversations so a single # SELECT can feed both the model-fed and display views. _CONVERSATION_ROW_COLUMNS = ( - "role, content, tool_call_id, tool_calls, tool_name, effect_disposition, " + "id, role, content, tool_call_id, tool_calls, tool_name, effect_disposition, " "finish_reason, reasoning, reasoning_content, reasoning_details, " "codex_reasoning_items, codex_message_items, platform_message_id, observed, timestamp, " "api_content, display_kind, display_metadata" @@ -6209,6 +6414,12 @@ class SessionDB(SessionSearchMixin, SessionSchemaMixin, SessionPortabilityMixin) if row["role"] in {"user", "assistant"} and isinstance(content, str): content = sanitize_context(content).strip() msg = {"role": row["role"], "content": content} + # Durable per-message identity for surfaces that need to address a + # specific row later (desktop reactions). Underscore-prefixed so + # every transport's convert_messages() strips it before the wire — + # the established escape hatch for agent-internal bookkeeping. + if row["id"] is not None: + msg["_row_id"] = row["id"] # api_content is the byte-fidelity sidecar: the exact string sent # to the API when it differed from the clean content. Returned # VERBATIM — no sanitize_context, no strip — because the replay diff --git a/tests/test_message_reactions.py b/tests/test_message_reactions.py new file mode 100644 index 00000000000..c9b5346a8f6 --- /dev/null +++ b/tests/test_message_reactions.py @@ -0,0 +1,169 @@ +"""Message reactions: persistence, tapback semantics, and cache safety. + +Behavior contracts, not snapshots — these assert how reactions must RELATE to +the transcript (one per author, announced once, never mutating history), so +they survive refactors of where reactions are stored. +""" + +import pytest + +from hermes_state import SessionDB + + +@pytest.fixture +def db(tmp_path, monkeypatch): + monkeypatch.setenv("HERMES_HOME", str(tmp_path)) + return SessionDB(db_path=tmp_path / "state.db") + + +@pytest.fixture +def session(db): + key = db.create_session("react-test", "test") + db.append_message(key, "user", "how do i center a div") + db.append_message(key, "assistant", "use flexbox") + rows = [m["_row_id"] for m in db.get_messages_as_conversation(key)] + + return key, rows + + +def test_conversation_rows_carry_durable_row_id(session, db): + """Every projected message exposes its messages.id — reactions key off it.""" + key, rows = session + + assert all(isinstance(r, int) for r in rows) + assert rows == sorted(rows), "row ids must follow insertion order" + assert len(set(rows)) == len(rows), "row ids must be unique" + + +def test_one_reaction_per_author(session, db): + """A second emoji from the same author REPLACES the first (iOS Tapback).""" + key, rows = session + + db.set_message_reaction(key, rows[0], "\u2764\ufe0f", author="user") + reactions = db.set_message_reaction(key, rows[0], "\U0001f602", author="user") + + assert [r["emoji"] for r in reactions] == ["\U0001f602"] + + +def test_repeating_an_emoji_retracts_it(session, db): + """Tapping the live reaction again clears it.""" + key, rows = session + + db.set_message_reaction(key, rows[0], "\U0001f44d", author="user") + reactions = db.set_message_reaction(key, rows[0], "\U0001f44d", author="user") + + assert reactions == [] + assert db.get_message_reactions(key, rows[0]) == [] + + +def test_authors_are_independent(session, db): + """User and agent each hold their own slot on the same message.""" + key, rows = session + + db.set_message_reaction(key, rows[0], "\u2764\ufe0f", author="user") + reactions = db.set_message_reaction(key, rows[0], "\U0001f525", author="agent") + + assert {r["author"] for r in reactions} == {"user", "agent"} + + remaining = db.set_message_reaction(key, rows[0], None, author="user") + assert [r["author"] for r in remaining] == ["agent"] + + +def test_rejects_rows_outside_the_session(session, db): + """A row id from another conversation is never writable.""" + key, rows = session + other = db.create_session("other", "test") + db.append_message(other, "user", "elsewhere") + + assert db.set_message_reaction(key, 9999, "\u2764\ufe0f") is None + assert db.set_message_reaction("no-such-session", rows[0], "\u2764\ufe0f") is None + + +def test_clearing_every_reaction_leaves_no_metadata(session, db): + """An empty reaction set removes the key instead of persisting `[]`.""" + key, rows = session + + db.set_message_reaction(key, rows[0], "\u2764\ufe0f", author="user") + db.set_message_reaction(key, rows[0], None, author="user") + + message = db.get_messages_as_conversation(key)[0] + assert "display_metadata" not in message + + +def test_reactions_survive_reload(session, db): + """Reactions are durable, not in-memory display state.""" + key, rows = session + db.set_message_reaction(key, rows[1], "\U0001f525", author="agent") + + reopened = SessionDB(db_path=db.db_path) + assert [r["emoji"] for r in reopened.get_message_reactions(key, rows[1])] == ["\U0001f525"] + + +def test_unseen_reactions_are_taken_exactly_once(session, db): + """The model is told about a reaction on ONE turn, never twice.""" + key, rows = session + db.set_message_reaction(key, rows[1], "\u2764\ufe0f", author="user") + + first = db.take_unseen_reactions(key, author="user") + assert [e["emoji"] for e in first] == ["\u2764\ufe0f"] + assert first[0]["row_id"] == rows[1] + assert first[0]["text"] == "use flexbox" + + assert db.take_unseen_reactions(key, author="user") == [] + + +def test_a_new_reaction_becomes_unseen_again(session, db): + """Replacing a seen reaction re-arms the announcement.""" + key, rows = session + db.set_message_reaction(key, rows[1], "\u2764\ufe0f", author="user") + db.take_unseen_reactions(key, author="user") + + db.set_message_reaction(key, rows[1], "\U0001f525", author="user") + + assert [e["emoji"] for e in db.take_unseen_reactions(key, author="user")] == ["\U0001f525"] + + +def test_take_unseen_filters_by_author(session, db): + """The agent's own reactions are never fed back to it as user input.""" + key, rows = session + db.set_message_reaction(key, rows[0], "\U0001f60a", author="agent") + + assert db.take_unseen_reactions(key, author="user") == [] + + +def test_reacting_never_mutates_message_content(session, db): + """CACHE SAFETY: reacting must not rewrite any already-sent message. + + Rewriting a past message would invalidate the provider's cached prefix for + the whole conversation — the reason reactions ride display_metadata and are + announced on the NEXT turn instead. + """ + key, rows = session + before = [m["content"] for m in db.get_messages_as_conversation(key)] + + db.set_message_reaction(key, rows[0], "\u2764\ufe0f", author="user") + db.set_message_reaction(key, rows[1], "\U0001f525", author="agent") + db.take_unseen_reactions(key, author="user") + + after = [m["content"] for m in db.get_messages_as_conversation(key)] + assert before == after + + +def test_latest_user_message_is_the_agents_default_target(session, db): + """The agent reacts to "the message that triggered me" without an id.""" + key, rows = session + assert db.latest_user_message_row_id(key) == rows[0] + + db.append_message(key, "user", "thanks!") + newest = db.get_messages_as_conversation(key)[-1]["_row_id"] + + assert db.latest_user_message_row_id(key) == newest + + +def test_row_id_never_reaches_the_provider(session, db): + """_row_id is underscore-prefixed so transports strip it before the wire.""" + key, _rows = session + + for message in db.get_messages_as_conversation(key): + assert "_row_id" in message + assert all(not k.startswith("_") or k == "_row_id" for k in message) diff --git a/tools/react_to_message_tool.py b/tools/react_to_message_tool.py new file mode 100644 index 00000000000..b2b37178cce --- /dev/null +++ b/tools/react_to_message_tool.py @@ -0,0 +1,154 @@ +#!/usr/bin/env python3 +"""Let the agent react to a message with an emoji in the Hermes desktop app. + +The conversational counterpart to the user's tapback: the same reaction store, +the same one-per-author semantics, just written with ``author="agent"``. + +Gated on ``HERMES_DESKTOP`` (like the other GUI affordances) so it costs nothing +on every other surface — the platform adapters already expose reactions through +``send_message(action="react")``, and this is the desktop's equivalent. + +Defaults to the message that triggered this turn (the photon precedent: the +model shouldn't have to thread row ids through tool calls), and emits +``message.reaction`` so the renderer paints it without waiting for a resume. +""" + +import json + +from gateway.session_context import get_session_env +from tools import desktop_ui +from tools.registry import registry, tool_error +from utils import env_var_enabled + + +def _open_session_db(): + """Open the SessionDB for the profile owning this turn, or ``None``.""" + try: + from hermes_state import SessionDB + + return SessionDB() + except Exception: + return None + + +def react_to_message_tool(emoji: str, message_row_id=None, messages_back=None) -> str: + """Attach (or with an empty ``emoji`` retract) the agent's reaction.""" + emoji = (emoji or "").strip() + session_key = get_session_env("HERMES_SESSION_KEY", "") or get_session_env( + "HERMES_SESSION_ID", "" + ) + + if not session_key: + return tool_error("No active session — reactions need a persisted conversation.") + + db = _open_session_db() + if db is None: + return tool_error("Session storage is unavailable.") + + row_id = message_row_id + target_role = "user" + if row_id is None: + # Default target: the latest user message. `messages_back` steps to + # earlier user turns (1 = the one before, etc.) for retroactive + # reactions — quoting text would be ambiguous, ids aren't visible to + # the model, but "two messages ago" is how a person thinks about it. + back = max(0, int(messages_back or 0)) + row_id = db.latest_message_row_id(session_key, role="user", offset=back) + if row_id is None: + return tool_error( + f"No user message found {back} back." if back else "No user message to react to yet." + ) + else: + row = db.get_message_role(session_key, int(row_id)) + target_role = row or "user" + + try: + reactions = db.set_message_reaction( + session_key, int(row_id), emoji or None, author="agent" + ) + except Exception as exc: + return tool_error(f"Failed to set the reaction: {exc}") + + if reactions is None: + return tool_error(f"Message {row_id} is not part of this conversation.") + + # Paint it live. A missing bridge (non-desktop surface) is not an error — + # the reaction is persisted either way and shows on the next load. + # `role` lets the renderer match a live message that doesn't know its + # durable row id yet (it only learns rowId on resume). + try: + desktop_ui.emit( + "message.reaction", + {"row_id": int(row_id), "reactions": reactions, "role": target_role}, + ) + except Exception: + pass + + return json.dumps( + {"success": True, "row_id": int(row_id), "reactions": reactions}, ensure_ascii=False + ) + + +def check_react_requirements() -> bool: + """Desktop GUI only — HERMES_DESKTOP is set on the gateway the app spawns.""" + return env_var_enabled("HERMES_DESKTOP") + + +REACT_TO_MESSAGE_SCHEMA = { + "name": "react_to_message", + "description": ( + "React to a message with a single emoji, the way you'd tapback in iMessage. " + "Reach for it when a reaction is what a person would do: something funny gets " + "a 😂, warmth gets a ❤️, a plan you're on board with gets a 👍 — then just " + "carry on with whatever the message actually needs. If a reaction says it " + "all, it can BE the reply (skip the redundant 'sounds good!' turn). Use it " + "like a person would: occasionally, when felt — not on every message, and " + "never as a status signal. NEVER narrate or explain a reaction ('I reacted " + "with...', 'Reacting now') — the emoji appearing on the bubble is the whole " + "point, and commentary kills it. Defaults to the user's most recent message. " + "One reaction per message: a different emoji replaces yours, an empty string " + "retracts it." + ), + "parameters": { + "type": "object", + "properties": { + "emoji": { + "type": "string", + "description": ( + "The emoji to react with (e.g. '❤️', '😂', '👍'). Pass an empty " + "string to remove your reaction." + ), + }, + "message_row_id": { + "type": "integer", + "description": ( + "Optional. The specific message to react to. Omit to react to the " + "user's latest message, which is almost always what you want." + ), + }, + "messages_back": { + "type": "integer", + "description": ( + "Optional. React to an EARLIER user message: 1 = the one before " + "the latest, 2 = two before, and so on. For when something lands " + "late — the joke you only got after answering." + ), + }, + }, + "required": ["emoji"], + }, +} + + +registry.register( + name="react_to_message", + toolset="terminal", + schema=REACT_TO_MESSAGE_SCHEMA, + handler=lambda args, **kw: react_to_message_tool( + emoji=args.get("emoji", ""), + message_row_id=args.get("message_row_id"), + messages_back=args.get("messages_back"), + ), + check_fn=check_react_requirements, + emoji="💛", +) diff --git a/toolsets.py b/toolsets.py index 75219f719cf..4c7588b0abd 100644 --- a/toolsets.py +++ b/toolsets.py @@ -34,9 +34,10 @@ _HERMES_CORE_TOOLS = [ # Terminal + process management "terminal", "process", # Desktop GUI affordances: read the embedded terminal pane, close an agent's - # read-only terminal tab, open a URL/file in the preview pane, and focus a - # pane (all gated on HERMES_DESKTOP via check_fn — hidden outside the GUI). - "read_terminal", "close_terminal", "open_preview", "focus_pane", + # read-only terminal tab, open a URL/file in the preview pane, focus a + # pane, and react to a message with an emoji (all gated on HERMES_DESKTOP + # via check_fn — hidden outside the GUI). + "read_terminal", "close_terminal", "open_preview", "focus_pane", "react_to_message", # File manipulation "read_file", "write_file", "patch", "search_files", # Vision + image generation diff --git a/tui_gateway/methods_prompt.py b/tui_gateway/methods_prompt.py index cc311de6cb4..804c7f44da0 100644 --- a/tui_gateway/methods_prompt.py +++ b/tui_gateway/methods_prompt.py @@ -6,11 +6,55 @@ are rebound onto server.py's globals at install time — see method_ctx.py. from .method_ctx import HandlerRegistry +import types + _registry = HandlerRegistry() method = _registry.method _profile_scoped = _registry.profile_scoped +def _pending_reaction_notes(session: dict) -> str: + """Note block describing reactions the user added since the last turn, or "". + + Applied to the MODEL INPUT only (``run_message``, beside the + speech-interrupted note) — never to the text that gets persisted. Prefixing + the persisted prompt bakes scaffolding into the transcript, which every + surface then renders as a garbled user message on reload. Each reaction is + announced once — the row is stamped ``seen`` on read. + """ + session_key = str(session.get("session_key") or "") + if not session_key: + return "" + + try: + with _session_db(session) as db: + if db is None: + return "" + pending = db.take_unseen_reactions(session_key, author="user") + except Exception: + logger.debug("Failed to read pending reactions", exc_info=True) + return "" + + if not pending: + return "" + + notes = [] + for entry in pending: + snippet = (entry.get("text") or "").strip().replace("\n", " ") + if len(snippet) > 120: + snippet = snippet[:120] + "…" + emoji = entry.get("emoji") or "" + whose = "their own" if entry.get("role") == "user" else "your" + if snippet: + notes.append(f'[The user reacted {emoji} to {whose} message: "{snippet}"]') + else: + # A row with no plain text (attachment-only, or a tool-call-only + # assistant turn) — an empty quote reads worse than no quote. + notes.append(f"[The user reacted {emoji} to {whose} earlier message]") + + return "\n".join(notes) + + @method("prompt.submit") def _(rid, params: dict) -> dict: from hermes_cli.input_sanitize import sanitize_user_prompt_text @@ -833,3 +877,13 @@ def _(rid, params: dict) -> dict: def register(server) -> None: """Bind this module's handlers onto ``server``'s globals and registry.""" _registry.install(server) + # Module-level helpers aren't @method handlers, so install() doesn't see + # them — but server.py's run path calls this one (run_message enrichment, + # beside the speech-interrupted note). Rebind and publish it the same way. + server._pending_reaction_notes = types.FunctionType( + _pending_reaction_notes.__code__, + vars(server), + _pending_reaction_notes.__name__, + _pending_reaction_notes.__defaults__, + _pending_reaction_notes.__closure__, + ) diff --git a/tui_gateway/methods_session.py b/tui_gateway/methods_session.py index ea0e1720f66..2dd9796ad58 100644 --- a/tui_gateway/methods_session.py +++ b/tui_gateway/methods_session.py @@ -919,6 +919,59 @@ def _(rid, params: dict) -> dict: return _err(rid, 5007, str(e)) +@method("message.react") +def _(rid, params: dict) -> dict: + """Set or clear one author's emoji reaction on a persisted message. + + iOS Tapback semantics, enforced in the DB layer: one reaction per author + per message, re-sending the same emoji retracts it. ``emoji: null`` clears + unconditionally. ``row_id`` is the durable ``messages.id`` forwarded by + ``_history_to_messages`` — the renderer's own message ids are ephemeral. + """ + session, err = _sess_nowait(params, rid) + if err: + return err + + # A live message hasn't round-tripped through a resume, so the desktop has + # no durable row id for it yet. It can instead name the ROLE whose newest + # row it means — which is the message the user just reacted to. + newest_role = str(params.get("newest_role") or "").strip() + row_id = params.get("row_id") + if row_id is None and newest_role not in {"user", "assistant"}: + return _err(rid, 4023, "row_id or newest_role required") + + emoji = params.get("emoji") + if emoji is not None: + emoji = str(emoji).strip() + if not emoji: + return _err(rid, 4024, "emoji must be a non-empty string or null") + + author = str(params.get("author") or "user").strip() + if author not in {"user", "agent"}: + return _err(rid, 4025, "author must be 'user' or 'agent'") + + with _session_db(session) as db: + if db is None: + return _db_unavailable_error(rid, code=5007) + try: + if row_id is None: + row_id = db.latest_message_row_id( + session["session_key"], role=newest_role + ) + if row_id is None: + return _err(rid, 4040, "no message to react to yet") + reactions = db.set_message_reaction( + session["session_key"], int(row_id), emoji, author=author + ) + except Exception as e: + return _err(rid, 5007, str(e)) + + if reactions is None: + return _err(rid, 4040, "message not found in this session") + + return _ok(rid, {"row_id": int(row_id), "reactions": reactions}) + + @method("llm.oneshot") def _(rid, params: dict) -> dict: """Run a single stateless LLM request outside any conversation. diff --git a/tui_gateway/server.py b/tui_gateway/server.py index b3a0ffa6187..8b7928da3be 100644 --- a/tui_gateway/server.py +++ b/tui_gateway/server.py @@ -6655,6 +6655,13 @@ def _history_to_messages(history: list[dict]) -> list[dict]: if not content_text.strip() and not has_reasoning: continue msg = {"role": role, "text": content_text} + # Durable row identity, stamped by _rows_to_conversation. The renderer's + # own message ids are ephemeral (timestamp+index derived, and a + # different shape for live vs rehydrated vs optimistic rows), so + # anything that addresses a specific persisted message later — message + # reactions — needs this instead. + if m.get("_row_id") is not None: + msg["row_id"] = m["_row_id"] if role == "user": invocation = _skill_scaffold_projection(content_text) if invocation: @@ -9132,6 +9139,17 @@ def _run_prompt_submit( elif isinstance(run_message, list): run_message = [{"type": "text", "text": SPEECH_INTERRUPTED_NOTE}, *run_message] + # Reactions the user added since the last turn ride the MODEL INPUT + # only (same enrichment channel as the speech-interrupted note); + # persist_user_message below stays the clean prompt, so no + # scaffolding reaches the transcript. Cache-safe: annotating the + # NEW turn never rewrites an already-sent message. + if reaction_notes := _pending_reaction_notes(session): + if isinstance(run_message, str): + run_message = f"{reaction_notes}\n\n{run_message}" + elif isinstance(run_message, list): + run_message = [{"type": "text", "text": reaction_notes}, *run_message] + def _stream(delta): with session["history_lock"]: _append_inflight_delta(session, delta)