fix(gateway): audio attachment note no longer steers the agent into punting

Sibling site of the PDF/DOCX note fixed in PR #44175: the audio file
attachment context note led with "Ask the user what they'd like you to
do with it", steering the model into asking instead of transcribing.
Rewritten to instruct the agent to transcribe/process the file itself
when the request involves its content, only asking when intent is
genuinely unclear. Contract assertion added to the existing audio
attachment note test.
This commit is contained in:
Teknium 2026-06-11 11:39:35 -07:00
parent 4e9be3ee32
commit 13650ab7f8
2 changed files with 9 additions and 1 deletions

View file

@ -7727,7 +7727,11 @@ class GatewayRunner(GatewayAuthorizationMixin, GatewayKanbanWatchersMixin, Gatew
_note = (
f"[The user sent an audio file attachment: '{_display}'. "
f"It is saved at: {_agent_path}. "
f"Ask the user what they'd like you to do with it, or pass the path to a transcription or media tool.]"
f"Its content is not inlined here. If the user's request involves "
f"what the audio contains, transcribe or process it yourself — for "
f"example by passing the path to a transcription or media tool — "
f"instead of asking the user to describe it. Only ask what to do "
f"with it if their intent is genuinely unclear.]"
)
message_text = f"{_note}\n\n{message_text}"

View file

@ -134,6 +134,10 @@ async def test_audio_attachment_context_note_format():
assert "audio file attachment" in result.lower()
# Should NOT contain the voice-message transcription wrapper text
assert "voice message" not in result.lower()
# Guides the agent to transcribe/process the file itself rather than
# punting back to the user (same bug class as the PDF/DOCX note).
assert "transcri" in result.lower()
assert "ask the user what they'd like" not in result.lower()
# ---------------------------------------------------------------------------