mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-20 15:33:54 +00:00
9e992df8ae
Port from nearai/ironclaw#2304: Telegram's 4096 character limit is measured in UTF-16 code units, not Unicode codepoints. Characters outside the Basic Multilingual Plane (emoji like 😀, CJK Extension B, musical symbols) are surrogate pairs: 1 Python char but 2 UTF-16 units. Previously, truncate_message() used Python's len() which counts codepoints. This could produce chunks exceeding Telegram's actual limit when messages contain many astral-plane characters. Changes: - Add utf16_len() helper and _prefix_within_utf16_limit() for UTF-16-aware string measurement and truncation - Add _custom_unit_to_cp() binary-search helper that maps a custom-unit budget to the largest safe codepoint slice position - Update truncate_message() to accept optional len_fn parameter - Telegram adapter now passes len_fn=utf16_len when splitting messages - Fix fallback truncation in Telegram error handler to use _prefix_within_utf16_limit instead of codepoint slicing - Update send_message_tool.py to use utf16_len for Telegram platform - Add comprehensive tests: utf16_len, _prefix_within_utf16_limit, truncate_message with len_fn (emoji splitting, content preservation, code block handling) - Update mock lambdas in reply_mode tests to accept **kw for len_fn |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| ADDING_A_PLATFORM.md | ||
| api_server.py | ||
| base.py | ||
| bluebubbles.py | ||
| dingtalk.py | ||
| discord.py | ||
| email.py | ||
| feishu.py | ||
| helpers.py | ||
| homeassistant.py | ||
| matrix.py | ||
| mattermost.py | ||
| signal.py | ||
| slack.py | ||
| sms.py | ||
| telegram.py | ||
| telegram_network.py | ||
| webhook.py | ||
| wecom.py | ||
| wecom_callback.py | ||
| wecom_crypto.py | ||
| weixin.py | ||
| whatsapp.py | ||