fix(install): emit UTF-8 from skills_sync on non-UTF-8 Windows locales

On Windows with a non-UTF-8 system locale (e.g. CP936/GBK on zh-CN),
Python defaults stdout/stderr to the active codepage. tools/skills_sync.py
prints glyphs such as checkmark (U+2713) and up-arrow (U+2191) that GBK
cannot encode, raising UnicodeEncodeError mid-run.

The installer (scripts/install.ps1) captures this script's stdout and the
Rust bootstrap parses it as UTF-8 expecting a JSON result frame. A GBK
byte stream (or the traceback it triggers) surfaces as:

  WARN stdout read error: stream did not contain valid UTF-8
  stage=config-templates state=Failed
  error=install.ps1 -Stage config-templates produced no JSON result frame
        (exit=Some(0))

i.e. the stage fails even though the script exits 0. install.ps1 already
sets [Console]::OutputEncoding = UTF8, but that does not propagate to the
python.exe child (Python reads PYTHONIOENCODING / locale, not the console
encoding).

Fix in two places for defense in depth:
- tools/skills_sync.py: reconfigure sys.stdout/stderr to UTF-8 at import so
  output is valid UTF-8 regardless of caller or active codepage.
- scripts/install.ps1: set PYTHONIOENCODING=utf-8 and PYTHONUTF8=1 (scoped
  to the call, restored afterwards) around the skills_sync.py invocation.
This commit is contained in:
kael-odin 2026-06-29 19:42:42 +08:00 committed by Teknium
parent cf35fd6de5
commit 3f1de1fc4a
2 changed files with 32 additions and 1 deletions

View file

@ -2435,7 +2435,22 @@ You are Hermes Agent, an intelligent AI assistant created by Nous Research. You
$pythonExe = "$InstallDir\venv\Scripts\python.exe"
if (Test-Path $pythonExe) {
try {
& $pythonExe "$InstallDir\tools\skills_sync.py" 2>$null
# Force the child python.exe to emit UTF-8 on its stdout/stderr.
# On non-UTF-8 Windows locales (CP936/GBK zh-CN) Python defaults
# its stream encoding to the active codepage and crashes on glyphs
# like ✓ (U+2713) that the codepage can't encode; the resulting
# non-UTF-8 bytes break this script's JSON result frame on stdout
# and abort the config-templates stage. Scope to this call only.
$prevPythonioencoding = $env:PYTHONIOENCODING
$prevPythonutf8 = $env:PYTHONUTF8
$env:PYTHONIOENCODING = "utf-8"
$env:PYTHONUTF8 = "1"
try {
& $pythonExe "$InstallDir\tools\skills_sync.py" 2>$null
} finally {
$env:PYTHONIOENCODING = $prevPythonioencoding
$env:PYTHONUTF8 = $prevPythonutf8
}
Write-Success "Skills synced to $HermesHome\skills"
} catch {
# Fallback: simple directory copy

View file

@ -26,8 +26,24 @@ import json
import logging
import os
import shutil
import sys
from datetime import datetime, timezone
from pathlib import Path, PurePosixPath
# Force stdout/stderr to UTF-8. On non-UTF-8 Windows locales (e.g. CP936/GBK
# on zh-CN), Python's default stream encoding can't represent the checkmark /
# arrow glyphs this script prints (✓ U+2713, ↑ U+2191), raising
# UnicodeEncodeError mid-run. The bootstrap installer (install.ps1) captures
# this script's stdout and parses it as UTF-8; a GBK byte stream then surfaces
# as "stream did not contain valid UTF-8" and aborts the config-templates
# stage even though the script itself exits 0. Reconfigure unconditionally so
# output is valid UTF-8 regardless of the active codepage or caller.
for _stream in (sys.stdout, sys.stderr):
if hasattr(_stream, "reconfigure"):
try:
_stream.reconfigure(encoding="utf-8", errors="replace")
except (ValueError, TypeError):
pass
from hermes_constants import get_bundled_skills_dir, get_hermes_home, get_optional_skills_dir
from agent.skill_utils import is_excluded_skill_path
from typing import Dict, List, Optional, Set, Tuple