mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-31 19:16:29 +00:00
f13f845116
Integration layer for the cjk_unicode61 tokenizer, rebuilt on the v23 schema (the contributed integration in PR #65544 predated it): - messages_fts_cjk: external-content FTS5 over a tool-row-excluding view (same v23 storage discipline as the trigram index it supersedes — zero inline text copies). Serves EVERY CJK query shape the legacy routing split between trigram (>=3 chars/token) and LIKE full scans (1-2 char tokens). Lone 1-char CJK runs and role_filter=['tool'] queries keep their legacy routes. - Dedicated marker pair (fts_cjk_rebuild_high_water/progress) gates the id-scoped triggers, so a cjk-only backfill never gates the complete messages_fts/trigram triggers. - Transitions ride (the existing throttled/resumable chunk engine): fresh DBs are born with the index; legacy v22 DBs land on v23+cjk in one run; already-optimized v23 DBs gaining the tokenizer get a marker-gated backfill; live writes are indexed immediately in every case. - Tokenizer-loss self-heal: a process that can't load the extension drops the cjk triggers (writes keep working), leaves a stale breadcrumb, and the index is rebuilt from scratch on the next optimize run — triggers are never reinstalled over a gap (external-content 'delete' on an unindexed rowid is the FTS5 corruption hazard the marker gating exists to prevent). - Capability classification: 'no such tokenizer: cjk_unicode61' joins the degraded-runtime error class everywhere (read probe, write probe, repair) so tokenizer absence is never misclassified as corruption. - Config: sessions.cjk_fts (default on, inert without the .so) and sessions.search_slow_ms in config.yaml, bridged to env by CLI + gateway (startup + per-turn reload). build.sh falls back to vendored SQLite headers so no libsqlite3-dev is needed. Slow-query log path attribution updated: fts_cjk / fts5 / trigram / like_scan. Tests: 14 lifecycle tests (fresh/legacy/stale/backfill paths, tokenizer-loss round-trip) + 5 config-bridge tests + slow-log suite.
172 lines
4.8 KiB
Text
172 lines
4.8 KiB
Text
.DS_Store
|
|
/venv/
|
|
/venv.old/
|
|
/_pycache/
|
|
*.pyc*
|
|
__pycache__/
|
|
act/
|
|
.act-sandbox-agent.*
|
|
.venv/
|
|
.venv
|
|
.vscode/
|
|
.env
|
|
.op.env
|
|
.env.local
|
|
.env.development.local
|
|
.env.test.local
|
|
.env.production.local
|
|
.env.development
|
|
.env.test
|
|
.hermes-docker/
|
|
.notebooklm-home/
|
|
.notebooklm-cli-venv/
|
|
.notebooklm-playwright/
|
|
.pip-cache/
|
|
.uv-cache/
|
|
compose.hermes.local.yml
|
|
export*
|
|
__pycache__/model_tools.cpython-310.pyc
|
|
__pycache__/web_tools.cpython-310.pyc
|
|
logs/
|
|
data/
|
|
.pytest_cache/
|
|
test_durations.json
|
|
.pytest-cache/
|
|
tmp/
|
|
temp_vision_images/
|
|
hermes-*/*
|
|
examples/
|
|
tests/quick_test_dataset.jsonl
|
|
tests/sample_dataset.jsonl
|
|
run_datagen_kimik2-thinking.sh
|
|
run_datagen_megascience_glm4-6.sh
|
|
run_datagen_sonnet.sh
|
|
source-data/*
|
|
run_datagen_megascience_glm4-6.sh
|
|
data/*
|
|
# No trailing slash: also matches node_modules SYMLINKS (worktrees often
|
|
# symlink node_modules to the main checkout; the dir-only pattern let one
|
|
# slip into a commit and break `npm ci` on CI with ENOTDIR).
|
|
node_modules
|
|
browser-use/
|
|
agent-browser/
|
|
# Private keys
|
|
*.ppk
|
|
*.pem
|
|
privvy*
|
|
images/
|
|
__pycache__/
|
|
hermes_agent.egg-info/
|
|
wandb/
|
|
testlogs
|
|
playwright-report/
|
|
test-results/
|
|
# Playwright visual regression baselines — cached from main in CI, not committed
|
|
*-snapshots/
|
|
|
|
# CLI config (may contain sensitive SSH paths)
|
|
cli-config.yaml
|
|
|
|
# Skills Hub state (lives in ~/.hermes/skills/.hub/ at runtime, but just in case)
|
|
skills/.hub/
|
|
ignored/
|
|
.worktrees/
|
|
environments/benchmarks/evals/
|
|
|
|
# Web UI build output
|
|
hermes_cli/web_dist/
|
|
# Cross-process web UI build lock (flock target, always empty)
|
|
.web_ui_build.lock
|
|
apps/desktop/build/
|
|
apps/desktop/dist/
|
|
|
|
# tsc-emitted artifacts (a stray `tsc -b` compiles into src/, and vite then
|
|
# resolves the stale .js OVER the .tsx — never track these)
|
|
apps/desktop/src/**/*.js
|
|
apps/desktop/src/**/*.js.map
|
|
apps/desktop/src/**/*.d.ts
|
|
!apps/desktop/src/global.d.ts
|
|
!apps/desktop/src/vite-env.d.ts
|
|
apps/shared/src/**/*.js
|
|
apps/shared/src/**/*.js.map
|
|
apps/shared/src/**/*.d.ts
|
|
apps/desktop/release/
|
|
*.tsbuildinfo
|
|
|
|
# Web UI assets — synced from @nous-research/ui at build time via
|
|
# `npm run sync-assets` (see web/package.json).
|
|
web/public/fonts/
|
|
web/public/ds-assets/
|
|
|
|
# Release script temp files
|
|
.release_notes.md
|
|
mini-swe-agent/
|
|
|
|
# Nix
|
|
.direnv/
|
|
.nix-stamps/
|
|
result
|
|
website/static/api/skills-index.json
|
|
# skills.json + skills-meta.json are build artifacts emitted by
|
|
# website/scripts/extract-skills.py during prebuild — keep them out of
|
|
# git for the same reason as skills-index.json (large, generated, change
|
|
# every build).
|
|
website/static/api/skills.json
|
|
website/static/api/skills-meta.json
|
|
# automation-blueprints-index.json is a build artifact emitted by
|
|
# website/scripts/extract-automation-blueprints.py during prebuild.
|
|
website/static/api/automation-blueprints-index.json
|
|
models-dev-upstream/
|
|
|
|
# Local editor / agent tooling (machine-specific; keep in global config, not the repo)
|
|
.codex/
|
|
.cursor/
|
|
.gemini/
|
|
.zed/
|
|
.mcp.json
|
|
opencode.json
|
|
config/mcporter.json
|
|
|
|
hermes_cli/tui_dist/*
|
|
hermes_cli/scripts/
|
|
docs/superpowers/*
|
|
# Working directory for the Hermes Agent's session state (~/.hermes/ at runtime;
|
|
# also created in-repo when an agent operates in this checkout). Plans, audit
|
|
# logs, and per-session caches are never artifacts of the codebase.
|
|
.hermes/
|
|
|
|
# Desktop/bootstrap install marker written into the managed checkout root by the
|
|
# bootstrap installer. It is Hermes-managed runtime state, never a code change —
|
|
# ignore it so `hermes update`'s `git stash push --include-untracked` does not
|
|
# treat it as a local edit and autostash it on every run (#38529).
|
|
.hermes-bootstrap-complete
|
|
|
|
# Persistent dev sandbox dir (scripts/dev-sandbox.sh --persistent)
|
|
.hermes-sandbox/
|
|
|
|
# Interrupted-update breadcrumb + recovery lock written next to the shared venv
|
|
# by `hermes update` / launch-time self-heal. Runtime state, never a code change
|
|
# — ignore so `git status` stays clean and update's autostash skips them.
|
|
.update-incomplete
|
|
.update-incomplete.lock
|
|
|
|
# Tool Search live-test harness output — non-deterministic model transcripts,
|
|
# regenerated by scripts/tool_search_livetest.py. Never an artifact of the repo.
|
|
scripts/out/
|
|
|
|
# Per-release changelog drafts. These exist only transiently during a release
|
|
# cut (passed to `gh release create --notes-file`); the GitHub Release itself
|
|
# stores the published notes. They are not a build artifact and must never be
|
|
# committed to the repo root. See the hermes-release skill.
|
|
RELEASE_v*.md
|
|
|
|
# Desktop demo-run scratch output (hermes writes demo/*.txt during recorded
|
|
# walkthroughs). Throwaway artifacts, never part of the app.
|
|
apps/desktop/demo/
|
|
|
|
# PR infographics are rendered locally and embedded in PR descriptions via the
|
|
# image-provider (fal.media) URL — they are NEVER committed to the repo. The
|
|
# PR body is the archive. See the hermes-agent-dev skill's
|
|
# pr-infographic-workflow reference (storage rule + lapse #8 / #COMMIT-1).
|
|
infographic/
|
|
native/fts5_cjk/*.so
|