Commit graph

12 commits

Author SHA1 Message Date
Ben Barclay
e327eaa2a0 feat(sync): default the sync plane to production
Skill Sync had no default base URL, so a user with no `sync.base_url` in
config.yaml and no HERMES_SYNC_BASE_URL got:

    sync inert: no sync base URL configured (config.yaml sync.base_url
    or HERMES_SYNC_BASE_URL).

Every sync command was unusable out of the box. The URL was left unset
because the plane did not exist yet when the client was written; it does now.

- Adds DEFAULT_SYNC_BASE_URL = "https://gateway-gateway.nousresearch.com" and
  returns it as the last step of resolve_sync_base_url().
- Resolution order is unchanged otherwise: HERMES_SYNC_BASE_URL ->
  config.yaml sync.base_url -> production default. The env var and config key
  now exist to point a dev/staging build at another plane rather than to make
  the feature work at all.
- Follows the existing precedent for production endpoints in this codebase
  (DEFAULT_NOUS_PORTAL_URL in hermes_cli/auth.py, HERMES_DIAGNOSTICS_BASE_URL
  in diagnostics_upload.py): a module constant with env/config override.

The "no sync base URL configured" guards are kept — they are now unreachable
in practice but remain correct if the default is ever blanked.

Tests: 3 new — the default is returned when nothing is configured, config
still overrides it, and the constant is a bare https origin (no trailing
slash, no path) since the client appends /v1/sync/. 2349 passed / 0 failed
across 56 suites via scripts/run_tests.sh.

Verified against a temp HERMES_HOME with no config: resolves to the
production plane; HERMES_SYNC_BASE_URL and sync.base_url both still win, and
trailing slashes are stripped.
2026-07-29 08:14:03 -07:00
Ben Barclay
f9c4d835f9 refactor(sync): name the access gate for what it is (Nous admin)
The client called its gate "the DEV-PHASE gate (tool_gateway_admin)", which
reads as though Skill Sync is gated on an unrelated service's admin right.
It isn't. NAS populates that claim from Permissions.ADMIN_ACCESS — the global
portal admin permission that guards /admin/* — so the gate is "is this user a
Nous admin?". The claim is simply named for its first consumer, the tool
gateway.

Renamed on this side to say what it means, while keeping the wire string
(other services read it):

- DEV_GATE_CLAIM -> NOUS_ADMIN_CLAIM (value unchanged: "tool_gateway_admin",
  with a comment recording why the wire name differs).
- identity/status key dev_gate_ok -> nous_admin, across the client, the CLI
  consumers, and the tests.
- The module docstring now states where the claim comes from, that the wire
  name is misleading, and that this gate is pre-launch containment rather
  than the shipping entitlement — admin status conflates "may administer
  Nous" with "has Skill Sync enabled" and has no middle setting for a beta
  cohort. Choosing the real entitlement is left as a separate decision.

Naming only — no behaviour change, and no change to which accounts can sync.
The user-facing messages stay deliberately vague ("not enabled for your
account yet") rather than telling users they need portal admin.

Verified: 2346 passed / 0 failed across 56 suites via scripts/run_tests.sh;
`hermes sync status` against a live token reports "nous_admin": true. Zero
stale dev_gate_ok / DEV_GATE_CLAIM references remain.
2026-07-29 07:57:25 +10:00
Ben Barclay
4f990ec09e refactor(sync): put every Skill Sync verb under hermes sync; drop HSP naming
Encapsulates the feature behind one command for launch, and adopts the
official product name.

One command:
- `propose` moves from `hermes skills propose` to `hermes sync propose`, so
  the whole feature is one command to learn and one to document. Its handler
  moves from cmd_skills to cmd_sync accordingly.
- The `hermes sync` parser now documents both halves plainly: personal sync
  across your devices, and sharing with your organisation. Added an examples
  epilog; rewrote the verb help in user language ("Include a skill in your
  sync" rather than "Opt a skill into sync").
- Every user-facing string that pointed at `hermes skills propose` now points
  at `hermes sync propose` (8 sites, including the agent-visible guidance
  returned by skill_manage and the org provenance header).

This also clears the way for #39343, which adds its own top-level `sync` for
git-repo profile backup — that feature nests under `skills`, this one owns
`sync`.

Naming:
- HSP / "Hermes Sync Protocol" is gone from prose, docstrings, and comments.
  The feature is "Skill Sync".
- Public identifiers renamed: HSPClient -> SyncClient, HSPError -> SyncError,
  HSPConflict -> SyncConflict, hsp_address -> wire_address, HSP_VERSION ->
  WIRE_VERSION.
- The WIRE names are deliberately NOT renamed: the `hsp_version` capability
  field and the `x-hsp-object-type` response header are set by the deployed
  gateway-gateway sync plane (verified in src/sync/syncRouter.ts), so
  renaming them client-side would break sync against a live server. A comment
  at the version constant records why they differ from the product name.
- The version-mismatch error is now actionable ("this server speaks sync
  version X, but this Hermes speaks Y — update Hermes to sync with it")
  instead of leaking the protocol acronym.

Also fixes a wiring gap found on the way: the gateway housekeeping tick
pulled personal skills but never org skills — the same defect already fixed
for the CLI. Org pull now runs there too, gated on real org membership.

Tests: the jargon guard now also fails on a bare "HSP". The two tests that
asserted the old cross-command structure are replaced by three asserting the
new one (propose IS under sync, propose is NOT under skills, sync usage
lists it). 2294 passed / 0 failed across all 51 suites that import the
changed modules, via scripts/run_tests.sh.

Verified by running the real CLI: `hermes sync --help` lists all eight verbs,
`hermes skills --help` no longer mentions propose, `hermes sync propose
--help` parses, and `hermes sync status` still reports live org state.
2026-07-29 07:47:06 +10:00
Ben Barclay
981feb6730 feat(skills): org skills are editable in place; local edits survive org updates
The read-only org mirror broke the learning loop precisely where it matters
most. The system prompt tells every agent to patch a skill the moment it
finds a gap, and shared skills are the ones the most people use — but every
write to _org/ was refused, and the curator was excluded from them outright.
So org skills froze while personal skills kept improving, and the offered
alternative ("fork it into a personal skill, then propose the fork") is not
something an agent does mid-task. The refusal WAS the feature; improvements
were simply lost, and manual forks would have fragmented the shared set.

Edit in place:
- skill_manage patch/edit/write_file now work on org skills. Only delete is
  still refused (the mirror is a view of org HEAD — a local delete returns on
  the next pull; removing a shared skill is an admin action).
- Org skills are curation-eligible again, so the curator can improve the
  highest-leverage skills in the system instead of skipping them.
- The load-time provenance header now says edits are allowed and kept,
  instead of instructing the agent not to edit.

Local edits are never overwritten:
- pull_org_skills previously rmtree'd each skill dir and re-materialized it,
  silently destroying local work on the next session start. It now records a
  content fingerprint per skill (.org-baseline.json) when it writes one, and
  SKIPS any skill whose local content diverges from that baseline.
- When upstream ALSO changed such a skill, it is reported in the pull
  result's "conflicted" list and left untouched for the user to resolve
  deliberately (propose the local version, or delete it and re-pull to take
  theirs). A missing baseline is treated as unmodified so pre-existing
  mirrors do not raise phantom conflicts.
- Fingerprints are content-based (path + bytes, sorted), so a touch/mtime
  change is not mistaken for an edit.

Sharing back:
- Default: the edit stays local and the tool result tells the user to run
  "hermes skills propose <skill>".
- Opt-in sync.org_auto_propose / HERMES_SYNC_ORG_AUTO_PROPOSE submits each
  edit immediately. Defaults OFF — pushing every agent edit to a whole
  organisation is not a safe default. A failed submission never fails the
  edit; the change is saved and can be proposed later.
- "hermes sync status" lists org skills with unshared local edits;
  "hermes sync pull" reports conflicts it declined to overwrite.

Tests: 25 in the namespace suite (was 15). The two that asserted the old
read-only behaviour now assert the opposite. New coverage for edit-applied,
share-back guidance, delete-still-refused, curation-allowed, edit detection,
missing-baseline tolerance, mtime-insensitivity, and the auto-propose
default. 428 passed / 0 failed via scripts/run_tests.sh.

Verified through the REAL pull path against a mock plane: pull v1 -> edit in
place -> upstream ships v2 -> pull leaves the local edit intact, reports the
conflict, and surfaces it in status.
2026-07-28 09:08:04 +10:00
Ben Barclay
d60d981281 fix(sync): don't require a base URL when an explicit client is supplied
pull_org_skills/propose_skill resolved and demanded HERMES_SYNC_BASE_URL
before using the caller-provided `client`, which already carries its own
base URL. Only resolve/require it on the path that actually constructs a
client.

Caught by scripts/run_tests.sh, which blanks env vars to match CI. Plain
`pytest` masked it: my shell had HERMES_SYNC_BASE_URL exported from live
testing, so the redundant check silently passed. Reproduced deliberately
with `env -u HERMES_SYNC_BASE_URL pytest` before fixing.

371 passed / 0 failed across the sync, skills, prompt, and skill-utils
suites via the canonical runner.
2026-07-27 16:44:27 +10:00
Ben Barclay
797c52b571 fix(sync): wire org skill pull into the runtime; scrub internal jargon
Two defects found by manual testing on the branch.

1. ORG SYNC NEVER RAN. The org pull/mirror/gating machinery was fully
   implemented and unit-tested but had ZERO runtime callers —
   maybe_pull_org_skills() was referenced only inside a comment, and
   `hermes sync` had no org path at all. Every code path fell through to
   personal sync (refs/user/<sub>/), so org skills never loaded and the
   feature looked like 'everything syncs to my personal org' even with a
   valid org token. The unit tests could not catch this: they invoked the
   functions directly, which is exactly the gap they left open.

   - cli.py session startup now calls maybe_pull_org_skills() alongside the
     personal maybe_pull_skills(), fail-quiet.
   - Auto-pull is gated on real org membership: resolve_org_identity()
     requires an org role on the token, only issued for multi-member orgs,
     so a solo account never reaches the network.
   - `hermes sync pull` refreshes the org mirror too (one pull, both
     surfaces) and reports what it refreshed.
   - `hermes sync status` exposes org_available/org_id/org_role/org_skills
     plus a plain-language summary, so a user can tell whether the org
     workflow applies instead of it being invisible.

2. INTERNAL JARGON LEAKED TO USERS. Help text and errors exposed internal
   milestone/spec coordinates: 'Propose a skill ... (M2)', 'Personal skill
   sync (HSP/1)', 'DEV-PHASE gate closed: your token lacks
   tool_gateway_admin', 'contract §4.3', and an inert message describing our
   internal personal-vs-multi-member design split. All rewritten in user
   language. Feature-local comments/docstrings lost their internal
   coordinates (§N, M1/M2, design.md, PR numbers) while keeping the
   explanatory prose. Pre-existing issue references elsewhere in the tree
   were deliberately left untouched.

Tests: 4 new guards, including two that assert the CALL SITES exist so the
org pull cannot silently become dead code again (verified failing when the
wiring is removed) and one that fails if user-facing help leaks jargon.
344 passed across the sync/skills/prompt suites.

Verified against live staging with a real org token: sync status reports
org_available=true, org_role=OWNER; sync pull performs the org refresh; the
.active_org marker is written with the org id from the token.
2026-07-27 16:40:16 +10:00
Ben Barclay
78598d091a feat(skills): org-skill namespace — token-gated discovery, fail-loud collisions, provenance (M2)
Implements the agreed design (2026-07-23): org skills are FIRST-CLASS (bare
names) with three hard companions.

1. TOKEN-GATED RESOLUTION: _org/<org_id>/ mirrors resolve ONLY while marked
   active. pull_org_skills (which runs only after the token's org_id+org_role
   verified) writes _org/.active_org; discovery (iter_skill_index_files,
   _find_skill_dir, snapshot manifest) prunes every other mirror. Leave the
   org (verified personal token in maybe_pull_org_skills) => marker cleared
   => org skills stop resolving; offline => marker untouched (grace).
   Snapshot manifest includes the marker so org switches invalidate the
   prompt snapshot; _SKILLS_SNAPSHOT_VERSION bumped to 2.

2. FAIL-LOUD COLLISIONS: listing pass unified across snapshot/scan paths;
   a personal/org name clash flags BOTH entries '[name collision — load via
   category path]' — neither side silently wins (personal-wins = silent
   divergence from the org set; org-wins = shadowed personal work).
   skill_view's existing multi-candidate refusal already rejects the
   ambiguous bare name.

3. PROVENANCE: org entries list under an org:<org_id> category with
   '[org-shared: by <author>]' tags; skill_view prepends a load-time header
   (org, author, as-of + read-only/fork-and-propose guidance) INTO the
   content the model consumes, plus an org_provenance result field. Author
   comes from the pull-time .org-provenance.json sidecar (HEAD commit author
   — token-verified at push by the plane's author_mismatch guard, gg #166).

4. READ-ONLY MIRROR: skill_manage patch/edit/delete/write_file refuse org-
   mirror targets with fork-and-propose guidance; org skills are curation-
   exempt (is_curation_eligible False — the org HEAD owns them).

Tests: 11 new (tests/agent/test_org_skill_namespace.py) covering gating,
stale-mirror pruning, org-switch flip, snapshot provenance, listing labels,
both-sides collision flags, read-only guard, curation exemption; 448 green
across skills/prompt/sync suites. Live E2E (real modules, temp HERMES_HOME,
mock plane): merge -> pull -> marker+sidecar -> labeled listing -> exactly-2
collision flags -> load-time header -> edit refused -> marker cleared =>
org skills vanish, personal survive.
2026-07-24 11:49:33 +10:00
Ben Barclay
bdef497a5a feat(sync): M2 org-skills client — _org/ pull, hermes skills propose, 202 handling
Client leg of M2 org-shared skills (hsp-1-contract.md §11), pairing with
gateway-gateway #162 and NAS #768 (both merged).

- resolve_org_identity(): org_id + org_role from the token claims. NO
  org_role claim (personal org — NAS only stamps it for multi-member orgs)
  => SyncInertError => every org surface is inert; personal M1 sync
  untouched (contract §11.1 REFINED). org_sync_available() for callers.
- pull_org_skills(): materialize the org canonical set (refs/org/<org_id>/
  HEAD) into ~/.hermes/skills/_org/<org_id>/ — fast-forward only, no client
  merge on the org path (design.md §2.6); read-only mirror by convention
  (§7.1: a local edit is a personal fork until proposed). maybe_pull_org_
  skills() best-effort hook (never raises; inert without the claim).
- propose_skill(): snapshot the LOCAL skill dir, splice it into the org
  HEAD's skill-tree map (per-skill delta, never wholesale replace), upload
  ?scope=org, CAS the org HEAD. ADMIN => direct merge; MEMBER => server
  converts to a proposal — cas_ref now surfaces 202 as
  {proposal_pending: True, proposal_id, ref} (success-shaped, NEVER
  presented as live). Non-interactive by design for the future automated
  submitter (Ben's trajectory note).
- put_objects(org_scope=True) adds ?scope=org (contract §11.5).
- is_sync_eligible(): skills under _org/ are excluded from PERSONAL sync —
  enterprise content never rides a personal push (§11.11).
- CLI: hermes skills propose <name> [-m msg] — prints 'pending admin
  review' for 202, 'merged' for admin, and a plain 'org sync unavailable'
  for personal orgs instead of a raw 403.

Tests: 56 in the sync client suite (+10 org: identity gate both ways, _org/
personal-sync exclusion, admin direct merge, member 202 w/ HEAD untouched +
proposal ref parked + never-merged, splice-not-replace root, pull mirror
materialization, no-head noop, org-feature gate, maybe_pull inert). Mock
server extended (org feature flag, member-CAS→202). 103 across skills
suites. Live CLI smoke: propose --help + personal-org inert path verified.
2026-07-23 20:02:26 +10:00
Ben Barclay
ce4a08d6e1 feat(sync): descriptive device names (hostname default, --name, Cloud env seed)
Commit author.device was an opaque uuid4 hex, so the sync console showed a hash
per device. Make it human-friendly:

- Default: seed new devices from the short hostname + a short random suffix
  (e.g. bens-macbook-a1b2c3) instead of a bare uuid. Existing .sync_device_id
  files are honored verbatim (a machine keeps its id) — backward-compatible.
- hermes sync device [--name N]: show or set an explicit label
  (set_device_name(), written to ~/.hermes/skills/.sync_device_id). New commits
  use it; past commits keep their old label (author.device is immutable).
- Hermes Cloud: HERMES_SYNC_DEVICE_NAME env seeds the first-use label so a
  hosted instance shows a recognizable name with no CLI call. Precedence:
  explicit .sync_device_id file > HERMES_SYNC_DEVICE_NAME env > hostname default.
  Env seeds first-use only, then persists, so a later --name still wins locally.

Tests: 46 pass (+5: hostname default, file-wins precedence, env first-use seed +
persistence, set/trim, empty-name reject). CLI verb smoke-verified end-to-end.
2026-07-23 11:50:10 +10:00
Ben Barclay
1f33fb2d00 feat(sync): env-configurable sync defaults + rename local .sync_manifest
Two changes:

1) Rename the client-local head-bookkeeping file .sync_manifest -> .sync_state
   (read_sync_state/write_sync_state) to remove the name collision with the
   §2.8 plane 'sync-manifest' OBJECT. read_sync_state migrates an existing
   .sync_manifest on first read so no device loses its head record.

2) Make the knobs a Hermes Cloud instance needs env-configurable, so an
   instance can be set up to use sync BY DEFAULT with no config.yaml edit or
   per-skill CLI call. Precedence: HERMES_SYNC_* env -> config.yaml sync.* ->
   built-in default (mirrors the existing HERMES_SYNC_BASE_URL bridge).
     - HERMES_SYNC_ENABLED        -> sync.enabled        (master on/off; def off)
       gated in maybe_push/maybe_pull alongside the dev-gate + base_url.
     - HERMES_SYNC_DEFAULT_OPT_IN -> sync.default_opt_in  (M1-D policy; def off)
       opt-out mode: every eligible skill syncs unless usage rec says sync:false;
       'your skills follow you with no setup' — the Cloud default. opt-in mode
       (default) unchanged: only sync:true skills sync.
   sync_status() + 'hermes sync status' surface feature_enabled/default_opt_in.

Cross-repo byte-compat re-verified: client manifest bytes still parse cleanly
through gateway-gateway parseSyncManifest.

Tests: 41 pass (+7: env precedence, opt-out/opt-in policy, rename migration).
2026-07-23 08:14:53 +10:00
Ben Barclay
3c4154aca4 feat(sync): opt-in as content sync-manifest (design.md §2.8), cross-device
Replace the device-local opt-in model (`.usage.json` `sync` flag as the sole
source of truth) with the §2.8 content model: a root-level `sync-manifest` blob
in the tree at `refs/user/<owner>/HEAD` recording per-skill {name, enabled},
matching gateway-gateway src/sync/manifest.ts byte-for-byte.

- build/parse_sync_manifest: canonical {type,version:1,skills:[{name,enabled}]},
  strict parse (malformed != empty).
- snapshot_profile embeds the manifest as a root-level blob alongside skill
  subtrees; the skill walk skips it (blob, not a SKILL.md-bearing tree).
- pull reconciles local opt-in intent FROM the plane manifest, so a skill opted
  in on one device becomes opted in on the others (opt-in is now cross-device,
  not per-device). Never silently disables a locally-enabled skill on pull.
- .usage.json `sync` flag kept as the editable local intent; plane manifest is
  authoritative.

Cross-repo byte-compat verified: the Python client's manifest bytes parse
cleanly through gateway-gateway's real parseSyncManifest (tsx harness).

Tests: 34 pass incl. 5 new (roundtrip, wire shape, strict-reject, root-blob
embed, pull adopts opt-in from manifest).
2026-07-23 08:01:05 +10:00
Ben Barclay
eaa539552c feat(sync): HSP/1 personal skill sync client (Milestone 1, client strand)
Implements the hermes-agent HSP/1 sync CLIENT against the frozen wire
contract (~/src/specs/collective-wisdom/hsp-1-contract.md §8), tested
against an in-process mock HSP server.

tools/skills_sync_client.py (new, low-level; does NOT import the CLI):
  * Full 64-hex sha256 content addressing + canonical JSON (§2.1/§2.5,
    OI-5) — kept distinct from the truncated local content_hash namespace.
  * HSPClient: capabilities/refs/objects GET, batch object upload
    (multipart, raw bytes per §1/§4.2), CAS ref (§4.4) with 409->HSPConflict.
  * Object building: skill dir -> blob/tree/commit; exec-bit preserved,
    symlinks skipped, oversize (413) surfaced; profile-root category trees.
  * push/pull + three-way merge (M1-C): reuses the origin/user/incoming
    decision semantics of skills_sync.py; non-overlap -> merge commit +
    retry CAS; true overlap -> refs/user/<owner>/conflict/<n> + surface.
  * DEV-PHASE gate: sync is INERT unless the resolved Nous token carries
    tool_gateway_admin===true (decoded from the bearer; server re-verifies).
  * Auth reuses resolve_nous_runtime_credentials() (no refresh reimpl).
  * maybe_push_skills / maybe_pull_skills gate-and-swallow entrypoints.

Opt-in (M1-D): tools/skill_usage.set_sync / is_sync_enabled — a `sync`
flag on the .usage.json sidecar; nothing syncs unless opted in. Only
agent-created/user-authored skills are eligible (bundled/hub excluded).

Hooks:
  * Debounced push in skill_manage success block (after the write gate).
  * Periodic pull at the two curator tick sites (gateway housekeeping loop
    + CLI startup).

CLI: hermes sync status|pull|push|now|enable|disable
  (hermes_cli/subcommands/sync.py + cmd_sync in main.py).

Tests: tests/tools/test_skills_sync_client.py — 29 tests (addressing,
canonicalization, dev gate, opt-in, object building, merge decisions, and
e2e push/pull/idempotency/conflict against a stdlib mock HSP server).
2026-07-18 13:55:09 +10:00