diff --git a/optional-skills/creative/video-orchestrator/SKILL.md b/optional-skills/creative/video-orchestrator/SKILL.md new file mode 100644 index 0000000000..53b75cda4a --- /dev/null +++ b/optional-skills/creative/video-orchestrator/SKILL.md @@ -0,0 +1,206 @@ +--- +name: video-orchestrator +description: Plan, set up, and monitor a multi-agent video production pipeline backed by Hermes Kanban. Use when the user wants to make ANY video — narrative film, product/marketing, music video, explainer, ASCII/terminal art, abstract/generative loop, comic, 3D, real-time/installation — and the work warrants decomposition into specialized profiles (writer, designer, animator, renderer, voice, editor, etc.) coordinated through a kanban board. Performs adaptive discovery to scope the brief, designs an appropriate team for the requested style, generates the setup script that creates Hermes profiles + initial kanban task, then helps monitor execution and intervene when tasks stall or fail. Routes scenes to whichever Hermes rendering / audio / design skill fits each beat (`ascii-video`, `manim-video`, `p5js`, `comfyui`, `touchdesigner-mcp`, `blender-mcp`, `pixel-art`, `baoyu-comic`, `claude-design`, `excalidraw`, `songsee`, `heartmula`, …) plus external APIs for TTS, image-gen, and image-to-video as needed. +version: 1.0.0 +author: [SHL0MS, alt-glitch] +license: MIT +metadata: + hermes: + tags: [video, kanban, multi-agent, orchestration, production-pipeline] + related_skills: [kanban-orchestrator, kanban-worker, ascii-video, manim-video, p5js, comfyui, touchdesigner-mcp, blender-mcp, pixel-art, ascii-art, songwriting-and-ai-music, heartmula, songsee, spotify, youtube-content, claude-design, excalidraw, architecture-diagram, concept-diagrams, baoyu-comic, baoyu-infographic, humanizer, gif-search, meme-generation] + credits: | + The single-project workspace layout, profile-config patching pattern, + SOUL.md-per-profile model, TEAM.md task-graph convention, and + `--workspace dir:` discipline are adapted from alt-glitch's + original multi-agent video pipeline at + https://github.com/NousResearch/kanban-video-pipeline. +--- + +# Video Orchestrator + +Wrap any video request — from a 15-second product teaser to a 5-minute narrative +short to a music video to an ASCII loop — in a Hermes Kanban pipeline that +decomposes the work to specialized agent profiles. + +This skill does **not** render anything itself. It is a meta-pipeline that: + +1. **Scopes** the request through targeted discovery +2. **Designs** an appropriate team (which roles, which tools per role) based on the style +3. **Generates** a setup script that creates Hermes profiles, project workspace, and the initial kanban task +4. **Hands off** to the director profile, which decomposes via the kanban +5. **Monitors** execution, helps intervene when tasks stall or fail + +The actual rendering happens inside the kanban once it's running, via whichever +existing skills + tools fit the scenes — `ascii-video`, `manim-video`, `p5js`, +`comfyui`, `touchdesigner-mcp`, `blender-mcp`, `songwriting-and-ai-music`, +`heartmula`, external APIs, or plain Python with PIL + ffmpeg. + +## When NOT to use this skill + +- The video is one continuous procedural project that needs no specialists. Just write the code directly. +- The user wants a quick one-shot conversion (e.g. "convert this mp4 to a GIF") — use ffmpeg directly. +- The output is a static image, GIF, or audio-only artifact — use the matching specific skill (`ascii-art`, `gifs`, `meme-generation`, `songwriting-and-ai-music`). +- The work fits a single existing skill cleanly (e.g. a pure ASCII video — just use `ascii-video`). + +## Workflow + +``` +DISCOVER → BRIEF → TEAM DESIGN → SETUP → EXECUTE → MONITOR +``` + +### Step 1 — Discover (ask the right questions) + +The discovery process is **adaptive**: ask only what is actually needed. Always +start with three questions to identify the broad shape: + +- **What is the video?** (one-sentence brief) +- **How long?** (5-30s teaser / 30-90s short / 90s-3min explainer / 3-10min film / longer) +- **What aspect ratio + target platform?** (1:1 / 9:16 / 16:9; X, IG, YouTube, internal, etc.) + +From the answer, classify the style category. The style determines which +follow-up questions to ask. **Do not ask all questions at once.** Ask 2-4 at a +time, listen, then proceed. Make reasonable assumptions whenever the user +implies an answer. + +For complete intake patterns and per-style question banks, see +**[references/intake.md](references/intake.md)**. + +### Step 2 — Brief + +Once enough is known, produce a structured `brief.md` using the template in +`assets/brief.md.tmpl`. Stages: + +1. **Concept** — the one-sentence pitch + emotional north star +2. **Scope** — duration, aspect, platform, deadline +3. **Style** — visual references, brand constraints, tone +4. **Scenes** — beat-by-beat breakdown (durations, content, target tool) +5. **Audio** — narration / music / SFX / silent (per scene if needed) +6. **Deliverables** — file format, resolution, optional alternates (vertical cut, GIF, etc.) + +Show the brief to the user for confirmation before designing the team. **The +brief is the contract** — every downstream task references it. + +### Step 3 — Team design + +Pick role archetypes from the library that fit this video. **Compose, don't +clone.** Most videos need 4-7 profiles. The director is always present; the +rest are picked by what the brief actually requires. + +For the role library and per-style team compositions, see +**[references/role-archetypes.md](references/role-archetypes.md)**. + +For mapping role → which Hermes skills + toolsets it loads, see +**[references/tool-matrix.md](references/tool-matrix.md)**. + +### Step 4 — Setup + +Generate a setup script (`setup.sh`) and run it. The script: + +1. Creates the project workspace (`~/projects/video-pipeline//`) +2. Copies any provided assets into `taste/`, `audio/`, `assets/` +3. Creates each Hermes profile via `hermes profile create --clone` +4. Writes per-profile `SOUL.md` (personality + role definition) +5. Configures profile YAML (toolsets, always_load skills, cwd) +6. Writes `brief.md`, `TEAM.md`, and `taste/` content +7. Fires the initial `hermes kanban create` task assigned to the director + +Use `scripts/bootstrap_pipeline.py` to generate setup.sh from a brief + +team-design JSON. See **[references/kanban-setup.md](references/kanban-setup.md)** +for the setup script structure, profile config patterns, and the critical +"shared workspace" rule. + +### Step 5 — Execute + +Run `setup.sh`. Then provide the user with monitoring commands: + +```bash +hermes kanban watch --tenant # live events +hermes kanban list --tenant # board snapshot +hermes dashboard # visual board UI +``` + +The director profile takes over from here, decomposing the work and routing +tasks to specialist profiles via the kanban toolset. + +### Step 6 — Monitor and intervene + +Stay engaged — the kanban runs autonomously but a stuck task or bad output +needs human (or AI) judgment. + +Monitoring patterns: poll `kanban list` periodically, inspect any RUNNING task +that exceeds its expected duration with `kanban show `, and check +heartbeats. When a worker's output fails review, the standard interventions are: + +1. Comment on the worker's task with specific feedback (`kanban_comment`) +2. Create a re-run task with the original as parent +3. Adjust the brief's scope and let the director re-decompose + +For diagnostic patterns, intervention recipes, and the "task is stuck" +playbook, see **[references/monitoring.md](references/monitoring.md)**. + +## Reference: worked examples + +Six concrete pipelines covering very different video styles — narrative film, +product/marketing, music video, math/algorithm explainer, ASCII video, real-time +installation — showing how the same workflow yields very different teams and +task graphs. See **[references/examples.md](references/examples.md)**. + +## Critical rules + +1. **Discovery before action.** Never start generating a brief or team without + asking at least the three baseline questions. A bad brief cascades through + the entire pipeline. + +2. **Match the team to the video.** Don't reuse the same 4-profile setup for + every job. A music video that doesn't have a beat-analysis profile will + misfire. A narrative film that doesn't have a writer profile will produce + incoherent scenes. See `references/role-archetypes.md`. + +3. **One workspace per project.** All profiles for a given video share the same + `dir:` workspace. Tasks pass artifacts via shared filesystem and structured + handoffs. **Every** `kanban_create` call passes + `workspace_kind="dir"` + `workspace_path=""`. + +4. **Tenant every project.** Use a project-specific tenant + (`--tenant `). Keeps the dashboard scoped and prevents + cross-pollination with other ongoing kanbans. + +5. **Respect existing skills.** When a scene fits an existing skill, the + relevant renderer should load that skill via `--skill ` on its task + or `always_load` in its profile. Do not re-derive what a skill already + provides. + +6. **The director never executes.** Even with the full `kanban + terminal + + file` toolset, the director's `SOUL.md` rules forbid it from executing + work itself. It decomposes and routes only — every concrete task becomes + a `hermes kanban create` call to a specialist profile. The + `kanban-orchestrator` skill spells this out further. + +7. **Don't over-decompose.** A 30-second product video does NOT need 20 tasks. + Aim for the smallest task graph that still parallelizes well and exposes the + right human-review gates. + +8. **Verify API keys BEFORE firing.** External APIs (TTS, image-gen, + image-to-video) need keys in `~/.hermes/.env` or the user's secret store. + A worker that hits a missing-key error wastes a task slot. The setup + script's `check_key` helper aborts cleanly if a required key is missing. + +## File map + +``` +SKILL.md ← this file (workflow + rules) +references/ + intake.md ← discovery question banks per style + role-archetypes.md ← role library (writer, designer, animator, …) + tool-matrix.md ← skill + toolset mapping per role + kanban-setup.md ← setup script structure & profile config + monitoring.md ← watch + intervene patterns + examples.md ← six worked pipelines +assets/ + brief.md.tmpl ← brief skeleton + setup.sh.tmpl ← setup script skeleton + soul.md.tmpl ← profile personality skeleton +scripts/ + bootstrap_pipeline.py ← generate setup.sh from brief + team JSON + monitor.py ← polling + intervention helpers +``` diff --git a/optional-skills/creative/video-orchestrator/assets/brief.md.tmpl b/optional-skills/creative/video-orchestrator/assets/brief.md.tmpl new file mode 100644 index 0000000000..fbe8d8cbfb --- /dev/null +++ b/optional-skills/creative/video-orchestrator/assets/brief.md.tmpl @@ -0,0 +1,79 @@ +# Video Brief — {{TITLE}} + +> Slug: `{{SLUG}}` · Tenant: `{{TENANT}}` · Project workspace: `{{WORKSPACE}}` + +## 1. Concept + +**One-line pitch.** {{ONE_LINE_PITCH}} + +**Emotional north star.** {{EMOTIONAL_NORTH_STAR}} +*(What should the viewer feel walking away?)* + +## 2. Scope + +| | | +|---|---| +| Duration | {{DURATION_S}} seconds | +| Aspect ratio | {{ASPECT}} | +| Resolution | {{RESOLUTION}} | +| Frame rate | {{FPS}} fps | +| Target platforms | {{PLATFORMS}} | +| Deadline | {{DEADLINE}} | +| Quality bar | {{QUALITY_BAR}} *(rough draft / polished / archival)* | + +## 3. Style + +**Visual references.** {{VISUAL_REFS}} + +**Tone.** {{TONE}} + +**Brand constraints.** {{BRAND_CONSTRAINTS}} +*(colors, typography, motion language; or "n/a")* + +**Aesthetic rules.** +{{AESTHETIC_RULES}} + +## 4. Scenes + +Beat-by-beat breakdown. Each scene gets a row. + +| # | Time | Content | Target tool / skill | Audio | Notes | +|---|------|---------|---------------------|-------|-------| +| 1 | 0:00–0:0X | {{SCENE_1_CONTENT}} | {{SCENE_1_TOOL}} | {{SCENE_1_AUDIO}} | {{SCENE_1_NOTES}} | +| 2 | 0:0X–0:0Y | ... | ... | ... | ... | + +## 5. Audio + +**Approach.** {{AUDIO_APPROACH}} +*(narration / music-only / synced to track / silent / mixed)* + +**Voiceover.** {{VO_DETAILS}} +*(provider, voice, language, script source — "n/a" if no VO)* + +**Music.** {{MUSIC_DETAILS}} +*(provided track path / commission via Suno / commission via heartmula / +license-free / "n/a")* + +**SFX.** {{SFX_DETAILS}} +*(generated, library, or "n/a")* + +## 6. Deliverables + +| Format | Resolution | Notes | +|--------|-----------|-------| +| {{PRIMARY_FORMAT}} | {{PRIMARY_RES}} | The main output | +| {{ALT_FORMAT_1}} | {{ALT_RES_1}} | {{ALT_NOTES_1}} | + +**Final filename.** `output/final.mp4` +*(plus optional `output/final-9x16.mp4`, `output/captions.srt`, etc.)* + +## 7. Constraints + +- API keys required: {{API_KEYS_REQUIRED}} +- External dependencies: {{EXT_DEPS}} +- Source assets to incorporate: {{SOURCE_ASSETS}} + +--- + +**This brief is the contract. The director and every downstream profile read +it. If the brief changes, the kanban must be re-fired — don't edit live.** diff --git a/optional-skills/creative/video-orchestrator/assets/setup.sh.tmpl b/optional-skills/creative/video-orchestrator/assets/setup.sh.tmpl new file mode 100644 index 0000000000..bab87ea972 --- /dev/null +++ b/optional-skills/creative/video-orchestrator/assets/setup.sh.tmpl @@ -0,0 +1,185 @@ +#!/usr/bin/env bash +# ═══════════════════════════════════════════════════════════════════════ +# Video Pipeline Setup — {{TITLE}} +# +# Generated by video-orchestrator skill. +# +# Slug: {{SLUG}} +# Workspace: {{WORKSPACE}} +# Tenant: {{TENANT}} +# ═══════════════════════════════════════════════════════════════════════ +set -euo pipefail + +PROJECT_SLUG="{{SLUG}}" +WORKSPACE="$HOME/projects/video-pipeline/${PROJECT_SLUG}" +TENANT="{{TENANT}}" + +# ───────────────────────────────────────────────────────────────────── +# 1. Verify required API keys +# ───────────────────────────────────────────────────────────────────── +echo "═══ Checking required API keys ═══" + +check_key() { + local var="$1" + local kc_account="${2:-hermes}" + local kc_service="${3:-$1}" + if grep -q "^${var}=" "$HOME/.hermes/.env" 2>/dev/null && \ + [ -n "$(grep "^${var}=" "$HOME/.hermes/.env" | cut -d= -f2-)" ]; then + echo " ✓ ${var} (env)" + return 0 + fi + if command -v security >/dev/null 2>&1 && \ + security find-generic-password -a "${kc_account}" -s "${kc_service}" -w >/dev/null 2>&1; then + echo " ✓ ${var} (Keychain ${kc_account}/${kc_service})" + return 0 + fi + echo " ✗ ${var} not set in ~/.hermes/.env or Keychain (${kc_account}/${kc_service})" + return 1 +} + +# Customize this list per project — only check keys actually used: +{{KEY_CHECKS}} + +# ───────────────────────────────────────────────────────────────────── +# 2. Create project workspace +# ───────────────────────────────────────────────────────────────────── +echo "═══ Creating project workspace ═══" +mkdir -p "$WORKSPACE"/{taste,audio/{voiceover,sfx},assets,scenes,checkpoints,tools,output} +{{SCENE_DIRS}} +echo " ✓ $WORKSPACE" + +# ───────────────────────────────────────────────────────────────────── +# 3. Create Hermes profiles +# ───────────────────────────────────────────────────────────────────── +echo "═══ Creating Hermes profiles ═══" + +{{PROFILE_CREATE_COMMANDS}} + +# ───────────────────────────────────────────────────────────────────── +# 4. Configure profiles (toolsets, skills, cwd) +# ───────────────────────────────────────────────────────────────────── +echo "═══ Configuring profiles ═══" + +configure_profile() { + local profile="$1" + local toolsets_json="$2" # JSON array string, e.g. '["kanban","terminal","file"]' + local skills_json="$3" # JSON array string, e.g. '["kanban-worker","ascii-video"]' + python3 - "$profile" "$toolsets_json" "$skills_json" "$WORKSPACE" <<'PY' +"""Patch a Hermes profile config.yaml using PyYAML so we don't depend on the +exact default-config string format. Validates the patch took effect and exits +non-zero if anything's off.""" +import json +import os +import sys + +try: + import yaml +except ImportError: + print("ERROR: PyYAML required. pip install pyyaml", file=sys.stderr) + sys.exit(1) + +profile, toolsets_json, skills_json, workspace = sys.argv[1:5] +toolsets = json.loads(toolsets_json) +skills = json.loads(skills_json) + +p = os.path.expanduser(f"~/.hermes/profiles/{profile}/config.yaml") +if not os.path.exists(p): + print(f" ✗ profile config not found: {p}", file=sys.stderr) + sys.exit(1) + +with open(p) as f: + cfg = yaml.safe_load(f) or {} + +# Apply our changes — only the keys we actually want to set. +cfg["toolsets"] = toolsets +cfg.setdefault("skills", {}) +cfg["skills"]["always_load"] = skills + +# Note: we do NOT touch cfg["approvals"] — that's a security-sensitive +# setting (manual confirmation of tool calls). Workspace cwd is overridden +# per-task by `--workspace dir:` on `hermes kanban create`, so we +# don't need to mutate cfg["terminal"]["cwd"] either. + +with open(p, "w") as f: + yaml.safe_dump(cfg, f, sort_keys=False) + +# Validate +with open(p) as f: + after = yaml.safe_load(f) +errors = [] +if after.get("toolsets") != toolsets: + errors.append(f"toolsets mismatch: {after.get('toolsets')!r}") +if after.get("skills", {}).get("always_load") != skills: + errors.append(f"skills.always_load mismatch: {after.get('skills', {}).get('always_load')!r}") +if errors: + print(f" ✗ {profile}: " + "; ".join(errors), file=sys.stderr) + sys.exit(1) +PY + if [ $? -ne 0 ]; then + echo " ✗ failed to configure ${profile}" >&2 + exit 1 + fi + echo " ✓ ${profile}" +} + +{{PROFILE_CONFIG_COMMANDS}} + +# ───────────────────────────────────────────────────────────────────── +# 5. Write SOUL.md per profile +# ───────────────────────────────────────────────────────────────────── +echo "═══ Writing profile personalities ═══" + +{{SOUL_WRITES}} + +# ───────────────────────────────────────────────────────────────────── +# 6. Copy brief, TEAM.md, and any provided assets +# ───────────────────────────────────────────────────────────────────── +echo "═══ Writing brief + taste ═══" + +cat > "$WORKSPACE/brief.md" <<'BRIEF_EOF' +{{BRIEF_CONTENTS}} +BRIEF_EOF + +cat > "$WORKSPACE/TEAM.md" <<'TEAM_EOF' +{{TEAM_CONTENTS}} +TEAM_EOF + +{{TASTE_WRITES}} + +{{ASSET_COPIES}} + +# ───────────────────────────────────────────────────────────────────── +# 7. Fire the initial kanban task +# ───────────────────────────────────────────────────────────────────── +echo "═══ Firing initial kanban task ═══" + +hermes kanban create "Direct production of {{TITLE}}" \ + --assignee director \ + --workspace dir:"$WORKSPACE" \ + --tenant "$TENANT" \ + --priority 2 \ + --max-runtime 4h \ + --body "$(cat < **Credit:** the single-project-workspace layout, profile-config patching +> approach, SOUL.md-per-profile convention, and `--workspace dir:` rule +> are adapted from alt-glitch's original multi-agent video pipeline: +> [NousResearch/kanban-video-pipeline](https://github.com/NousResearch/kanban-video-pipeline). +> This skill generalizes those patterns across video styles and replaces the +> string-replacement config patcher with a PyYAML-based one. + +## Project workspace structure + +Every video project gets one workspace under `~/projects/video-pipeline//`: + +``` +~/projects/video-pipeline// +├── brief.md ← the contract; all tasks reference +├── TEAM.md ← team composition + task graph (director reads this) +├── taste/ +│ ├── brand-guide.md ← color, typography, motion rules +│ ├── emotional-dna.md ← what the piece should FEEL like +│ └── style-frames/ ← optional: visual references +├── audio/ +│ ├── track.mp3 ← provided music (if any) +│ ├── voiceover/ ← per-line TTS clips +│ └── sfx/ ← sound effects +├── assets/ +│ ├── logos/ +│ ├── fonts/ +│ └── existing-footage/ ← reusable provided clips +├── scenes/ +│ ├── scene-01/ +│ │ ├── VISUAL_SPEC.md ← cinematographer's per-scene spec +│ │ ├── render.py ← renderer's code (or sketch.html, etc.) +│ │ ├── checkpoints/ ← preview frames for QA +│ │ └── clip.mp4 ← the deliverable for this scene +│ ├── scene-02/... +│ └── ... +├── checkpoints/ ← global review frames +├── tools/ ← optional project-local helpers +└── output/ + ├── final.mp4 ← stitched + audio + ├── final-noaudio.mp4 + ├── final-9x16.mp4 ← optional: vertical alternate + └── captions.srt ← optional: subtitle file +``` + +**The slug** is derived from the brief title: lowercase, hyphen-separated. +Example: `q3-product-teaser`, `ascii-mood-loop`, `interview-cut-2026-q1`. + +## The setup.sh script + +The setup script does six things in order: + +1. **Create workspace tree** — all directories above +2. **Create profiles** — `hermes profile create --clone` +3. **Configure profiles** — patch each profile's + `~/.hermes/profiles//config.yaml` to set toolsets, always_load skills, + and `cwd` +4. **Write SOUL.md per profile** — the personality + role definition +5. **Copy any provided assets + write `brief.md`, `TEAM.md`, and `taste/`** +6. **Fire the initial kanban task** — `hermes kanban create` assigned to the director + +See `assets/setup.sh.tmpl` for the skeleton. + +### Profile creation pattern + +```bash +hermes profile create director --clone 2>/dev/null || true +``` + +The `--clone` flag clones from the active profile (preserving model, base +config). The `|| true` makes the script idempotent — re-running won't error if +the profile already exists. + +### Profile config patching + +Each profile has a YAML config at `~/.hermes/profiles//config.yaml`. The +setup script edits exactly two keys: + +1. `toolsets:` — replace the default with the role's required toolsets +2. `skills.always_load:` — list the role's must-load skills (may be empty) + +**Do NOT** modify `approvals.mode` (controls user-confirmation of tool calls +— a security setting that must stay as the user configured it). **Do NOT** +modify `terminal.cwd` — the kanban dispatcher overrides cwd per-task via +`--workspace dir:`, so the profile's cwd is irrelevant to the kanban +work and changing it could break the user's interactive use of the profile. + +Use **PyYAML**, not string replacement, so the patch is robust against +default-config schema drift: + +```bash +configure_profile() { + local profile="$1" + local toolsets_json="$2" # JSON array, e.g. '["kanban","terminal","file"]' + local skills_json="$3" # JSON array, e.g. '["kanban-worker","ascii-video"]' + python3 - "$profile" "$toolsets_json" "$skills_json" <<'PY' +import json, os, sys, yaml +profile, ts_json, sk_json = sys.argv[1:4] +p = os.path.expanduser(f"~/.hermes/profiles/{profile}/config.yaml") +with open(p) as f: + cfg = yaml.safe_load(f) or {} +cfg["toolsets"] = json.loads(ts_json) +cfg.setdefault("skills", {})["always_load"] = json.loads(sk_json) +with open(p, "w") as f: + yaml.safe_dump(cfg, f, sort_keys=False) +PY +} +``` + +PyYAML must be installed in the user's Python (it ships with most Hermes +installs). If absent: `pip install pyyaml`. + +The setup script should also **validate** the patch by re-reading the file +and comparing — see `assets/setup.sh.tmpl` for the validation pattern. + +### SOUL.md per profile + +Each profile gets a `SOUL.md` at `~/.hermes/profiles//SOUL.md` that +defines its role, voice, and rules. See `assets/soul.md.tmpl` for the +template. Customize per role and per project. + +The director's SOUL.md should be the most opinionated — its voice flavors +the entire production. **Critical content for the director's SOUL.md:** + +- **Anti-temptation rules:** "Do not execute the work yourself. For every + concrete task, create a kanban task and assign it. Decompose, route, comment, + approve — that's the whole job." (The `kanban-orchestrator` skill provides + the deeper playbook; load it.) +- **Decomposition steps:** Read `brief.md`, `TEAM.md`, `taste/`. Use the team + graph in `TEAM.md` to fan out tasks. +- **The workspace_path rule** (see below). + +Other profiles' SOUL.md is briefer; mostly mechanical: who you are, what you +read, what you produce, what skills/tools to use, where to write outputs. +Most non-director profiles should `always_load: kanban-worker` for the +deeper-than-baseline kanban guidance. + +### Initial kanban task + +The final action of setup.sh is firing the kanban: + +```bash +hermes kanban create "Direct production of