From 6f27390fae352cc4e2aa5f41f1cbd35139656fb5 Mon Sep 17 00:00:00 2001 From: kshitijk4poor Date: Fri, 17 Apr 2026 21:50:35 +0530 Subject: [PATCH] feat: rewrite TouchDesigner skill for twozero MCP (v2.0.0) Major rewrite of the TouchDesigner skill: - Replace custom API handler with twozero MCP (36 native tools) - Add audio-reactive GLSL proven recipe (spectrum chain, pitfalls) - Add recording checklist (FPS>0, non-black, audio cueing) - Expand pitfalls: 38 entries from real sessions (was 20) - Update network-patterns with MCP-native build scripts - Rewrite mcp-tools reference for twozero v2.774+ - Update troubleshooting for MCP-based workflow - Remove obsolete custom_api_handler.py - Generalize Environment section for all users - Remove session-specific Paired Skills section - Bump version to 2.0.0 --- skills/creative/touchdesigner/SKILL.md | 427 +++++++----- .../touchdesigner/references/mcp-tools.md | 645 +++++++----------- .../references/network-patterns.md | 286 ++++---- .../touchdesigner/references/pitfalls.md | 622 +++++++++++------ .../touchdesigner/references/python-api.md | 24 +- .../references/troubleshooting.md | 438 ++++++------ .../scripts/custom_api_handler.py | 140 ---- .../creative/touchdesigner/scripts/setup.sh | 236 +++---- 8 files changed, 1398 insertions(+), 1420 deletions(-) delete mode 100644 skills/creative/touchdesigner/scripts/custom_api_handler.py diff --git a/skills/creative/touchdesigner/SKILL.md b/skills/creative/touchdesigner/SKILL.md index 0f464193f..88fc79b2e 100644 --- a/skills/creative/touchdesigner/SKILL.md +++ b/skills/creative/touchdesigner/SKILL.md @@ -1,278 +1,339 @@ --- name: touchdesigner -description: "Control a running TouchDesigner instance programmatically — create operators, set parameters, wire connections, execute Python, build real-time visuals. Covers: GLSL shaders, audio-reactive, generative art, video processing, instancing, and live performance." -version: 3.0.0 +description: "Control a running TouchDesigner instance via twozero MCP — create operators, set parameters, wire connections, execute Python, build real-time visuals. 36 native tools." +version: 1.0.0 author: Hermes Agent license: MIT metadata: hermes: - tags: [TouchDesigner, MCP, creative-coding, real-time-visuals, generative-art, audio-reactive, VJ, installation, GLSL] + tags: [TouchDesigner, MCP, twozero, creative-coding, real-time-visuals, generative-art, audio-reactive, VJ, installation, GLSL] related_skills: [native-mcp, ascii-video, manim-video, hermes-video] - security: - allow_network: true - allow_install: true - allow_config_write: true + --- -# TouchDesigner Integration +# TouchDesigner Integration (twozero MCP) + +## CRITICAL RULES + +1. **NEVER guess parameter names.** Call `td_get_par_info` for the op type FIRST. Your training data is wrong for TD 2025.32. +2. **If `tdAttributeError` fires, STOP.** Call `td_get_operator_info` on the failing node before continuing. +3. **NEVER hardcode absolute paths** in script callbacks. Use `me.parent()` / `scriptOp.parent()`. +4. **Prefer native MCP tools over td_execute_python.** Use `td_create_operator`, `td_set_operator_pars`, `td_get_errors` etc. Only fall back to `td_execute_python` for complex multi-step logic. +5. **Call `td_get_hints` before building.** It returns patterns specific to the op type you're working with. ## Architecture -Hermes Agent -> HTTP REST (curl) -> TD WebServer DAT (port 9981) -> TD Python environment. +``` +Hermes Agent -> MCP (Streamable HTTP) -> twozero.tox (port 40404) -> TD Python +``` -The agent controls a **running TouchDesigner instance** via a REST API on port 9981. It does NOT generate .toe files from scratch. +36 native tools. Free plugin (no payment/license — confirmed April 2026). +Context-aware (knows selected OP, current network). +Hub health check: `GET http://localhost:40404/mcp` returns JSON with instance PID, project name, TD version. -## First-Time Setup (one-time, persists across sessions) +## Setup (Automated) -### 1. Verify TD is running and check for existing API +Run the setup script to handle everything: ```bash -lsof -i :9981 -P -n | grep LISTEN # TD listening? -curl -s --max-time 5 http://127.0.0.1:9981/api/td/server/td # API working? +bash ~/.hermes/skills/creative/touchdesigner/scripts/setup.sh ``` -If HTTP 200 + JSON → skip to **Discovery**. Setup is already done. +The script will: +1. Check if TD is running +2. Download twozero.tox if not already cached +3. Add `twozero_td` MCP server to Hermes config (if missing) +4. Test the MCP connection on port 40404 +5. Report what manual steps remain (drag .tox into TD, enable MCP toggle) -### 2. If no API: deploy the custom handler +### Manual steps (one-time, cannot be automated) -The user must paste ONE line into TD Textport (Alt+T / Dialogs > Textport and DATs): +1. **Drag `~/Downloads/twozero.tox` into the TD network editor** → click Install +2. **Enable MCP:** click twozero icon → Settings → mcp → "auto start MCP" → Yes +3. **Restart Hermes session** to pick up the new MCP server -``` -exec(open('PATH_TO_SKILL/scripts/custom_api_handler.py').read()) -``` - -Copy this to their clipboard with `pbcopy`. This creates a WebServer DAT + callback handler pair in `/project1` that implements the REST API. No external dependencies. - -**Why not the official .tox?** The `mcp_webserver_base.tox` from 8beeeaaat/touchdesigner-mcp frequently fails to import its Python modules after drag-drop (relative path resolution issue). Our custom handler is self-contained and more reliable. See `references/pitfalls.md` #1-2. - -### 3. Save the project to persist the API - -After the handler is running, save the project so the API auto-starts on every future TD launch: - -```python -td_exec("project.save(os.path.expanduser('~/Documents/HermesAgent.toe'))") -``` - -TD auto-opens the last saved project on launch. From now on, `open /Applications/TouchDesigner.app` → port 9981 is live → agent can connect immediately. - -To launch TD with this project explicitly: +After setup, verify: ```bash -open /Applications/TouchDesigner.app ~/Documents/HermesAgent.toe +nc -z 127.0.0.1 40404 && echo "twozero MCP: READY" ``` -### 4. Optional: Configure Hermes MCP +## Environment Notes -Add under `mcp_servers:` in the user's Hermes config: -```yaml -touchdesigner: - command: npx - args: ["-y", "touchdesigner-mcp-server@latest"] - env: - TD_API_URL: "http://127.0.0.1:9981" - timeout: 120 -``` - -This is optional — the agent works fully via `curl` to the REST API using `execute_code`. MCP tools are a convenience layer. - -## Talking to TD (the td_exec pattern) - -All communication uses this pattern in `execute_code`: - -```python -import json, shlex -from hermes_tools import terminal - -API = "http://127.0.0.1:9981" -def td_exec(script): - payload = json.dumps({"script": script}) - cmd = f"curl -s --max-time 15 -X POST -H 'Content-Type: application/json' -d {shlex.quote(payload)} '{API}/api/td/server/exec'" - r = terminal(cmd, timeout=20) - return json.loads(r['output']) - -# Returns: {"result": , "stdout": "...", "stderr": "..."} -``` - -For large GLSL shaders: write to a temp file, then `td_exec("op('...').text = open('/tmp/shader.glsl').read()")`. +- **Non-Commercial TD** caps resolution at 1280×1280. Use `outputresolution = 'custom'` and set width/height explicitly. +- **Codecs:** `prores` (preferred on macOS) or `mjpa` as fallback. H.264/H.265/AV1 require a Commercial license. +- Always call `td_get_par_info` before setting params — names vary by TD version (see CRITICAL RULES #1). ## Workflow -### Step 0: Discovery (MANDATORY — never skip) +### Step 0: Discover (before building anything) -**Never hardcode parameter names.** They change between TD versions. Run this first: - -```python -td_exec(""" -import sys -info = {'version': str(app.version), 'platform': sys.platform} -root = op('/project1') -for name, optype in [('glslTOP', glslTOP), ('constantTOP', constantTOP), - ('blurTOP', blurTOP), ('textTOP', textTOP), - ('levelTOP', levelTOP), ('compositeTOP', compositeTOP), - ('transformTOP', transformTOP), ('feedbackTOP', feedbackTOP), - ('windowCOMP', windowCOMP)]: - n = root.create(optype, '_d_' + name) - kw = ['color','size','font','dat','alpha','opacity','resolution','text', - 'extend','operand','top','pixel','format','win','type'] - info[name] = [p.name for p in n.pars() if any(k in p.name.lower() for k in kw)] - n.destroy() -result = info -""") +``` +Call td_get_par_info with op_type for each type you plan to use. +Call td_get_hints with the topic you're building (e.g. "glsl", "audio reactive", "feedback"). +Call td_get_focus to see where the user is and what's selected. +Call td_get_network to see what already exists. ``` -Use the returned param names for ALL subsequent calls. Store them in your session context. +No temp nodes, no cleanup. This replaces the old discovery dance entirely. ### Step 1: Clean + Build -Build the entire network in ONE `td_exec` call (batching avoids round-trip overhead and ensures TD advances frames between calls): +**IMPORTANT: Split cleanup and creation into SEPARATE MCP calls.** Destroying and recreating same-named nodes in one `td_execute_python` script causes "Invalid OP object" errors. See pitfalls #10b. + +Use `td_create_operator` for each node (handles viewport positioning automatically): + +``` +td_create_operator(type="noiseTOP", parent="/project1", name="bg", parameters={"resolutionw": 1280, "resolutionh": 720}) +td_create_operator(type="levelTOP", parent="/project1", name="brightness") +td_create_operator(type="nullTOP", parent="/project1", name="out") +``` + +For bulk creation or wiring, use `td_execute_python`: ```python -td_exec(""" +# td_execute_python script: root = op('/project1') -keep = {'api_server', 'api_handler'} -for child in list(root.children): # snapshot before destroying - if child.name not in keep and child.valid: - child.destroy() - -# Create nodes, set params (using discovered names), wire, verify -... -result = {'nodes': len(list(root.children)), 'errors': [...]} -""") +nodes = [] +for name, optype in [('bg', noiseTOP), ('fx', levelTOP), ('out', nullTOP)]: + n = root.create(optype, name) + nodes.append(n.path) +# Wire chain +for i in range(len(nodes)-1): + op(nodes[i]).outputConnectors[0].connect(op(nodes[i+1]).inputConnectors[0]) +result = {'created': nodes} ``` -### Step 2: Wire connections +### Step 2: Set Parameters -```python -gl.outputConnectors[0].connect(comp.inputConnectors[0]) +Prefer the native tool (validates params, won't crash): + +``` +td_set_operator_pars(path="/project1/bg", parameters={"roughness": 0.6, "monochrome": true}) ``` -### Step 3: Verify +For expressions or modes, use `td_execute_python`: ```python -for c in list(root.children): - e = c.errors(); w = c.warnings() - if e: print(c.name, 'ERR:', e) +op('/project1/time_driver').par.colorr.expr = "absTime.seconds % 1000.0" ``` -### Step 4: Display +### Step 3: Wire + +Use `td_execute_python` — no native wire tool exists: ```python -win = root.create(windowCOMP, 'display') -win.par.winop = out.path # discovered param name +op('/project1/bg').outputConnectors[0].connect(op('/project1/fx').inputConnectors[0]) +``` + +### Step 4: Verify + +``` +td_get_errors(path="/project1", recursive=true) +td_get_perf() +td_get_operator_info(path="/project1/out", detail="full") +``` + +### Step 5: Display / Capture + +``` +td_get_screenshot(path="/project1/out") +``` + +Or open a window via script: + +```python +win = op('/project1').create(windowCOMP, 'display') +win.par.winop = op('/project1/out').path win.par.winw = 1280; win.par.winh = 720 win.par.winopen.pulse() ``` +## MCP Tool Quick Reference + +**Core (use these most):** +| Tool | What | +|------|------| +| `td_execute_python` | Run arbitrary Python in TD. Full API access. | +| `td_create_operator` | Create node with params + auto-positioning | +| `td_set_operator_pars` | Set params safely (validates, won't crash) | +| `td_get_operator_info` | Inspect one node: connections, params, errors | +| `td_get_operators_info` | Inspect multiple nodes in one call | +| `td_get_network` | See network structure at a path | +| `td_get_errors` | Find errors/warnings recursively | +| `td_get_par_info` | Get param names for an OP type (replaces discovery) | +| `td_get_hints` | Get patterns/tips before building | +| `td_get_focus` | What network is open, what's selected | + +**Read/Write:** +| Tool | What | +|------|------| +| `td_read_dat` | Read DAT text content | +| `td_write_dat` | Write/patch DAT content | +| `td_read_chop` | Read CHOP channel values | +| `td_read_textport` | Read TD console output | + +**Visual:** +| Tool | What | +|------|------| +| `td_get_screenshot` | Capture one OP viewer to file | +| `td_get_screenshots` | Capture multiple OPs at once | +| `td_get_screen_screenshot` | Capture actual screen via TD | +| `td_navigate_to` | Jump network editor to an OP | + +**Search:** +| Tool | What | +|------|------| +| `td_find_op` | Find ops by name/type across project | +| `td_search` | Search code, expressions, string params | + +**System:** +| Tool | What | +|------|------| +| `td_get_perf` | Performance profiling (FPS, slow ops) | +| `td_list_instances` | List all running TD instances | +| `td_get_docs` | In-depth docs on a TD topic | +| `td_agents_md` | Read/write per-COMP markdown docs | +| `td_reinit_extension` | Reload extension after code edit | +| `td_clear_textport` | Clear console before debug session | + +**Input Automation:** +| Tool | What | +|------|------| +| `td_input_execute` | Send mouse/keyboard to TD | +| `td_input_status` | Poll input queue status | +| `td_input_clear` | Stop input automation | +| `td_op_screen_rect` | Get screen coords of a node | +| `td_click_screen_point` | Click a point in a screenshot | + +See `references/mcp-tools.md` for full parameter schemas. + ## Key Implementation Rules -**Always clean safely:** `list(root.children)` before iterating + `child.valid` check. - -**GLSL time:** No `uTDCurrentTime` in TD 099. Feed time via 1x1 Constant TOP. -**CRITICAL: must use `rgba32float` format** — the default 8-bit format clamps values to 0-1, so `absTime.seconds % 1000.0` becomes 1.0 and the shader appears frozen: +**GLSL time:** No `uTDCurrentTime` in GLSL TOP. Use the Values page: ```python -t = root.create(constantTOP, 'time_driver') -t.par.format = 'rgba32float' # ← REQUIRED or time is stuck at 1.0 -t.par.outputresolution = 'custom' -t.par.resolutionw = 1 -t.par.resolutionh = 1 -t.par.colorr.expr = "absTime.seconds % 1000.0" -t.par.colorg.expr = "int(absTime.seconds / 1000.0)" -t.outputConnectors[0].connect(glsl.inputConnectors[0]) -# In GLSL: vec4 td = texture(sTD2DInputs[0], vec2(.5)); float t = td.r + td.g*1000.; +# Call td_get_par_info(op_type="glslTOP") first to confirm param names +td_set_operator_pars(path="/project1/shader", parameters={"value0name": "uTime"}) +# Then set expression via script: +# op('/project1/shader').par.value0.expr = "absTime.seconds" +# In GLSL: uniform float uTime; ``` -**Feedback TOP:** Use `top` parameter reference (not direct input wire). The "Not enough sources" error resolves after first cook. The "Cook dependency loop" warning is expected. +Fallback: Constant TOP in `rgba32float` format (8-bit clamps to 0-1, freezing the shader). + +**Feedback TOP:** Use `top` parameter reference, not direct input wire. "Not enough sources" resolves after first cook. "Cook dependency loop" warning is expected. **Resolution:** Non-Commercial caps at 1280×1280. Use `outputresolution = 'custom'`. -**Large shaders:** Write GLSL to `/tmp/file.glsl`, then `td_exec("op('shader').text = open('/tmp/file.glsl').read()")`. +**Large shaders:** Write GLSL to `/tmp/file.glsl`, then use `td_write_dat` or `td_execute_python` to load. -**WebServer DAT quirk:** Response body goes in `response['data']` not `response['body']`. Request POST body comes as bytes in `request['data']`. +**Vertex/Point access (TD 2025.32):** `point.P[0]`, `point.P[1]`, `point.P[2]` — NOT `.x`, `.y`, `.z`. + +**Extensions:** `ext0object` format is `"op('./datName').module.ClassName(me)"` in CONSTANT mode. After editing extension code with `td_write_dat`, call `td_reinit_extension`. + +**Script callbacks:** ALWAYS use relative paths via `me.parent()` / `scriptOp.parent()`. + +**Cleaning nodes:** Always `list(root.children)` before iterating + `child.valid` check. ## Recording / Exporting Video -To capture TD output as video or image sequence for external use (e.g., ASCII video pipeline): - -### Movie Recording (recommended) - ```python -# Put a Null TOP before the recorder (official best practice) +# via td_execute_python: +root = op('/project1') rec = root.create(moviefileoutTOP, 'recorder') -null_out.outputConnectors[0].connect(rec.inputConnectors[0]) - +op('/project1/out').outputConnectors[0].connect(rec.inputConnectors[0]) rec.par.type = 'movie' rec.par.file = '/tmp/output.mov' -rec.par.videocodec = 'mjpa' # Motion JPEG — works on Non-Commercial - -# Start/stop recording (par.record is a toggle, NOT .record() method) +rec.par.videocodec = 'prores' # Apple ProRes — NOT license-restricted on macOS rec.par.record = True # start -# ... wait ... -rec.par.record = False # stop +# rec.par.record = False # stop (call separately later) ``` -**H.264/H.265 require a Commercial license** — use `mjpa` (Motion JPEG) or `prores` on Non-Commercial. Extract frames afterward with ffmpeg if needed: -```bash -ffmpeg -i /tmp/output.mov -vframes 120 /tmp/frames/frame_%06d.png -``` +H.264/H.265/AV1 need Commercial license. Use `prores` on macOS or `mjpa` as fallback. +Extract frames: `ffmpeg -i /tmp/output.mov -vframes 120 /tmp/frames/frame_%06d.png` -### Image Sequence Export +**TOP.save() is useless for animation** — captures same GPU texture every time. Always use MovieFileOut. -```python -rec.par.type = 'imagesequence' -rec.par.imagefiletype = 'png' -rec.par.file.expr = "'/tmp/frames/out' + me.fileSuffix" # fileSuffix is REQUIRED -rec.par.record = True -``` +### Before Recording: Checklist -### Pitfalls - -- **Race condition:** When setting `par.file` and starting recording in the same script, use `run("...", delayFrames=2)` so the file path is applied before recording begins. -- **TOP.save() is useless for animation:** Calling `op('null1').save(path)` in a loop or rapid API calls captures the same GPU texture every time — TD doesn't cook new frames between save calls. Always use MovieFileOut for animated output. -- See `references/pitfalls.md` #25-27 for full details. +1. **Verify FPS > 0** via `td_get_perf`. If FPS=0 the recording will be empty. See pitfalls #37-38. +2. **Verify shader output is not black** via `td_get_screenshot`. Black output = shader error or missing input. See pitfalls #7, #39. +3. **If recording with audio:** cue audio to start first, then delay recording by 3 frames. See pitfalls #18. +4. **Set output path before starting record** — setting both in the same script can race. ## Audio-Reactive GLSL (Proven Recipe) -Complete chain for music-driven visuals: AudioFileIn → AudioSpectrum → Math (boost) → Resample (256) → CHOP To TOP → GLSL TOP (spectrum sampled per-pixel). See `references/network-patterns.md` Pattern 3b for the full working recipe with shader code. +### Correct signal chain (tested April 2026) -## Audio-Reactive Visuals - -The most powerful TD workflow for the agent: play an audio file, analyze its spectrum, and drive a GLSL shader in real-time. The agent builds the entire signal chain programmatically. - -**Signal chain:** ``` -AudioFileIn CHOP → AudioSpectrum CHOP → Math CHOP (gain=5) - → Resample CHOP (256) → CHOP To TOP (spectrum texture) - ↓ (GLSL input 1) - Constant TOP (rgba32float, time) → GLSL TOP → Null TOP → MovieFileOut - (input 0) +AudioFileIn CHOP (playmode=sequential) + → AudioSpectrum CHOP (FFT=512, outputmenu=setmanually, outlength=256, timeslice=ON) + → Math CHOP (gain=10) + → CHOP to TOP (dataformat=r, layout=rowscropped) + → GLSL TOP input 1 (spectrum texture, 256x2) + +Constant TOP (rgba32float, time) → GLSL TOP input 0 +GLSL TOP → Null TOP → MovieFileOut ``` -**Key technique:** The spectrum becomes a 256×1 texture. In GLSL, `texture(sTD2DInputs[1], vec2(x, 0.0)).r` samples frequency at position x (0=bass, 1=treble). This lets the shader react per-pixel to different frequency bands. +### Critical audio-reactive rules (empirically verified) -**Smoothing is critical:** Raw FFT jitters. Use `Math CHOP` gain to boost weak signal, then the GLSL shader's own temporal integration (via feedback or time-smoothed params) handles visual smoothing. +1. **TimeSlice must stay ON** for AudioSpectrum. OFF = processes entire audio file → 24000+ samples → CHOP to TOP overflow. +2. **Set Output Length manually** to 256 via `outputmenu='setmanually'` and `outlength=256`. Default outputs 22050 samples. +3. **DO NOT use Lag CHOP for spectrum smoothing.** Lag CHOP operates in timeslice mode and expands 256 samples to 2400+, averaging all values to near-zero (~1e-06). The shader receives no usable data. This was the #1 audio sync failure in testing. +4. **DO NOT use Filter CHOP either** — same timeslice expansion problem with spectrum data. +5. **Smoothing belongs in the GLSL shader** if needed, via temporal lerp with a feedback texture: `mix(prevValue, newValue, 0.3)`. This gives frame-perfect sync with zero pipeline latency. +6. **CHOP to TOP dataformat = 'r'**, layout = 'rowscropped'. Spectrum output is 256x2 (stereo). Sample at y=0.25 for first channel. +7. **Math gain = 10** (not 5). Raw spectrum values are ~0.19 in bass range. Gain of 10 gives usable ~5.0 for the shader. +8. **No Resample CHOP needed.** Control output size via AudioSpectrum's `outlength` param directly. -See `references/network-patterns.md` Pattern 9b for the complete build script + shader code. +### GLSL spectrum sampling + +```glsl +// Input 0 = time (1x1 rgba32float), Input 1 = spectrum (256x2) +float iTime = texture(sTD2DInputs[0], vec2(0.5)).r; + +// Sample multiple points per band and average for stability: +// NOTE: y=0.25 for first channel (stereo texture is 256x2, first row center is 0.25) +float bass = (texture(sTD2DInputs[1], vec2(0.02, 0.25)).r + + texture(sTD2DInputs[1], vec2(0.05, 0.25)).r) / 2.0; +float mid = (texture(sTD2DInputs[1], vec2(0.2, 0.25)).r + + texture(sTD2DInputs[1], vec2(0.35, 0.25)).r) / 2.0; +float hi = (texture(sTD2DInputs[1], vec2(0.6, 0.25)).r + + texture(sTD2DInputs[1], vec2(0.8, 0.25)).r) / 2.0; +``` + +See `references/network-patterns.md` for complete build scripts + shader code. ## Operator Quick Reference -| Family | Color | Examples | Suffix | -|--------|-------|----------|--------| -| TOP | Purple | noiseTop, glslTop, compositeTop, levelTop, blurTop, textTop, nullTop, feedbackTop, renderTop | TOP | -| CHOP | Green | audiofileinChop, audiospectrumChop, mathChop, lfoChop, constantChop | CHOP | -| SOP | Blue | gridSop, sphereSop, transformSop, noiseSop | SOP | -| DAT | White | textDat, tableDat, scriptDat, webserverDAT | DAT | -| MAT | Yellow | phongMat, pbrMat, glslMat, constMat | MAT | -| COMP | Gray | geometryComp, containerComp, cameraComp, lightComp, windowCOMP | COMP | +| Family | Color | Python class / MCP type | Suffix | +|--------|-------|-------------|--------| +| TOP | Purple | noiseTOP, glslTOP, compositeTOP, levelTop, blurTOP, textTOP, nullTOP | TOP | +| CHOP | Green | audiofileinCHOP, audiospectrumCHOP, mathCHOP, lfoCHOP, constantCHOP | CHOP | +| SOP | Blue | gridSOP, sphereSOP, transformSOP, noiseSOP | SOP | +| DAT | White | textDAT, tableDAT, scriptDAT, webserverDAT | DAT | +| MAT | Yellow | phongMAT, pbrMAT, glslMAT, constMAT | MAT | +| COMP | Gray | geometryCOMP, containerCOMP, cameraCOMP, lightCOMP, windowCOMP | COMP | -See `references/operators.md` for full catalog. See `references/network-patterns.md` for recipes. +## Security Notes + +- MCP runs on localhost only (port 40404). No authentication — any local process can send commands. +- `td_execute_python` has unrestricted access to the TD Python environment and filesystem as the TD process user. +- `setup.sh` downloads twozero.tox from the official 404zero.com URL. Verify the download if concerned. +- The skill never sends data outside localhost. All MCP communication is local. ## References | File | What | |------|------| -| `references/pitfalls.md` | **READ FIRST** — 31 hard-won lessons from real sessions | +| `references/pitfalls.md` | Hard-won lessons from real sessions | | `references/operators.md` | All operator families with params and use cases | -| `references/network-patterns.md` | Recipes: audio-reactive, generative, video, GLSL, instancing | -| `references/mcp-tools.md` | MCP tool schemas (optional — curl works without MCP) | +| `references/network-patterns.md` | Recipes: audio-reactive, generative, GLSL, instancing | +| `references/mcp-tools.md` | Full twozero MCP tool parameter schemas | | `references/python-api.md` | TD Python: op(), scripting, extensions | -| `references/troubleshooting.md` | Connection diagnostics, param debugging, performance | -| `scripts/custom_api_handler.py` | Self-contained REST API handler for TD WebServer DAT | +| `references/troubleshooting.md` | Connection diagnostics, debugging | +| `scripts/setup.sh` | Automated setup script | + +--- + +> You're not writing code. You're conducting light. diff --git a/skills/creative/touchdesigner/references/mcp-tools.md b/skills/creative/touchdesigner/references/mcp-tools.md index 5e4ad98d5..ec90076cb 100644 --- a/skills/creative/touchdesigner/references/mcp-tools.md +++ b/skills/creative/touchdesigner/references/mcp-tools.md @@ -1,501 +1,382 @@ -# TouchDesigner MCP Tools Reference +# twozero MCP Tools Reference -Complete parameter schemas and usage examples for all 13 MCP tools from the 8beeeaaat/touchdesigner-mcp server. +36 tools from twozero MCP v2.774+ (April 2026). +All tools accept an optional `target_instance` param for multi-TD-instance scenarios. -## Hermes Configuration +## Execution & Scripting -Add a `touchdesigner` entry under the `mcp_servers` section of your Hermes config. Example YAML block: +### td_execute_python -```yaml -# Under mcp_servers: in config.yaml -mcp_servers: - touchdesigner: - command: npx - args: ["-y", "touchdesigner-mcp-server@latest"] - env: - TD_API_URL: "http://127.0.0.1:9981" - timeout: 120 - connect_timeout: 60 -``` +Execute Python code inside TouchDesigner and return the result. Has full access to TD Python API (op, project, app, etc). Print statements and the last expression value are captured. Best for: wiring connections (inputConnectors), setting expressions (par.X.expr/mode), querying parameter names, and batch creation scripts (5+ operators). For creating 1-4 operators, prefer td_create_operator instead. -For a locally built server, point `command` to `node` and `args` to the built server index.js path. Set `TD_API_URL` to the TouchDesigner WebServer DAT address (default port 9981). +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `code` | string | yes | Python code to execute in TouchDesigner | -For the documentation/knowledge server (no running TD needed), add a `td_docs` entry using `touchdesigner-mcp-server` as the npx package. +## Network & Structure -Tools are registered as `mcp_touchdesigner_` in Hermes. +### td_get_network -**If MCP tools are not available as direct function calls** (common when the MCP server connects but Hermes doesn't expose them as callable tools), use the custom API handler directly via `curl` in `execute_code` or `terminal`: +Get the operator network structure in TouchDesigner (TD) at a given path. Returns compact list: name OPType flags. First line is full path of queried op. Flags: ch:N=children count, !cook=allowCooking off, bypass, private=isPrivate, blocked:reason, "comment text". depth=0 (default) = current level only. depth=1 = one level of children (indented). To explore deeper, call again on a specific COMP path. System operators (/ui, /sys) are hidden by default. -```python -import json, shlex -from hermes_tools import terminal +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | no | Network path to inspect, e.g. '/' or '/project1' | +| `depth` | integer | no | How many levels deep to recurse. 0=current level only (recommended), 1=include direct children of COMPs | +| `includeSystem` | boolean | no | Include system operators (/ui, /sys). Default false. | +| `nodeXY` | boolean | no | Include nodeX,nodeY coordinates. Default false. | -def td_exec(script): - """Execute Python in TouchDesigner via the REST API.""" - escaped = json.dumps({"script": script}) - cmd = f"curl -s --max-time 15 -X POST -H 'Content-Type: application/json' -d {shlex.quote(escaped)} 'http://127.0.0.1:9981/api/td/server/exec'" - r = terminal(cmd, timeout=20) - return json.loads(r['output']) +### td_create_operator -# Example: list all nodes -result = td_exec('result = [c.name for c in op("/project1").children]') -print(result) # {"result": ["node1", "node2", ...], "stdout": "", "stderr": ""} -``` +Create a new operator (node) in TouchDesigner (TD). Preferred way to create operators — handles viewport positioning, viewer flag, and docked ops automatically. For batch creation (5+ ops), you may use td_execute_python with a script instead, but then call td_get_hints('construction') first for correct parameter names and layout rules. Supports all TD operator types: TOP, CHOP, SOP, DAT, COMP, MAT. If parent is omitted, creates in the currently open network at the user's viewport position. When building a container: first create baseCOMP (no parent), then create children with parent=compPath. -This `td_exec` helper works with both the official .tox handler and the custom API handler from `scripts/custom_api_handler.py`. +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `type` | string | yes | Operator type, e.g. 'textDAT', 'constantCHOP', 'noiseTOP', 'transformTOP', 'baseCOMP' | +| `parent` | string | no | Path to the parent operator. If omitted, uses the currently open network in TD. | +| `name` | string | no | Name for the new operator (optional, TD auto-names if omitted) | +| `parameters` | object | no | Key-value pairs of parameters to set on the created operator | -Tools are registered as `mcp_touchdesigner_` in Hermes. +### td_find_op -## Common Formatting Parameters +Find operators by name and/or type across the project. Returns TSV: path, OPType, flags. Flags: bypass, !cook, private, blocked:reason. Use td_search to search inside code/expressions; use td_find_op to find operators themselves. -Most tools accept these optional formatting parameters: +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | string | no | Substring to match in operator name (case-insensitive). E.g. 'noise' finds noise1, noise2, myNoise. | +| `type` | string | no | Substring to match in OPType (case-insensitive). E.g. 'noiseTOP', 'baseCOMP', 'CHOP'. Use exact type for precision or partial for broader matches. | +| `root` | string | no | Root operator path to search from. Default '/project1'. | +| `max_results` | number | no | Maximum results to return. Default 50. | +| `max_depth` | number | no | Max recursion depth from root. Default unlimited. | +| `detail` | `basic` / `summary` | no | Result detail level. 'basic' = name/path/type (fast). 'summary' = + connections, non-default pars, expressions. Default 'basic'. | -| Parameter | Type | Values | Description | -|-----------|------|--------|-------------| -| `detailLevel` | string | `"minimal"`, `"summary"`, `"detailed"` | Response verbosity | -| `responseFormat` | string | `"json"`, `"yaml"`, `"markdown"` | Output format | -| `limit` | integer | 1-500 | Max items (on list-type tools only) | +### td_search -These are client-side formatting — they control how the MCP server formats the response text, not what data TD returns. +Search for text across all code (DAT scripts), parameter expressions, and string parameter values in the TD project. Returns TSV: path, kind (code/expression/parameter/ref), line, text. JSON when context>0. Words are OR-matched. Use quotes for exact phrases: 'GetLogin "op('login')"'. Use count_only=true to quickly check if something is referenced without fetching full results. ---- +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `query` | string | yes | Search query. Multiple words = OR (any match). Wrap in quotes for exact phrase. Example: 'GetLogin getLogin' finds either. | +| `root` | string | no | Root operator path to search from. Default '/project1'. | +| `scope` | `all` / `code` / `editable` / `expressions` / `parameters` | no | What to search. 'code' = DAT scripts only (fast, ~0.05s). 'editable' = only editable code (skips inherited/ref DATs). 'expressions' = parameter expressions only. 'parameters' = string parameter values only. 'all' = everything (slow, ~1.5s due to parameter scan). Default 'all'. | +| `case_sensitive` | boolean | no | Case-sensitive matching. Default false. | +| `max_results` | number | no | Maximum results to return. Default 50. | +| `context` | number | no | Lines to show before/after each code match. Saves td_read_dat calls. Default 0. | +| `count_only` | boolean | no | Return only match count, not results. Fast existence check. | +| `max_depth` | number | no | Max recursion depth from root. Default unlimited. | -## Tool 1: describe_td_tools +### td_navigate_to -**Purpose:** Meta-tool — lists all available TouchDesigner MCP tools with descriptions and parameters. +Navigate the TouchDesigner Network Editor viewport to show a specific operator. Opens the operator's parent network and centers the view on it. Use this to show the user where a problem is, or to navigate to an operator before modifying it. -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `filter` | string | No | Keyword to filter tools by name, description, or parameter | -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | yes | Path to the operator to navigate to, e.g. '/project1/noise1' | -**Example:** Find tools related to node creation -``` -describe_td_tools(filter="create") -``` +## Operator Inspection -**Note:** This tool runs entirely in the MCP server — it does NOT contact TouchDesigner. Use it to discover what's available. +### td_get_operator_info ---- +Get information about a specific operator (node) in TouchDesigner (TD). detail='summary': connections, non-default pars, expressions, CHOP channels (compact). detail='full': all of the above PLUS every parameter with value/default/label. -## Tool 2: get_td_info +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | yes | Full path to the operator, e.g. '/project1/noise1' | +| `detail` | `summary` / `full` | no | Level of detail. 'summary' = connections, expressions, non-default pars, custom pars (pulse marked), CHOP channels. 'full' = summary + all parameters. Default 'full'. | -**Purpose:** Get TouchDesigner server information (version, OS, build). +### td_get_operators_info -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | +Get information about multiple operators in one call. Returns an array of operator info objects. Use instead of calling td_get_operator_info multiple times. -**Example:** Check TD is running and get version -``` -get_td_info() -``` +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `paths` | array | yes | Array of full operator paths, e.g. ['/project1/null1', '/project1/null2'] | +| `detail` | `summary` / `full` | no | Level of detail. Default 'summary'. | -**Returns:** TD version, build number, OS name/version, MCP API version. +### td_get_par_info -**Use this first** to verify the connection is working before building networks. +Get parameter names and details for a TouchDesigner operator type. Without specific pars: returns compact list of all parameters with their names, types, and menu options. With pars: returns full details (help text, menu values, style) for specific parameters. Use this when you need to know exact parameter names before setting them. ---- +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `op_type` | string | yes | TD operator type name, e.g. 'noiseTOP', 'blurTOP', 'lfoCHOP', 'compositeTOP' | +| `pars` | array | no | Optional list of specific parameter names to get full details for | -## Tool 3: execute_python_script +## Parameter Setting -**Purpose:** Execute arbitrary Python code inside TouchDesigner's Python environment. +### td_set_operator_pars -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `script` | string | **Yes** | Python code to execute | -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | +Set parameters and flags on an operator in TouchDesigner (TD). Safer than td_execute_python for simple parameter changes. Can set values, toggle bypass/viewer, without writing Python code. -**Available globals in the script:** -- `op` — find operators by path -- `ops` — find multiple operators by pattern -- `me` — the WebServer DAT running the script -- `parent` — me.parent() -- `project` — root project component -- `td` — the full td module -- `result` — set this to explicitly return a value +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | yes | Path to the operator | +| `parameters` | object | no | Key-value pairs of parameters to set | +| `bypass` | boolean | no | Set bypass state of the operator (not available on COMPs) | +| `viewer` | boolean | no | Set viewer state of the operator | +| `allowCooking` | boolean | no | Set cooking flag on a COMP. When False, internal network stops cooking (0 CPU). COMP-only. | -**Execution behavior:** -- Single-line scripts: tries `eval()` first (returns value), falls back to `exec()` -- Multi-line scripts: uses `exec()` always -- stdout/stderr are captured and returned separately -- If `result` is not set, tries to evaluate the last expression as the return value +## Data Read/Write -**Examples:** +### td_read_dat -```python -# Simple query -execute_python_script(script="op('/project1/noise1').par.seed.val") -# Returns: {"result": 42, "stdout": "", "stderr": ""} +Read the text content of a DAT operator in TouchDesigner (TD). Returns content with line numbers. Use to read scripts, extensions, GLSL shaders, table data. -# Multi-line script -execute_python_script(script=""" -nodes = op('/project1').findChildren(type=TOP) -result = [{'name': n.name, 'type': n.OPType} for n in nodes] -""") +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | yes | Path to the DAT operator | +| `start_line` | integer | no | Start line (1-based). Omit to read from beginning. | +| `end_line` | integer | no | End line (inclusive). Omit to read to end. | -# Connect two operators -execute_python_script(script="op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1'))") +### td_write_dat -# Create and configure in one script -execute_python_script(script=""" -parent = op('/project1') -n = parent.create(noiseTop, 'my_noise') -n.par.seed.val = 42 -n.par.monochrome.val = True -n.par.resolutionw.val = 1920 -n.par.resolutionh.val = 1080 -result = {'path': n.path, 'type': n.OPType} -""") +Write or patch text content of a DAT operator in TouchDesigner (TD). Can do full replacement or StrReplace-style patching (old_text -> new_text). Use for editing scripts, extensions, shaders. Does NOT reinit extensions automatically. -# Batch wire a chain -execute_python_script(script=""" -chain = ['noise1', 'level1', 'blur1', 'composite1', 'null_out'] -for i in range(len(chain) - 1): - src = op(f'/project1/{chain[i]}') - dst = op(f'/project1/{chain[i+1]}') - if src and dst: - src.outputConnectors[0].connect(dst) -result = 'Wired chain: ' + ' -> '.join(chain) -""") -``` +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | yes | Path to the DAT operator | +| `text` | string | no | Full replacement text. Use this OR old_text+new_text, not both. | +| `old_text` | string | no | Text to find and replace (must be unique in the DAT) | +| `new_text` | string | no | Replacement text | +| `replace_all` | boolean | no | If true, replaces ALL occurrences of old_text (default: false, requires unique match) | -**When to use:** Wiring connections, complex logic, batch operations, querying state that other tools don't cover. This is the most powerful and flexible tool. +### td_read_chop ---- +Read CHOP channel sample data. Returns channel values as arrays. Use when you need the actual sample values (animation curves, lookup tables, waveforms), not just the summary from td_get_operator_info. -## Tool 4: create_td_node +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | yes | Path to the CHOP operator | +| `channels` | array | no | Channel names to read. Omit to read all channels. | +| `start` | integer | no | Start sample index (0-based). Omit to read from beginning. | +| `end` | integer | no | End sample index (inclusive). Omit to read to end. | -**Purpose:** Create a new operator in TouchDesigner. +### td_read_textport -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `parentPath` | string | **Yes** | Path to parent (e.g., `/project1`) | -| `nodeType` | string | **Yes** | Operator type (e.g., `noiseTop`, `mathChop`) | -| `nodeName` | string | No | Custom name (auto-generated if omitted) | -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | +Read the last N lines from the TouchDesigner (TD) log/textport (console output). Use this to see errors, warnings and print output from TD. -**Examples:** +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `lines` | integer | no | Number of recent lines to return | -``` -create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="bg_noise") -create_td_node(parentPath="/project1", nodeType="compositeTop") # auto-named -create_td_node(parentPath="/project1/audio_chain", nodeType="audiospectrumChop", nodeName="spectrum") -``` +### td_clear_textport -**Returns:** Node summary with id, name, path, opType, and all default parameter values. +Clear the MCP textport log buffer. Use this before starting a debug session or an edit-run-check loop to keep td_read_textport output focused and minimal. -**Node type naming convention:** camelCase family suffix — `noiseTop`, `mathChop`, `gridSop`, `tableDat`, `phongMat`, `geometryComp`. See `references/operators.md` for the full list. +No parameters (other than optional `target_instance`). ---- +## Visual Capture -## Tool 5: delete_td_node +### td_get_screenshot -**Purpose:** Delete an existing operator. +Get a screenshot of an operator's viewer in TouchDesigner (TD). Saves the image to a file and returns the file path. Use your file-reading tool to view the image. Shows what the operator looks like in its viewer (TOP output, CHOP waveform graph, SOP geometry, DAT table, parameter UI, etc). Use this to visually inspect any operator, or to generate images via TD for use in your project. TWO-STEP ASYNC USAGE: Step 1 — call with 'path' to start: returns {'status': 'pending', 'requestId': '...'}. Step 2 — call with 'request_id' to retrieve: returns {'file': '/tmp/.../opname_id.jpg'}. Then read the file to see the image. If step 2 still returns pending, make one other tool call then retry. -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `nodePath` | string | **Yes** | Absolute path to node (e.g., `/project1/noise1`) | -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | no | Full operator path to screenshot, e.g. '/project1/noise1'. Required for step 1. | +| `request_id` | string | no | Request ID from step 1 to retrieve the completed screenshot. | +| `max_size` | integer | no | Max pixel size for the longer side (default 512). Use 0 for original operator resolution (useful for pixel-accurate UI work). Higher values (e.g. 1024) for more detail. | +| `output_path` | string | no | Optional absolute path where the image should be saved (e.g. '/Users/me/project/render.png'). If omitted, saved to /tmp/pisang_mcp/screenshots/. Use absolute paths — TD's working directory may differ from the agent's. | +| `as_top` | boolean | no | If true, captures the operator directly as a TOP (bypasses the viewer renderer), preserving alpha/transparency. Only works for TOP operators — if the target is not a TOP, falls back to the viewer automatically. Use this when you need a clean PNG with alpha, e.g. to save a generated image for use in another project. | +| `format` | `auto` / `jpg` / `png` | no | Image format. 'auto' (default): JPEG for viewer mode, PNG for as_top=true. 'jpg': always JPEG (smaller). 'png': always PNG (lossless). | -**Example:** +### td_get_screenshots -``` -delete_td_node(nodePath="/project1/noise1") -``` +Get screenshots of multiple operators in one batch. Saves images to files and returns file paths. Use your file-reading tool to view images. TWO-STEP ASYNC USAGE: Step 1 — call with 'paths' array to start: returns {'status': 'pending', 'batchId': '...', 'total': N}. Step 2 — call with 'batch_id' to retrieve: returns {'files': [{op, file}, ...]}. Then read the files to see the images. If still processing returns {'status': 'pending', 'ready': K, 'total': N}. -**Returns:** Confirmation with the deleted node's summary (captured before deletion). +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `paths` | array | no | List of full operator paths to screenshot. Required for step 1. | +| `batch_id` | string | no | Batch ID from step 1 to retrieve completed screenshots. | +| `max_size` | integer | no | Max pixel size for longer side (default 512). Use 0 for original resolution. | +| `as_top` | boolean | no | If true, captures TOP operators directly (preserves alpha). Non-TOP operators fall back to viewer. | +| `output_dir` | string | no | Optional absolute path to a directory. Each screenshot saved as .jpg or .png inside it and kept on disk. | +| `format` | `auto` / `jpg` / `png` | no | Image format. 'auto' (default): JPEG for viewer mode, PNG for as_top=true. 'jpg': always JPEG (smaller). 'png': always PNG (lossless). | ---- +### td_get_screen_screenshot -## Tool 6: get_td_nodes +Capture a screenshot of the actual screen via TD's screenGrabTOP. Saves the image to a file and returns the file path. Use your file-reading tool to view the image. Unlike td_get_screenshot (operator viewer), this shows what the user literally sees on their monitor — TD windows, UI panels, everything. Use when simulating mouse/keyboard input to verify what happened on screen. Workflow: td_get_screen_screenshot → read file → td_input_execute → wait idle → td_get_screen_screenshot again. TWO-STEP ASYNC: Step 1 — call without request_id: returns {'status':'pending','requestId':'...'}. Step 2 — call with request_id: returns {'file': '/tmp/.../screen_id.jpg', 'info': '...metadata...'}. Then read the file to see the image. The requestId also stays usable with td_screen_point_to_global for later coordinate lookup. crop_x/y/w/h are in ACTUAL SCREEN PIXELS (not image pixels). Crops exceeding screen bounds are auto-clamped. SMART DEFAULTS: max_size is auto when omitted — 1920 for full screen (good overview), max(crop_w,crop_h) for cropped (guarantees 1:1 scale). At 1:1 scale: screen_coord = crop_origin + image_pixel. Otherwise use the formula from metadata. -**Purpose:** List operators under a path with optional filtering. +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `request_id` | string | no | Request ID from step 1 to retrieve the completed screenshot. | +| `max_size` | integer | no | Max pixel size for the longer side. Auto when omitted: 1920 for full screen, max(crop_w,crop_h) for cropped (1:1). Set explicitly to override. | +| `crop_x` | integer | no | Left edge in screen pixels. | +| `crop_y` | integer | no | Top edge in screen pixels (y=0 at top of screen). | +| `crop_w` | integer | no | Width in pixels. | +| `crop_h` | integer | no | Height in pixels. | +| `display` | integer | no | Screen index (default 0 = primary display). | -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `parentPath` | string | **Yes** | Parent path (e.g., `/project1`) | -| `pattern` | string | No | Glob pattern for name filtering (default: `*`) | -| `includeProperties` | boolean | No | Include full parameter values (default: false) | -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | -| `limit` | integer | No | Max items (1-500) | +## Context & Focus -**Examples:** +### td_get_focus -``` -# List all direct children of /project1 -get_td_nodes(parentPath="/project1") +Get the current user focus in TouchDesigner (TD): which network is open, selected operators, current operator, and rollover (what is under the mouse cursor). IMPORTANT: when the user says 'this operator' or 'вот этот', they mean the SELECTED/CURRENT operator, NOT the rollover. Rollover is just incidental mouse position and should be ignored for intent. Pass screenshots=true to immediately start a screenshot batch for all selected operators — response includes a 'screenshots' field with batchId; retrieve with td_get_screenshots(batch_id=...). -# Find all noise operators -get_td_nodes(parentPath="/project1", pattern="noise*") +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `screenshots` | boolean | no | If true, start a screenshot batch for all selected operators. Retrieve with td_get_screenshots(batch_id=...). | +| `max_size` | integer | no | Max screenshot size when screenshots=true (default 512). | +| `as_top` | boolean | no | Passed to the screenshot batch when screenshots=true. | -# Get full parameter details -get_td_nodes(parentPath="/project1", pattern="*", includeProperties=true, limit=20) -``` +### td_get_errors -**Returns:** List of node summaries. With `includeProperties=false` (default): id, name, path, opType only. With `includeProperties=true`: full parameter values included. +Find errors and warnings in TouchDesigner (TD) operators. Checks operator errors, warnings, AND broken parameter expressions (missing channels, bad references, etc). Also includes recent script errors from the log (tracebacks), grouped and deduplicated — e.g. 1000 identical mouse-move errors shown as ×1000 with one entry. If path is given, checks that operator and its children. If no path, checks the currently open network. Use '/' for entire project. Use when user says something is broken, has errors, red nodes, горит ошибка, etc. TIP: call td_clear_textport before reproducing an error to keep log focused. TIP: combine with td_get_perf when user says 'тупит/лагает' to check both errors and performance. ---- +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | no | Path to check. If omitted, checks the current network. Use '/' to scan entire project. | +| `recursive` | boolean | no | Check children recursively (default true) | +| `include_log` | boolean | no | Include recent script errors from log, grouped by unique signature (default true). Use td_clear_textport before reproducing an error to keep results focused. | -## Tool 7: get_td_node_parameters +### td_get_perf -**Purpose:** Get detailed parameters of a specific node. +Get performance data from TouchDesigner (TD). Returns TSV: header with fps/budget/memory summary, then slowest operators sorted by cook time. Columns: path, OPType, cpu/cook(ms), gpu/cook(ms), cpu/s, gpu/s, rate, flags. Use when user reports lag, low FPS, slow performance, тупит, тормозит. -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `nodePath` | string | **Yes** | Node path (e.g., `/project1/noise1`) | -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | -| `limit` | integer | No | Max parameters (1-500) | +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | no | Path to profile. If omitted, profiles the current network. Use '/' for entire project. | +| `top` | integer | no | Number of slowest operators to return | -**Example:** +## Documentation -``` -get_td_node_parameters(nodePath="/project1/noise1") -``` +### td_get_docs -**Returns:** All parameter name-value pairs for the node. Use this to discover available parameters before calling update_td_node_parameters. +Get comprehensive documentation on a TouchDesigner topic. Unlike td_get_hints (compact tips), this returns in-depth reference material. Call without arguments to see available topics with descriptions. Call with a topic name to get the full documentation. ---- +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `topic` | string | no | Topic to get docs for. Omit to list available topics. | -## Tool 8: get_td_node_errors +### td_get_hints -**Purpose:** Check for errors on a node and all its descendants (recursive). +Get TouchDesigner tips and common patterns for a topic. Call this BEFORE creating operators or writing TD Python code to learn correct parameter names, expressions, and idiomatic approaches. Available topics: animation, noise, connections, parameters, scripting, construction, ui_analysis, panel_layout, screenshots, input_simulation, undo. IMPORTANT: always call with topic='construction' before building multi-operator setups to get correct TOP/CHOP parameter names, compositeTOP input ordering, and layout guidelines. IMPORTANT: always call with topic='input_simulation' before using td_input_execute to learn focus recovery, coordinate systems, and testing workflow. -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `nodePath` | string | **Yes** | Absolute path to inspect (e.g., `/project1`) | -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | -| `limit` | integer | No | Max error items (1-500) | +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `topic` | string | yes | Topic to get hints for. Available: 'animation', 'noise', 'connections', 'parameters', 'scripting', 'construction', 'ui_analysis', 'panel_layout', 'screenshots', 'input_simulation', 'undo', 'networking', 'all' | -**Examples:** +### td_agents_md -``` -# Check entire project for errors -get_td_node_errors(nodePath="/project1") +Read, write, or update the agents_md documentation inside a COMP container. agents_md is a Markdown textDAT describing the container's purpose, structure, and conventions. action='read': returns content + staleness check (compares documented children vs live state). action='update': refreshes auto-generated sections (children list, connections) from live state, preserves human-written sections. action='write': sets full content, creates the DAT if missing. -# Check a specific chain -get_td_node_errors(nodePath="/project1/audio_chain") -``` +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | yes | Path to the COMP container | +| `action` | `read` / `update` / `write` | yes | read=get content+staleness, update=refresh auto sections, write=set content | +| `content` | string | no | Markdown content (only for action='write') | -**Returns:** Error count, hasErrors boolean, and list of errors each with nodePath, nodeName, opType, and error message. +## Input Automation -**Always call this after building a network** to catch wiring mistakes, missing references, and configuration errors. +### td_input_execute ---- +Send a sequence of mouse/keyboard commands to TouchDesigner. Commands execute sequentially with smooth bezier movement. Returns immediately — poll td_input_status() until status='idle' before proceeding. Command types: 'focus' — bring TD to foreground. 'move' — smooth mouse move: {type,x,y,duration,easing}. 'click' — click: {type,x,y,button,hold,duration,easing}. hold=seconds to hold down. duration=smooth move before click. 'dblclick' — double click: {type,x,y,duration}. 'mousedown'/'mouseup' — {type,x,y,button}. 'key' — keystroke: {type,keys} e.g. 'ctrl+z','tab','escape','shift+f5'. Requires Accessibility permission on Mac. 'type' — human-like typing: {type,text,wpm,variance} — layout-independent Unicode, variable timing. 'wait' — pause: {type,duration}. 'scroll' — {type,x,y,dx,dy,steps} — human-like scroll: moves mouse to (x,y) first, then sends dy (vertical, +up) and dx (horizontal, +right) as multiple ticks with natural timing. steps=4 by default. Mouse commands may include coord_space='logical' (default) or coord_space='physical'. On macOS, 'physical' means actual screen pixels from td_get_screen_screenshot and is converted to CGEvent logical coords automatically. Top-level coord_space applies to commands that do not override it. on_error: 'stop' (default) clears queue on error; 'continue' skips failed command. IMPORTANT: call td_get_hints('input_simulation') before first use to learn focus recovery, coordinate systems, and testing workflow. -## Tool 9: update_td_node_parameters +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `commands` | array | yes | List of command dicts to execute in sequence. | +| `coord_space` | `logical` / `physical` | no | Default coordinate space for mouse commands that do not specify their own coord_space. 'logical' uses CGEvent coords directly. 'physical' uses actual screen pixels from td_get_screen_screenshot and is auto-converted on macOS. | +| `on_error` | `stop` / `continue` | no | What to do on error. Default 'stop'. | -**Purpose:** Update parameters on an existing node. +### td_input_status -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `nodePath` | string | **Yes** | Path to node (e.g., `/project1/noise1`) | -| `properties` | object | **Yes** | Key-value pairs to update (e.g., `{"seed": 42, "monochrome": true}`) | -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | +Get current status of the td_input command queue. Poll this after td_input_execute until status='idle'. Returns: status ('idle'/'running'), current command, queue_remaining, last error. -**Examples:** +No parameters (other than optional `target_instance`). -``` -# Set noise parameters -update_td_node_parameters( - nodePath="/project1/noise1", - properties={"seed": 42, "monochrome": false, "period": 4.0, "harmonics": 3, - "resolutionw": 1920, "resolutionh": 1080} -) +### td_input_clear -# Set a file path -update_td_node_parameters( - nodePath="/project1/moviefilein1", - properties={"file": "/Users/me/Videos/clip.mp4", "play": true} -) +Clear the td_input command queue and stop current execution immediately. -# Set compositing mode -update_td_node_parameters( - nodePath="/project1/composite1", - properties={"operand": 0} # 0=Over, 1=Under, 3=Add, 18=Multiply, 27=Screen -) -``` +No parameters (other than optional `target_instance`). -**Returns:** List of successfully updated properties and any that failed (with reasons). Raises error if zero properties were updated. +### td_op_screen_rect -**Parameter value types:** Floats, ints, booleans, and strings are all accepted. For menu parameters, use either the string label or the integer index. +Get the screen coordinates of an operator node in the network editor. Returns {x,y,w,h,cx,cy} where cx,cy is the center for clicking. Use this to find where to click on a specific operator. Only works if the operator's parent network is currently open in a network editor pane. ---- +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | yes | Full path to the operator, e.g. '/project1/myComp/noise1' | -## Tool 10: exec_node_method +### td_click_screen_point -**Purpose:** Call a Python method directly on a specific node. +Resolve a point inside a previous td_get_screen_screenshot result and click it. Pass the screenshot request_id plus either normalized u/v or image_x/image_y. Queues a td_input click using physical screen coordinates, so it works directly with screenshot-derived points. Use duration/easing to control the cursor travel before the click. -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `nodePath` | string | **Yes** | Path to node | -| `method` | string | **Yes** | Method name to call | -| `args` | array | No | Positional arguments (strings, numbers, booleans) | -| `kwargs` | object | No | Keyword arguments | -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `request_id` | string | yes | Request ID originally returned by td_get_screen_screenshot. | +| `u` | number | no | Normalized horizontal position inside the screenshot region (0=left, 1=right). Use with v. | +| `v` | number | no | Normalized vertical position inside the screenshot region (0=top, 1=bottom). Use with u. | +| `image_x` | number | no | Horizontal pixel coordinate inside the returned screenshot image. Use with image_y. | +| `image_y` | number | no | Vertical pixel coordinate inside the returned screenshot image. Use with image_x. | +| `button` | `left` / `right` / `middle` | no | Mouse button to click. Default left. | +| `hold` | number | no | Seconds to hold the mouse button down before releasing. | +| `duration` | number | no | Seconds for the cursor to travel to the target before clicking. | +| `easing` | `linear` / `ease-in` / `ease-out` / `ease-in-out` | no | Cursor movement easing for the pre-click travel. | +| `focus` | boolean | no | If true, bring TD to the front before clicking and wait briefly for focus to settle. | -**Examples:** +### td_screen_point_to_global -``` -# Get all children of a component -exec_node_method(nodePath="/project1", method="findChildren") +Convert a point inside a previous td_get_screen_screenshot result into absolute screen coordinates. Pass the screenshot request_id plus either normalized u/v (0..1 inside that screenshot region) or image_x/image_y in returned image pixels. Returns absolute physical screen coordinates, logical coordinates, and a ready-to-use td_input_execute payload. Metadata is kept for the most recent screen screenshots so multiple agents can resolve points later by request_id. -# Find specific children -exec_node_method(nodePath="/project1", method="findChildren", - kwargs={"name": "noise*", "depth": 1}) +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `request_id` | string | yes | Request ID originally returned by td_get_screen_screenshot. | +| `u` | number | no | Normalized horizontal position inside the screenshot region (0=left, 1=right). Use with v. | +| `v` | number | no | Normalized vertical position inside the screenshot region (0=top, 1=bottom). Use with u. | +| `image_x` | number | no | Horizontal pixel coordinate inside the returned screenshot image. Use with image_y. | +| `image_y` | number | no | Vertical pixel coordinate inside the returned screenshot image. Use with image_x. | -# Get node errors -exec_node_method(nodePath="/project1/noise1", method="errors") +## System -# Get node warnings -exec_node_method(nodePath="/project1/noise1", method="warnings") +### td_list_instances -# Save a component as .tox -exec_node_method(nodePath="/project1/myContainer", method="save", - args=["/path/to/component.tox"]) -``` +List all running TouchDesigner (TD) instances with active MCP servers. Returns port, project name, PID, and instanceId for each instance. Call this at the start of every conversation to discover available instances and choose which one to work with. instanceId is stable for the lifetime of a TD process and is used as target_instance in all other tool calls. -**Returns:** Processed return value of the method call. TD operators are serialized to their path strings, iterables to lists, etc. +No parameters (other than optional `target_instance`). ---- +### td_project_quit -## Tool 11: get_td_classes +Save and/or close the current TouchDesigner (TD) project. Can save before closing. Reports if project has unsaved changes. To close a different instance, pass target_instance=instanceId. WARNING: this will shut down the MCP server on that instance. -**Purpose:** List available TouchDesigner Python classes and modules. +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `save` | boolean | no | Save the project before closing. Default true. | +| `force` | boolean | no | Force close without save dialog. Default false. | -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | -| `limit` | integer | No | Max items (default: 50) | +### td_reinit_extension -**Example:** +Reinitialize an extension on a COMP in TouchDesigner (TD). Call this AFTER finishing all code edits via td_write_dat to apply changes. Do NOT call after every small edit - batch your changes first. -``` -get_td_classes(limit=100) -``` +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `path` | string | yes | Path to the COMP with the extension | -**Returns:** List of class/module names and their docstrings from the td module. Useful for discovering what's available in TD's Python environment. +### td_dev_log ---- +Read the last N entries from the MCP dev log. Only available when Devmode is enabled. Shows request/response history. -## Tool 12: get_td_class_details +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `count` | integer | no | Number of recent log entries to return | -**Purpose:** Get methods and properties of a specific TD Python class. +### td_clear_dev_log -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `className` | string | **Yes** | Class name (e.g., `noiseTop`, `OP`, `COMP`) | -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | -| `limit` | integer | No | Max methods/properties (default: 30) | +Clear the current MCP dev log by closing the old file and starting a fresh one. Only available when Devmode is enabled. -**Examples:** +No parameters (other than optional `target_instance`). -``` -# Inspect the noiseTop class -get_td_class_details(className="noiseTop") +### td_test_session -# Inspect the base OP class (all operators inherit from this) -get_td_class_details(className="OP", limit=50) +Manage test sessions, bug reports, and conversation export. IMPORTANT: Do NOT proactively suggest exporting chat or submitting reports. These are tools for specific situations: - export_chat / submit_report: ONLY when the user encounters a BUG with the plugin or TouchDesigner and wants to report it, or when the user explicitly asks to export the conversation. Never suggest this at session end or as routine action. USER PHRASES → ACTIONS: 'разбор тестовых сессий' / 'analyze test sessions' → list, then pull, read meta.json → index.jsonl → calls/. 'разбор репортов' / 'analyze user reports' → list with session='user', then pull by name. 'экспортируй чат' / 'export chat' → (1) export_chat_id → marker, (2) export_chat with session=marker. 'сообщи о проблеме' / 'report bug' → export chat, review for privacy, then submit_report with summary + tags + result_op=file_path. ACTIONS: export_chat_id | export_chat | submit_report | start | note | import_chat | end | list | pull. list: default=auto-detect repo. session='user' for user_reports (dev only). pull: auto-searches both repos. Auto-detects dev vs user Hub access. -# Inspect COMP (component) class -get_td_class_details(className="COMP") -``` - -**Returns:** Class name, type, description, methods (name + description + type), and properties (name + description + type). - ---- - -## Tool 13: get_td_module_help - -**Purpose:** Retrieve Python help() text for any TD module, class, or function. - -**Parameters:** -| Name | Type | Required | Description | -|------|------|----------|-------------| -| `moduleName` | string | **Yes** | Module/class name (e.g., `noiseCHOP`, `tdu`, `td.OP`) | -| `detailLevel` | string | No | Response verbosity | -| `responseFormat` | string | No | Output format | - -**Examples:** - -``` -# Get help for the noise CHOP class -get_td_module_help(moduleName="noiseCHOP") - -# Get help for the tdu utilities module -get_td_module_help(moduleName="tdu") - -# Dotted name resolution works -get_td_module_help(moduleName="td.OP") -``` - -**Returns:** Full Python help() text output, cleaned of backspace characters. - ---- - -## Workflow: Building a Complete Network - -Typical sequence of tool calls to build a project: - -1. `get_td_info` — verify connection -2. `get_td_nodes(parentPath="/project1")` — see what already exists -3. `create_td_node` (multiple) — create all operators -4. `update_td_node_parameters` (multiple) — configure each operator -5. `execute_python_script` — wire all connections in one batch script -6. `get_td_node_errors(nodePath="/project1")` — check for problems -7. `get_td_node_parameters` — verify specific nodes if needed -8. Iterate: adjust parameters, add operators, fix errors - -## TD Documentation MCP Server Tools - -The bottobot/touchdesigner-mcp-server provides 21 reference/knowledge tools (no running TD needed): - -| Tool | Purpose | -|------|---------| -| `get_operator` | Get full documentation for a specific operator | -| `search_operators` | Search operators by keyword | -| `list_operators` | List all operators (filterable by family) | -| `compare_operators` | Compare two operators side by side | -| `get_operator_examples` | Get usage examples for an operator | -| `suggest_workflow` | Get workflow suggestions for a task | -| `get_tutorial` | Get a full TD tutorial | -| `list_tutorials` | List available tutorials | -| `search_tutorials` | Search tutorial content | -| `get_python_api` | Get Python API class documentation | -| `search_python_api` | Search Python API | -| `list_python_classes` | List all documented Python classes | -| `get_version_info` | Get TD version release notes | -| `list_versions` | List all documented TD versions | -| `get_experimental_techniques` | Get advanced technique guides (GLSL, ML, generative, etc.) | -| `search_experimental` | Search experimental techniques | -| `get_glsl_pattern` | Get GLSL code patterns (SDF, color, math utilities) | -| `get_operator_connections` | Get common operator wiring patterns | -| `get_network_template` | Get complete network templates with Python generation scripts | -| `get_experimental_build` | Get experimental build info | -| `list_experimental_builds` | List experimental builds | - -This server contains 630 operator docs, 14 tutorials, 69 Python API classes, and 7 experimental technique categories with working code. +| Param | Type | Required | Description | +|-------|------|----------|-------------| +| `action` | `export_chat_id` / `export_chat` / `submit_report` / `start` / `note` / `import_chat` / `end` / `list` / `pull` | yes | Action: export_chat_id / export_chat / submit_report / start / note / import_chat / end / list / pull | +| `prompt` | string | no | (start) The test prompt/task description | +| `tags` | array | no | (start) Tags for categorization, e.g. ['ui', 'layout'] | +| `text` | string | no | (note) Observation text. (import_chat) Full conversation text. | +| `outcome` | `success` / `partial` / `failure` | no | (end) Result: success / partial / failure | +| `summary` | string | no | (end) Brief summary of what happened | +| `result_op` | string | no | (end) Path to operator to save as result.tox | +| `session` | string | no | (pull) Session name or substring to download | diff --git a/skills/creative/touchdesigner/references/network-patterns.md b/skills/creative/touchdesigner/references/network-patterns.md index 7afa24150..cb04fd54d 100644 --- a/skills/creative/touchdesigner/references/network-patterns.md +++ b/skills/creative/touchdesigner/references/network-patterns.md @@ -20,32 +20,32 @@ Audio File In CHOP -> Audio Spectrum CHOP -> Math CHOP (scale) **MCP Build Sequence:** ``` -1. create_td_node(parentPath="/project1", nodeType="audiofileinChop", nodeName="audio_in") -2. create_td_node(parentPath="/project1", nodeType="audiospectrumChop", nodeName="spectrum") -3. create_td_node(parentPath="/project1", nodeType="mathChop", nodeName="spectrum_scale") -4. create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="noise1") -5. create_td_node(parentPath="/project1", nodeType="levelTop", nodeName="level1") -6. create_td_node(parentPath="/project1", nodeType="feedbackTop", nodeName="feedback1") -7. create_td_node(parentPath="/project1", nodeType="compositeTop", nodeName="comp1") -8. create_td_node(parentPath="/project1", nodeType="nullTop", nodeName="out") +1. td_create_operator(parent="/project1", type="audiofileinChop", name="audio_in") +2. td_create_operator(parent="/project1", type="audiospectrumChop", name="spectrum") +3. td_create_operator(parent="/project1", type="mathChop", name="spectrum_scale") +4. td_create_operator(parent="/project1", type="noiseTop", name="noise1") +5. td_create_operator(parent="/project1", type="levelTop", name="level1") +6. td_create_operator(parent="/project1", type="feedbackTop", name="feedback1") +7. td_create_operator(parent="/project1", type="compositeTop", name="comp1") +8. td_create_operator(parent="/project1", type="nullTop", name="out") -9. update_td_node_parameters(nodePath="/project1/audio_in", +9. td_set_operator_pars(path="/project1/audio_in", properties={"file": "/path/to/music.wav", "play": true}) -10. update_td_node_parameters(nodePath="/project1/spectrum", +10. td_set_operator_pars(path="/project1/spectrum", properties={"size": 512}) -11. update_td_node_parameters(nodePath="/project1/spectrum_scale", +11. td_set_operator_pars(path="/project1/spectrum_scale", properties={"gain": 2.0, "postoff": 0.0}) -12. update_td_node_parameters(nodePath="/project1/noise1", - properties={"type": 1, "monochrome": false, "resolutionw": 1920, "resolutionh": 1080, +12. td_set_operator_pars(path="/project1/noise1", + properties={"type": 1, "monochrome": false, "resolutionw": 1280, "resolutionh": 720, "period": 4.0, "harmonics": 3, "amp": 1.0}) -13. update_td_node_parameters(nodePath="/project1/level1", +13. td_set_operator_pars(path="/project1/level1", properties={"opacity": 0.95, "gamma1": 0.75}) -14. update_td_node_parameters(nodePath="/project1/feedback1", +14. td_set_operator_pars(path="/project1/feedback1", properties={"top": "/project1/comp1"}) -15. update_td_node_parameters(nodePath="/project1/comp1", +15. td_set_operator_pars(path="/project1/comp1", properties={"operand": 0}) -16. execute_python_script: """ +16. td_execute_python: """ op('/project1/audio_in').outputConnectors[0].connect(op('/project1/spectrum')) op('/project1/spectrum').outputConnectors[0].connect(op('/project1/spectrum_scale')) op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1')) @@ -54,7 +54,7 @@ op('/project1/feedback1').outputConnectors[0].connect(op('/project1/comp1').inpu op('/project1/comp1').outputConnectors[0].connect(op('/project1/out')) """ -17. execute_python_script: """ +17. td_execute_python: """ # Export spectrum values to drive noise parameters # This makes the noise react to audio frequencies op('/project1/noise1').par.seed.expr = "op('/project1/spectrum_scale')['chan1']" @@ -85,7 +85,7 @@ Math CHOP: chanop=1 (Add channels), range1low=0, range1high=10 Trigger CHOP: attack=0.02, peak=1.0, decay=0.3, sustain=0.0, release=0.1 # Export to visual: Scale, brightness, or color intensity -execute_python_script: "op('/project1/level1').par.brightness1.expr = \"1.0 + op('/project1/trigger1')['chan1'] * 0.5\"" +td_execute_python: "op('/project1/level1').par.brightness1.expr = \"1.0 + op('/project1/trigger1')['chan1'] * 0.5\"" ``` ### Pattern 3: Multi-Band Audio -> Multi-Layer Visuals @@ -109,26 +109,26 @@ Audio In -> Spectrum -> Audio Band EQ (3 bands: bass, mid, treble) Out ``` -### Pattern 3b: Audio-Reactive GLSL Fractal (Proven td_exec Recipe) +### Pattern 3b: Audio-Reactive GLSL Fractal (Proven Recipe) -Complete working recipe tested in TD 099. Plays an MP3, runs FFT, feeds spectrum as a texture into a GLSL shader where inner fractal reacts to bass, outer to treble. +Complete working recipe. Plays an MP3, runs FFT, feeds spectrum as a texture into a GLSL shader where inner fractal reacts to bass, outer to treble. **Network:** ``` -AudioFileIn CHOP → AudioSpectrum CHOP → Math CHOP (boost) → Resample CHOP (256) - ↓ - CHOP To TOP (256x1 spectrum texture) - ↓ +AudioFileIn CHOP → AudioSpectrum CHOP (FFT=512, outlength=256) + → Math CHOP (gain=10) → CHOP To TOP (256x2 spectrum texture, dataformat=r) + ↓ Constant TOP (time, rgba32float) → GLSL TOP (input 0=time, input 1=spectrum) → Null → MovieFileOut ↓ AudioFileIn CHOP → Audio Device Out CHOP Record to .mov ``` -**Build via td_exec (one call per step for reliability):** +**Build via td_execute_python (one call per step for reliability):** ```python # Step 1: Audio chain -td_exec(""" +# td_execute_python script: +td_execute_python(code=""" root = op('/project1') audio = root.create(audiofileinCHOP, 'audio_in') audio.par.file = '/path/to/music.mp3' @@ -148,7 +148,7 @@ resamp.par.timeslice = True resamp.par.rate = 256 chop2top = root.create(choptoTOP, 'spectrum_tex') -resamp.outputConnectors[0].connect(chop2top.inputConnectors[0]) +chop2top.par.chop = resamp # CHOP To TOP has NO input connectors — use par.chop reference # Audio output (hear the music) aout = root.create(audiodeviceoutCHOP, 'audio_out') @@ -156,8 +156,9 @@ audio.outputConnectors[0].connect(aout.inputConnectors[0]) result = 'audio chain ok' """) -# Step 2: Time driver (MUST be rgba32float — see pitfalls #12) -td_exec(""" +# Step 2: Time driver (MUST be rgba32float — see pitfalls #6) +# td_execute_python script: +td_execute_python(code=""" root = op('/project1') td = root.create(constantTOP, 'time_driver') td.par.format = 'rgba32float' @@ -170,7 +171,8 @@ result = 'time ok' """) # Step 3: GLSL shader (write to /tmp, load from file) -td_exec(""" +# td_execute_python script: +td_execute_python(code=""" root = op('/project1') glsl = root.create(glslTOP, 'audio_shader') glsl.par.outputresolution = 'custom' @@ -188,7 +190,8 @@ result = 'glsl ok' """) # Step 4: Output + recorder -td_exec(""" +# td_execute_python script: +td_execute_python(code=""" root = op('/project1') out = root.create(nullTOP, 'output') op('/project1/audio_shader').outputConnectors[0].connect(out.inputConnectors[0]) @@ -214,7 +217,7 @@ vec3 palette(float t) { void main() { // Input 0 = time (1x1 rgba32float constant) - // Input 1 = audio spectrum (256x1 CHOP To TOP) + // Input 1 = audio spectrum (256x2 CHOP To TOP, stereo — sample at y=0.25 for first channel) vec4 td = texture(sTD2DInputs[0], vec2(0.5)); float t = td.r + td.g * 1000.0; @@ -223,15 +226,15 @@ void main() { vec2 uv0 = uv; vec3 finalColor = vec3(0.0); - float bass = texture(sTD2DInputs[1], vec2(0.05, 0.0)).r; - float mids = texture(sTD2DInputs[1], vec2(0.25, 0.0)).r; + float bass = texture(sTD2DInputs[1], vec2(0.05, 0.25)).r; + float mids = texture(sTD2DInputs[1], vec2(0.25, 0.25)).r; for (float i = 0.0; i < 4.0; i++) { uv = fract(uv * (1.4 + bass * 0.3)) - 0.5; float d = length(uv) * exp(-length(uv0)); // Sample spectrum at distance: inner=bass, outer=treble - float freq = texture(sTD2DInputs[1], vec2(clamp(d * 0.5, 0.0, 1.0), 0.0)).r; + float freq = texture(sTD2DInputs[1], vec2(clamp(d * 0.5, 0.0, 1.0), 0.25)).r; vec3 col = palette(length(uv0) + i * 0.4 + t * 0.35); d = sin(d * (7.0 + bass * 4.0) + t * 1.5) / 8.0; @@ -247,7 +250,7 @@ void main() { ``` **Key insights from testing:** -- `spectrum_tex` (CHOP To TOP) produces a 256x1 texture — x position = frequency +- `spectrum_tex` (CHOP To TOP) produces a 256x2 texture — x position = frequency, y=0.25 for first channel - Sampling at `vec2(0.05, 0.0)` gets bass, `vec2(0.65, 0.0)` gets treble - Sampling based on pixel distance (`d * 0.5`) makes inner fractal react to bass, outer to treble - `bass * 0.3` in the `fract()` zoom makes the fractal breathe with kicks @@ -269,26 +272,26 @@ Noise TOP -> Composite TOP -> Level TOP -> Null TOP (out) **MCP Build Sequence:** ``` -1. create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="seed_noise") -2. create_td_node(parentPath="/project1", nodeType="compositeTop", nodeName="mix") -3. create_td_node(parentPath="/project1", nodeType="transformTop", nodeName="evolve") -4. create_td_node(parentPath="/project1", nodeType="feedbackTop", nodeName="fb") -5. create_td_node(parentPath="/project1", nodeType="levelTop", nodeName="color_correct") -6. create_td_node(parentPath="/project1", nodeType="nullTop", nodeName="out") +1. td_create_operator(parent="/project1", type="noiseTop", name="seed_noise") +2. td_create_operator(parent="/project1", type="compositeTop", name="mix") +3. td_create_operator(parent="/project1", type="transformTop", name="evolve") +4. td_create_operator(parent="/project1", type="feedbackTop", name="fb") +5. td_create_operator(parent="/project1", type="levelTop", name="color_correct") +6. td_create_operator(parent="/project1", type="nullTop", name="out") -7. update_td_node_parameters(nodePath="/project1/seed_noise", +7. td_set_operator_pars(path="/project1/seed_noise", properties={"type": 1, "monochrome": false, "period": 2.0, "amp": 0.3, - "resolutionw": 1920, "resolutionh": 1080}) -8. update_td_node_parameters(nodePath="/project1/mix", + "resolutionw": 1280, "resolutionh": 720}) +8. td_set_operator_pars(path="/project1/mix", properties={"operand": 27}) # 27 = Screen blend -9. update_td_node_parameters(nodePath="/project1/evolve", +9. td_set_operator_pars(path="/project1/evolve", properties={"sx": 1.003, "sy": 1.003, "rz": 0.5, "extend": 2}) # slight zoom + rotate, repeat edges -10. update_td_node_parameters(nodePath="/project1/fb", +10. td_set_operator_pars(path="/project1/fb", properties={"top": "/project1/mix"}) -11. update_td_node_parameters(nodePath="/project1/color_correct", +11. td_set_operator_pars(path="/project1/color_correct", properties={"opacity": 0.98, "gamma1": 0.85}) -12. execute_python_script: """ +12. td_execute_python: """ op('/project1/seed_noise').outputConnectors[0].connect(op('/project1/mix').inputConnectors[0]) op('/project1/fb').outputConnectors[0].connect(op('/project1/evolve')) op('/project1/evolve').outputConnectors[0].connect(op('/project1/mix').inputConnectors[1]) @@ -319,15 +322,15 @@ Table DAT (instance data) -> DAT to CHOP -> Geometry COMP (instancing on) -> Ren **MCP Build Sequence:** ``` -1. create_td_node(parentPath="/project1", nodeType="tableDat", nodeName="instance_data") -2. create_td_node(parentPath="/project1", nodeType="geometryComp", nodeName="geo1") -3. create_td_node(parentPath="/project1/geo1", nodeType="sphereSop", nodeName="sphere") -4. create_td_node(parentPath="/project1", nodeType="constMat", nodeName="mat1") -5. create_td_node(parentPath="/project1", nodeType="cameraComp", nodeName="cam1") -6. create_td_node(parentPath="/project1", nodeType="lightComp", nodeName="light1") -7. create_td_node(parentPath="/project1", nodeType="renderTop", nodeName="render1") +1. td_create_operator(parent="/project1", type="tableDat", name="instance_data") +2. td_create_operator(parent="/project1", type="geometryComp", name="geo1") +3. td_create_operator(parent="/project1/geo1", type="sphereSop", name="sphere") +4. td_create_operator(parent="/project1", type="constMat", name="mat1") +5. td_create_operator(parent="/project1", type="cameraComp", name="cam1") +6. td_create_operator(parent="/project1", type="lightComp", name="light1") +7. td_create_operator(parent="/project1", type="renderTop", name="render1") -8. execute_python_script: """ +8. td_execute_python: """ import random, math dat = op('/project1/instance_data') dat.clear() @@ -346,15 +349,15 @@ for i in range(500): ]) """ -9. update_td_node_parameters(nodePath="/project1/geo1", +9. td_set_operator_pars(path="/project1/geo1", properties={"instancing": true, "instancechop": "", "instancedat": "/project1/instance_data", "material": "/project1/mat1"}) -10. update_td_node_parameters(nodePath="/project1/render1", +10. td_set_operator_pars(path="/project1/render1", properties={"camera": "/project1/cam1", "geometry": "/project1/geo1", "light": "/project1/light1", - "resolutionw": 1920, "resolutionh": 1080}) -11. update_td_node_parameters(nodePath="/project1/cam1", + "resolutionw": 1280, "resolutionh": 720}) +11. td_set_operator_pars(path="/project1/cam1", properties={"tz": 10}) ``` @@ -369,7 +372,7 @@ Text DAT (GLSL code) -> GLSL TOP (resolution, dat reference) -> Feedback TOP Level TOP (out) ``` -**Key GLSL code (write to Text DAT via execute_python_script):** +**Key GLSL code (write to Text DAT via td_execute_python):** ```glsl // Gray-Scott reaction-diffusion @@ -422,26 +425,26 @@ Movie File In TOP -> HSV Adjust TOP -> Level TOP -> Blur TOP -> Composite TOP -> **MCP Build Sequence:** ``` -1. create_td_node(parentPath="/project1", nodeType="moviefileinTop", nodeName="video_in") -2. create_td_node(parentPath="/project1", nodeType="hsvadjustTop", nodeName="color") -3. create_td_node(parentPath="/project1", nodeType="levelTop", nodeName="levels") -4. create_td_node(parentPath="/project1", nodeType="blurTop", nodeName="blur") -5. create_td_node(parentPath="/project1", nodeType="compositeTop", nodeName="overlay") -6. create_td_node(parentPath="/project1", nodeType="textTop", nodeName="title") -7. create_td_node(parentPath="/project1", nodeType="nullTop", nodeName="out") +1. td_create_operator(parent="/project1", type="moviefileinTop", name="video_in") +2. td_create_operator(parent="/project1", type="hsvadjustTop", name="color") +3. td_create_operator(parent="/project1", type="levelTop", name="levels") +4. td_create_operator(parent="/project1", type="blurTop", name="blur") +5. td_create_operator(parent="/project1", type="compositeTop", name="overlay") +6. td_create_operator(parent="/project1", type="textTop", name="title") +7. td_create_operator(parent="/project1", type="nullTop", name="out") -8. update_td_node_parameters(nodePath="/project1/video_in", +8. td_set_operator_pars(path="/project1/video_in", properties={"file": "/path/to/video.mp4", "play": true}) -9. update_td_node_parameters(nodePath="/project1/color", +9. td_set_operator_pars(path="/project1/color", properties={"hueoffset": 0.1, "saturationmult": 1.3}) -10. update_td_node_parameters(nodePath="/project1/levels", +10. td_set_operator_pars(path="/project1/levels", properties={"brightness1": 1.1, "contrast": 1.2, "gamma1": 0.9}) -11. update_td_node_parameters(nodePath="/project1/blur", +11. td_set_operator_pars(path="/project1/blur", properties={"sizex": 2, "sizey": 2}) -12. update_td_node_parameters(nodePath="/project1/title", +12. td_set_operator_pars(path="/project1/title", properties={"text": "My Video", "fontsizex": 48, "alignx": 1, "aligny": 1}) -13. execute_python_script: """ +13. td_execute_python: """ chain = ['video_in', 'color', 'levels', 'blur'] for i in range(len(chain) - 1): op(f'/project1/{chain[i]}').outputConnectors[0].connect(op(f'/project1/{chain[i+1]}')) @@ -460,7 +463,7 @@ Record the output to a file. **H.264/H.265 require a Commercial license** — us ``` ```python -# Build via td_exec(): +# Build via td_execute_python: root = op('/project1') # Always put a Null TOP before the recorder @@ -488,25 +491,73 @@ rec.par.record = False - `TOP.save()` called rapidly always captures the same frame — use MovieFileOut for animation - See `pitfalls.md` #25-27 for full details -### Pattern 8b: TD → External Pipeline (e.g., ASCII Video) +### Pattern 8b: TD → External Pipeline (FFmpeg / Python / Post-Processing) -Export TD visuals for use in another tool (ffmpeg, Python, ASCII art, etc.): +Export TD visuals for use in another tool (ffmpeg, Python, ASCII art, etc.). This is the standard workflow when you need to composite TD output with external processing (ASCII conversion, Python shader chains, ML inference, etc.). + +**Step 1: Record to video in TD** ```python -# 1. Record with MovieFileOut (MJPEG) -rec.par.videocodec = 'mjpa' +# Preferred: ProRes on macOS (lossless, Non-Commercial OK, ~55MB/s at 1280x720) +rec.par.videocodec = 'prores' +# Fallback for non-macOS: mjpa (Motion JPEG) +# rec.par.videocodec = 'mjpa' rec.par.record = True # ... wait N seconds ... rec.par.record = False - -# 2. Extract frames with ffmpeg (outside TD) -# ffmpeg -i /tmp/output.mov -vframes 120 /tmp/frames/frame_%06d.png - -# 3. Load frames in Python for processing -# from PIL import Image -# img = Image.open('/tmp/frames/frame_000001.png') ``` +**Step 2: Extract frames with ffmpeg** + +```bash +# Extract all frames at 30fps +ffmpeg -y -i /tmp/output.mov -vf 'fps=30' /tmp/frames/frame_%06d.png + +# Or extract a specific duration +ffmpeg -y -i /tmp/output.mov -t 25 -vf 'fps=30' /tmp/frames/frame_%06d.png + +# Or extract specific frame range +ffmpeg -y -i /tmp/output.mov -vf 'select=between(n\,0\,749)' -vsync vfr /tmp/frames/frame_%06d.png +``` + +**Step 3: Process frames in Python** + +```python +from PIL import Image +import os + +frames_dir = '/tmp/frames' +output_dir = '/tmp/processed' +os.makedirs(output_dir, exist_ok=True) + +for fname in sorted(os.listdir(frames_dir)): + if not fname.endswith('.png'): + continue + img = Image.open(os.path.join(frames_dir, fname)) + # ... apply your processing ... + img.save(os.path.join(output_dir, fname)) +``` + +**Step 4: Mux processed frames back with audio** + +```bash +# Create video from processed frames + audio with fade-out +ffmpeg -y \ + -framerate 30 -i /tmp/processed/frame_%06d.png \ + -i /tmp/audio.mp3 \ + -c:v libx264 -pix_fmt yuv420p -crf 18 \ + -c:a aac -b:a 192k \ + -shortest \ + -af 'afade=t=out:st=23:d=2' \ + /tmp/final_output.mp4 +``` + +**Key considerations:** +- Use ProRes for the TD recording step to avoid generation loss during compositing +- Extract at the target output framerate (not TD's render framerate) +- For audio-synced content, analyze the audio file separately in Python (scipy FFT) to get per-frame features (rms, spectral bands, beats) and drive compositing parameters +- Always verify TD FPS > 0 before recording (see pitfalls #37, #38) + ## Data Visualization ### Pattern 9: Table Data -> Bar Chart via Instancing @@ -524,7 +575,7 @@ Box SOP -> Geometry COMP (instancing from CHOP) -> Render TOP -> Null TOP (out) ```python # Script DAT code to transform data to instance positions -execute_python_script: """ +td_execute_python: """ source = op('/project1/data_table') instance = op('/project1/instance_transform') instance.clear() @@ -545,21 +596,23 @@ for i in range(1, source.numRows): ### Pattern 9b: Audio-Reactive GLSL Fractal (Proven Recipe) -Audio spectrum drives a GLSL fractal shader directly via a spectrum texture input. Bass thickens inner fractal lines, mids twist rotation, highs light outer edges. Tested and working on TD 099 Non-Commercial. +Audio spectrum drives a GLSL fractal shader directly via a spectrum texture input. Bass thickens inner fractal lines, mids twist rotation, highs light outer edges. **Always run discovery (SKILL.md Step 0) before using any param names from these recipes — they may differ in your TD version.** ``` -Audio File In CHOP → Audio Spectrum CHOP → Math CHOP (boost gain=5) - → Resample CHOP (256 samples) → CHOP To TOP (spectrum texture, 256x1) +Audio File In CHOP → Audio Spectrum CHOP (FFT=512, outlength=256) + → Math CHOP (gain=10) + → CHOP To TOP (spectrum texture, 256x2, dataformat=r) ↓ (input 1) Constant TOP (rgba32float, time) → GLSL TOP (audio-reactive shader) → Null TOP (input 0) ↑ Text DAT (shader code) ``` -**Build via td_exec (complete working script):** +**Build via td_execute_python (complete working script):** ```python -td_exec(""" +# td_execute_python script: +td_execute_python(code=""" import os root = op('/project1') @@ -568,25 +621,24 @@ audio = root.create(audiofileinCHOP, 'audio_in') audio.par.file = '/path/to/music.mp3' audio.par.playmode = 0 # Locked to timeline -# FFT analysis +# FFT analysis (output length manually set to 256 bins) spectrum = root.create(audiospectrumCHOP, 'spectrum') audio.outputConnectors[0].connect(spectrum.inputConnectors[0]) +spectrum.par.fftsize = '512' +spectrum.par.outputmenu = 'setmanually' +spectrum.par.outlength = 256 -# Normalize + boost +# THEN boost gain on the raw spectrum (NO Lag CHOP — see pitfall #34) math = root.create(mathCHOP, 'math_norm') spectrum.outputConnectors[0].connect(math.inputConnectors[0]) -math.par.gain = 5 +math.par.gain = 10 -# Resample to 256 bins for texture -resample = root.create(resampleCHOP, 'resample_spec') -math.outputConnectors[0].connect(resample.inputConnectors[0]) -resample.par.timeslice = True -resample.par.rate = 256 - -# Spectrum → texture (256x1 image) +# Spectrum → texture (256x2 image — stereo, sample at y=0.25 for first channel) # NOTE: choptoTOP has NO input connectors — use par.chop reference! spec_tex = root.create(choptoTOP, 'spectrum_tex') -spec_tex.par.chop = resample +spec_tex.par.chop = math +spec_tex.par.dataformat = 'r' +spec_tex.par.layout = 'rowscropped' # Time driver (rgba32float to avoid 0-1 clamping!) time_drv = root.create(constantTOP, 'time_driver') @@ -640,9 +692,9 @@ void main() { vec2 uv0 = uv; vec3 finalColor = vec3(0.0); - float bass = texture(sTD2DInputs[1], vec2(0.05, 0.0)).r; - float mids = texture(sTD2DInputs[1], vec2(0.25, 0.0)).r; - float highs = texture(sTD2DInputs[1], vec2(0.65, 0.0)).r; + float bass = texture(sTD2DInputs[1], vec2(0.05, 0.25)).r; + float mids = texture(sTD2DInputs[1], vec2(0.25, 0.25)).r; + float highs = texture(sTD2DInputs[1], vec2(0.65, 0.25)).r; float ca = cos(t * (0.15 + mids * 0.3)); float sa = sin(t * (0.15 + mids * 0.3)); @@ -651,7 +703,7 @@ void main() { for (float i = 0.0; i < 4.0; i++) { uv = fract(uv * (1.4 + bass * 0.3)) - 0.5; float d = length(uv) * exp(-length(uv0)); - float freq = texture(sTD2DInputs[1], vec2(clamp(d*0.5, 0.0, 1.0), 0.0)).r; + float freq = texture(sTD2DInputs[1], vec2(clamp(d*0.5, 0.0, 1.0), 0.25)).r; vec3 col = palette(length(uv0) + i * 0.4 + t * 0.35); d = sin(d * (7.0 + bass * 4.0) + t * 1.5) / 8.0; d = abs(d); @@ -769,7 +821,7 @@ Multi Touch In DAT -> Script CHOP (parse touches) -> [export to visual params] ```python # Normalize mouse position to 0-1 range -execute_python_script: """ +td_execute_python: """ op('/project1/noise1').par.offsetx.expr = "op('/project1/mouse_norm')['tx']" op('/project1/noise1').par.offsety.expr = "op('/project1/mouse_norm')['ty']" """ @@ -782,12 +834,12 @@ OSC In CHOP (port 7000) -> Select CHOP (pick channels) -> [export to visual para ``` ``` -1. create_td_node(parentPath="/project1", nodeType="oscinChop", nodeName="osc_in") -2. update_td_node_parameters(nodePath="/project1/osc_in", properties={"port": 7000}) +1. td_create_operator(parent="/project1", type="oscinChop", name="osc_in") +2. td_set_operator_pars(path="/project1/osc_in", properties={"port": 7000}) # OSC messages like /frequency 440 will appear as channel "frequency" with value 440 # Export to any parameter: -3. execute_python_script: "op('/project1/noise1').par.period.expr = \"op('/project1/osc_in')['frequency']\"" +3. td_execute_python: "op('/project1/noise1').par.period.expr = \"op('/project1/osc_in')['frequency']\"" ``` ### Pattern 14: MIDI Control (DJ/VJ) @@ -815,12 +867,12 @@ Source C (camera) --------+ ```python # MIDI CC1 controls which source is active (0-127 -> 0-2) -execute_python_script: """ +td_execute_python: """ op('/project1/switch1').par.index.expr = "int(op('/project1/midi_in')['cc1'] / 42)" """ # MIDI CC2 controls crossfade between current and next -execute_python_script: """ +td_execute_python: """ op('/project1/cross1').par.cross.expr = "op('/project1/midi_in')['cc2'] / 127.0" """ ``` @@ -851,7 +903,7 @@ Script CHOP (cue state: current_cue, progress, next_cue_trigger) ``` ```python -execute_python_script: """ +td_execute_python: """ # Simple cue system cue_table = op('/project1/cue_list') cue_state = op('/project1/cue_state') @@ -900,7 +952,7 @@ WebSocket DAT -> Script DAT (parse JSON messages) -> [update visuals] ``` ```python -execute_python_script: """ +td_execute_python: """ ws = op('/project1/websocket1') ws.par.address = 'ws://localhost:8080' ws.par.active = True diff --git a/skills/creative/touchdesigner/references/pitfalls.md b/skills/creative/touchdesigner/references/pitfalls.md index 862bdc563..5883ed72c 100644 --- a/skills/creative/touchdesigner/references/pitfalls.md +++ b/skills/creative/touchdesigner/references/pitfalls.md @@ -2,93 +2,16 @@ Hard-won knowledge from real TD sessions. Read this before building anything. -## Setup & Connection - -### 1. The .tox from the git repo is BROKEN - -The `td/mcp_webserver_base.tox` in the `8beeeaaat/touchdesigner-mcp` git clone is **incomplete**. It's missing the `td_server` Python module (generated by `npm run gen:webserver` which requires Docker). Port 9981 opens, but every route returns 404. - -**Always download the release zip:** -```bash -curl -L -o td.zip \ - "https://github.com/8beeeaaat/touchdesigner-mcp/releases/latest/download/touchdesigner-mcp-td.zip" -unzip -o td.zip -d touchdesigner-mcp-td -``` - -### 2. The release .tox also breaks (frequently) - -Even the correct release .tox fails after drag-and-drop import because `import_modules.py` resolves `modules/` via `parent().par.externaltox.eval()` — a relative path that often goes wrong. Symptoms: port 9981 listens, all routes 404, TD Textport shows `[ERROR] Failed to setup modules`. - -**The custom API handler (`scripts/custom_api_handler.py`) is more reliable.** It has zero external module dependencies — just a WebServer DAT + Text DAT callback. The skill's setup workflow should try the .tox first, test with `curl`, and auto-deploy the handler if 404. - -### 3. You CANNOT automate the .tox import from outside TD - -TD has no CLI flag to import a .tox. macOS blocks keystroke injection via System Events for security. The only way to get code into TD from outside is: -- Have a WebServer DAT already running (chicken-and-egg) -- AppleScript to open Textport + clipboard paste (fragile, not always reliable) -- User manually drags the .tox or pastes a script - -**Plan for one manual step** from the user (either .tox drag-drop or Textport paste). Make it as frictionless as possible: `open -R /path/to/file` to reveal in Finder. - -### 4. The npm package name is `touchdesigner-mcp-server` (not `@anthropic/...`) - -The Hermes config should use: -```yaml -command: npx -args: ["-y", "touchdesigner-mcp-server@latest"] -``` - -### 5. MCP tools may register but not be callable - -Hermes may report "17 MCP tool(s) now available" but the tools aren't exposed as function calls. Use the REST API directly via `curl` in `execute_code` as a reliable fallback: -```python -def td_exec(script): - escaped = json.dumps({"script": script}) - cmd = f"curl -s -X POST -H 'Content-Type: application/json' -d {shlex.quote(escaped)} 'http://127.0.0.1:9981/api/td/server/exec'" - return json.loads(terminal(cmd)['output']) -``` - -## TD WebServer DAT Quirks - -### 6. Response body goes in `response['data']`, NOT `response['body']` - -When writing custom WebServer DAT handlers, the response payload must be set on the `data` key: -```python -response['data'] = json.dumps({"result": 42}) # ✓ works -response['body'] = json.dumps({"result": 42}) # ✗ ignored -``` - -### 7. Request POST body comes as BYTES in `request['data']` - -Not `request['body']`, and it's `bytes` not `str`: -```python -raw = request.get('data', b'') -if isinstance(raw, bytes): - raw = raw.decode('utf-8') -body = json.loads(raw) if raw else {} -``` - -### 8. Non-Commercial license caps resolution at 1280×1280 - -Setting `resolutionw=1920` silently clamps to 1280. Always check effective resolution after creation: -```python -n.cook(force=True) -actual = str(n.width) + 'x' + str(n.height) -``` - ## Parameter Names -### 9. NEVER hardcode parameter names — always discover +### 1. NEVER hardcode parameter names — always discover -Parameter names change between TD versions. What works in 099 may not work in 098 or 2023.x. Always run discovery first: -```python -n = root.create(glslTOP, '_test') -pars = [(p.name, type(p.val).__name__) for p in n.pars()] -n.destroy() -``` +Parameter names change between TD versions. What works in one build may not work in another. ALWAYS use td_get_par_info to discover actual names from TD. -Known differences from docs/online references: -| What docs say | TD 099 actual | Notes | +The agent's LLM training data contains WRONG parameter names. Do not trust them. + +Known historical differences (may vary further — always verify): +| What docs/training say | Actual in some versions | Notes | |---------------|---------------|-------| | `dat` | `pixeldat` | GLSL TOP pixel shader DAT | | `colora` | `alpha` | Constant TOP alpha | @@ -98,7 +21,15 @@ Known differences from docs/online references: | `bgcolora` | `bgalpha` | Text TOP bg alpha | | `value1name` | `vec0name` | GLSL TOP uniform name | -### 10. Use `safe_par()` pattern for cross-version compatibility +### 2. twozero td_execute_python response format + +When calling `td_execute_python` via twozero MCP, successful responses return `(ok)` followed by FPS/error summary (e.g. `[fps 60.0/60] [0 err/0 warn]`), NOT the raw Python `result` dict. If you're parsing responses programmatically, check for the `(ok)` prefix — don't pattern-match on Python variable names from the script. Use `td_get_operator_info` or separate inspection calls to read back values. + +### 3. When using td_set_operator_pars, param names must match exactly + +Use td_get_par_info to discover them. The MCP tool validates parameter names and returns clear errors explaining what went wrong, unlike raw Python which crashes the whole script with tdAttributeError and stops execution. Always discover before setting. + +### 3. Use `safe_par()` pattern for cross-version compatibility ```python def safe_par(node, name, value): @@ -109,36 +40,65 @@ def safe_par(node, name, value): return False ``` -### 11. `td.tdAttributeError` crashes the whole script +### 4. `td.tdAttributeError` crashes the whole script — use defensive access -If you do `node.par.nonexistent = value`, TD raises `tdAttributeError` and **stops the entire script**. There's no way to catch it with try/except in some TD versions. Always check with `getattr` first or use `safe_par()`. +If you do `node.par.nonexistent = value`, TD raises `tdAttributeError` and stops the entire script. Prevention is better than catching: +- Use `op()` instead of `opex()` — `op()` returns None on failure, `opex()` raises +- Use `hasattr(node.par, 'name')` before accessing any parameter +- Use `getattr(node.par, 'name', None)` with a default +- Use the `safe_par()` pattern from pitfall #3 + +```python +# WRONG — crashes if param doesn't exist: +node.par.nonexistent = value + +# CORRECT — defensive access: +if hasattr(node.par, 'nonexistent'): + node.par.nonexistent = value +``` + +### 5. `outputresolution` is a string menu, not an integer + +``` +menuNames: ['useinput','eighth','quarter','half','2x','4x','8x','fit','limit','custom','parpanel'] +``` +Always use the string form. Setting `outputresolution = 9` may silently fail. +```python +node.par.outputresolution = 'custom' # correct +node.par.resolutionw = 1280; node.par.resolutionh = 720 +``` +Discover valid values: `list(node.par.outputresolution.menuNames)` ## GLSL Shaders -### 12. `uTDCurrentTime` does NOT exist in TD 099 +### 6. `uTDCurrentTime` does NOT exist in GLSL TOP -The GLSL builtin for time was removed or never existed in some builds. Feed time via a 1×1 Constant TOP input. **CRITICAL: set format to `rgba32float`** — the default 8-bit format clamps values to 0-1, so `absTime.seconds % 1000.0` gets clamped and the GLSL shader sees a frozen time value of 1.0: +There is NO built-in time uniform for GLSL TOPs. GLSL MAT has `uTDGeneral.seconds` but that's NOT available in GLSL TOP context. + +**PRIMARY — GLSL TOP Vectors/Values page:** +```python +gl.par.value0name = 'uTime' +gl.par.value0.expr = "absTime.seconds" +# In GLSL: uniform float uTime; +``` + +**FALLBACK — Constant TOP texture (for complex time data):** + +CRITICAL: set format to `rgba32float` — default 8-bit clamps to 0-1: ```python t = root.create(constantTOP, 'time_driver') -t.par.format = 'rgba32float' # ← REQUIRED! Without this, time is stuck at 1.0 +t.par.format = 'rgba32float' t.par.outputresolution = 'custom' -t.par.resolutionw = 1 -t.par.resolutionh = 1 +t.par.resolutionw = 1; t.par.resolutionh = 1 t.par.colorr.expr = "absTime.seconds % 1000.0" -t.par.colorg.expr = "int(absTime.seconds / 1000.0)" t.outputConnectors[0].connect(glsl.inputConnectors[0]) ``` -In GLSL: -```glsl -vec4 td = texture(sTD2DInputs[0], vec2(0.5)); -float t = td.r + td.g * 1000.0; -``` -### 13. GLSL compile errors are silent in the API +### 7. GLSL compile errors are silent in the API The GLSL TOP shows a yellow warning triangle in the UI but `node.errors()` may return empty string. Check `node.warnings()` too, and create an Info DAT pointed at the GLSL TOP to read the actual compiler output. -### 14. TD GLSL uses `vUV.st` not `gl_FragCoord` +### 8. TD GLSL uses `vUV.st` not `gl_FragCoord` — and REQUIRES `TDOutputSwizzle()` on macOS Standard GLSL patterns don't work. TD provides: - `vUV.st` — UV coordinates (0-1) @@ -146,9 +106,26 @@ Standard GLSL patterns don't work. TD provides: - `sTD2DInputs[0]` — input textures - `layout(location = 0) out vec4 fragColor` — output +CRITICAL on macOS: Always wrap output with `TDOutputSwizzle()`: +```glsl +fragColor = TDOutputSwizzle(color); +``` +TD uses GLSL 4.60 (Vulkan backend). GLSL 3.30 and earlier removed. + +### 9. Large GLSL shaders — write to temp file + +GLSL code with special characters can corrupt JSON payloads. Write the shader to a temp file and load it in TD: +```python +# Agent side: write shader to /tmp/shader.glsl via write_file +# TD side: +sd = root.create(textDAT, 'shader_code') +with open('/tmp/shader.glsl', 'r') as f: + sd.text = f.read() +``` + ## Node Management -### 15. Destroying nodes while iterating `root.children` causes `tdError` +### 10. Destroying nodes while iterating `root.children` causes `tdError` The iterator is invalidated when a child is destroyed. Always snapshot first: ```python @@ -158,9 +135,34 @@ for child in kids: child.destroy() ``` -### 16. Feedback TOP: use `top` parameter, NOT direct input wire +### 10b. Split cleanup and creation into SEPARATE td_execute_python calls -In TD 099, the feedbackTOP's `top` parameter references which TOP to delay. **Do not also wire that TOP directly into the feedback's input** — this creates a real cook dependency loop (warning flood, potential crash). The "Not enough sources" error on feedbackTOP is benign and resolves after a few frames of playback. +Creating nodes with the same names you just destroyed in the SAME script causes "Invalid OP object" errors — even with `list()` snapshot. TD's internal references can go stale within one execution context. + +**WRONG (single call):** +```python +# td_execute_python: +for c in list(root.children): + if c.valid and c.name.startswith('promo_'): + c.destroy() +# ... then create promo_audio, promo_shader etc. in same script → CRASHES +``` + +**CORRECT (two separate calls):** +```python +# Call 1: td_execute_python — clean only +for c in list(root.children): + if c.valid and c.name.startswith('promo_'): + c.destroy() + +# Call 2: td_execute_python — build (separate MCP call) +audio = root.create(audiofileinCHOP, 'promo_audio') +# ... rest of build +``` + +### 11. Feedback TOP: use `top` parameter, NOT direct input wire + +The feedbackTOP's `top` parameter references which TOP to delay. Do NOT also wire that TOP directly into the feedback's input — this creates a real cook dependency loop. Correct setup: ```python @@ -169,23 +171,173 @@ fb.par.top = comp.path # reference only — no wire to fb input fb.outputConnectors[0].connect(xf) # fb output -> transform -> fade -> comp ``` -The resulting "Cook dependency loop detected" **warning** on the transform/fade chain is expected and correct — that's what feedback loops do. It's informational, not an error. +The "Cook dependency loop detected" warning on the transform/fade chain is expected. -### 16. GLSL TOP auto-creates companion nodes +### 12. GLSL TOP auto-creates companion nodes -Creating a `glslTOP` also creates `name_pixel` (Text DAT), `name_info` (Info DAT), and `name_compute` (Text DAT). These are visible in the network and count toward node totals. Don't be alarmed by "extra" nodes. +Creating a `glslTOP` also creates `name_pixel` (Text DAT), `name_info` (Info DAT), and `name_compute` (Text DAT). These are visible in the network. Don't be alarmed by "extra" nodes. -### 17. The default project root is `/project1` +### 13. The default project root is `/project1` New TD files start with `/project1` as the main container. System nodes live at `/`, `/ui`, `/sys`, `/local`, `/perform`. Don't create user nodes outside `/project1`. -### 18. `open -R` reveals the file but can't automate the drag +### 14. Non-Commercial license caps resolution at 1280x1280 -Use `open -R /path/to/file.tox` to open Finder highlighting the file. The user must then drag it into TD manually. No AppleScript workaround exists on modern macOS due to accessibility restrictions. +Setting `resolutionw=1920` silently clamps to 1280. Always check effective resolution after creation: +```python +n.cook(force=True) +actual = str(n.width) + 'x' + str(n.height) +``` + +## Recording & Codecs + +### 15. MovieFileOut TOP: H.264/H.265/AV1 requires Commercial license + +In Non-Commercial TD, these codecs produce an error. Recommended alternatives: +- `prores` — Apple ProRes, **best on macOS**, HW accelerated, NOT license-restricted. ~55MB/s at 1280x720 but lossless quality. **Use this as default on macOS.** +- `cineform` — GoPro Cineform, supports alpha +- `hap` — GPU-accelerated playback, large files +- `notchlc` — GPU-accelerated, good quality +- `mjpa` — Motion JPEG, legacy fallback (lossy, use only if ProRes unavailable) + +For image sequences: `rec.par.type = 'imagesequence'`, `rec.par.imagefiletype = 'png'` + +### 16. MovieFileOut `.record()` method may not exist + +Use the toggle parameter instead: +```python +rec.par.record = True # start recording +rec.par.record = False # stop recording +``` + +When setting file path and starting recording in the same script, use delayFrames: +```python +rec.par.file = '/tmp/new_output.mov' +run("op('/project1/recorder').par.record = True", delayFrames=2) +``` + +### 17. TOP.save() captures same frame when called rapidly + +Use MovieFileOut for real-time recording. Set `project.realTime = False` for frame-accurate output. + +### 18. AudioFileIn CHOP: cue and recording sequence matters + +The recording sequence must be done in exact order, or the recording will be empty, audio will start mid-file, or the file won't be written. + +**Proven recording sequence:** + +```python +# Step 1: Stop any existing recording +rec.par.record = False + +# Step 2: Reset audio to beginning +audio.par.play = False +audio.par.cue = True +audio.par.cuepoint = 0 # may need cuepointunit=0 too +# Verify: audio.par.cue.eval() should be True + +# Step 3: Set output file path +rec.par.file = '/tmp/output.mov' + +# Step 4: Release cue + start playing + start recording (with frame delay) +audio.par.cue = False +audio.par.play = True +audio.par.playmode = 2 # Sequential — plays once through +run("op('/project1/recorder').par.record = True", delayFrames=3) +``` + +**Why each step matters:** +- `rec.par.record = False` first — if a previous recording is active, setting `par.file` may fail silently +- `audio.par.cue = True` + `cuepoint = 0` — guarantees audio starts from the beginning, otherwise the spectrum may be silent for the first few seconds +- `delayFrames=3` on the record start — setting `par.file` and `par.record = True` in the same script can race; the file path needs a frame to register before recording starts +- `playmode = 2` (Sequential) — plays the file once. Use `playmode = 0` (Locked to Timeline) if you want TD's timeline to control position + +## TD Python API Patterns + +### 19. COMP extension setup: ext0object format is CRITICAL + +`ext0object` expects a CONSTANT string (NOT expression mode): +```python +comp.par.ext0object = "op('./myExtensionDat').module.MyClassName(me)" +``` +NEVER set as just the DAT name. NEVER use ParMode.EXPRESSION. ALWAYS ensure the DAT has `par.language='python'`. + +### 20. td.Panel is NOT subscriptable — use attribute access + +```python +comp.panel.select # correct (attribute access, returns float) +comp.panel['select'] # WRONG — 'td.Panel' object is not subscriptable +``` + +### 21. ALWAYS use relative paths in script callbacks + +In scriptTOP/CHOP/SOP/DAT callbacks, use paths relative to `scriptOp` or `me`: +```python +root = scriptOp.parent().parent() +dat = root.op('pixel_data') +``` +NEVER hardcode absolute paths like `op('/project1/myComp/child')` — they break when containers are renamed or copied. + +### 22. keyboardinCHOP channel names have 'k' prefix + +Channel names are `kup`, `kdown`, `kleft`, `kright`, `ka`, `kb`, etc. — NOT `up`, `down`, `a`, `b`. Always verify with: +```python +channels = [c.name for c in op('/project1/keyboard1').chans()] +``` + +### 23. expressCHOP cook-only properties — false positive errors + +`me.inputVal`, `me.chanIndex`, `me.sampleIndex` work ONLY in cook-context. Calling `par.expr0expr.eval()` from outside always raises an error — this is NOT a real operator error. Ignore these in error scans. + +### 24. td.Vertex attributes — use index access not named attributes + +In TD 2025.32, `td.Vertex` objects do NOT have `.x`, `.y`, `.z` attributes: +```python +# WRONG — crashes: +vertex.x, vertex.y, vertex.z + +# CORRECT — index-based: +vertex.point.P[0], vertex.point.P[1], vertex.point.P[2] +# Or for SOP point positions: +pt = sop.points()[i] +pos = pt.P # use P[0], P[1], P[2] +``` + +## Audio + +### 25. Audio Spectrum CHOP output is weak — boost it + +Raw output is very small (0.001-0.05). Use built-in boost: `spectrum.par.highfrequencyboost = 3.0` + +If still weak, add Math CHOP in Range mode: `fromrangehi=0.05, torangehi=1.0` + +### 26. AudioSpectrum CHOP: timeslice and sample count are the #1 gotcha + +AudioSpectrum at 44100Hz with `timeslice=False` outputs the ENTIRE audio file as samples (~24000+). CHOP-to-TOP then exceeds texture resolution max and warns/fails. + +**Fix:** Keep `timeslice = True` (default) for real-time per-frame FFT. Set `fftsize` to control bin count (it's a STRING enum: `'256'` not `256`). + +If the CHOP-to-TOP still gets too many samples, set `layout = 'rowscropped'` on the choptoTOP. + +```python +spectrum.par.fftsize = '256' # STRING, not int — enum values +spectrum.par.timeslice = True # MUST be True for real-time audio reactivity +spectex.par.layout = 'rowscropped' # handles oversized CHOP inputs +``` + +**resampleCHOP has NO `numsamples` param.** It uses `rate`, `start`, `end`, `method`. Don't guess — always `td_get_par_info('resampleCHOP')` first. + +### 27. CHOP To TOP has NO input connectors — use par.chop reference + +```python +spec_tex = root.create(choptoTOP, 'spectrum_tex') +spec_tex.par.chop = resample # correct: parameter reference +# NOT: resample.outputConnectors[0].connect(spec_tex.inputConnectors[0]) # WRONG +``` ## Workflow -### 19. Always verify after building — errors are silent +### 28. Always verify after building — errors are silent Node errors and broken connections produce no output. Always check: ```python @@ -196,141 +348,161 @@ for c in list(root.children): if w: print(c.name, 'WARN:', w) ``` -### 20. Build in one big `execute_python_script` call, not many small ones +### 29. Window COMP param for display target is `winop` -Each API round-trip adds latency. Bundle node creation + parameter setting + wiring into a single script that creates everything at once, then verify in one final call. - -### 21. Window COMP param for display target is `winop` (not `top` or `window`) - -To display output in a separate window: ```python win = root.create(windowCOMP, 'display') -win.par.winop = '/project1/logo_out' # ← this is the correct param +win.par.winop = '/project1/logo_out' win.par.winw = 1280; win.par.winh = 720 -win.par.winopen.pulse() # open the window +win.par.winopen.pulse() ``` -### 22. Save the project to make API persistent across TD restarts +### 30. `sample()` returns frozen pixels in rapid calls -After deploying the custom API handler, save the project: +`out.sample(x, y)` returns pixels from a single cook snapshot. Compare samples with 2+ second delays, or use screencapture on the display window. + +### 31. Audio-reactive GLSL: dual-layer sync pipeline + +For audio-synced visuals, use BOTH layers for maximum effect: + +**Layer 1 (TD-side, real-time):** AudioFileIn → AudioSpectrum(timeslice=True, fftsize='256') → Math(gain=5) → choptoTOP(par.chop=math, layout='rowscropped') → GLSL input. The shader samples `sTD2DInputs[1]` at different x positions for bass/mid/hi. Record the TD output with MovieFileOut. + +**Layer 2 (Python-side, post-hoc):** scipy FFT on the SAME audio file → per-frame features (rms, bass, mid, hi, beat detection) → drive ASCII brightness, chromatic aberration, beat flashes during the render pass. + +Both layers locked to the same audio file = visuals genuinely sync to the beat at two independent stages. + +**Key gotcha:** AudioFileIn must be cued (`par.cue=True` → `par.cuepulse.pulse()`) then uncued (`par.cue=False`, `par.play=True`) before recording starts. Otherwise the spectrum is silent for the first few seconds. + +### 32. twozero MCP: benchmark and prefer native tools + +Benchmarked April 2026: twozero MCP with 36 native tools. The old curl/REST method (port 9981) had zero native tools. + +**Always prefer native MCP tools over td_execute_python:** +- `td_create_operator` over `root.create()` scripts (handles viewport positioning) +- `td_set_operator_pars` over `node.par.X = Y` scripts (validates param names) +- `td_get_par_info` over temp-node discovery dance (instant, no cleanup) +- `td_get_errors` over manual `c.errors()` loops +- `td_get_focus` for context awareness (no equivalent in old method) + +Only fall back to `td_execute_python` for multi-step logic (wiring chains, conditional builds, loops). + +### 33. twozero td_execute_python response wrapping + +twozero wraps `td_execute_python` responses with status info: `(ok)\n\n[fps 60.0/60] [0 err/0 warn]`. Your Python `result` variable value may not appear verbatim in the response text. If you need to check results programmatically, use `print()` statements in the script — they appear in the response. Don't rely on string-matching the `result` dict. + +### 34. Audio-reactive chain: DO NOT use Lag CHOP or Filter CHOP for spectrum smoothing + +The Derivative docs and tutorials suggest using Lag CHOP (lag1=0.2, lag2=0.5) to smooth raw FFT output before passing to a shader. **This does NOT work with AudioSpectrum → CHOP to TOP → GLSL.** + +What happens: Lag CHOP operates in timeslice mode. A 256-sample spectrum input gets expanded to 1600-2400 samples. The Lag averaging drives all values to near-zero (~1e-06). The CHOP to TOP produces a 2400x2 texture instead of 256x2. The shader receives effectively zero audio data. + +**The correct chain is: Spectrum(outlength=256) → Math(gain=10) → CHOPtoTOP → GLSL.** No CHOP smoothing at all. If you need smoothing, do it in the GLSL shader via temporal lerp with a feedback texture. + +Verified values with audio playing: +- Without Lag CHOP: bass bins = 5.0-5.4, mid bins = 1.0-1.7 (strong, usable) +- With Lag CHOP: ALL bins = 0.000001-0.00004 (dead, zero audio reactivity) + +### 35. AudioSpectrum Output Length: set manually to avoid CHOP to TOP overflow + +AudioSpectrum in Visualization mode with FFT 8192 outputs 22,050 samples by default (1 per Hz, 0–22050). CHOP to TOP cannot handle this — you get "Number of samples exceeded texture resolution max". + +Fix: `spectrum.par.outputmenu = 'setmanually'` and `spectrum.par.outlength = 256`. This gives 256 frequency bins — plenty for visual FFT. + +DO NOT set `timeslice = False` as a workaround — that processes the entire audio file at once and produces even more samples. + +### 36. GLSL spectrum texture from CHOP to TOP is 256x2 not 256x1 + +AudioSpectrum outputs 2 channels (stereo: chan1, chan2). CHOP to TOP with `dataformat='r'` creates a 256x2 texture — one row per channel. Sample the first channel at `y=0.25` (center of first row), NOT `y=0.5` (boundary between rows): + +```glsl +float bass = texture(sTD2DInputs[1], vec2(0.05, 0.25)).r; // correct +float bass = texture(sTD2DInputs[1], vec2(0.05, 0.5)).r; // WRONG — samples between rows +``` + +### 37. FPS=0 doesn't mean ops aren't cooking — check play state + +TD can show `fps:0` in `td_get_perf` while ops still cook and `TOP.save()` still produces valid screenshots. The two most common causes: + +**a) Project is paused (playbar stopped).** TD's playbar can be toggled with spacebar. The `root` at `/` has no `.playbar` attribute (it's on the perform COMP). The easiest fix is sending a spacebar keypress via `td_input_execute`, though this tool can sometimes error. As a workaround, `TOP.save()` always works regardless of play state — use it to verify rendering is actually happening before spending time debugging FPS. + +**b) Audio device CHOP blocking the main thread.** An `audiooutCHOP` with an active audio device can consume 300-400ms/s (2000%+ of frame budget), stalling the cook loop at FPS=0. Fix: keep the CHOP active but set `volume=0` to prevent the audio driver from blocking. Disabling it entirely (`active=False`) may also work but can prevent downstream audio processing CHOPs from cooking. + +Diagnostic sequence when FPS=0: +1. `td_get_perf` — check if any op has extreme CPU/s +2. `TOP.save()` on the output — if it produces a valid image, the pipeline works, just not at real-time rate +3. Check for blocking CHOPs (audioout, audiodevin, etc.) +4. Toggle play state (spacebar, or check if absTime.seconds is advancing) + +### 38. Recording while FPS=0 produces empty or near-empty files + +This is the #1 cause of "I recorded for 30 seconds but got a 2-frame video." If TD's cook loop is stalled (FPS=0 or very low), MovieFileOut has nothing to record. Unlike `TOP.save()` which captures the last cooked frame regardless, MovieFileOut only writes frames that actually cook. + +**Always verify FPS before starting a recording:** ```python -td_exec("project.save(os.path.expanduser('~/Documents/HermesAgent.toe'))") +# Check via td_get_perf first +# If FPS < 30, do NOT start recording — fix the performance issue first +# If FPS=0, the playbar is likely paused — see pitfall #37 ``` -TD auto-opens the last saved project on launch. The API handler is now baked into the .toe file — next time TD opens, port 9981 is live with zero manual steps. To explicitly launch with this project: `open /Applications/TouchDesigner.app ~/Documents/HermesAgent.toe` -### 23. `sample()` returns frozen pixels when called from WebServer DAT callback +Common causes of recording empty video: +- Playbar paused (FPS=0) — see pitfall #37 +- Audio device CHOP blocking the main thread — see pitfall #37b +- Recording started before audio was cued — audio is silent, GLSL outputs black, MovieFileOut records black frames that look empty +- `par.file` set in the same script as `par.record = True` — see pitfall #18 -`out.sample(x, y)` called from inside the API handler's `exec()` returns pixels from a single cook snapshot. It does NOT update between multiple API calls in quick succession. To verify animation is working, either: -- Compare samples with a 2+ second delay between separate `td_exec()` calls -- Use `screencapture` on the display window -- Check `absTime.seconds` is advancing and shader uses time correctly +### 39. GLSL shader produces black output — test before committing to a long render -### 22. `outputresolution` is a string menu, not an integer +New GLSL shaders can fail silently (see pitfall #7). Before recording a long take, always: -### 25. MovieFileOut TOP: H.264/H.265 requires Commercial license +1. **Write a minimal test shader first** that just outputs a solid color or pass-through: +```glsl +void main() { + vec2 uv = vUV.st; + fragColor = TDOutputSwizzle(vec4(uv, 0.0, 1.0)); +} +``` -In Non-Commercial TD 099, encoding with H.264 or H.265 produces an error: "GPU Accelerated H.264/H.265 Encoding requires a Commercial license". Use Motion JPEG instead: +2. **Verify the test renders correctly** via `td_get_screenshot` on the GLSL TOP's output. + +3. **Swap in the real shader** and screenshot again immediately. If black, the shader has a compile error or logic issue. + +4. **Only then start recording.** A 90-second ProRes recording is ~5GB. Recording black frames wastes disk and time. + +Common causes of black GLSL output: +- Missing `TDOutputSwizzle()` on macOS (pitfall #8) +- Time uniform not connected — shader uses default 0.0, fractal stays at origin +- Spectrum texture not connected — audio values all 0.0, driving everything to black +- Integer division where float division was expected (`1/2 = 0` not `0.5`) +- `absTime.seconds % 1000.0` rolled over past 1000 and the modulo produces unexpected values + +### 40. td_write_dat uses `text` parameter, NOT `content` + +The MCP tool `td_write_dat` expects a `text` parameter for full replacement. Passing `content` returns an error: `"Provide either 'text' for full replace, or 'old_text'+'new_text' for patching"`. + +If `td_write_dat` fails, fall back to `td_execute_python`: ```python -rec = root.create(moviefileoutTOP, 'recorder') -rec.par.type = 'movie' -rec.par.file = '/tmp/output.mov' -rec.par.videocodec = 'mjpa' # Motion JPEG — works on Non-Commercial +op("/project1/shader_code").text = shader_string ``` -For image sequences, use `type = 'imagesequence'` and the file param **must** use `me.fileSuffix`: +### 41. td_execute_python does NOT return stdout or print() output + +Despite what earlier versions of pitfall #33 stated, `print()` and `debug()` output from `td_execute_python` scripts does NOT appear in the MCP response. The response is always just `(ok)` + FPS/error summary. To read values back, use dedicated inspection tools (`td_get_operator_info`, `td_read_dat`, `td_read_chop`) instead of trying to print from within a script. + +### 42. td_get_operator_info JSON is appended with `[fps X.X/X]` — breaks json.loads() + +The response text from `td_get_operator_info` has `[fps 60.0/60]` appended after the JSON object. This causes `json.loads()` to fail with "Extra data" errors. Strip it before parsing: ```python -rec.par.type = 'imagesequence' -rec.par.imagefiletype = 'png' -rec.par.file.expr = "'/tmp/frames/out' + me.fileSuffix" +clean = response_text.rsplit('[fps', 1)[0] +data = json.loads(clean) ``` -### 26. MovieFileOut `.record()` method may not exist +### 43. td_get_screenshot is asynchronous — returns `{"status": "pending"}` -In TD 099, there is no `.record()` method on moviefileoutTOP. Use the toggle parameter instead: -```python -rec.par.record = True # start recording -rec.par.record = False # stop recording -``` - -When setting the file path and starting recording in the same script, use `run()` with `delayFrames` to avoid a race condition where the old filename is used: -```python -rec.par.file = '/tmp/new_output.mov' -run("op('/project1/recorder').par.record = True", delayFrames=2) -``` - -### 27. TOP.save() captures same frame when called rapidly - -`op('null1').save(path)` captures the current GPU texture at call time. When called multiple times in a single script (or rapid API calls), TD doesn't cook new frames between saves — all exported PNGs will be identical. To get unique frames, use the MovieFileOut TOP which records in real-time from TD's cook cycle. - -### 28. AudioFileIn CHOP: cue before recording for sync - -When recording audio-reactive visuals, always cue the audio to the start before beginning the recording. Otherwise the visuals are synced to wherever the audio happens to be in its playback: -```python -op('/project1/audio_in').par.cue.pulse() # reset to start -run("op('/project1/recorder').par.record = True", delayFrames=3) -``` -The audio plays via `playmode=0` (Locked to Timeline), so it stays in sync with TD's frame clock. Use `audiodeviceoutCHOP` to hear the audio during recording. - -### 29. Audio Spectrum CHOP output is weak — boost with Math CHOP - -The raw AudioSpectrum CHOP output has very small values (often 0.001-0.05 range). When fed directly to CHOP To TOP → GLSL, the shader barely reacts. Always insert a Math CHOP with `gain=5` (or higher) between the spectrum and the CHOP To TOP to get usable 0-1 range values in the shader. - -### 30. CHOP To TOP texture size — Resample to 256 first - -`choptoTOP` creates a texture where width = number of samples. An AudioSpectrum CHOP at 44100Hz has ~24000 samples — creating a 24000×1 texture is wasteful. Use a Resample CHOP set to 256 or 512 samples before the CHOP To TOP for an efficient spectrum texture. - -### 31. CHOP To TOP has NO input connectors — use par.chop reference - -`choptoTOP` does NOT have input connectors. `resample.outputConnectors[0].connect(chop_to_top.inputConnectors[0])` silently does nothing. Use the `chop` parameter instead: -```python -spec_tex = root.create(choptoTOP, 'spectrum_tex') -spec_tex.par.chop = resample # ← correct: parameter reference -# NOT: resample.outputConnectors[0].connect(spec_tex.inputConnectors[0]) # ← WRONG: no connectors -``` - -### 22. `outputresolution` is a string menu, not an integer - -The `outputresolution` param is a menu with string values: -``` -menuNames: ['useinput','eighth','quarter','half','2x','4x','8x','fit','limit','custom','parpanel'] -``` -Always use the string form. Setting `outputresolution = 9` may silently fail. -```python -node.par.outputresolution = 'custom' # ✓ correct -node.par.resolutionw = 1280; node.par.resolutionh = 720 -``` -Discover valid values: `list(node.par.outputresolution.menuNames)` - -### 23. Large GLSL shaders break curl JSON escaping - -GLSL code full of single/double quotes, backslashes, and special chars will corrupt the JSON payload when sent via `curl -d`. **Write the shader to a temp file and load it in TD:** -```python -# Agent side: write shader to /tmp/shader.glsl via write_file -# TD side (via td_exec): -sd = root.create(textDAT, 'shader_code') -with open('/tmp/shader.glsl', 'r') as f: - sd.text = f.read() -``` -This avoids all escaping issues. The TD Python environment has full filesystem access. - -### 24. TD crashes lose everything — the WebServer DAT must be re-deployed - -If TD crashes (common with heavy GLSL or rapid-fire API calls), all nodes including the WebServer DAT are lost. On relaunch, port 9981 is dead. Recovery: -1. Detect: `curl` returns exit code 7 (connection refused) or `lsof -i :9981` shows nothing -2. Check: `pgrep TouchDesigner` to confirm TD is running -3. Re-deploy: user must paste `exec(open('...custom_api_handler.py').read())` into Textport again -4. Verify: poll port 9981 until API responds - -The `td_exec()` helper should handle this gracefully: -```python -def td_exec(script): - escaped = json.dumps({"script": script}) - cmd = f"curl -s --max-time 15 -X POST -H 'Content-Type: application/json' -d {shlex.quote(escaped)} 'http://127.0.0.1:9981/api/td/server/exec'" - r = terminal(cmd, timeout=20) - if r.get('exit_code') == 7: - return {'error': 'TD not responding — WebServer DAT may need re-deploy'} - try: - return json.loads(r['output']) - except: - return {'error': 'Bad response', 'raw': r['output'][:200]} +Screenshots don't complete instantly. The tool returns `{"status": "pending", "requestId": "..."}` and the actual file appears later. Wait a few seconds before checking for the file. There is no callback or completion notification — poll the filesystem. + +### 44. Recording duration is manual — no auto-stop at audio end + +MovieFileOut records until `par.record = False` is set. If audio ends before you stop recording, the file keeps growing with repeated frames. Always stop recording promptly after the audio duration. For precision: set a timer on the agent side matching the audio length, then send `par.record = False`. Trim excess with ffmpeg as a safety net: +```bash +ffmpeg -i raw.mov -t 25 -c copy trimmed.mov ``` diff --git a/skills/creative/touchdesigner/references/python-api.md b/skills/creative/touchdesigner/references/python-api.md index 2b8d8847f..f2955110b 100644 --- a/skills/creative/touchdesigner/references/python-api.md +++ b/skills/creative/touchdesigner/references/python-api.md @@ -7,7 +7,7 @@ TouchDesigner's Python environment auto-imports the `td` module. All TD-specific When using the MCP `execute_python_script` tool, these globals are pre-loaded: - `op` — shortcut for `td.op()`, finds operators by path - `ops` — shortcut for `td.ops()`, finds multiple operators by pattern -- `me` — the operator running the script (not meaningful via MCP — will be the WebServer DAT) +- `me` — the operator running the script (via MCP this is the twozero internal executor) - `parent` — shortcut for `me.parent()` - `project` — the root project component - `td` — the full td module @@ -432,7 +432,7 @@ for path, params in settings.items(): ## Python Version and Packages -TouchDesigner bundles Python 3.11+ (as of TD 2024) with these pre-installed: +TouchDesigner bundles Python 3.11+ with these pre-installed: - **numpy** — array operations, fast math - **scipy** — signal processing, FFT - **OpenCV** (cv2) — computer vision @@ -440,4 +440,24 @@ TouchDesigner bundles Python 3.11+ (as of TD 2024) with these pre-installed: - **requests** — HTTP client - **json**, **re**, **os**, **sys** — standard library +**IMPORTANT:** Parameter names in examples below are illustrative. Always run discovery (SKILL.md Step 0) to get actual names for your TD version. Do NOT copy param names from these examples verbatim. + Custom packages can be installed to TD's Python site-packages directory. See TD documentation for the exact path per platform. + +## SOP Vertex/Point Access (TD 2025.32) + +In TD 2025.32, `td.Vertex` does NOT have `.x`, `.y`, `.z` attributes. Use index access: + +```python +# WRONG — crashes in TD 2025.32: +vertex.x, vertex.y, vertex.z + +# CORRECT — index/attribute access: +pt = sop.points()[i] +pos = pt.P # Position object +x, y, z = pos[0], pos[1], pos[2] + +# Always introspect first: +dir(sop.points()[0]) # see what attributes actually exist +dir(sop.points()[0].P) # see Position object interface +``` diff --git a/skills/creative/touchdesigner/references/troubleshooting.md b/skills/creative/touchdesigner/references/troubleshooting.md index 30ad580f4..c9817ebe0 100644 --- a/skills/creative/touchdesigner/references/troubleshooting.md +++ b/skills/creative/touchdesigner/references/troubleshooting.md @@ -1,274 +1,244 @@ -# TouchDesigner Troubleshooting +# TouchDesigner Troubleshooting (twozero MCP) > See `references/pitfalls.md` for the comprehensive lessons-learned list. -## Quick Connection Diagnostic +## 1. Connection Issues -```bash -lsof -i :9981 -P -n | grep LISTEN # Step 1: Is TD listening? -curl -s http://127.0.0.1:9981/api/td/server/td # Step 2: API working? +### Port 40404 not responding + +Check these in order: + +1. Is TouchDesigner running? + ```bash + pgrep TouchDesigner + ``` + +1b. Quick hub health check (no JSON-RPC needed): + A plain GET to the MCP URL returns instance info: + ``` + curl -s http://localhost:40404/mcp + ``` + Returns: `{"hub": true, "pid": ..., "instances": {"127.0.0.1_PID": {"project": "...", "tdVersion": "...", ...}}}` + If this returns JSON but `instances` is empty, TD is running but twozero hasn't registered yet. + +2. Is twozero installed in TD? + Open TD Palette Browser > twozero should be listed. If not, install it. + +3. Is MCP enabled in twozero settings? + In TD, open twozero preferences and confirm MCP server is toggled ON. + +4. Test the port directly: + ```bash + nc -z 127.0.0.1 40404 + ``` + +5. Test the MCP endpoint: + ```bash + curl -s http://localhost:40404/mcp + ``` + Should return JSON with hub info. If it does, the server is running. + +### Hub responds but no TD instances + +The twozero MCP hub is running but TD hasn't registered. Causes: +- TD project not loaded yet (still on splash screen) +- twozero COMP not initialized in the current project +- twozero version mismatch + +Fix: Open/reload a TD project that contains the twozero COMP. Use td_list_instances +to check which TD instances are registered. + +### Multi-instance setup + +twozero auto-assigns ports for multiple TD instances: +- First instance: 40404 +- Second instance: 40405 +- Third instance: 40406 +- etc. + +Use `td_list_instances` to discover all running instances and their ports. + +## 2. MCP Tool Errors + +### td_execute_python returns error + +The error message from td_execute_python often contains the Python traceback. +If it's unclear, use `td_read_textport` to see the full TD console output — +Python exceptions are always printed there. + +Common causes: +- Syntax error in the script +- Referencing a node that doesn't exist (op() returns None, then you call .par on None) +- Using wrong parameter names (see pitfalls.md) + +### td_set_operator_pars fails + +Parameter name mismatch is the #1 cause. The tool validates param names and +returns clear errors, but you must use exact names. + +Fix: ALWAYS call `td_get_par_info` first to discover the real parameter names: +``` +td_get_par_info(op_type='glslTOP') +td_get_par_info(op_type='noiseTOP') ``` -| Symptom | Cause | Fix | -|---------|-------|-----| -| Connection refused | No WebServer DAT | Deploy `scripts/custom_api_handler.py` in TD Textport | -| HTTP 404 on all routes | .tox module import failed | Deploy custom handler (pitfalls #1-2) | -| HTTP 200, empty body | Response in wrong key | Handler uses `response['data']` not `response['body']` (pitfalls #6) | -| HTTP 200, JSON body | Working | Proceed to discovery | -| MCP tools not callable | Normal — use curl instead | `td_exec()` pattern in SKILL.md works without MCP | +### td_create_operator type name errors -## Node Creation Issues +Operator type names use camelCase with family suffix: +- CORRECT: noiseTOP, glslTOP, levelTOP, compositeTOP, audiospectrumCHOP +- WRONG: NoiseTOP, noise_top, NOISE TOP, Noise + +### td_get_operator_info for deep inspection + +If unsure about any aspect of an operator (params, inputs, outputs, state): +``` +td_get_operator_info(path='/project1/noise1', detail='full') +``` + +## 3. Parameter Discovery + +CRITICAL: ALWAYS use td_get_par_info to discover parameter names. + +The agent's LLM training data contains WRONG parameter names for TouchDesigner. +Do not trust them. Known wrong names include dat vs pixeldat, colora vs alpha, +sizex vs size, and many more. See pitfalls.md for the full list. + +Workflow: +1. td_get_par_info(op_type='glslTOP') — get all params for a type +2. td_get_operator_info(path='/project1/mynode', detail='full') — get params for a specific instance +3. Use ONLY the names returned by these tools + +## 4. Performance + +### Diagnosing slow performance + +Use `td_get_perf` to see which operators are slow. Look at cook times — +anything over 1ms per frame is worth investigating. + +Common causes: +- Resolution too high (especially on Non-Commercial) +- Complex GLSL shaders +- Too many TOP-to-CHOP or CHOP-to-TOP transfers (GPU-CPU memory copies) +- Feedback loops without decay (values accumulate, memory grows) + +### Non-Commercial license restrictions + +- Resolution cap: 1280x1280. Setting resolutionw=1920 silently clamps to 1280. +- H.264/H.265/AV1 encoding requires Commercial license. Use ProRes or Hap instead. +- No commercial use of output. + +Always check effective resolution after creation: +```python +n.cook(force=True) +actual = str(n.width) + 'x' + str(n.height) +``` + +## 5. Hermes Configuration + +### Config location + +~/.hermes/config.yaml + +### MCP entry format + +The twozero TD entry should look like: +```yaml +mcpServers: + twozero_td: + url: http://localhost:40404/mcp +``` + +### After config changes + +Restart the Hermes session for changes to take effect. The MCP connection is +established at session startup. + +### Verifying MCP tools are available + +After restarting, the session log should show twozero MCP tools registered. +If tools show as registered but aren't callable, check: +- The twozero MCP hub is still running (curl test above) +- TD is still running with a project loaded +- No firewall blocking localhost:40404 + +## 6. Node Creation Issues ### "Node type not found" error -**Cause:** Wrong `nodeType` string in `create_td_node`. +Wrong type string. Use camelCase with family suffix: +- Wrong: NoiseTop, noise_top, NOISE TOP +- Right: noiseTOP -**Fix:** Use camelCase with family suffix. Common mistakes: -- Wrong: `NoiseTop`, `noise_top`, `NOISE TOP`, `Noise` -- Right: `noiseTop` -- Wrong: `AudioSpectrum`, `audio_spectrum_chop` -- Right: `audiospectrumChop` +### Node created but not visible -**Discovery method:** Use `get_td_classes` to see available types, or `execute_python_script` with `dir(td)` filtered for operator classes. - -### Node created but not visible in TD - -**Cause:** Node was created in a different container than expected, or TD viewport is looking at a different network. - -**Fix:** Check `parentPath` — use absolute paths like `/project1`. Verify with `get_td_nodes(parentPath="/project1")`. +Check parentPath — use absolute paths like /project1. The default project +root is /project1. System nodes live at /, /ui, /sys, /local, /perform. +Don't create user nodes outside /project1. ### Cannot create node inside a non-COMP -**Cause:** Only COMP operators (Container, Base, Geometry, etc.) can contain child operators. You cannot create nodes inside a TOP, CHOP, SOP, DAT, or MAT. +Only COMP operators (Container, Base, Geometry, etc.) can contain children. +You cannot create nodes inside a TOP, CHOP, SOP, DAT, or MAT. -**Fix:** Create a Container COMP or Base COMP first, then create nodes inside it. +## 7. Wiring Issues -## Parameter Issues +### Cross-family wiring -### Parameter not updating +TOPs connect to TOPs, CHOPs to CHOPs, SOPs to SOPs, DATs to DATs. +Use converter operators to bridge: choptoTOP, topToCHOP, soptoDAT, etc. -**Causes:** -1. **Wrong parameter name.** TD parameter names change across versions. Run the discovery script (SKILL.md Step 0) or use `get_td_node_parameters` to discover exact names for your TD version. Never trust online docs or this skill's tables — always verify. -2. **Parameter is read-only.** Some parameters are computed/locked. -3. **Wrong value type.** Menu parameters need integer index or exact string label. -4. **Parameter has an expression.** If `node.par.X.expr` is set, `.val` is ignored. Clear the expression first. - -**Discovery-based approach (preferred):** +Note: choptoTOP has NO input connectors. Use par.chop reference instead: ```python -execute_python_script(script=""" -n = op('/project1/mynode') -pars = [(p.name, type(p.val).__name__, p.val) for p in n.pars() - if any(k in p.name.lower() for k in ['color', 'size', 'dat', 'font', 'alpha'])] -result = pars -""") +spec_tex.par.chop = resample_node # correct +# NOT: resample.outputConnectors[0].connect(spec_tex.inputConnectors[0]) ``` -**Safe parameter setter pattern:** +### Feedback loops + +Never create A -> B -> A directly. Use a Feedback TOP: ```python -def safe_par(node, name, value): - p = getattr(node.par, name, None) - if p is not None: - p.val = value - return True - return False # param doesn't exist in this TD version +fb = root.create(feedbackTOP, 'fb') +fb.par.top = comp.path # reference only, no wire to fb input +fb.outputConnectors[0].connect(next_node) ``` +"Cook dependency loop detected" warning on the chain is expected and correct. -### Common parameter name gotchas +## 8. GLSL Issues -| What you expect | Actual name | Notes | -|----------------|-------------|-------| -| `width` | `resolutionw` | TOP resolution width | -| `height` | `resolutionh` | TOP resolution height | -| `filepath` | `file` | File path parameter | -| `color` | `colorr`, `colorg`, `colorb`, `colora` | Separate RGBA components | -| `position_x` | `tx` | Translate X | -| `rotation` | `rz` | Rotate Z (2D rotation) | -| `scale` | `sx`, `sy` | Separate X/Y scale | -| `blend_mode` | `operand` | Composite TOP blend mode (integer) | -| `opacity` | `opacity` | On Level TOP (this one is correct!) | +### Shader compilation errors are silent -### Composite TOP operand values +GLSL TOP shows a yellow warning in the UI but node.errors() may return empty. +Check node.warnings() too. Create an Info DAT pointed at the GLSL TOP for +full compiler output. -| Mode | Index | -|------|-------| -| Over | 0 | -| Under | 1 | -| Inside | 2 | -| Add | 3 | -| Subtract | 4 | -| Difference | 5 | -| Multiply | 18 | -| Screen | 27 | -| Maximum | 13 | -| Minimum | 14 | -| Average | 28 | +### TD GLSL specifics -## Connection/Wiring Issues +- Uses GLSL 4.60 (Vulkan backend). GLSL 3.30 and earlier removed. +- UV coordinates: vUV.st (not gl_FragCoord) +- Input textures: sTD2DInputs[0] +- Output: layout(location = 0) out vec4 fragColor +- macOS CRITICAL: Always wrap output with TDOutputSwizzle(color) +- No built-in time uniform. Pass time via GLSL TOP Values page or Constant TOP. -### Connections not working +## 9. Recording Issues -**Causes:** -1. **Cross-family wiring.** TOPs can only connect to TOPs, CHOPs to CHOPs, etc. Use converter operators to bridge families. -2. **Wrong connector index.** Most operators have one output connector (index 0). Multi-output operators may need index 1, 2, etc. -3. **Node path wrong.** Verify paths are absolute and correctly spelled. +### H.264/H.265/AV1 requires Commercial license -**Verify connections:** +Use Apple ProRes on macOS (hardware accelerated, not license-restricted): ```python -execute_python_script(script=""" -node = op('/project1/level1') -result = { - 'inputs': [i.path if i else None for i in node.inputs], - 'outputs': [o.path if o else None for o in node.outputs] -} -""") +rec.par.videocodec = 'prores' # Preferred on macOS — lossless, Non-Commercial OK +# rec.par.videocodec = 'mjpa' # Fallback — lossy, works everywhere ``` -### Feedback loops causing errors +### MovieFileOut has no .record() method -**Symptom:** "Circular dependency" or infinite cook loop. - -**Fix:** Always use a Feedback TOP (or a Null TOP with a one-frame delay) to break the loop: -``` -A -> B -> Feedback(references B) -> A -``` -Never create A -> B -> A directly. - -## Performance Issues - -### Low FPS / choppy output - -**Common causes and fixes:** - -1. **Resolution too high.** Start at 1920x1080, only go higher if GPU handles it. -2. **Too many operators.** Each operator has GPU/CPU overhead. Consolidate where possible. -3. **Expensive shader.** GLSL TOPs with complex math per-pixel drain GPU. Profile with TD's Performance Monitor (F2). -4. **No GPU instancing.** Rendering 1000 separate geometry objects is much slower than 1 instanced geometry. -5. **Unnecessary cooks.** Operators that don't change frame-to-frame still recook if inputs change. Use Null TOPs to cache stable results. -6. **Large texture transfers.** TOP to CHOP and CHOP to TOP involve GPU-CPU memory transfers. Minimize these. - -**Performance Monitor:** +Use the toggle parameter: ```python -execute_python_script(script="td.performanceMonitor = True") -# After testing: -execute_python_script(script="td.performanceMonitor = False") +rec.par.record = True # start +rec.par.record = False # stop ``` -### Memory growing over time +### All exported frames identical -**Causes:** -- Cache TOPs with high `length` value -- Feedback loops without brightness decay (values accumulate) -- Table DATs growing without clearing -- Movie File In loading many unique frames - -**Fix:** Always add slight decay in feedback loops (Level TOP with `opacity=0.98` or multiply blend). Clear tables periodically. - -## Export / Recording Issues - -### Movie File Out not recording - -**Checklist:** -1. Is the `record` parameter toggled on? `update_td_node_parameters(properties={"record": true})` -2. Is an input connected? The Movie File Out needs a TOP input. -3. Is the output path valid and writable? Check `file` parameter. -4. Is the codec available? H.264 (type 4) is most reliable. - -### Exported video is black - -**Causes:** -1. The TOP chain output is all black (brightness too low). -2. The input TOP has errors (check with `get_td_node_errors`). -3. Resolution mismatch — the output may be wrong resolution. - -**Debug:** Check the input TOP's actual pixel values: -```python -execute_python_script(script=""" -import numpy as np -top = op('/project1/out') -arr = top.numpyArray(delayed=True) -result = {'mean': float(arr.mean()), 'max': float(arr.max()), 'shape': list(arr.shape)} -""") -``` - -### .tox export losing connections - -**Note:** When saving a component as .tox, only the component and its internal children are saved. External connections (wires to operators outside the component) are lost. Design self-contained components. - -## Python Scripting Issues - -### execute_python_script returns empty result - -**Causes:** -1. The script used `exec()` semantics (multi-line) but didn't set `result`. -2. The last expression has no return value (e.g., `print()` returns None). - -**Fix:** Explicitly set `result`: -```python -execute_python_script(script=""" -nodes = op('/project1').findChildren(type=TOP) -result = len(nodes) # explicitly set return value -""") -``` - -### Script errors not clear - -**Check stderr in the response.** The MCP server captures both stdout and stderr from script execution. Error tracebacks appear in stderr. - -### Module not found in TD Python - -**Cause:** TD's Python environment may not have the module. TD bundles numpy, scipy, opencv, Pillow, and requests. Other packages need manual installation. - -**Check available packages:** -```python -execute_python_script(script=""" -import sys -result = [p for p in sys.path] -""") -``` - -## Common Workflow Pitfalls - -### Building before verifying connection - -Always call `get_td_info` first. If TD isn't running or the WebServer DAT isn't loaded, all subsequent tool calls will fail. - -### Not checking errors after building - -Always call `get_td_node_errors(nodePath="/project1")` after creating and wiring a network. Broken connections and missing references are silent until you check. - -### Creating too many operators in one go - -When building complex networks, create in logical groups: -1. Create all operators in a section -2. Wire that section -3. Verify with `get_td_node_errors` -4. Move to the next section - -Don't create 50 operators, wire them all, then discover something was wrong 30 operators ago. - -### Parameter expressions vs static values - -If you set `node.par.X.val = 5` but there's an expression on that parameter (`node.par.X.expr`), the expression wins. To use a static value, clear the expression first: -```python -execute_python_script(script=""" -op('/project1/noise1').par.seed.expr = '' # clear expression -op('/project1/noise1').par.seed.val = 42 # now static value works -""") -``` - -### Forgetting to start audio playback - -Audio File In CHOP won't produce data unless `play` is True and a valid `file` is set: -``` -update_td_node_parameters(nodePath="/project1/audio_in", - properties={"file": "/path/to/music.wav", "play": true}) -``` - -### GLSL shader compilation errors - -If a GLSL TOP shows errors after setting shader code: -1. Check the shader code in the Text DAT for syntax errors -2. Ensure the GLSL version is compatible (TD uses GLSL 3.30+) -3. Input sampler name must be `sTD2DInputs[0]` (not custom names) -4. Output must use `layout(location = 0) out vec4 fragColor` -5. UV coordinates come from `vUV.st` (not `gl_FragCoord`) +TOP.save() captures same frame when called rapidly. Use MovieFileOut for +real-time recording. Set project.realTime = False for frame-accurate output. diff --git a/skills/creative/touchdesigner/scripts/custom_api_handler.py b/skills/creative/touchdesigner/scripts/custom_api_handler.py deleted file mode 100644 index fd3772a87..000000000 --- a/skills/creative/touchdesigner/scripts/custom_api_handler.py +++ /dev/null @@ -1,140 +0,0 @@ -""" -Custom API Handler for TouchDesigner WebServer DAT -=================================================== -Use this when mcp_webserver_base.tox fails to load its modules -(common — the .tox relies on relative paths to a modules/ folder -that often break during import). - -Paste into TD Textport or run via exec(open('...').read()): - Creates a WebServer DAT + Text DAT callback handler on port 9981. - Implements the core endpoints the MCP server expects. - -After running, test with: - curl http://127.0.0.1:9981/api/td/server/td -""" - -root = op('/project1') - -# Remove broken webserver if present -old = op('/project1/mcp_webserver_base') -if old and old.valid: - old.destroy() - -# Create WebServer DAT -ws = root.create(webserverDAT, 'api_server') -ws.par.port = 9981 -ws.par.active = True -ws.nodeX = -800; ws.nodeY = 500 - -# Create callback handler -cb = root.create(textDAT, 'api_handler') -cb.nodeX = -800; cb.nodeY = 400 -cb.text = r''' -import json, traceback, io, sys - -def onHTTPRequest(webServerDAT, request, response): - uri = request.get('uri', '') - method = request.get('method', 'GET') - response['statusCode'] = 200 - response['statusReason'] = 'OK' - response['headers'] = {'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*'} - - try: - # TD sends POST body as bytes in request['data'] - raw = request.get('data', request.get('body', '')) - if isinstance(raw, bytes): - raw = raw.decode('utf-8') - body = {} - if raw and isinstance(raw, str) and raw.strip(): - body = json.loads(raw) - pars = request.get('pars', {}) - - if uri == '/api/td/server/td': - response['data'] = json.dumps({ - 'version': str(app.version), - 'osName': sys.platform, - 'apiVersion': '1.4.3', - 'product': 'TouchDesigner' - }) - - elif uri == '/api/td/server/exec': - script = body.get('script', '') - old_stdout = sys.stdout - sys.stdout = buf = io.StringIO() - result_val = None - err_text = '' - try: - globs = {'op': op, 'ops': ops, 'me': webServerDAT, 'parent': parent, - 'project': project, 'td': td, 'result': None, - 'app': app, 'absTime': absTime} - lines = script.strip().split('\n') - if len(lines) == 1: - try: - result_val = eval(script, globs) - except SyntaxError: - exec(script, globs) - result_val = globs.get('result') - else: - exec(script, globs) - result_val = globs.get('result') - except Exception as e: - err_text = traceback.format_exc() - finally: - captured = buf.getvalue() - sys.stdout = old_stdout - response['data'] = json.dumps({ - 'result': _serialize(result_val), - 'stdout': captured, - 'stderr': err_text - }) - - elif uri == '/api/nodes': - pp = pars.get('parentPath', ['/project1'])[0] - p = op(pp) - nodes = [] - if p: - for c in p.children: - nodes.append({'name': c.name, 'path': c.path, - 'opType': c.OPType, 'family': c.family}) - response['data'] = json.dumps({'data': nodes}) - - elif uri == '/api/nodes/errors': - np = pars.get('nodePath', ['/project1'])[0] - n = op(np) - errors = [] - if n: - def _collect(node, depth=0): - if depth > 10: return - e = node.errors() - if e: - errors.append({'nodePath': node.path, 'nodeName': node.name, - 'opType': node.OPType, 'errors': str(e)}) - if hasattr(node, 'children'): - for c in node.children: _collect(c, depth+1) - _collect(n) - response['data'] = json.dumps({'data': errors, 'hasErrors': len(errors)>0, - 'errorCount': len(errors)}) - - else: - response['statusCode'] = 404 - response['data'] = json.dumps({'error': 'Unknown: ' + uri}) - - except Exception as e: - response['statusCode'] = 500 - response['data'] = json.dumps({'error': str(e), 'trace': traceback.format_exc()}) - - return response - -def _serialize(v): - if v is None: return None - if isinstance(v, (int, float, bool, str)): return v - if isinstance(v, (list, tuple)): return [_serialize(i) for i in v] - if isinstance(v, dict): return {str(k): _serialize(vv) for k, vv in v.items()} - return str(v) -''' - -# Point webserver to callback -ws.par.callbacks = cb.path - -print("Custom API server created on port 9981") -print("Test: curl http://127.0.0.1:9981/api/td/server/td") diff --git a/skills/creative/touchdesigner/scripts/setup.sh b/skills/creative/touchdesigner/scripts/setup.sh index ce8b56870..f6bab2f50 100644 --- a/skills/creative/touchdesigner/scripts/setup.sh +++ b/skills/creative/touchdesigner/scripts/setup.sh @@ -1,152 +1,114 @@ #!/usr/bin/env bash -# TouchDesigner MCP Setup Verification Script -# Checks all prerequisites and guides configuration - +# setup.sh — Automated setup for twozero MCP plugin for TouchDesigner +# Idempotent: safe to run multiple times. set -euo pipefail -RED='\033[0;31m' -GREEN='\033[0;32m' -YELLOW='\033[1;33m' -BLUE='\033[0;34m' -NC='\033[0m' +GREEN='\033[0;32m'; RED='\033[0;31m'; YELLOW='\033[1;33m'; CYAN='\033[0;36m'; NC='\033[0m' +OK="${GREEN}✔${NC}"; FAIL="${RED}✘${NC}"; WARN="${YELLOW}⚠${NC}" -pass() { echo -e " ${GREEN}✓${NC} $1"; } -fail() { echo -e " ${RED}✗${NC} $1"; } -warn() { echo -e " ${YELLOW}!${NC} $1"; } -info() { echo -e " ${BLUE}→${NC} $1"; } +TWOZERO_URL="https://www.404zero.com/pisang/twozero.tox" +TOX_PATH="$HOME/Downloads/twozero.tox" +HERMES_CFG="$HOME/.hermes/config.yaml" +MCP_PORT=40404 +MCP_ENDPOINT="http://localhost:${MCP_PORT}/mcp" -echo "" -echo "TouchDesigner MCP Setup Check" -echo "==============================" -echo "" +manual_steps=() -ERRORS=0 +echo -e "\n${CYAN}═══ twozero MCP for TouchDesigner — Setup ═══${NC}\n" -# 1. Check Node.js -echo "1. Node.js" -if command -v node &>/dev/null; then - NODE_VER=$(node --version 2>/dev/null || echo "unknown") - MAJOR=$(echo "$NODE_VER" | sed 's/^v//' | cut -d. -f1) - if [ "$MAJOR" -ge 18 ] 2>/dev/null; then - pass "Node.js $NODE_VER (>= 18 required)" +# ── 1. Check if TouchDesigner is running ── +if pgrep -if "TouchDesigner" >/dev/null 2>&1; then + echo -e " ${OK} TouchDesigner is running" + td_running=true +else + echo -e " ${WARN} TouchDesigner is not running" + td_running=false +fi + +# ── 2. Ensure twozero.tox exists ── +if [[ -f "$TOX_PATH" ]]; then + echo -e " ${OK} twozero.tox already exists at ${TOX_PATH}" +else + echo -e " ${WARN} twozero.tox not found — downloading..." + if curl -fSL -o "$TOX_PATH" "$TWOZERO_URL" 2>/dev/null; then + echo -e " ${OK} Downloaded twozero.tox to ${TOX_PATH}" else - fail "Node.js $NODE_VER (>= 18 required, please upgrade)" - ERRORS=$((ERRORS + 1)) + echo -e " ${FAIL} Failed to download twozero.tox from ${TWOZERO_URL}" + echo " Please download manually and place at ${TOX_PATH}" + manual_steps+=("Download twozero.tox from ${TWOZERO_URL} to ${TOX_PATH}") + fi +fi + +# ── 3. Ensure Hermes config has twozero_td MCP entry ── +if [[ ! -f "$HERMES_CFG" ]]; then + echo -e " ${FAIL} Hermes config not found at ${HERMES_CFG}" + manual_steps+=("Create ${HERMES_CFG} with twozero_td MCP server entry") +elif grep -q 'twozero_td' "$HERMES_CFG" 2>/dev/null; then + echo -e " ${OK} twozero_td MCP entry exists in Hermes config" +else + echo -e " ${WARN} Adding twozero_td MCP entry to Hermes config..." + python3 -c " +import yaml, sys, copy + +cfg_path = '$HERMES_CFG' +with open(cfg_path, 'r') as f: + cfg = yaml.safe_load(f) or {} + +if 'mcp_servers' not in cfg: + cfg['mcp_servers'] = {} + +if 'twozero_td' not in cfg['mcp_servers']: + cfg['mcp_servers']['twozero_td'] = { + 'url': '${MCP_ENDPOINT}', + 'timeout': 120, + 'connect_timeout': 60 + } + with open(cfg_path, 'w') as f: + yaml.dump(cfg, f, default_flow_style=False, sort_keys=False) + print('added') +else: + print('exists') +" 2>/dev/null && echo -e " ${OK} twozero_td MCP entry added to config" \ + || { echo -e " ${FAIL} Could not update config (is PyYAML installed?)"; \ + manual_steps+=("Add twozero_td MCP entry to ${HERMES_CFG} manually"); } + manual_steps+=("Restart Hermes session to pick up config change") +fi + +# ── 4. Test if MCP port is responding ── +if nc -z 127.0.0.1 "$MCP_PORT" 2>/dev/null; then + echo -e " ${OK} Port ${MCP_PORT} is open" + + # ── 5. Verify MCP endpoint responds ── + resp=$(curl -s --max-time 3 "$MCP_ENDPOINT" 2>/dev/null || true) + if [[ -n "$resp" ]]; then + echo -e " ${OK} MCP endpoint responded at ${MCP_ENDPOINT}" + else + echo -e " ${WARN} Port open but MCP endpoint returned empty response" + manual_steps+=("Verify MCP is enabled in twozero settings") fi else - fail "Node.js not found" - info "Install: https://nodejs.org/ or 'brew install node'" - ERRORS=$((ERRORS + 1)) -fi - -# 2. Check npm/npx -echo "2. npm/npx" -if command -v npx &>/dev/null; then - NPX_VER=$(npx --version 2>/dev/null || echo "unknown") - pass "npx $NPX_VER" -else - fail "npx not found (usually comes with Node.js)" - ERRORS=$((ERRORS + 1)) -fi - -# 3. Check MCP Python package -echo "3. MCP Python package" -HERMES_HOME="${HERMES_HOME:-$HOME/.hermes}" -VENV_PYTHON="" - -# Try to find the Hermes venv Python -if [ -f "$HERMES_HOME/hermes-agent/.venv/bin/python" ]; then - VENV_PYTHON="$HERMES_HOME/hermes-agent/.venv/bin/python" -elif [ -f "$HERMES_HOME/hermes-agent/venv/bin/python" ]; then - VENV_PYTHON="$HERMES_HOME/hermes-agent/venv/bin/python" -fi - -if [ -n "$VENV_PYTHON" ]; then - if $VENV_PYTHON -c "import mcp" 2>/dev/null; then - MCP_VER=$($VENV_PYTHON -c "import importlib.metadata; print(importlib.metadata.version('mcp'))" 2>/dev/null || echo "installed") - pass "mcp package ($MCP_VER) in Hermes venv" + echo -e " ${WARN} Port ${MCP_PORT} is not open" + if [[ "$td_running" == true ]]; then + manual_steps+=("In TD: drag twozero.tox into network editor → click Install") + manual_steps+=("Enable MCP: twozero icon → Settings → mcp → 'auto start MCP' → Yes") else - fail "mcp package not installed in Hermes venv" - info "Install: $VENV_PYTHON -m pip install mcp" - ERRORS=$((ERRORS + 1)) + manual_steps+=("Launch TouchDesigner") + manual_steps+=("Drag twozero.tox into the TD network editor and click Install") + manual_steps+=("Enable MCP: twozero icon → Settings → mcp → 'auto start MCP' → Yes") fi +fi + +# ── Status Report ── +echo -e "\n${CYAN}═══ Status Report ═══${NC}\n" + +if [[ ${#manual_steps[@]} -eq 0 ]]; then + echo -e " ${OK} ${GREEN}Fully configured! twozero MCP is ready to use.${NC}\n" + exit 0 else - warn "Could not find Hermes venv — check mcp package manually" -fi - -# 4. Check TouchDesigner -echo "4. TouchDesigner" -TD_FOUND=false - -# macOS -if [ -d "/Applications/TouchDesigner.app" ]; then - TD_FOUND=true - pass "TouchDesigner found at /Applications/TouchDesigner.app" -fi - -# Linux (common install locations) -if command -v TouchDesigner &>/dev/null; then - TD_FOUND=true - pass "TouchDesigner found in PATH" -fi - -if [ -d "$HOME/TouchDesigner" ]; then - TD_FOUND=true - pass "TouchDesigner found at ~/TouchDesigner" -fi - -if [ "$TD_FOUND" = false ]; then - warn "TouchDesigner not detected (may be installed elsewhere)" - info "Download from: https://derivative.ca/download" - info "Free Non-Commercial license available" -fi - -# 5. Check TD WebServer DAT reachability -echo "5. TouchDesigner WebServer DAT" -TD_URL="${TD_API_URL:-http://127.0.0.1:9981}" -if command -v curl &>/dev/null; then - HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --connect-timeout 3 "$TD_URL/api/td/server/td" 2>/dev/null || echo "000") - if [ "$HTTP_CODE" = "200" ]; then - TD_INFO=$(curl -s --connect-timeout 3 "$TD_URL/api/td/server/td" 2>/dev/null || echo "{}") - pass "TD WebServer DAT responding at $TD_URL" - info "Response: $TD_INFO" - elif [ "$HTTP_CODE" = "000" ]; then - warn "Cannot reach TD WebServer DAT at $TD_URL" - info "Make sure TouchDesigner is running with mcp_webserver_base.tox imported" - else - warn "TD WebServer DAT returned HTTP $HTTP_CODE at $TD_URL" - fi -else - warn "curl not found — cannot test TD connection" -fi - -# 6. Check Hermes config -echo "6. Hermes MCP config" -CONFIG_FILE="$HERMES_HOME/config.yaml" -if [ -f "$CONFIG_FILE" ]; then - if grep -q "touchdesigner" "$CONFIG_FILE" 2>/dev/null; then - pass "TouchDesigner MCP server configured in config.yaml" - else - warn "No 'touchdesigner' entry found in mcp_servers config" - info "Add a touchdesigner entry under mcp_servers: in $CONFIG_FILE" - info "See references/mcp-tools.md for the configuration block" - fi -else - warn "No Hermes config.yaml found at $CONFIG_FILE" -fi - -# Summary -echo "" -echo "==============================" -if [ $ERRORS -eq 0 ]; then - echo -e "${GREEN}All critical checks passed!${NC}" + echo -e " ${WARN} ${YELLOW}Manual steps remaining:${NC}\n" + for i in "${!manual_steps[@]}"; do + echo -e " $((i+1)). ${manual_steps[$i]}" + done echo "" - echo "Next steps:" - echo " 1. Open TouchDesigner and import mcp_webserver_base.tox" - echo " 2. Add the MCP server config to Hermes (see references/mcp-tools.md)" - echo " 3. Restart Hermes and test: 'Get TouchDesigner server info'" -else - echo -e "${RED}$ERRORS critical issue(s) found.${NC}" - echo "Fix the issues above, then re-run this script." + exit 1 fi -echo ""