From 7a5371b20d2e8226a3ec61f0320b4cb57d68e88f Mon Sep 17 00:00:00 2001 From: kshitijk4poor Date: Wed, 15 Apr 2026 10:33:15 +0530 Subject: [PATCH] feat: add TouchDesigner integration skill MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New skill: creative/touchdesigner — control a running TouchDesigner instance via REST API. Build real-time visual networks programmatically. Architecture: Hermes Agent -> HTTP REST (curl) -> TD WebServer DAT -> TD Python env Key features: - Custom API handler (scripts/custom_api_handler.py) that creates a self-contained WebServer DAT + callback in TD. More reliable than the official mcp_webserver_base.tox which frequently fails module imports. - Discovery-first workflow: never hardcode TD parameter names. Always probe the running instance first since names change across versions. - Persistent setup: save the TD project once with the API handler baked in. TD auto-opens the last project on launch, so port 9981 is live with zero manual steps after first-time setup. - Works via curl in execute_code (no MCP dependency required). - Optional MCP server config for touchdesigner-mcp-server npm package. Skill structure (2823 lines total): SKILL.md (209 lines) — setup, workflow, key rules, operator reference references/pitfalls.md (276 lines) — 24 hard-won lessons references/operators.md (239 lines) — all 6 operator families references/network-patterns.md (589 lines) — audio-reactive, generative, video processing, GLSL, instancing, live performance recipes references/mcp-tools.md (501 lines) — 13 MCP tool schemas references/python-api.md (443 lines) — TD Python scripting patterns references/troubleshooting.md (274 lines) — connection diagnostics scripts/custom_api_handler.py (140 lines) — REST API handler for TD scripts/setup.sh (152 lines) — prerequisite checker Tested on TouchDesigner 099 Non-Commercial (macOS/darwin). --- skills/creative/touchdesigner/SKILL.md | 278 ++++++ .../touchdesigner/references/mcp-tools.md | 501 ++++++++++ .../references/network-patterns.md | 914 ++++++++++++++++++ .../touchdesigner/references/operators.md | 239 +++++ .../touchdesigner/references/pitfalls.md | 336 +++++++ .../touchdesigner/references/python-api.md | 443 +++++++++ .../references/troubleshooting.md | 274 ++++++ .../scripts/custom_api_handler.py | 140 +++ .../creative/touchdesigner/scripts/setup.sh | 152 +++ 9 files changed, 3277 insertions(+) create mode 100644 skills/creative/touchdesigner/SKILL.md create mode 100644 skills/creative/touchdesigner/references/mcp-tools.md create mode 100644 skills/creative/touchdesigner/references/network-patterns.md create mode 100644 skills/creative/touchdesigner/references/operators.md create mode 100644 skills/creative/touchdesigner/references/pitfalls.md create mode 100644 skills/creative/touchdesigner/references/python-api.md create mode 100644 skills/creative/touchdesigner/references/troubleshooting.md create mode 100644 skills/creative/touchdesigner/scripts/custom_api_handler.py create mode 100644 skills/creative/touchdesigner/scripts/setup.sh diff --git a/skills/creative/touchdesigner/SKILL.md b/skills/creative/touchdesigner/SKILL.md new file mode 100644 index 000000000..0f464193f --- /dev/null +++ b/skills/creative/touchdesigner/SKILL.md @@ -0,0 +1,278 @@ +--- +name: touchdesigner +description: "Control a running TouchDesigner instance programmatically — create operators, set parameters, wire connections, execute Python, build real-time visuals. Covers: GLSL shaders, audio-reactive, generative art, video processing, instancing, and live performance." +version: 3.0.0 +author: Hermes Agent +license: MIT +metadata: + hermes: + tags: [TouchDesigner, MCP, creative-coding, real-time-visuals, generative-art, audio-reactive, VJ, installation, GLSL] + related_skills: [native-mcp, ascii-video, manim-video, hermes-video] + security: + allow_network: true + allow_install: true + allow_config_write: true +--- + +# TouchDesigner Integration + +## Architecture + +Hermes Agent -> HTTP REST (curl) -> TD WebServer DAT (port 9981) -> TD Python environment. + +The agent controls a **running TouchDesigner instance** via a REST API on port 9981. It does NOT generate .toe files from scratch. + +## First-Time Setup (one-time, persists across sessions) + +### 1. Verify TD is running and check for existing API + +```bash +lsof -i :9981 -P -n | grep LISTEN # TD listening? +curl -s --max-time 5 http://127.0.0.1:9981/api/td/server/td # API working? +``` + +If HTTP 200 + JSON → skip to **Discovery**. Setup is already done. + +### 2. If no API: deploy the custom handler + +The user must paste ONE line into TD Textport (Alt+T / Dialogs > Textport and DATs): + +``` +exec(open('PATH_TO_SKILL/scripts/custom_api_handler.py').read()) +``` + +Copy this to their clipboard with `pbcopy`. This creates a WebServer DAT + callback handler pair in `/project1` that implements the REST API. No external dependencies. + +**Why not the official .tox?** The `mcp_webserver_base.tox` from 8beeeaaat/touchdesigner-mcp frequently fails to import its Python modules after drag-drop (relative path resolution issue). Our custom handler is self-contained and more reliable. See `references/pitfalls.md` #1-2. + +### 3. Save the project to persist the API + +After the handler is running, save the project so the API auto-starts on every future TD launch: + +```python +td_exec("project.save(os.path.expanduser('~/Documents/HermesAgent.toe'))") +``` + +TD auto-opens the last saved project on launch. From now on, `open /Applications/TouchDesigner.app` → port 9981 is live → agent can connect immediately. + +To launch TD with this project explicitly: +```bash +open /Applications/TouchDesigner.app ~/Documents/HermesAgent.toe +``` + +### 4. Optional: Configure Hermes MCP + +Add under `mcp_servers:` in the user's Hermes config: +```yaml +touchdesigner: + command: npx + args: ["-y", "touchdesigner-mcp-server@latest"] + env: + TD_API_URL: "http://127.0.0.1:9981" + timeout: 120 +``` + +This is optional — the agent works fully via `curl` to the REST API using `execute_code`. MCP tools are a convenience layer. + +## Talking to TD (the td_exec pattern) + +All communication uses this pattern in `execute_code`: + +```python +import json, shlex +from hermes_tools import terminal + +API = "http://127.0.0.1:9981" +def td_exec(script): + payload = json.dumps({"script": script}) + cmd = f"curl -s --max-time 15 -X POST -H 'Content-Type: application/json' -d {shlex.quote(payload)} '{API}/api/td/server/exec'" + r = terminal(cmd, timeout=20) + return json.loads(r['output']) + +# Returns: {"result": , "stdout": "...", "stderr": "..."} +``` + +For large GLSL shaders: write to a temp file, then `td_exec("op('...').text = open('/tmp/shader.glsl').read()")`. + +## Workflow + +### Step 0: Discovery (MANDATORY — never skip) + +**Never hardcode parameter names.** They change between TD versions. Run this first: + +```python +td_exec(""" +import sys +info = {'version': str(app.version), 'platform': sys.platform} +root = op('/project1') +for name, optype in [('glslTOP', glslTOP), ('constantTOP', constantTOP), + ('blurTOP', blurTOP), ('textTOP', textTOP), + ('levelTOP', levelTOP), ('compositeTOP', compositeTOP), + ('transformTOP', transformTOP), ('feedbackTOP', feedbackTOP), + ('windowCOMP', windowCOMP)]: + n = root.create(optype, '_d_' + name) + kw = ['color','size','font','dat','alpha','opacity','resolution','text', + 'extend','operand','top','pixel','format','win','type'] + info[name] = [p.name for p in n.pars() if any(k in p.name.lower() for k in kw)] + n.destroy() +result = info +""") +``` + +Use the returned param names for ALL subsequent calls. Store them in your session context. + +### Step 1: Clean + Build + +Build the entire network in ONE `td_exec` call (batching avoids round-trip overhead and ensures TD advances frames between calls): + +```python +td_exec(""" +root = op('/project1') +keep = {'api_server', 'api_handler'} +for child in list(root.children): # snapshot before destroying + if child.name not in keep and child.valid: + child.destroy() + +# Create nodes, set params (using discovered names), wire, verify +... +result = {'nodes': len(list(root.children)), 'errors': [...]} +""") +``` + +### Step 2: Wire connections + +```python +gl.outputConnectors[0].connect(comp.inputConnectors[0]) +``` + +### Step 3: Verify + +```python +for c in list(root.children): + e = c.errors(); w = c.warnings() + if e: print(c.name, 'ERR:', e) +``` + +### Step 4: Display + +```python +win = root.create(windowCOMP, 'display') +win.par.winop = out.path # discovered param name +win.par.winw = 1280; win.par.winh = 720 +win.par.winopen.pulse() +``` + +## Key Implementation Rules + +**Always clean safely:** `list(root.children)` before iterating + `child.valid` check. + +**GLSL time:** No `uTDCurrentTime` in TD 099. Feed time via 1x1 Constant TOP. +**CRITICAL: must use `rgba32float` format** — the default 8-bit format clamps values to 0-1, so `absTime.seconds % 1000.0` becomes 1.0 and the shader appears frozen: +```python +t = root.create(constantTOP, 'time_driver') +t.par.format = 'rgba32float' # ← REQUIRED or time is stuck at 1.0 +t.par.outputresolution = 'custom' +t.par.resolutionw = 1 +t.par.resolutionh = 1 +t.par.colorr.expr = "absTime.seconds % 1000.0" +t.par.colorg.expr = "int(absTime.seconds / 1000.0)" +t.outputConnectors[0].connect(glsl.inputConnectors[0]) +# In GLSL: vec4 td = texture(sTD2DInputs[0], vec2(.5)); float t = td.r + td.g*1000.; +``` + +**Feedback TOP:** Use `top` parameter reference (not direct input wire). The "Not enough sources" error resolves after first cook. The "Cook dependency loop" warning is expected. + +**Resolution:** Non-Commercial caps at 1280×1280. Use `outputresolution = 'custom'`. + +**Large shaders:** Write GLSL to `/tmp/file.glsl`, then `td_exec("op('shader').text = open('/tmp/file.glsl').read()")`. + +**WebServer DAT quirk:** Response body goes in `response['data']` not `response['body']`. Request POST body comes as bytes in `request['data']`. + +## Recording / Exporting Video + +To capture TD output as video or image sequence for external use (e.g., ASCII video pipeline): + +### Movie Recording (recommended) + +```python +# Put a Null TOP before the recorder (official best practice) +rec = root.create(moviefileoutTOP, 'recorder') +null_out.outputConnectors[0].connect(rec.inputConnectors[0]) + +rec.par.type = 'movie' +rec.par.file = '/tmp/output.mov' +rec.par.videocodec = 'mjpa' # Motion JPEG — works on Non-Commercial + +# Start/stop recording (par.record is a toggle, NOT .record() method) +rec.par.record = True # start +# ... wait ... +rec.par.record = False # stop +``` + +**H.264/H.265 require a Commercial license** — use `mjpa` (Motion JPEG) or `prores` on Non-Commercial. Extract frames afterward with ffmpeg if needed: +```bash +ffmpeg -i /tmp/output.mov -vframes 120 /tmp/frames/frame_%06d.png +``` + +### Image Sequence Export + +```python +rec.par.type = 'imagesequence' +rec.par.imagefiletype = 'png' +rec.par.file.expr = "'/tmp/frames/out' + me.fileSuffix" # fileSuffix is REQUIRED +rec.par.record = True +``` + +### Pitfalls + +- **Race condition:** When setting `par.file` and starting recording in the same script, use `run("...", delayFrames=2)` so the file path is applied before recording begins. +- **TOP.save() is useless for animation:** Calling `op('null1').save(path)` in a loop or rapid API calls captures the same GPU texture every time — TD doesn't cook new frames between save calls. Always use MovieFileOut for animated output. +- See `references/pitfalls.md` #25-27 for full details. + +## Audio-Reactive GLSL (Proven Recipe) + +Complete chain for music-driven visuals: AudioFileIn → AudioSpectrum → Math (boost) → Resample (256) → CHOP To TOP → GLSL TOP (spectrum sampled per-pixel). See `references/network-patterns.md` Pattern 3b for the full working recipe with shader code. + +## Audio-Reactive Visuals + +The most powerful TD workflow for the agent: play an audio file, analyze its spectrum, and drive a GLSL shader in real-time. The agent builds the entire signal chain programmatically. + +**Signal chain:** +``` +AudioFileIn CHOP → AudioSpectrum CHOP → Math CHOP (gain=5) + → Resample CHOP (256) → CHOP To TOP (spectrum texture) + ↓ (GLSL input 1) + Constant TOP (rgba32float, time) → GLSL TOP → Null TOP → MovieFileOut + (input 0) +``` + +**Key technique:** The spectrum becomes a 256×1 texture. In GLSL, `texture(sTD2DInputs[1], vec2(x, 0.0)).r` samples frequency at position x (0=bass, 1=treble). This lets the shader react per-pixel to different frequency bands. + +**Smoothing is critical:** Raw FFT jitters. Use `Math CHOP` gain to boost weak signal, then the GLSL shader's own temporal integration (via feedback or time-smoothed params) handles visual smoothing. + +See `references/network-patterns.md` Pattern 9b for the complete build script + shader code. + +## Operator Quick Reference + +| Family | Color | Examples | Suffix | +|--------|-------|----------|--------| +| TOP | Purple | noiseTop, glslTop, compositeTop, levelTop, blurTop, textTop, nullTop, feedbackTop, renderTop | TOP | +| CHOP | Green | audiofileinChop, audiospectrumChop, mathChop, lfoChop, constantChop | CHOP | +| SOP | Blue | gridSop, sphereSop, transformSop, noiseSop | SOP | +| DAT | White | textDat, tableDat, scriptDat, webserverDAT | DAT | +| MAT | Yellow | phongMat, pbrMat, glslMat, constMat | MAT | +| COMP | Gray | geometryComp, containerComp, cameraComp, lightComp, windowCOMP | COMP | + +See `references/operators.md` for full catalog. See `references/network-patterns.md` for recipes. + +## References + +| File | What | +|------|------| +| `references/pitfalls.md` | **READ FIRST** — 31 hard-won lessons from real sessions | +| `references/operators.md` | All operator families with params and use cases | +| `references/network-patterns.md` | Recipes: audio-reactive, generative, video, GLSL, instancing | +| `references/mcp-tools.md` | MCP tool schemas (optional — curl works without MCP) | +| `references/python-api.md` | TD Python: op(), scripting, extensions | +| `references/troubleshooting.md` | Connection diagnostics, param debugging, performance | +| `scripts/custom_api_handler.py` | Self-contained REST API handler for TD WebServer DAT | diff --git a/skills/creative/touchdesigner/references/mcp-tools.md b/skills/creative/touchdesigner/references/mcp-tools.md new file mode 100644 index 000000000..5e4ad98d5 --- /dev/null +++ b/skills/creative/touchdesigner/references/mcp-tools.md @@ -0,0 +1,501 @@ +# TouchDesigner MCP Tools Reference + +Complete parameter schemas and usage examples for all 13 MCP tools from the 8beeeaaat/touchdesigner-mcp server. + +## Hermes Configuration + +Add a `touchdesigner` entry under the `mcp_servers` section of your Hermes config. Example YAML block: + +```yaml +# Under mcp_servers: in config.yaml +mcp_servers: + touchdesigner: + command: npx + args: ["-y", "touchdesigner-mcp-server@latest"] + env: + TD_API_URL: "http://127.0.0.1:9981" + timeout: 120 + connect_timeout: 60 +``` + +For a locally built server, point `command` to `node` and `args` to the built server index.js path. Set `TD_API_URL` to the TouchDesigner WebServer DAT address (default port 9981). + +For the documentation/knowledge server (no running TD needed), add a `td_docs` entry using `touchdesigner-mcp-server` as the npx package. + +Tools are registered as `mcp_touchdesigner_` in Hermes. + +**If MCP tools are not available as direct function calls** (common when the MCP server connects but Hermes doesn't expose them as callable tools), use the custom API handler directly via `curl` in `execute_code` or `terminal`: + +```python +import json, shlex +from hermes_tools import terminal + +def td_exec(script): + """Execute Python in TouchDesigner via the REST API.""" + escaped = json.dumps({"script": script}) + cmd = f"curl -s --max-time 15 -X POST -H 'Content-Type: application/json' -d {shlex.quote(escaped)} 'http://127.0.0.1:9981/api/td/server/exec'" + r = terminal(cmd, timeout=20) + return json.loads(r['output']) + +# Example: list all nodes +result = td_exec('result = [c.name for c in op("/project1").children]') +print(result) # {"result": ["node1", "node2", ...], "stdout": "", "stderr": ""} +``` + +This `td_exec` helper works with both the official .tox handler and the custom API handler from `scripts/custom_api_handler.py`. + +Tools are registered as `mcp_touchdesigner_` in Hermes. + +## Common Formatting Parameters + +Most tools accept these optional formatting parameters: + +| Parameter | Type | Values | Description | +|-----------|------|--------|-------------| +| `detailLevel` | string | `"minimal"`, `"summary"`, `"detailed"` | Response verbosity | +| `responseFormat` | string | `"json"`, `"yaml"`, `"markdown"` | Output format | +| `limit` | integer | 1-500 | Max items (on list-type tools only) | + +These are client-side formatting — they control how the MCP server formats the response text, not what data TD returns. + +--- + +## Tool 1: describe_td_tools + +**Purpose:** Meta-tool — lists all available TouchDesigner MCP tools with descriptions and parameters. + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `filter` | string | No | Keyword to filter tools by name, description, or parameter | +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | + +**Example:** Find tools related to node creation +``` +describe_td_tools(filter="create") +``` + +**Note:** This tool runs entirely in the MCP server — it does NOT contact TouchDesigner. Use it to discover what's available. + +--- + +## Tool 2: get_td_info + +**Purpose:** Get TouchDesigner server information (version, OS, build). + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | + +**Example:** Check TD is running and get version +``` +get_td_info() +``` + +**Returns:** TD version, build number, OS name/version, MCP API version. + +**Use this first** to verify the connection is working before building networks. + +--- + +## Tool 3: execute_python_script + +**Purpose:** Execute arbitrary Python code inside TouchDesigner's Python environment. + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `script` | string | **Yes** | Python code to execute | +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | + +**Available globals in the script:** +- `op` — find operators by path +- `ops` — find multiple operators by pattern +- `me` — the WebServer DAT running the script +- `parent` — me.parent() +- `project` — root project component +- `td` — the full td module +- `result` — set this to explicitly return a value + +**Execution behavior:** +- Single-line scripts: tries `eval()` first (returns value), falls back to `exec()` +- Multi-line scripts: uses `exec()` always +- stdout/stderr are captured and returned separately +- If `result` is not set, tries to evaluate the last expression as the return value + +**Examples:** + +```python +# Simple query +execute_python_script(script="op('/project1/noise1').par.seed.val") +# Returns: {"result": 42, "stdout": "", "stderr": ""} + +# Multi-line script +execute_python_script(script=""" +nodes = op('/project1').findChildren(type=TOP) +result = [{'name': n.name, 'type': n.OPType} for n in nodes] +""") + +# Connect two operators +execute_python_script(script="op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1'))") + +# Create and configure in one script +execute_python_script(script=""" +parent = op('/project1') +n = parent.create(noiseTop, 'my_noise') +n.par.seed.val = 42 +n.par.monochrome.val = True +n.par.resolutionw.val = 1920 +n.par.resolutionh.val = 1080 +result = {'path': n.path, 'type': n.OPType} +""") + +# Batch wire a chain +execute_python_script(script=""" +chain = ['noise1', 'level1', 'blur1', 'composite1', 'null_out'] +for i in range(len(chain) - 1): + src = op(f'/project1/{chain[i]}') + dst = op(f'/project1/{chain[i+1]}') + if src and dst: + src.outputConnectors[0].connect(dst) +result = 'Wired chain: ' + ' -> '.join(chain) +""") +``` + +**When to use:** Wiring connections, complex logic, batch operations, querying state that other tools don't cover. This is the most powerful and flexible tool. + +--- + +## Tool 4: create_td_node + +**Purpose:** Create a new operator in TouchDesigner. + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `parentPath` | string | **Yes** | Path to parent (e.g., `/project1`) | +| `nodeType` | string | **Yes** | Operator type (e.g., `noiseTop`, `mathChop`) | +| `nodeName` | string | No | Custom name (auto-generated if omitted) | +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | + +**Examples:** + +``` +create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="bg_noise") +create_td_node(parentPath="/project1", nodeType="compositeTop") # auto-named +create_td_node(parentPath="/project1/audio_chain", nodeType="audiospectrumChop", nodeName="spectrum") +``` + +**Returns:** Node summary with id, name, path, opType, and all default parameter values. + +**Node type naming convention:** camelCase family suffix — `noiseTop`, `mathChop`, `gridSop`, `tableDat`, `phongMat`, `geometryComp`. See `references/operators.md` for the full list. + +--- + +## Tool 5: delete_td_node + +**Purpose:** Delete an existing operator. + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `nodePath` | string | **Yes** | Absolute path to node (e.g., `/project1/noise1`) | +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | + +**Example:** + +``` +delete_td_node(nodePath="/project1/noise1") +``` + +**Returns:** Confirmation with the deleted node's summary (captured before deletion). + +--- + +## Tool 6: get_td_nodes + +**Purpose:** List operators under a path with optional filtering. + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `parentPath` | string | **Yes** | Parent path (e.g., `/project1`) | +| `pattern` | string | No | Glob pattern for name filtering (default: `*`) | +| `includeProperties` | boolean | No | Include full parameter values (default: false) | +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | +| `limit` | integer | No | Max items (1-500) | + +**Examples:** + +``` +# List all direct children of /project1 +get_td_nodes(parentPath="/project1") + +# Find all noise operators +get_td_nodes(parentPath="/project1", pattern="noise*") + +# Get full parameter details +get_td_nodes(parentPath="/project1", pattern="*", includeProperties=true, limit=20) +``` + +**Returns:** List of node summaries. With `includeProperties=false` (default): id, name, path, opType only. With `includeProperties=true`: full parameter values included. + +--- + +## Tool 7: get_td_node_parameters + +**Purpose:** Get detailed parameters of a specific node. + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `nodePath` | string | **Yes** | Node path (e.g., `/project1/noise1`) | +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | +| `limit` | integer | No | Max parameters (1-500) | + +**Example:** + +``` +get_td_node_parameters(nodePath="/project1/noise1") +``` + +**Returns:** All parameter name-value pairs for the node. Use this to discover available parameters before calling update_td_node_parameters. + +--- + +## Tool 8: get_td_node_errors + +**Purpose:** Check for errors on a node and all its descendants (recursive). + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `nodePath` | string | **Yes** | Absolute path to inspect (e.g., `/project1`) | +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | +| `limit` | integer | No | Max error items (1-500) | + +**Examples:** + +``` +# Check entire project for errors +get_td_node_errors(nodePath="/project1") + +# Check a specific chain +get_td_node_errors(nodePath="/project1/audio_chain") +``` + +**Returns:** Error count, hasErrors boolean, and list of errors each with nodePath, nodeName, opType, and error message. + +**Always call this after building a network** to catch wiring mistakes, missing references, and configuration errors. + +--- + +## Tool 9: update_td_node_parameters + +**Purpose:** Update parameters on an existing node. + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `nodePath` | string | **Yes** | Path to node (e.g., `/project1/noise1`) | +| `properties` | object | **Yes** | Key-value pairs to update (e.g., `{"seed": 42, "monochrome": true}`) | +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | + +**Examples:** + +``` +# Set noise parameters +update_td_node_parameters( + nodePath="/project1/noise1", + properties={"seed": 42, "monochrome": false, "period": 4.0, "harmonics": 3, + "resolutionw": 1920, "resolutionh": 1080} +) + +# Set a file path +update_td_node_parameters( + nodePath="/project1/moviefilein1", + properties={"file": "/Users/me/Videos/clip.mp4", "play": true} +) + +# Set compositing mode +update_td_node_parameters( + nodePath="/project1/composite1", + properties={"operand": 0} # 0=Over, 1=Under, 3=Add, 18=Multiply, 27=Screen +) +``` + +**Returns:** List of successfully updated properties and any that failed (with reasons). Raises error if zero properties were updated. + +**Parameter value types:** Floats, ints, booleans, and strings are all accepted. For menu parameters, use either the string label or the integer index. + +--- + +## Tool 10: exec_node_method + +**Purpose:** Call a Python method directly on a specific node. + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `nodePath` | string | **Yes** | Path to node | +| `method` | string | **Yes** | Method name to call | +| `args` | array | No | Positional arguments (strings, numbers, booleans) | +| `kwargs` | object | No | Keyword arguments | +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | + +**Examples:** + +``` +# Get all children of a component +exec_node_method(nodePath="/project1", method="findChildren") + +# Find specific children +exec_node_method(nodePath="/project1", method="findChildren", + kwargs={"name": "noise*", "depth": 1}) + +# Get node errors +exec_node_method(nodePath="/project1/noise1", method="errors") + +# Get node warnings +exec_node_method(nodePath="/project1/noise1", method="warnings") + +# Save a component as .tox +exec_node_method(nodePath="/project1/myContainer", method="save", + args=["/path/to/component.tox"]) +``` + +**Returns:** Processed return value of the method call. TD operators are serialized to their path strings, iterables to lists, etc. + +--- + +## Tool 11: get_td_classes + +**Purpose:** List available TouchDesigner Python classes and modules. + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | +| `limit` | integer | No | Max items (default: 50) | + +**Example:** + +``` +get_td_classes(limit=100) +``` + +**Returns:** List of class/module names and their docstrings from the td module. Useful for discovering what's available in TD's Python environment. + +--- + +## Tool 12: get_td_class_details + +**Purpose:** Get methods and properties of a specific TD Python class. + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `className` | string | **Yes** | Class name (e.g., `noiseTop`, `OP`, `COMP`) | +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | +| `limit` | integer | No | Max methods/properties (default: 30) | + +**Examples:** + +``` +# Inspect the noiseTop class +get_td_class_details(className="noiseTop") + +# Inspect the base OP class (all operators inherit from this) +get_td_class_details(className="OP", limit=50) + +# Inspect COMP (component) class +get_td_class_details(className="COMP") +``` + +**Returns:** Class name, type, description, methods (name + description + type), and properties (name + description + type). + +--- + +## Tool 13: get_td_module_help + +**Purpose:** Retrieve Python help() text for any TD module, class, or function. + +**Parameters:** +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `moduleName` | string | **Yes** | Module/class name (e.g., `noiseCHOP`, `tdu`, `td.OP`) | +| `detailLevel` | string | No | Response verbosity | +| `responseFormat` | string | No | Output format | + +**Examples:** + +``` +# Get help for the noise CHOP class +get_td_module_help(moduleName="noiseCHOP") + +# Get help for the tdu utilities module +get_td_module_help(moduleName="tdu") + +# Dotted name resolution works +get_td_module_help(moduleName="td.OP") +``` + +**Returns:** Full Python help() text output, cleaned of backspace characters. + +--- + +## Workflow: Building a Complete Network + +Typical sequence of tool calls to build a project: + +1. `get_td_info` — verify connection +2. `get_td_nodes(parentPath="/project1")` — see what already exists +3. `create_td_node` (multiple) — create all operators +4. `update_td_node_parameters` (multiple) — configure each operator +5. `execute_python_script` — wire all connections in one batch script +6. `get_td_node_errors(nodePath="/project1")` — check for problems +7. `get_td_node_parameters` — verify specific nodes if needed +8. Iterate: adjust parameters, add operators, fix errors + +## TD Documentation MCP Server Tools + +The bottobot/touchdesigner-mcp-server provides 21 reference/knowledge tools (no running TD needed): + +| Tool | Purpose | +|------|---------| +| `get_operator` | Get full documentation for a specific operator | +| `search_operators` | Search operators by keyword | +| `list_operators` | List all operators (filterable by family) | +| `compare_operators` | Compare two operators side by side | +| `get_operator_examples` | Get usage examples for an operator | +| `suggest_workflow` | Get workflow suggestions for a task | +| `get_tutorial` | Get a full TD tutorial | +| `list_tutorials` | List available tutorials | +| `search_tutorials` | Search tutorial content | +| `get_python_api` | Get Python API class documentation | +| `search_python_api` | Search Python API | +| `list_python_classes` | List all documented Python classes | +| `get_version_info` | Get TD version release notes | +| `list_versions` | List all documented TD versions | +| `get_experimental_techniques` | Get advanced technique guides (GLSL, ML, generative, etc.) | +| `search_experimental` | Search experimental techniques | +| `get_glsl_pattern` | Get GLSL code patterns (SDF, color, math utilities) | +| `get_operator_connections` | Get common operator wiring patterns | +| `get_network_template` | Get complete network templates with Python generation scripts | +| `get_experimental_build` | Get experimental build info | +| `list_experimental_builds` | List experimental builds | + +This server contains 630 operator docs, 14 tutorials, 69 Python API classes, and 7 experimental technique categories with working code. diff --git a/skills/creative/touchdesigner/references/network-patterns.md b/skills/creative/touchdesigner/references/network-patterns.md new file mode 100644 index 000000000..7afa24150 --- /dev/null +++ b/skills/creative/touchdesigner/references/network-patterns.md @@ -0,0 +1,914 @@ +# TouchDesigner Network Patterns + +Complete network recipes for common creative coding tasks. Each pattern shows the operator chain, MCP tool calls to build it, and key parameter settings. + +## Audio-Reactive Visuals + +### Pattern 1: Audio Spectrum -> Noise Displacement + +Audio drives noise parameters for organic, music-responsive textures. + +``` +Audio File In CHOP -> Audio Spectrum CHOP -> Math CHOP (scale) + | + v (export to noise params) + Noise TOP -> Level TOP -> Feedback TOP -> Composite TOP -> Null TOP (out) + ^ | + |________________| +``` + +**MCP Build Sequence:** + +``` +1. create_td_node(parentPath="/project1", nodeType="audiofileinChop", nodeName="audio_in") +2. create_td_node(parentPath="/project1", nodeType="audiospectrumChop", nodeName="spectrum") +3. create_td_node(parentPath="/project1", nodeType="mathChop", nodeName="spectrum_scale") +4. create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="noise1") +5. create_td_node(parentPath="/project1", nodeType="levelTop", nodeName="level1") +6. create_td_node(parentPath="/project1", nodeType="feedbackTop", nodeName="feedback1") +7. create_td_node(parentPath="/project1", nodeType="compositeTop", nodeName="comp1") +8. create_td_node(parentPath="/project1", nodeType="nullTop", nodeName="out") + +9. update_td_node_parameters(nodePath="/project1/audio_in", + properties={"file": "/path/to/music.wav", "play": true}) +10. update_td_node_parameters(nodePath="/project1/spectrum", + properties={"size": 512}) +11. update_td_node_parameters(nodePath="/project1/spectrum_scale", + properties={"gain": 2.0, "postoff": 0.0}) +12. update_td_node_parameters(nodePath="/project1/noise1", + properties={"type": 1, "monochrome": false, "resolutionw": 1920, "resolutionh": 1080, + "period": 4.0, "harmonics": 3, "amp": 1.0}) +13. update_td_node_parameters(nodePath="/project1/level1", + properties={"opacity": 0.95, "gamma1": 0.75}) +14. update_td_node_parameters(nodePath="/project1/feedback1", + properties={"top": "/project1/comp1"}) +15. update_td_node_parameters(nodePath="/project1/comp1", + properties={"operand": 0}) + +16. execute_python_script: """ +op('/project1/audio_in').outputConnectors[0].connect(op('/project1/spectrum')) +op('/project1/spectrum').outputConnectors[0].connect(op('/project1/spectrum_scale')) +op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1')) +op('/project1/level1').outputConnectors[0].connect(op('/project1/comp1').inputConnectors[0]) +op('/project1/feedback1').outputConnectors[0].connect(op('/project1/comp1').inputConnectors[1]) +op('/project1/comp1').outputConnectors[0].connect(op('/project1/out')) +""" + +17. execute_python_script: """ +# Export spectrum values to drive noise parameters +# This makes the noise react to audio frequencies +op('/project1/noise1').par.seed.expr = "op('/project1/spectrum_scale')['chan1']" +op('/project1/noise1').par.period.expr = "tdu.remap(op('/project1/spectrum_scale')['chan1'].eval(), 0, 1, 1, 8)" +""" +``` + +### Pattern 2: Beat Detection -> Visual Pulses + +Detect beats from audio and trigger visual events. + +``` +Audio Device In CHOP -> Audio Spectrum CHOP -> Math CHOP (isolate bass) + | + Trigger CHOP (envelope) + | + [export to visual params] +``` + +**Key parameter settings:** + +``` +# Isolate bass frequencies (20-200 Hz) +Math CHOP: chanop=1 (Add channels), range1low=0, range1high=10 + (first 10 FFT bins = bass frequencies with 512 FFT at 44100Hz) + +# ADSR envelope on each beat +Trigger CHOP: attack=0.02, peak=1.0, decay=0.3, sustain=0.0, release=0.1 + +# Export to visual: Scale, brightness, or color intensity +execute_python_script: "op('/project1/level1').par.brightness1.expr = \"1.0 + op('/project1/trigger1')['chan1'] * 0.5\"" +``` + +### Pattern 3: Multi-Band Audio -> Multi-Layer Visuals + +Split audio into frequency bands, drive different visual layers per band. + +``` +Audio In -> Spectrum -> Audio Band EQ (3 bands: bass, mid, treble) + | + +---------+---------+ + | | | + Bass Mids Treble + | | | + Noise TOP Circle TOP Text TOP + (slow,dark) (mid,warm) (fast,bright) + | | | + +-----+----+----+----+ + | | + Composite Composite + | + Out +``` + +### Pattern 3b: Audio-Reactive GLSL Fractal (Proven td_exec Recipe) + +Complete working recipe tested in TD 099. Plays an MP3, runs FFT, feeds spectrum as a texture into a GLSL shader where inner fractal reacts to bass, outer to treble. + +**Network:** +``` +AudioFileIn CHOP → AudioSpectrum CHOP → Math CHOP (boost) → Resample CHOP (256) + ↓ + CHOP To TOP (256x1 spectrum texture) + ↓ +Constant TOP (time, rgba32float) → GLSL TOP (input 0=time, input 1=spectrum) → Null → MovieFileOut + ↓ +AudioFileIn CHOP → Audio Device Out CHOP Record to .mov +``` + +**Build via td_exec (one call per step for reliability):** + +```python +# Step 1: Audio chain +td_exec(""" +root = op('/project1') +audio = root.create(audiofileinCHOP, 'audio_in') +audio.par.file = '/path/to/music.mp3' +audio.par.playmode = 0 # Locked to timeline +audio.par.volume = 0.5 + +spec = root.create(audiospectrumCHOP, 'spectrum') +audio.outputConnectors[0].connect(spec.inputConnectors[0]) + +math_n = root.create(mathCHOP, 'math_norm') +spec.outputConnectors[0].connect(math_n.inputConnectors[0]) +math_n.par.gain = 5 # boost signal + +resamp = root.create(resampleCHOP, 'resample_spec') +math_n.outputConnectors[0].connect(resamp.inputConnectors[0]) +resamp.par.timeslice = True +resamp.par.rate = 256 + +chop2top = root.create(choptoTOP, 'spectrum_tex') +resamp.outputConnectors[0].connect(chop2top.inputConnectors[0]) + +# Audio output (hear the music) +aout = root.create(audiodeviceoutCHOP, 'audio_out') +audio.outputConnectors[0].connect(aout.inputConnectors[0]) +result = 'audio chain ok' +""") + +# Step 2: Time driver (MUST be rgba32float — see pitfalls #12) +td_exec(""" +root = op('/project1') +td = root.create(constantTOP, 'time_driver') +td.par.format = 'rgba32float' +td.par.outputresolution = 'custom' +td.par.resolutionw = 1 +td.par.resolutionh = 1 +td.par.colorr.expr = "absTime.seconds % 1000.0" +td.par.colorg.expr = "int(absTime.seconds / 1000.0)" +result = 'time ok' +""") + +# Step 3: GLSL shader (write to /tmp, load from file) +td_exec(""" +root = op('/project1') +glsl = root.create(glslTOP, 'audio_shader') +glsl.par.outputresolution = 'custom' +glsl.par.resolutionw = 1280 +glsl.par.resolutionh = 720 + +sd = root.create(textDAT, 'shader_code') +sd.text = open('/tmp/my_shader.glsl').read() +glsl.par.pixeldat = sd + +# Wire: input 0 = time, input 1 = spectrum texture +op('/project1/time_driver').outputConnectors[0].connect(glsl.inputConnectors[0]) +op('/project1/spectrum_tex').outputConnectors[0].connect(glsl.inputConnectors[1]) +result = 'glsl ok' +""") + +# Step 4: Output + recorder +td_exec(""" +root = op('/project1') +out = root.create(nullTOP, 'output') +op('/project1/audio_shader').outputConnectors[0].connect(out.inputConnectors[0]) + +rec = root.create(moviefileoutTOP, 'recorder') +out.outputConnectors[0].connect(rec.inputConnectors[0]) +rec.par.type = 'movie' +rec.par.file = '/tmp/output.mov' +rec.par.videocodec = 'mjpa' +result = 'output ok' +""") +``` + +**GLSL shader pattern (audio-reactive fractal):** +```glsl +out vec4 fragColor; + +vec3 palette(float t) { + vec3 a = vec3(0.5); vec3 b = vec3(0.5); + vec3 c = vec3(1.0); vec3 d = vec3(0.263, 0.416, 0.557); + return a + b * cos(6.28318 * (c * t + d)); +} + +void main() { + // Input 0 = time (1x1 rgba32float constant) + // Input 1 = audio spectrum (256x1 CHOP To TOP) + vec4 td = texture(sTD2DInputs[0], vec2(0.5)); + float t = td.r + td.g * 1000.0; + + vec2 res = uTDOutputInfo.res.zw; + vec2 uv = (gl_FragCoord.xy * 2.0 - res) / min(res.x, res.y); + vec2 uv0 = uv; + vec3 finalColor = vec3(0.0); + + float bass = texture(sTD2DInputs[1], vec2(0.05, 0.0)).r; + float mids = texture(sTD2DInputs[1], vec2(0.25, 0.0)).r; + + for (float i = 0.0; i < 4.0; i++) { + uv = fract(uv * (1.4 + bass * 0.3)) - 0.5; + float d = length(uv) * exp(-length(uv0)); + + // Sample spectrum at distance: inner=bass, outer=treble + float freq = texture(sTD2DInputs[1], vec2(clamp(d * 0.5, 0.0, 1.0), 0.0)).r; + + vec3 col = palette(length(uv0) + i * 0.4 + t * 0.35); + d = sin(d * (7.0 + bass * 4.0) + t * 1.5) / 8.0; + d = abs(d); + d = pow(0.012 / d, 1.2 + freq * 0.8 + bass * 0.5); + finalColor += col * d; + } + + // Tone mapping + finalColor = finalColor / (finalColor + vec3(1.0)); + fragColor = TDOutputSwizzle(vec4(finalColor, 1.0)); +} +``` + +**Key insights from testing:** +- `spectrum_tex` (CHOP To TOP) produces a 256x1 texture — x position = frequency +- Sampling at `vec2(0.05, 0.0)` gets bass, `vec2(0.65, 0.0)` gets treble +- Sampling based on pixel distance (`d * 0.5`) makes inner fractal react to bass, outer to treble +- `bass * 0.3` in the `fract()` zoom makes the fractal breathe with kicks +- Math CHOP gain of 5 is needed because raw spectrum values are very small + +## Generative Art + +### Pattern 4: Feedback Loop with Transform + +Classic generative technique — texture evolves through recursive transformation. + +``` +Noise TOP -> Composite TOP -> Level TOP -> Null TOP (out) + ^ | + | v + Transform TOP <- Feedback TOP +``` + +**MCP Build Sequence:** + +``` +1. create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="seed_noise") +2. create_td_node(parentPath="/project1", nodeType="compositeTop", nodeName="mix") +3. create_td_node(parentPath="/project1", nodeType="transformTop", nodeName="evolve") +4. create_td_node(parentPath="/project1", nodeType="feedbackTop", nodeName="fb") +5. create_td_node(parentPath="/project1", nodeType="levelTop", nodeName="color_correct") +6. create_td_node(parentPath="/project1", nodeType="nullTop", nodeName="out") + +7. update_td_node_parameters(nodePath="/project1/seed_noise", + properties={"type": 1, "monochrome": false, "period": 2.0, "amp": 0.3, + "resolutionw": 1920, "resolutionh": 1080}) +8. update_td_node_parameters(nodePath="/project1/mix", + properties={"operand": 27}) # 27 = Screen blend +9. update_td_node_parameters(nodePath="/project1/evolve", + properties={"sx": 1.003, "sy": 1.003, "rz": 0.5, "extend": 2}) # slight zoom + rotate, repeat edges +10. update_td_node_parameters(nodePath="/project1/fb", + properties={"top": "/project1/mix"}) +11. update_td_node_parameters(nodePath="/project1/color_correct", + properties={"opacity": 0.98, "gamma1": 0.85}) + +12. execute_python_script: """ +op('/project1/seed_noise').outputConnectors[0].connect(op('/project1/mix').inputConnectors[0]) +op('/project1/fb').outputConnectors[0].connect(op('/project1/evolve')) +op('/project1/evolve').outputConnectors[0].connect(op('/project1/mix').inputConnectors[1]) +op('/project1/mix').outputConnectors[0].connect(op('/project1/color_correct')) +op('/project1/color_correct').outputConnectors[0].connect(op('/project1/out')) +""" +``` + +**Variations:** +- Change Transform: `rz` (rotation), `sx/sy` (zoom), `tx/ty` (drift) +- Change Composite operand: Screen (glow), Add (bright), Multiply (dark) +- Add HSV Adjust in the feedback loop for color evolution +- Add Blur for dreamlike softness +- Replace Noise with a GLSL TOP for custom seed patterns + +### Pattern 5: Instancing (Particle-Like Systems) + +Render thousands of copies of geometry, each with unique position/rotation/scale driven by CHOP data or DATs. + +``` +Table DAT (instance data) -> DAT to CHOP -> Geometry COMP (instancing on) -> Render TOP + + Sphere SOP (template geometry) + + Constant MAT (material) + + Camera COMP + + Light COMP +``` + +**MCP Build Sequence:** + +``` +1. create_td_node(parentPath="/project1", nodeType="tableDat", nodeName="instance_data") +2. create_td_node(parentPath="/project1", nodeType="geometryComp", nodeName="geo1") +3. create_td_node(parentPath="/project1/geo1", nodeType="sphereSop", nodeName="sphere") +4. create_td_node(parentPath="/project1", nodeType="constMat", nodeName="mat1") +5. create_td_node(parentPath="/project1", nodeType="cameraComp", nodeName="cam1") +6. create_td_node(parentPath="/project1", nodeType="lightComp", nodeName="light1") +7. create_td_node(parentPath="/project1", nodeType="renderTop", nodeName="render1") + +8. execute_python_script: """ +import random, math +dat = op('/project1/instance_data') +dat.clear() +dat.appendRow(['tx', 'ty', 'tz', 'sx', 'sy', 'sz', 'cr', 'cg', 'cb']) +for i in range(500): + angle = i * 0.1 + r = 2 + i * 0.01 + dat.appendRow([ + str(math.cos(angle) * r), + str(math.sin(angle) * r), + str((i - 250) * 0.02), + '0.05', '0.05', '0.05', + str(random.random()), + str(random.random()), + str(random.random()) + ]) +""" + +9. update_td_node_parameters(nodePath="/project1/geo1", + properties={"instancing": true, "instancechop": "", + "instancedat": "/project1/instance_data", + "material": "/project1/mat1"}) +10. update_td_node_parameters(nodePath="/project1/render1", + properties={"camera": "/project1/cam1", "geometry": "/project1/geo1", + "light": "/project1/light1", + "resolutionw": 1920, "resolutionh": 1080}) +11. update_td_node_parameters(nodePath="/project1/cam1", + properties={"tz": 10}) +``` + +### Pattern 6: Reaction-Diffusion (GLSL) + +Classic Gray-Scott reaction-diffusion system running on the GPU. + +``` +Text DAT (GLSL code) -> GLSL TOP (resolution, dat reference) -> Feedback TOP + ^ | + |_______________________________________| + Level TOP (out) +``` + +**Key GLSL code (write to Text DAT via execute_python_script):** + +```glsl +// Gray-Scott reaction-diffusion +uniform float feed; // 0.037 +uniform float kill; // 0.06 +uniform float dA; // 1.0 +uniform float dB; // 0.5 + +layout(location = 0) out vec4 fragColor; + +void main() { + vec2 uv = vUV.st; + vec2 texel = 1.0 / uTDOutputInfo.res.zw; + + vec4 c = texture(sTD2DInputs[0], uv); + float a = c.r; + float b = c.g; + + // Laplacian (9-point stencil) + float lA = 0.0, lB = 0.0; + for(int dx = -1; dx <= 1; dx++) { + for(int dy = -1; dy <= 1; dy++) { + float w = (dx == 0 && dy == 0) ? -1.0 : (abs(dx) + abs(dy) == 1 ? 0.2 : 0.05); + vec4 s = texture(sTD2DInputs[0], uv + vec2(dx, dy) * texel); + lA += s.r * w; + lB += s.g * w; + } + } + + float reaction = a * b * b; + float newA = a + (dA * lA - reaction + feed * (1.0 - a)); + float newB = b + (dB * lB + reaction - (kill + feed) * b); + + fragColor = vec4(clamp(newA, 0.0, 1.0), clamp(newB, 0.0, 1.0), 0.0, 1.0); +} +``` + +## Video Processing + +### Pattern 7: Video Effects Chain + +Apply a chain of effects to a video file. + +``` +Movie File In TOP -> HSV Adjust TOP -> Level TOP -> Blur TOP -> Composite TOP -> Null TOP (out) + ^ + Text TOP ---+ +``` + +**MCP Build Sequence:** + +``` +1. create_td_node(parentPath="/project1", nodeType="moviefileinTop", nodeName="video_in") +2. create_td_node(parentPath="/project1", nodeType="hsvadjustTop", nodeName="color") +3. create_td_node(parentPath="/project1", nodeType="levelTop", nodeName="levels") +4. create_td_node(parentPath="/project1", nodeType="blurTop", nodeName="blur") +5. create_td_node(parentPath="/project1", nodeType="compositeTop", nodeName="overlay") +6. create_td_node(parentPath="/project1", nodeType="textTop", nodeName="title") +7. create_td_node(parentPath="/project1", nodeType="nullTop", nodeName="out") + +8. update_td_node_parameters(nodePath="/project1/video_in", + properties={"file": "/path/to/video.mp4", "play": true}) +9. update_td_node_parameters(nodePath="/project1/color", + properties={"hueoffset": 0.1, "saturationmult": 1.3}) +10. update_td_node_parameters(nodePath="/project1/levels", + properties={"brightness1": 1.1, "contrast": 1.2, "gamma1": 0.9}) +11. update_td_node_parameters(nodePath="/project1/blur", + properties={"sizex": 2, "sizey": 2}) +12. update_td_node_parameters(nodePath="/project1/title", + properties={"text": "My Video", "fontsizex": 48, "alignx": 1, "aligny": 1}) + +13. execute_python_script: """ +chain = ['video_in', 'color', 'levels', 'blur'] +for i in range(len(chain) - 1): + op(f'/project1/{chain[i]}').outputConnectors[0].connect(op(f'/project1/{chain[i+1]}')) +op('/project1/blur').outputConnectors[0].connect(op('/project1/overlay').inputConnectors[0]) +op('/project1/title').outputConnectors[0].connect(op('/project1/overlay').inputConnectors[1]) +op('/project1/overlay').outputConnectors[0].connect(op('/project1/out')) +""" +``` + +### Pattern 8: Video Recording + +Record the output to a file. **H.264/H.265 require a Commercial license** — use Motion JPEG (`mjpa`) on Non-Commercial. + +``` +[any TOP chain] -> Null TOP -> Movie File Out TOP +``` + +```python +# Build via td_exec(): +root = op('/project1') + +# Always put a Null TOP before the recorder +null_out = root.op('out') # or create one +rec = root.create(moviefileoutTOP, 'recorder') +null_out.outputConnectors[0].connect(rec.inputConnectors[0]) + +rec.par.type = 'movie' +rec.par.file = '/tmp/output.mov' +rec.par.videocodec = 'mjpa' # Motion JPEG — works on Non-Commercial + +# Start recording (par.record is a toggle — .record() method may not exist) +rec.par.record = True +# ... let TD run for desired duration ... +rec.par.record = False + +# For image sequences: +# rec.par.type = 'imagesequence' +# rec.par.imagefiletype = 'png' +# rec.par.file.expr = "'/tmp/frames/out' + me.fileSuffix" # fileSuffix REQUIRED +``` + +**Pitfalls:** +- Setting `par.file` + `par.record = True` in the same script may race — use `run("...", delayFrames=2)` +- `TOP.save()` called rapidly always captures the same frame — use MovieFileOut for animation +- See `pitfalls.md` #25-27 for full details + +### Pattern 8b: TD → External Pipeline (e.g., ASCII Video) + +Export TD visuals for use in another tool (ffmpeg, Python, ASCII art, etc.): + +```python +# 1. Record with MovieFileOut (MJPEG) +rec.par.videocodec = 'mjpa' +rec.par.record = True +# ... wait N seconds ... +rec.par.record = False + +# 2. Extract frames with ffmpeg (outside TD) +# ffmpeg -i /tmp/output.mov -vframes 120 /tmp/frames/frame_%06d.png + +# 3. Load frames in Python for processing +# from PIL import Image +# img = Image.open('/tmp/frames/frame_000001.png') +``` + +## Data Visualization + +### Pattern 9: Table Data -> Bar Chart via Instancing + +Visualize tabular data as a 3D bar chart. + +``` +Table DAT (data) -> Script DAT (transform to instance format) -> DAT to CHOP + | +Box SOP -> Geometry COMP (instancing from CHOP) -> Render TOP -> Null TOP (out) + + PBR MAT + + Camera COMP + + Light COMP +``` + +```python +# Script DAT code to transform data to instance positions +execute_python_script: """ +source = op('/project1/data_table') +instance = op('/project1/instance_transform') +instance.clear() +instance.appendRow(['tx', 'ty', 'tz', 'sx', 'sy', 'sz', 'cr', 'cg', 'cb']) + +for i in range(1, source.numRows): + value = float(source[i, 'value']) + name = source[i, 'name'] + instance.appendRow([ + str(i * 1.5), # x position (spread bars) + str(value / 2), # y position (center bar vertically) + '0', # z position + '1', str(value), '1', # scale (height = data value) + '0.2', '0.6', '1.0' # color (blue) + ]) +""" +``` + +### Pattern 9b: Audio-Reactive GLSL Fractal (Proven Recipe) + +Audio spectrum drives a GLSL fractal shader directly via a spectrum texture input. Bass thickens inner fractal lines, mids twist rotation, highs light outer edges. Tested and working on TD 099 Non-Commercial. + +``` +Audio File In CHOP → Audio Spectrum CHOP → Math CHOP (boost gain=5) + → Resample CHOP (256 samples) → CHOP To TOP (spectrum texture, 256x1) + ↓ (input 1) +Constant TOP (rgba32float, time) → GLSL TOP (audio-reactive shader) → Null TOP + (input 0) ↑ + Text DAT (shader code) +``` + +**Build via td_exec (complete working script):** + +```python +td_exec(""" +import os +root = op('/project1') + +# Audio input +audio = root.create(audiofileinCHOP, 'audio_in') +audio.par.file = '/path/to/music.mp3' +audio.par.playmode = 0 # Locked to timeline + +# FFT analysis +spectrum = root.create(audiospectrumCHOP, 'spectrum') +audio.outputConnectors[0].connect(spectrum.inputConnectors[0]) + +# Normalize + boost +math = root.create(mathCHOP, 'math_norm') +spectrum.outputConnectors[0].connect(math.inputConnectors[0]) +math.par.gain = 5 + +# Resample to 256 bins for texture +resample = root.create(resampleCHOP, 'resample_spec') +math.outputConnectors[0].connect(resample.inputConnectors[0]) +resample.par.timeslice = True +resample.par.rate = 256 + +# Spectrum → texture (256x1 image) +# NOTE: choptoTOP has NO input connectors — use par.chop reference! +spec_tex = root.create(choptoTOP, 'spectrum_tex') +spec_tex.par.chop = resample + +# Time driver (rgba32float to avoid 0-1 clamping!) +time_drv = root.create(constantTOP, 'time_driver') +time_drv.par.format = 'rgba32float' +time_drv.par.outputresolution = 'custom' +time_drv.par.resolutionw = 1 +time_drv.par.resolutionh = 1 +time_drv.par.colorr.expr = "absTime.seconds % 1000.0" +time_drv.par.colorg.expr = "int(absTime.seconds / 1000.0)" + +# GLSL shader +glsl = root.create(glslTOP, 'audio_shader') +glsl.par.outputresolution = 'custom' +glsl.par.resolutionw = 1280; glsl.par.resolutionh = 720 + +shader_dat = root.create(textDAT, 'shader_code') +shader_dat.text = open('/tmp/shader.glsl').read() +glsl.par.pixeldat = shader_dat + +# Wire: input 0=time, input 1=spectrum +time_drv.outputConnectors[0].connect(glsl.inputConnectors[0]) +spec_tex.outputConnectors[0].connect(glsl.inputConnectors[1]) + +# Output + audio playback +out = root.create(nullTOP, 'output') +glsl.outputConnectors[0].connect(out.inputConnectors[0]) +audio_out = root.create(audiodeviceoutCHOP, 'audio_out') +audio.outputConnectors[0].connect(audio_out.inputConnectors[0]) + +result = 'network built' +""") +``` + +**GLSL shader (reads spectrum from input 1 texture):** + +```glsl +out vec4 fragColor; + +vec3 palette(float t) { + vec3 a = vec3(0.5); vec3 b = vec3(0.5); + vec3 c = vec3(1.0); vec3 d = vec3(0.263, 0.416, 0.557); + return a + b * cos(6.28318 * (c * t + d)); +} + +void main() { + vec4 td = texture(sTD2DInputs[0], vec2(0.5)); + float t = td.r + td.g * 1000.0; + + vec2 res = uTDOutputInfo.res.zw; + vec2 uv = (gl_FragCoord.xy * 2.0 - res) / min(res.x, res.y); + vec2 uv0 = uv; + vec3 finalColor = vec3(0.0); + + float bass = texture(sTD2DInputs[1], vec2(0.05, 0.0)).r; + float mids = texture(sTD2DInputs[1], vec2(0.25, 0.0)).r; + float highs = texture(sTD2DInputs[1], vec2(0.65, 0.0)).r; + + float ca = cos(t * (0.15 + mids * 0.3)); + float sa = sin(t * (0.15 + mids * 0.3)); + uv = mat2(ca, -sa, sa, ca) * uv; + + for (float i = 0.0; i < 4.0; i++) { + uv = fract(uv * (1.4 + bass * 0.3)) - 0.5; + float d = length(uv) * exp(-length(uv0)); + float freq = texture(sTD2DInputs[1], vec2(clamp(d*0.5, 0.0, 1.0), 0.0)).r; + vec3 col = palette(length(uv0) + i * 0.4 + t * 0.35); + d = sin(d * (7.0 + bass * 4.0) + t * 1.5) / 8.0; + d = abs(d); + d = pow(0.012 / d, 1.2 + freq * 0.8 + bass * 0.5); + finalColor += col * d; + } + + float glow = (0.03 + bass * 0.05) / (length(uv0) + 0.03); + finalColor += vec3(0.4, 0.1, 0.7) * glow * (0.6 + 0.4 * sin(t * 2.5)); + + float ring = abs(length(uv0) - 0.4 - mids * 0.3); + finalColor += vec3(0.1, 0.6, 0.8) * (0.005 / ring) * (0.2 + highs * 0.5); + + finalColor *= smoothstep(0.0, 1.0, 1.0 - dot(uv0*0.55, uv0*0.55)); + finalColor = finalColor / (finalColor + vec3(1.0)); + + fragColor = TDOutputSwizzle(vec4(finalColor, 1.0)); +} +``` + +**How spectrum sampling drives the visual:** +- `texture(sTD2DInputs[1], vec2(x, 0.0)).r` — x position = frequency (0=bass, 1=treble) +- Inner fractal iterations sample lower x → react to bass +- Outer iterations sample higher x → react to treble +- `bass * 0.3` on `fract()` scale → fractal zoom pulses with bass +- `bass * 4.0` on sin frequency → line density pulses with bass +- `mids * 0.3` on rotation speed → spiral twists faster during vocal/mid sections +- `highs * 0.5` on ring opacity → high-frequency sparkle on outer ring + +**Recording the output:** Use MovieFileOut TOP with `mjpa` codec (H.264 requires Commercial license). See pitfalls #25-27. + +## GLSL Shaders + +### Pattern 10: Custom Fragment Shader + +Write a custom visual effect as a GLSL fragment shader. + +``` +Text DAT (shader code) -> GLSL TOP -> Level TOP -> Null TOP (out) + + optional input TOPs for texture sampling +``` + +**Common GLSL uniforms available in TouchDesigner:** + +```glsl +// Automatically provided by TD +uniform vec4 uTDOutputInfo; // .res.zw = resolution + +// NOTE: uTDCurrentTime does NOT exist in TD 099! +// Feed time via a 1x1 Constant TOP (format=rgba32float): +// t.par.colorr.expr = "absTime.seconds % 1000.0" +// t.par.colorg.expr = "int(absTime.seconds / 1000.0)" +// Then read in GLSL: +// vec4 td = texture(sTD2DInputs[0], vec2(0.5)); +// float t = td.r + td.g * 1000.0; + +// Input textures (from connected TOP inputs) +uniform sampler2D sTD2DInputs[1]; // array of input samplers + +// From vertex shader +in vec3 vUV; // UV coordinates (0-1 range) +``` + +**Example: Plasma shader (using time from input texture)** + +```glsl +layout(location = 0) out vec4 fragColor; + +void main() { + vec2 uv = vUV.st; + // Read time from Constant TOP input 0 (rgba32float format) + vec4 td = texture(sTD2DInputs[0], vec2(0.5)); + float t = td.r + td.g * 1000.0; + + float v1 = sin(uv.x * 10.0 + t); + float v2 = sin(uv.y * 10.0 + t * 0.7); + float v3 = sin((uv.x + uv.y) * 10.0 + t * 1.3); + float v4 = sin(length(uv - 0.5) * 20.0 - t * 2.0); + + float v = (v1 + v2 + v3 + v4) * 0.25; + + vec3 color = vec3( + sin(v * 3.14159 + 0.0) * 0.5 + 0.5, + sin(v * 3.14159 + 2.094) * 0.5 + 0.5, + sin(v * 3.14159 + 4.189) * 0.5 + 0.5 + ); + + fragColor = vec4(color, 1.0); +} +``` + +### Pattern 11: Multi-Pass GLSL (Ping-Pong) + +For effects needing state across frames (particles, fluid, cellular automata), use GLSL Multi TOP with multiple passes or a Feedback TOP loop. + +``` +GLSL Multi TOP (pass 0: simulation, pass 1: rendering) + + Text DAT (simulation shader) + + Text DAT (render shader) + -> Level TOP -> Null TOP (out) + ^ + |__ Feedback TOP (feeds simulation state back) +``` + +## Interactive Installations + +### Pattern 12: Mouse/Touch -> Visual Response + +``` +Mouse In CHOP -> Math CHOP (normalize to 0-1) -> [export to visual params] + +# Or for touch/multi-touch: +Multi Touch In DAT -> Script CHOP (parse touches) -> [export to visual params] +``` + +```python +# Normalize mouse position to 0-1 range +execute_python_script: """ +op('/project1/noise1').par.offsetx.expr = "op('/project1/mouse_norm')['tx']" +op('/project1/noise1').par.offsety.expr = "op('/project1/mouse_norm')['ty']" +""" +``` + +### Pattern 13: OSC Control (from external software) + +``` +OSC In CHOP (port 7000) -> Select CHOP (pick channels) -> [export to visual params] +``` + +``` +1. create_td_node(parentPath="/project1", nodeType="oscinChop", nodeName="osc_in") +2. update_td_node_parameters(nodePath="/project1/osc_in", properties={"port": 7000}) + +# OSC messages like /frequency 440 will appear as channel "frequency" with value 440 +# Export to any parameter: +3. execute_python_script: "op('/project1/noise1').par.period.expr = \"op('/project1/osc_in')['frequency']\"" +``` + +### Pattern 14: MIDI Control (DJ/VJ) + +``` +MIDI In CHOP (device) -> Select CHOP -> [export channels to visual params] +``` + +Common MIDI mappings: +- CC channels (knobs/faders): continuous 0-127, map to float params +- Note On/Off: binary triggers, map to Trigger CHOP for envelopes +- Velocity: intensity/brightness + +## Live Performance + +### Pattern 15: Multi-Source VJ Setup + +``` +Source A (generative) ----+ +Source B (video) ---------+-- Switch/Cross TOP -- Level TOP -- Window COMP (output) +Source C (camera) --------+ + ^ + MIDI/OSC control selects active source and crossfade +``` + +```python +# MIDI CC1 controls which source is active (0-127 -> 0-2) +execute_python_script: """ +op('/project1/switch1').par.index.expr = "int(op('/project1/midi_in')['cc1'] / 42)" +""" + +# MIDI CC2 controls crossfade between current and next +execute_python_script: """ +op('/project1/cross1').par.cross.expr = "op('/project1/midi_in')['cc2'] / 127.0" +""" +``` + +### Pattern 16: Projection Mapping + +``` +Content TOPs ----+ + | +Stoner TOP (UV mapping) -> Composite TOP -> Window COMP (projector output) + or +Kantan Mapper COMP (external .tox) +``` + +For projection mapping, the key is: +1. Create your visual content as standard TOPs +2. Use Stoner TOP or a third-party mapping tool to UV-map content to physical surfaces +3. Output via Window COMP to the projector + +### Pattern 17: Cue System + +``` +Table DAT (cue list: cue_number, scene_name, duration, transition_type) + | +Script CHOP (cue state: current_cue, progress, next_cue_trigger) + | +[export to Switch/Cross TOPs to transition between scenes] +``` + +```python +execute_python_script: """ +# Simple cue system +cue_table = op('/project1/cue_list') +cue_state = op('/project1/cue_state') + +def advance_cue(): + current = int(cue_state.par.value0.val) + next_cue = min(current + 1, cue_table.numRows - 1) + cue_state.par.value0.val = next_cue + + scene = cue_table[next_cue, 'scene'] + duration = float(cue_table[next_cue, 'duration']) + + # Set crossfade target and duration + op('/project1/cross1').par.cross.val = 0 + # Animate cross to 1.0 over duration seconds + # (use a Timer CHOP or LFO CHOP for smooth animation) +""" +``` + +## Networking + +### Pattern 18: OSC Server/Client + +``` +# Sending OSC +OSC Out CHOP -> (network) -> external application + +# Receiving OSC +(network) -> OSC In CHOP -> Select CHOP -> [use values] +``` + +### Pattern 19: NDI Video Streaming + +``` +# Send video over network +[any TOP chain] -> NDI Out TOP (source name) + +# Receive video from network +NDI In TOP (select source) -> [process as normal TOP] +``` + +### Pattern 20: WebSocket Communication + +``` +WebSocket DAT -> Script DAT (parse JSON messages) -> [update visuals] +``` + +```python +execute_python_script: """ +ws = op('/project1/websocket1') +ws.par.address = 'ws://localhost:8080' +ws.par.active = True + +# In a DAT Execute callback (Script DAT watching WebSocket DAT): +# def onTableChange(dat): +# import json +# msg = json.loads(dat.text) +# op('/project1/noise1').par.seed.val = msg.get('seed', 0) +""" +``` diff --git a/skills/creative/touchdesigner/references/operators.md b/skills/creative/touchdesigner/references/operators.md new file mode 100644 index 000000000..6aa716cb9 --- /dev/null +++ b/skills/creative/touchdesigner/references/operators.md @@ -0,0 +1,239 @@ +# TouchDesigner Operator Reference + +## Operator Families Overview + +TouchDesigner has 6 operator families. Each family processes a specific data type and is color-coded in the UI. Operators can only connect to others of the SAME family (with cross-family converters as the bridge). + +## TOPs — Texture Operators (Purple) + +2D image/texture processing on the GPU. The workhorse of visual output. + +### Generators (create images from nothing) + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Noise TOP | `noiseTop` | `type` (0-6), `monochrome`, `seed`, `period`, `harmonics`, `exponent`, `amp`, `offset`, `resolutionw/h` | Procedural noise textures — Perlin, Simplex, Sparse, etc. Foundation of generative art. | +| Constant TOP | `constantTop` | `colorr/g/b/a`, `resolutionw/h` | Solid color. Use as background or blend input. | +| Text TOP | `textTop` | `text`, `fontsizex`, `fontfile`, `alignx/y`, `colorr/g/b` | Render text to texture. Supports multi-line, word wrap. | +| Ramp TOP | `rampTop` | `type` (0=horizontal, 1=vertical, 2=radial, 3=circular), `phase`, `period` | Gradient textures for masking, color mapping. | +| Circle TOP | `circleTop` | `radiusx/y`, `centerx/y`, `width` | Circles, rings, ellipses. | +| Rectangle TOP | `rectangleTop` | `sizex/y`, `centerx/y`, `softness` | Rectangles with optional softness. | +| GLSL TOP | `glslTop` | `dat` (points to shader DAT), `resolutionw/h`, `outputformat`, custom uniforms | Custom fragment shaders. Most powerful TOP for custom visuals. | +| GLSL Multi TOP | `glslmultiTop` | `dat`, `numinputs`, `numoutputs`, `numcomputepasses` | Multi-pass GLSL with compute shaders. Advanced. | +| Render TOP | `renderTop` | `camera`, `geometry`, `lights`, `resolutionw/h` | Renders 3D scenes (SOPs + MATs + Camera/Light COMPs). | + +### Filters (modify a single input) + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Level TOP | `levelTop` | `opacity`, `brightness1/2`, `gamma1/2`, `contrast`, `invert`, `blacklevel/whitelevel` | Brightness, contrast, gamma, levels. Essential color correction. | +| Blur TOP | `blurTop` | `sizex/y`, `type` (0=Gaussian, 1=Box, 2=Bartlett) | Gaussian/box blur. | +| Transform TOP | `transformTop` | `tx/ty`, `sx/sy`, `rz`, `pivotx/y`, `extend` (0=Hold, 1=Zero, 2=Repeat, 3=Mirror) | Translate, scale, rotate textures. | +| HSV Adjust TOP | `hsvadjustTop` | `hueoffset`, `saturationmult`, `valuemult` | HSV color adjustments. | +| Lookup TOP | `lookupTop` | (input: texture + lookup table) | Color remapping via lookup table texture. | +| Edge TOP | `edgeTop` | `type` (0=Sobel, 1=Frei-Chen) | Edge detection. | +| Displace TOP | `displaceTop` | `scalex/y` | Pixel displacement using a second input as displacement map. | +| Flip TOP | `flipTop` | `flipx`, `flipy`, `flop` (diagonal) | Mirror/flip textures. | +| Crop TOP | `cropTop` | `cropleft/right/top/bottom` | Crop region of texture. | +| Resolution TOP | `resolutionTop` | `resolutionw/h`, `outputresolution` | Resize textures. | +| Null TOP | `nullTop` | (none significant) | Pass-through. Use for organization, referencing, feedback delay. | +| Cache TOP | `cacheTop` | `length`, `step` | Store N frames of history. Useful for trails, time effects. | + +### Compositors (combine multiple inputs) + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Composite TOP | `compositeTop` | `operand` (0-31: Over, Add, Multiply, Screen, etc.) | Blend two textures with standard compositing modes. | +| Over TOP | `overTop` | (simple alpha compositing) | Layer with alpha. Simpler than Composite. | +| Add TOP | `addTop` | (additive blend) | Additive blending. Great for glow, light effects. | +| Multiply TOP | `multiplyTop` | (multiplicative blend) | Multiply blend. Good for masking, darkening. | +| Switch TOP | `switchTop` | `index` (0-based) | Switch between multiple inputs by index. | +| Cross TOP | `crossTop` | `cross` (0.0-1.0) | Crossfade between two inputs. | + +### I/O (input/output) + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Movie File In TOP | `moviefileinTop` | `file`, `speed`, `trim`, `index` | Load video files, image sequences. | +| Movie File Out TOP | `moviefileoutTop` | `file`, `type` (codec), `record` (toggle) | Record/export video files. | +| NDI In TOP | `ndiinTop` | `sourcename` | Receive NDI video streams. | +| NDI Out TOP | `ndioutTop` | `sourcename` | Send NDI video streams. | +| Syphon Spout In/Out TOP | `syphonspoutinTop` / `syphonspoutoutTop` | `servername` | Inter-app texture sharing. | +| Video Device In TOP | `videodeviceinTop` | `device` | Webcam/capture card input. | +| Feedback TOP | `feedbackTop` | `top` (path to the TOP to feed back) | One-frame delay feedback. Essential for recursive effects. | + +### Converters + +| Operator | Type Name | Direction | Use | +|----------|-----------|-----------|-----| +| CHOP to TOP | `choptopTop` | CHOP -> TOP | Visualize channel data as texture (waveform, spectrum display). | +| TOP to CHOP | `topchopChop` | TOP -> CHOP | Sample texture pixels as channel data. | + +## CHOPs — Channel Operators (Green) + +Time-varying numeric data: audio, animation curves, sensor data, control signals. + +### Generators + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Constant CHOP | `constantChop` | `name0/value0`, `name1/value1`... | Static named channels. Control panel for parameters. | +| LFO CHOP | `lfoChop` | `frequency`, `type` (0=Sin, 1=Tri, 2=Square, 3=Ramp, 4=Pulse), `amp`, `offset`, `phase` | Low frequency oscillator. Animation driver. | +| Noise CHOP | `noiseChop` | `type`, `roughness`, `period`, `amp`, `seed`, `channels` | Smooth random motion. Organic animation. | +| Pattern CHOP | `patternChop` | `type` (0=Sine, 1=Triangle, ...), `length`, `cycles` | Generate waveform patterns. | +| Timer CHOP | `timerChop` | `length`, `play`, `cue`, `cycles` | Countdown/count-up timer with cue points. | +| Count CHOP | `countChop` | `threshold`, `limittype`, `limitmin/max` | Event counter with wrapping/clamping. | + +### Audio + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Audio File In CHOP | `audiofileinChop` | `file`, `volume`, `play`, `speed`, `trim` | Play audio files. | +| Audio Device In CHOP | `audiodeviceinChop` | `device`, `channels` | Live microphone/line input. | +| Audio Spectrum CHOP | `audiospectrumChop` | `size` (FFT size), `outputformat` (0=Power, 1=Magnitude) | FFT frequency analysis. | +| Audio Band EQ CHOP | `audiobandeqChop` | `bands`, `gaindb` per band | Frequency band isolation. | +| Audio Device Out CHOP | `audiodeviceoutChop` | `device` | Audio playback output. | + +### Math/Logic + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Math CHOP | `mathChop` | `preoff`, `gain`, `postoff`, `chanop` (0=Off, 1=Add, 2=Subtract, 3=Multiply...) | Math operations on channels. The Swiss army knife. | +| Logic CHOP | `logicChop` | `preop` (0=Off, 1=AND, 2=OR, 3=XOR, 4=NAND), `convert` | Boolean logic on channels. | +| Filter CHOP | `filterChop` | `type` (0=Low Pass, 1=Band Pass, 2=High Pass, 3=Notch), `cutofffreq`, `filterwidth` | Smooth, dampen, filter signals. | +| Lag CHOP | `lagChop` | `lag1/2`, `overshoot1/2` | Smooth transitions with overshoot. | +| Limit CHOP | `limitChop` | `type` (0=Clamp, 1=Loop, 2=ZigZag), `min/max` | Clamp or wrap channel values. | +| Speed CHOP | `speedChop` | (none significant) | Integrate values (velocity to position, acceleration to velocity). | +| Trigger CHOP | `triggerChop` | `attack`, `peak`, `decay`, `sustain`, `release` | ADSR envelope from trigger events. | +| Select CHOP | `selectChop` | `chop` (path), `channames` | Reference channels from another CHOP. | +| Merge CHOP | `mergeChop` | `align` (0=Extend, 1=Trim to First, 2=Trim to Shortest) | Combine channels from multiple CHOPs. | +| Null CHOP | `nullChop` | (none significant) | Pass-through for organization and referencing. | + +### Input Devices + +| Operator | Type Name | Use | +|----------|-----------|-----| +| Mouse In CHOP | `mouseinChop` | Mouse position, buttons, wheel. | +| Keyboard In CHOP | `keyboardinChop` | Keyboard key states. | +| MIDI In CHOP | `midiinChop` | MIDI note/CC input. | +| OSC In CHOP | `oscinChop` | OSC message input (network). | + +## SOPs — Surface Operators (Blue) + +3D geometry: points, polygons, NURBS, meshes. + +### Generators + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Grid SOP | `gridSop` | `rows`, `cols`, `sizex/y`, `type` (0=Polygon, 1=Mesh, 2=NURBS) | Flat grid mesh. Foundation for displacement, instancing. | +| Sphere SOP | `sphereSop` | `type`, `rows`, `cols`, `radius` | Sphere geometry. | +| Box SOP | `boxSop` | `sizex/y/z` | Box geometry. | +| Torus SOP | `torusSop` | `radiusx/y`, `rows`, `cols` | Donut shape. | +| Circle SOP | `circleSop` | `type`, `radius`, `divs` | Circle/ring geometry. | +| Line SOP | `lineSop` | `dist`, `points` | Line segments. | +| Text SOP | `textSop` | `text`, `fontsizex`, `fontfile`, `extrude` | 3D text geometry. | + +### Modifiers + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Transform SOP | `transformSop` | `tx/ty/tz`, `rx/ry/rz`, `sx/sy/sz` | Transform geometry (translate, rotate, scale). | +| Noise SOP | `noiseSop` | `type`, `amp`, `period`, `roughness` | Deform geometry with noise. | +| Sort SOP | `sortSop` | `ptsort`, `primsort` | Reorder points/primitives. | +| Facet SOP | `facetSop` | `unique`, `consolidate`, `computenormals` | Normals, consolidation, unique points. | +| Merge SOP | `mergeSop` | (none significant) | Combine multiple geometry inputs. | +| Null SOP | `nullSop` | (none significant) | Pass-through. | + +## DATs — Data Operators (White) + +Text, tables, scripts, network data. + +### Core + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Table DAT | `tableDat` | (edit content directly) | Spreadsheet-like data tables. | +| Text DAT | `textDat` | (edit content directly) | Arbitrary text content. Shader code, configs, scripts. | +| Script DAT | `scriptDat` | `language` (0=Python, 1=C++) | Custom callbacks and DAT processing. | +| CHOP Execute DAT | `chopexecDat` | `chop` (path to watch), callbacks | Trigger Python on CHOP value changes. | +| DAT Execute DAT | `datexecDat` | `dat` (path to watch) | Trigger Python on DAT content changes. | +| Panel Execute DAT | `panelexecDat` | `panel` | Trigger Python on UI panel events. | + +### I/O + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Web DAT | `webDat` | `url`, `fetchmethod` (0=GET, 1=POST) | HTTP requests. API integration. | +| TCP/IP DAT | `tcpipDat` | `address`, `port`, `mode` | TCP networking. | +| OSC In DAT | `oscinDat` | `port` | Receive OSC as text messages. | +| Serial DAT | `serialDat` | `port`, `baudrate` | Serial port communication (Arduino, etc.). | +| File In DAT | `fileinDat` | `file` | Read text files. | +| File Out DAT | `fileoutDat` | `file`, `write` | Write text files. | + +### Conversions + +| Operator | Type Name | Direction | Use | +|----------|-----------|-----------|-----| +| DAT to CHOP | `dattochopChop` | DAT -> CHOP | Convert table data to channels. | +| CHOP to DAT | `choptodatDat` | CHOP -> DAT | Convert channel data to table rows. | +| SOP to DAT | `soptodatDat` | SOP -> DAT | Extract geometry data as table. | + +## MATs — Material Operators (Yellow) + +Materials for 3D rendering in Render TOP / Geometry COMP. + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Phong MAT | `phongMat` | `diff_colorr/g/b`, `spec_colorr/g/b`, `shininess`, `colormap`, `normalmap` | Classic Phong shading. Simple, fast. | +| PBR MAT | `pbrMat` | `basecolorr/g/b`, `metallic`, `roughness`, `normalmap`, `emitcolorr/g/b` | Physically-based rendering. Realistic materials. | +| GLSL MAT | `glslMat` | `dat` (shader DAT), custom uniforms | Custom vertex + fragment shaders for 3D. | +| Constant MAT | `constMat` | `colorr/g/b`, `colormap` | Flat unlit color/texture. No shading. | +| Point Sprite MAT | `pointspriteMat` | `colormap`, `scale` | Render points as camera-facing sprites. Great for particles. | +| Wireframe MAT | `wireframeMat` | `colorr/g/b`, `width` | Wireframe rendering. | +| Depth MAT | `depthMat` | `near`, `far` | Render depth buffer as grayscale. | + +## COMPs — Component Operators (Gray) + +Containers, 3D scene elements, UI components. + +### 3D Scene + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Geometry COMP | `geometryComp` | `material` (path), `instancechop` (path), `instancing` (toggle) | Renders geometry with material. Instancing host. | +| Camera COMP | `cameraComp` | `tx/ty/tz`, `rx/ry/rz`, `fov`, `near/far` | Camera for Render TOP. | +| Light COMP | `lightComp` | `lighttype` (0=Point, 1=Directional, 2=Spot, 3=Cone), `dimmer`, `colorr/g/b` | Lighting for 3D scenes. | +| Ambient Light COMP | `ambientlightComp` | `dimmer`, `colorr/g/b` | Ambient lighting. | +| Environment Light COMP | `envlightComp` | `envmap` | Image-based lighting (IBL). | + +### Containers + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Container COMP | `containerComp` | `w`, `h`, `bgcolor1/2/3` | UI container. Holds other COMPs for panel layouts. | +| Base COMP | `baseComp` | (none significant) | Generic container. Networks-inside-networks. | +| Replicator COMP | `replicatorComp` | `template`, `operatorsdat` | Clone a template operator N times from a table. | + +### Utilities + +| Operator | Type Name | Key Parameters | Use | +|----------|-----------|---------------|-----| +| Window COMP | `windowComp` | `winw/h`, `winoffsetx/y`, `monitor`, `borders` | Output window for display/projection. | +| Select COMP | `selectComp` | `rowcol`, `panel` | Select and display content from elsewhere. | +| Engine COMP | `engineComp` | `tox`, `externaltox` | Load external .tox components. Sub-process isolation. | + +## Cross-Family Converter Summary + +| From | To | Operator | Type Name | +|------|-----|----------|-----------| +| CHOP | TOP | CHOP to TOP | `choptopTop` | +| TOP | CHOP | TOP to CHOP | `topchopChop` | +| DAT | CHOP | DAT to CHOP | `dattochopChop` | +| CHOP | DAT | CHOP to DAT | `choptodatDat` | +| SOP | CHOP | SOP to CHOP | `soptochopChop` | +| CHOP | SOP | CHOP to SOP | `choptosopSop` | +| SOP | DAT | SOP to DAT | `soptodatDat` | +| DAT | SOP | DAT to SOP | `dattosopSop` | +| SOP | TOP | (use Render TOP + Geometry COMP) | — | +| TOP | SOP | TOP to SOP | `toptosopSop` | diff --git a/skills/creative/touchdesigner/references/pitfalls.md b/skills/creative/touchdesigner/references/pitfalls.md new file mode 100644 index 000000000..862bdc563 --- /dev/null +++ b/skills/creative/touchdesigner/references/pitfalls.md @@ -0,0 +1,336 @@ +# TouchDesigner MCP — Pitfalls & Lessons Learned + +Hard-won knowledge from real TD sessions. Read this before building anything. + +## Setup & Connection + +### 1. The .tox from the git repo is BROKEN + +The `td/mcp_webserver_base.tox` in the `8beeeaaat/touchdesigner-mcp` git clone is **incomplete**. It's missing the `td_server` Python module (generated by `npm run gen:webserver` which requires Docker). Port 9981 opens, but every route returns 404. + +**Always download the release zip:** +```bash +curl -L -o td.zip \ + "https://github.com/8beeeaaat/touchdesigner-mcp/releases/latest/download/touchdesigner-mcp-td.zip" +unzip -o td.zip -d touchdesigner-mcp-td +``` + +### 2. The release .tox also breaks (frequently) + +Even the correct release .tox fails after drag-and-drop import because `import_modules.py` resolves `modules/` via `parent().par.externaltox.eval()` — a relative path that often goes wrong. Symptoms: port 9981 listens, all routes 404, TD Textport shows `[ERROR] Failed to setup modules`. + +**The custom API handler (`scripts/custom_api_handler.py`) is more reliable.** It has zero external module dependencies — just a WebServer DAT + Text DAT callback. The skill's setup workflow should try the .tox first, test with `curl`, and auto-deploy the handler if 404. + +### 3. You CANNOT automate the .tox import from outside TD + +TD has no CLI flag to import a .tox. macOS blocks keystroke injection via System Events for security. The only way to get code into TD from outside is: +- Have a WebServer DAT already running (chicken-and-egg) +- AppleScript to open Textport + clipboard paste (fragile, not always reliable) +- User manually drags the .tox or pastes a script + +**Plan for one manual step** from the user (either .tox drag-drop or Textport paste). Make it as frictionless as possible: `open -R /path/to/file` to reveal in Finder. + +### 4. The npm package name is `touchdesigner-mcp-server` (not `@anthropic/...`) + +The Hermes config should use: +```yaml +command: npx +args: ["-y", "touchdesigner-mcp-server@latest"] +``` + +### 5. MCP tools may register but not be callable + +Hermes may report "17 MCP tool(s) now available" but the tools aren't exposed as function calls. Use the REST API directly via `curl` in `execute_code` as a reliable fallback: +```python +def td_exec(script): + escaped = json.dumps({"script": script}) + cmd = f"curl -s -X POST -H 'Content-Type: application/json' -d {shlex.quote(escaped)} 'http://127.0.0.1:9981/api/td/server/exec'" + return json.loads(terminal(cmd)['output']) +``` + +## TD WebServer DAT Quirks + +### 6. Response body goes in `response['data']`, NOT `response['body']` + +When writing custom WebServer DAT handlers, the response payload must be set on the `data` key: +```python +response['data'] = json.dumps({"result": 42}) # ✓ works +response['body'] = json.dumps({"result": 42}) # ✗ ignored +``` + +### 7. Request POST body comes as BYTES in `request['data']` + +Not `request['body']`, and it's `bytes` not `str`: +```python +raw = request.get('data', b'') +if isinstance(raw, bytes): + raw = raw.decode('utf-8') +body = json.loads(raw) if raw else {} +``` + +### 8. Non-Commercial license caps resolution at 1280×1280 + +Setting `resolutionw=1920` silently clamps to 1280. Always check effective resolution after creation: +```python +n.cook(force=True) +actual = str(n.width) + 'x' + str(n.height) +``` + +## Parameter Names + +### 9. NEVER hardcode parameter names — always discover + +Parameter names change between TD versions. What works in 099 may not work in 098 or 2023.x. Always run discovery first: +```python +n = root.create(glslTOP, '_test') +pars = [(p.name, type(p.val).__name__) for p in n.pars()] +n.destroy() +``` + +Known differences from docs/online references: +| What docs say | TD 099 actual | Notes | +|---------------|---------------|-------| +| `dat` | `pixeldat` | GLSL TOP pixel shader DAT | +| `colora` | `alpha` | Constant TOP alpha | +| `sizex` / `sizey` | `size` | Blur TOP (single value) | +| `fontr/g/b/a` | `fontcolorr/g/b/a` | Text TOP font color (r/g/b) | +| `fontcolora` | `fontalpha` | Text TOP font alpha (NOT `fontcolora`) | +| `bgcolora` | `bgalpha` | Text TOP bg alpha | +| `value1name` | `vec0name` | GLSL TOP uniform name | + +### 10. Use `safe_par()` pattern for cross-version compatibility + +```python +def safe_par(node, name, value): + p = getattr(node.par, name, None) + if p is not None: + p.val = value + return True + return False +``` + +### 11. `td.tdAttributeError` crashes the whole script + +If you do `node.par.nonexistent = value`, TD raises `tdAttributeError` and **stops the entire script**. There's no way to catch it with try/except in some TD versions. Always check with `getattr` first or use `safe_par()`. + +## GLSL Shaders + +### 12. `uTDCurrentTime` does NOT exist in TD 099 + +The GLSL builtin for time was removed or never existed in some builds. Feed time via a 1×1 Constant TOP input. **CRITICAL: set format to `rgba32float`** — the default 8-bit format clamps values to 0-1, so `absTime.seconds % 1000.0` gets clamped and the GLSL shader sees a frozen time value of 1.0: +```python +t = root.create(constantTOP, 'time_driver') +t.par.format = 'rgba32float' # ← REQUIRED! Without this, time is stuck at 1.0 +t.par.outputresolution = 'custom' +t.par.resolutionw = 1 +t.par.resolutionh = 1 +t.par.colorr.expr = "absTime.seconds % 1000.0" +t.par.colorg.expr = "int(absTime.seconds / 1000.0)" +t.outputConnectors[0].connect(glsl.inputConnectors[0]) +``` +In GLSL: +```glsl +vec4 td = texture(sTD2DInputs[0], vec2(0.5)); +float t = td.r + td.g * 1000.0; +``` + +### 13. GLSL compile errors are silent in the API + +The GLSL TOP shows a yellow warning triangle in the UI but `node.errors()` may return empty string. Check `node.warnings()` too, and create an Info DAT pointed at the GLSL TOP to read the actual compiler output. + +### 14. TD GLSL uses `vUV.st` not `gl_FragCoord` + +Standard GLSL patterns don't work. TD provides: +- `vUV.st` — UV coordinates (0-1) +- `uTDOutputInfo.res.zw` — resolution +- `sTD2DInputs[0]` — input textures +- `layout(location = 0) out vec4 fragColor` — output + +## Node Management + +### 15. Destroying nodes while iterating `root.children` causes `tdError` + +The iterator is invalidated when a child is destroyed. Always snapshot first: +```python +kids = list(root.children) # snapshot +for child in kids: + if child.valid: # check — earlier destroys may cascade + child.destroy() +``` + +### 16. Feedback TOP: use `top` parameter, NOT direct input wire + +In TD 099, the feedbackTOP's `top` parameter references which TOP to delay. **Do not also wire that TOP directly into the feedback's input** — this creates a real cook dependency loop (warning flood, potential crash). The "Not enough sources" error on feedbackTOP is benign and resolves after a few frames of playback. + +Correct setup: +```python +fb = root.create(feedbackTOP, 'fb_delay') +fb.par.top = comp.path # reference only — no wire to fb input +fb.outputConnectors[0].connect(xf) # fb output -> transform -> fade -> comp +``` + +The resulting "Cook dependency loop detected" **warning** on the transform/fade chain is expected and correct — that's what feedback loops do. It's informational, not an error. + +### 16. GLSL TOP auto-creates companion nodes + +Creating a `glslTOP` also creates `name_pixel` (Text DAT), `name_info` (Info DAT), and `name_compute` (Text DAT). These are visible in the network and count toward node totals. Don't be alarmed by "extra" nodes. + +### 17. The default project root is `/project1` + +New TD files start with `/project1` as the main container. System nodes live at `/`, `/ui`, `/sys`, `/local`, `/perform`. Don't create user nodes outside `/project1`. + +### 18. `open -R` reveals the file but can't automate the drag + +Use `open -R /path/to/file.tox` to open Finder highlighting the file. The user must then drag it into TD manually. No AppleScript workaround exists on modern macOS due to accessibility restrictions. + +## Workflow + +### 19. Always verify after building — errors are silent + +Node errors and broken connections produce no output. Always check: +```python +for c in list(root.children): + e = c.errors() + w = c.warnings() + if e: print(c.name, 'ERR:', e) + if w: print(c.name, 'WARN:', w) +``` + +### 20. Build in one big `execute_python_script` call, not many small ones + +Each API round-trip adds latency. Bundle node creation + parameter setting + wiring into a single script that creates everything at once, then verify in one final call. + +### 21. Window COMP param for display target is `winop` (not `top` or `window`) + +To display output in a separate window: +```python +win = root.create(windowCOMP, 'display') +win.par.winop = '/project1/logo_out' # ← this is the correct param +win.par.winw = 1280; win.par.winh = 720 +win.par.winopen.pulse() # open the window +``` + +### 22. Save the project to make API persistent across TD restarts + +After deploying the custom API handler, save the project: +```python +td_exec("project.save(os.path.expanduser('~/Documents/HermesAgent.toe'))") +``` +TD auto-opens the last saved project on launch. The API handler is now baked into the .toe file — next time TD opens, port 9981 is live with zero manual steps. To explicitly launch with this project: `open /Applications/TouchDesigner.app ~/Documents/HermesAgent.toe` + +### 23. `sample()` returns frozen pixels when called from WebServer DAT callback + +`out.sample(x, y)` called from inside the API handler's `exec()` returns pixels from a single cook snapshot. It does NOT update between multiple API calls in quick succession. To verify animation is working, either: +- Compare samples with a 2+ second delay between separate `td_exec()` calls +- Use `screencapture` on the display window +- Check `absTime.seconds` is advancing and shader uses time correctly + +### 22. `outputresolution` is a string menu, not an integer + +### 25. MovieFileOut TOP: H.264/H.265 requires Commercial license + +In Non-Commercial TD 099, encoding with H.264 or H.265 produces an error: "GPU Accelerated H.264/H.265 Encoding requires a Commercial license". Use Motion JPEG instead: +```python +rec = root.create(moviefileoutTOP, 'recorder') +rec.par.type = 'movie' +rec.par.file = '/tmp/output.mov' +rec.par.videocodec = 'mjpa' # Motion JPEG — works on Non-Commercial +``` + +For image sequences, use `type = 'imagesequence'` and the file param **must** use `me.fileSuffix`: +```python +rec.par.type = 'imagesequence' +rec.par.imagefiletype = 'png' +rec.par.file.expr = "'/tmp/frames/out' + me.fileSuffix" +``` + +### 26. MovieFileOut `.record()` method may not exist + +In TD 099, there is no `.record()` method on moviefileoutTOP. Use the toggle parameter instead: +```python +rec.par.record = True # start recording +rec.par.record = False # stop recording +``` + +When setting the file path and starting recording in the same script, use `run()` with `delayFrames` to avoid a race condition where the old filename is used: +```python +rec.par.file = '/tmp/new_output.mov' +run("op('/project1/recorder').par.record = True", delayFrames=2) +``` + +### 27. TOP.save() captures same frame when called rapidly + +`op('null1').save(path)` captures the current GPU texture at call time. When called multiple times in a single script (or rapid API calls), TD doesn't cook new frames between saves — all exported PNGs will be identical. To get unique frames, use the MovieFileOut TOP which records in real-time from TD's cook cycle. + +### 28. AudioFileIn CHOP: cue before recording for sync + +When recording audio-reactive visuals, always cue the audio to the start before beginning the recording. Otherwise the visuals are synced to wherever the audio happens to be in its playback: +```python +op('/project1/audio_in').par.cue.pulse() # reset to start +run("op('/project1/recorder').par.record = True", delayFrames=3) +``` +The audio plays via `playmode=0` (Locked to Timeline), so it stays in sync with TD's frame clock. Use `audiodeviceoutCHOP` to hear the audio during recording. + +### 29. Audio Spectrum CHOP output is weak — boost with Math CHOP + +The raw AudioSpectrum CHOP output has very small values (often 0.001-0.05 range). When fed directly to CHOP To TOP → GLSL, the shader barely reacts. Always insert a Math CHOP with `gain=5` (or higher) between the spectrum and the CHOP To TOP to get usable 0-1 range values in the shader. + +### 30. CHOP To TOP texture size — Resample to 256 first + +`choptoTOP` creates a texture where width = number of samples. An AudioSpectrum CHOP at 44100Hz has ~24000 samples — creating a 24000×1 texture is wasteful. Use a Resample CHOP set to 256 or 512 samples before the CHOP To TOP for an efficient spectrum texture. + +### 31. CHOP To TOP has NO input connectors — use par.chop reference + +`choptoTOP` does NOT have input connectors. `resample.outputConnectors[0].connect(chop_to_top.inputConnectors[0])` silently does nothing. Use the `chop` parameter instead: +```python +spec_tex = root.create(choptoTOP, 'spectrum_tex') +spec_tex.par.chop = resample # ← correct: parameter reference +# NOT: resample.outputConnectors[0].connect(spec_tex.inputConnectors[0]) # ← WRONG: no connectors +``` + +### 22. `outputresolution` is a string menu, not an integer + +The `outputresolution` param is a menu with string values: +``` +menuNames: ['useinput','eighth','quarter','half','2x','4x','8x','fit','limit','custom','parpanel'] +``` +Always use the string form. Setting `outputresolution = 9` may silently fail. +```python +node.par.outputresolution = 'custom' # ✓ correct +node.par.resolutionw = 1280; node.par.resolutionh = 720 +``` +Discover valid values: `list(node.par.outputresolution.menuNames)` + +### 23. Large GLSL shaders break curl JSON escaping + +GLSL code full of single/double quotes, backslashes, and special chars will corrupt the JSON payload when sent via `curl -d`. **Write the shader to a temp file and load it in TD:** +```python +# Agent side: write shader to /tmp/shader.glsl via write_file +# TD side (via td_exec): +sd = root.create(textDAT, 'shader_code') +with open('/tmp/shader.glsl', 'r') as f: + sd.text = f.read() +``` +This avoids all escaping issues. The TD Python environment has full filesystem access. + +### 24. TD crashes lose everything — the WebServer DAT must be re-deployed + +If TD crashes (common with heavy GLSL or rapid-fire API calls), all nodes including the WebServer DAT are lost. On relaunch, port 9981 is dead. Recovery: +1. Detect: `curl` returns exit code 7 (connection refused) or `lsof -i :9981` shows nothing +2. Check: `pgrep TouchDesigner` to confirm TD is running +3. Re-deploy: user must paste `exec(open('...custom_api_handler.py').read())` into Textport again +4. Verify: poll port 9981 until API responds + +The `td_exec()` helper should handle this gracefully: +```python +def td_exec(script): + escaped = json.dumps({"script": script}) + cmd = f"curl -s --max-time 15 -X POST -H 'Content-Type: application/json' -d {shlex.quote(escaped)} 'http://127.0.0.1:9981/api/td/server/exec'" + r = terminal(cmd, timeout=20) + if r.get('exit_code') == 7: + return {'error': 'TD not responding — WebServer DAT may need re-deploy'} + try: + return json.loads(r['output']) + except: + return {'error': 'Bad response', 'raw': r['output'][:200]} +``` diff --git a/skills/creative/touchdesigner/references/python-api.md b/skills/creative/touchdesigner/references/python-api.md new file mode 100644 index 000000000..2b8d8847f --- /dev/null +++ b/skills/creative/touchdesigner/references/python-api.md @@ -0,0 +1,443 @@ +# TouchDesigner Python API Reference + +## The td Module + +TouchDesigner's Python environment auto-imports the `td` module. All TD-specific classes, functions, and constants live here. Scripts inside TD (Script DATs, CHOP/DAT Execute callbacks, Extensions) have full access. + +When using the MCP `execute_python_script` tool, these globals are pre-loaded: +- `op` — shortcut for `td.op()`, finds operators by path +- `ops` — shortcut for `td.ops()`, finds multiple operators by pattern +- `me` — the operator running the script (not meaningful via MCP — will be the WebServer DAT) +- `parent` — shortcut for `me.parent()` +- `project` — the root project component +- `td` — the full td module + +## Finding Operators: op() and ops() + +### op(path) — Find a single operator + +```python +# Absolute path (always works from MCP) +node = op('/project1/noise1') + +# Relative path (relative to current operator — only in Script DATs) +node = op('noise1') # sibling +node = op('../noise1') # parent's sibling + +# Returns None if not found (does NOT raise) +node = op('/project1/nonexistent') # None +``` + +### ops(pattern) — Find multiple operators + +```python +# Glob patterns +nodes = ops('/project1/noise*') # all nodes starting with "noise" +nodes = ops('/project1/*') # all direct children +nodes = ops('/project1/container1/*') # all children of container1 + +# Returns a tuple of operators (may be empty) +for n in ops('/project1/*'): + print(n.name, n.OPType) +``` + +### Navigation from a node + +```python +node = op('/project1/noise1') + +node.name # 'noise1' +node.path # '/project1/noise1' +node.OPType # 'noiseTop' +node.type # +node.family # 'TOP' + +# Parent / children +node.parent() # the parent COMP +node.parent().children # all siblings + self +node.parent().findChildren(name='noise*') # filtered + +# Type checking +node.isTOP # True +node.isCHOP # False +node.isSOP # False +node.isDAT # False +node.isMAT # False +node.isCOMP # False +``` + +## Parameters + +Every operator has parameters accessed via the `.par` attribute. + +### Reading parameters + +```python +node = op('/project1/noise1') + +# Direct access +node.par.seed.val # current evaluated value (may be an expression result) +node.par.seed.eval() # same as .val +node.par.seed.default # default value +node.par.monochrome.val # boolean parameters: True/False + +# List all parameters +for p in node.pars(): + print(f"{p.name}: {p.val} (default: {p.default})") + +# Filter by page (parameter group) +for p in node.pars('Noise'): # page name + print(f"{p.name}: {p.val}") +``` + +### Setting parameters + +```python +# Direct value setting +node.par.seed.val = 42 +node.par.monochrome.val = True +node.par.resolutionw.val = 1920 +node.par.resolutionh.val = 1080 + +# String parameters +op('/project1/text1').par.text.val = 'Hello World' + +# File paths +op('/project1/moviefilein1').par.file.val = '/path/to/video.mp4' + +# Reference another operator (for "dat", "chop", "top" type parameters) +op('/project1/glsl1').par.dat.val = '/project1/shader_code' +``` + +### Parameter expressions + +```python +# Python expressions that evaluate dynamically +node.par.seed.expr = "me.time.frame" +node.par.tx.expr = "math.sin(me.time.seconds * 2)" + +# Reference another parameter +node.par.brightness1.expr = "op('/project1/constant1').par.value0.val" + +# Export (one-way binding from CHOP to parameter) +# This makes the parameter follow a CHOP channel value +op('/project1/noise1').par.seed.val # can also be driven by exports +``` + +### Parameter types + +| Type | Python Type | Example | +|------|------------|---------| +| Float | `float` | `node.par.brightness1.val = 0.5` | +| Int | `int` | `node.par.seed.val = 42` | +| Toggle | `bool` | `node.par.monochrome.val = True` | +| String | `str` | `node.par.text.val = 'hello'` | +| Menu | `int` (index) or `str` (label) | `node.par.type.val = 'sine'` | +| File | `str` (path) | `node.par.file.val = '/path/to/file'` | +| OP reference | `str` (path) | `node.par.dat.val = '/project1/text1'` | +| Color | separate r/g/b/a floats | `node.par.colorr.val = 1.0` | +| XY/XYZ | separate x/y/z floats | `node.par.tx.val = 0.5` | + +## Creating and Deleting Operators + +```python +# Create via parent component +parent = op('/project1') +new_node = parent.create(noiseTop) # using class reference +new_node = parent.create(noiseTop, 'my_noise') # with custom name + +# The MCP create_td_node tool handles this automatically: +# create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="my_noise") + +# Delete +node = op('/project1/my_noise') +node.destroy() + +# Copy +original = op('/project1/noise1') +copy = parent.copy(original, name='noise1_copy') +``` + +## Connections (Wiring Operators) + +### Output to Input connections + +```python +# Connect noise1's output to level1's input +op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1')) + +# Connect to specific input index (for multi-input operators like Composite) +op('/project1/noise1').outputConnectors[0].connect(op('/project1/composite1').inputConnectors[0]) +op('/project1/text1').outputConnectors[0].connect(op('/project1/composite1').inputConnectors[1]) + +# Disconnect all outputs +op('/project1/noise1').outputConnectors[0].disconnect() + +# Query connections +node = op('/project1/level1') +inputs = node.inputs # list of connected input operators +outputs = node.outputs # list of connected output operators +``` + +### Connection patterns for common setups + +```python +# Linear chain: A -> B -> C -> D +ops_list = [op(f'/project1/{name}') for name in ['noise1', 'level1', 'blur1', 'null1']] +for i in range(len(ops_list) - 1): + ops_list[i].outputConnectors[0].connect(ops_list[i+1]) + +# Fan-out: A -> B, A -> C, A -> D +source = op('/project1/noise1') +for target_name in ['level1', 'composite1', 'transform1']: + source.outputConnectors[0].connect(op(f'/project1/{target_name}')) + +# Merge: A + B + C -> Composite +comp = op('/project1/composite1') +for i, source_name in enumerate(['noise1', 'text1', 'ramp1']): + op(f'/project1/{source_name}').outputConnectors[0].connect(comp.inputConnectors[i]) +``` + +## DAT Content Manipulation + +### Text DATs + +```python +dat = op('/project1/text1') + +# Read +content = dat.text # full text as string + +# Write +dat.text = "new content" +dat.text = '''multi +line +content''' + +# Append +dat.text += "\nnew line" +``` + +### Table DATs + +```python +dat = op('/project1/table1') + +# Read cell +val = dat[0, 0] # row 0, col 0 +val = dat[0, 'name'] # row 0, column named 'name' +val = dat['key', 1] # row named 'key', col 1 + +# Write cell +dat[0, 0] = 'value' + +# Read row/col +row = dat.row(0) # list of Cell objects +col = dat.col('name') # list of Cell objects + +# Dimensions +rows = dat.numRows +cols = dat.numCols + +# Append row +dat.appendRow(['col1_val', 'col2_val', 'col3_val']) + +# Clear +dat.clear() + +# Set entire table +dat.clear() +dat.appendRow(['name', 'value', 'type']) +dat.appendRow(['frequency', '440', 'float']) +dat.appendRow(['amplitude', '0.8', 'float']) +``` + +## Time and Animation + +```python +# Global time +td.absTime.frame # absolute frame number (never resets) +td.absTime.seconds # absolute seconds + +# Timeline time (affected by play/pause/loop) +me.time.frame # current frame on timeline +me.time.seconds # current seconds on timeline +me.time.rate # FPS setting + +# Timeline control (via execute_python_script) +project.play = True +project.play = False +project.frameRange = (1, 300) # set timeline range + +# Cook frame (when operator was last computed) +node.cookFrame +node.cookTime +``` + +## Extensions (Custom Python Classes on Components) + +Extensions add custom Python methods and attributes to COMPs. + +```python +# Create extension on a Base COMP +base = op('/project1/myBase') + +# The extension class is defined in a Text DAT inside the COMP +# Typically named 'ExtClass' with the extension code: + +extension_code = ''' +class MyExtension: + def __init__(self, ownerComp): + self.ownerComp = ownerComp + self.counter = 0 + + def Reset(self): + self.counter = 0 + + def Increment(self): + self.counter += 1 + return self.counter + + @property + def Count(self): + return self.counter +''' + +# Write extension code to DAT inside the COMP +op('/project1/myBase/extClass').text = extension_code + +# Configure the extension on the COMP +base.par.extension1 = 'extClass' # name of the DAT +base.par.promoteextension1 = True # promote methods to parent + +# Call extension methods +base.Increment() # calls MyExtension.Increment() +count = base.Count # accesses MyExtension.Count property +base.Reset() +``` + +## Useful Built-in Modules + +### tdu — TouchDesigner Utilities + +```python +import tdu + +# Dependency tracking (reactive values) +dep = tdu.Dependency(initial_value) +dep.val = new_value # triggers dependents to recook + +# File path utilities +tdu.expandPath('$HOME/Desktop/output.mov') + +# Math +tdu.clamp(value, min, max) +tdu.remap(value, from_min, from_max, to_min, to_max) +``` + +### TDFunctions + +```python +from TDFunctions import * + +# Commonly used utilities +clamp(value, low, high) +remap(value, inLow, inHigh, outLow, outHigh) +interp(value1, value2, t) # linear interpolation +``` + +### TDStoreTools — Persistent Storage + +```python +from TDStoreTools import StorageManager + +# Store data that survives project reload +me.store('myKey', 'myValue') +val = me.fetch('myKey', default='fallback') + +# Storage dict +me.storage['key'] = value +``` + +## Common Patterns via execute_python_script + +### Build a complete chain + +```python +# Create a complete audio-reactive noise chain +parent = op('/project1') + +# Create operators +audio_in = parent.create(audiofileinChop, 'audio_in') +spectrum = parent.create(audiospectrumChop, 'spectrum') +chop_to_top = parent.create(choptopTop, 'chop_to_top') +noise = parent.create(noiseTop, 'noise1') +level = parent.create(levelTop, 'level1') +null_out = parent.create(nullTop, 'out') + +# Wire the chain +audio_in.outputConnectors[0].connect(spectrum) +spectrum.outputConnectors[0].connect(chop_to_top) +noise.outputConnectors[0].connect(level) +level.outputConnectors[0].connect(null_out) + +# Set parameters +audio_in.par.file = '/path/to/music.wav' +audio_in.par.play = True +spectrum.par.size = 512 +noise.par.type = 1 # Sparse +noise.par.monochrome = False +noise.par.resolutionw = 1920 +noise.par.resolutionh = 1080 +level.par.opacity = 0.8 +level.par.gamma1 = 0.7 +``` + +### Query network state + +```python +# Get all TOPs in the project +tops = [c for c in op('/project1').findChildren(type=TOP)] +for t in tops: + print(f"{t.path}: {t.OPType} {'ERROR' if t.errors() else 'OK'}") + +# Find all operators with errors +def find_errors(parent_path='/project1'): + parent = op(parent_path) + errors = [] + for child in parent.findChildren(depth=-1): + if child.errors(): + errors.append((child.path, child.errors())) + return errors + +result = find_errors() +``` + +### Batch parameter changes + +```python +# Set parameters on multiple nodes at once +settings = { + '/project1/noise1': {'seed': 42, 'monochrome': False, 'resolutionw': 1920}, + '/project1/level1': {'brightness1': 1.2, 'gamma1': 0.8}, + '/project1/blur1': {'sizex': 5, 'sizey': 5}, +} + +for path, params in settings.items(): + node = op(path) + if node: + for key, val in params.items(): + setattr(node.par, key, val) +``` + +## Python Version and Packages + +TouchDesigner bundles Python 3.11+ (as of TD 2024) with these pre-installed: +- **numpy** — array operations, fast math +- **scipy** — signal processing, FFT +- **OpenCV** (cv2) — computer vision +- **PIL/Pillow** — image processing +- **requests** — HTTP client +- **json**, **re**, **os**, **sys** — standard library + +Custom packages can be installed to TD's Python site-packages directory. See TD documentation for the exact path per platform. diff --git a/skills/creative/touchdesigner/references/troubleshooting.md b/skills/creative/touchdesigner/references/troubleshooting.md new file mode 100644 index 000000000..30ad580f4 --- /dev/null +++ b/skills/creative/touchdesigner/references/troubleshooting.md @@ -0,0 +1,274 @@ +# TouchDesigner Troubleshooting + +> See `references/pitfalls.md` for the comprehensive lessons-learned list. + +## Quick Connection Diagnostic + +```bash +lsof -i :9981 -P -n | grep LISTEN # Step 1: Is TD listening? +curl -s http://127.0.0.1:9981/api/td/server/td # Step 2: API working? +``` + +| Symptom | Cause | Fix | +|---------|-------|-----| +| Connection refused | No WebServer DAT | Deploy `scripts/custom_api_handler.py` in TD Textport | +| HTTP 404 on all routes | .tox module import failed | Deploy custom handler (pitfalls #1-2) | +| HTTP 200, empty body | Response in wrong key | Handler uses `response['data']` not `response['body']` (pitfalls #6) | +| HTTP 200, JSON body | Working | Proceed to discovery | +| MCP tools not callable | Normal — use curl instead | `td_exec()` pattern in SKILL.md works without MCP | + +## Node Creation Issues + +### "Node type not found" error + +**Cause:** Wrong `nodeType` string in `create_td_node`. + +**Fix:** Use camelCase with family suffix. Common mistakes: +- Wrong: `NoiseTop`, `noise_top`, `NOISE TOP`, `Noise` +- Right: `noiseTop` +- Wrong: `AudioSpectrum`, `audio_spectrum_chop` +- Right: `audiospectrumChop` + +**Discovery method:** Use `get_td_classes` to see available types, or `execute_python_script` with `dir(td)` filtered for operator classes. + +### Node created but not visible in TD + +**Cause:** Node was created in a different container than expected, or TD viewport is looking at a different network. + +**Fix:** Check `parentPath` — use absolute paths like `/project1`. Verify with `get_td_nodes(parentPath="/project1")`. + +### Cannot create node inside a non-COMP + +**Cause:** Only COMP operators (Container, Base, Geometry, etc.) can contain child operators. You cannot create nodes inside a TOP, CHOP, SOP, DAT, or MAT. + +**Fix:** Create a Container COMP or Base COMP first, then create nodes inside it. + +## Parameter Issues + +### Parameter not updating + +**Causes:** +1. **Wrong parameter name.** TD parameter names change across versions. Run the discovery script (SKILL.md Step 0) or use `get_td_node_parameters` to discover exact names for your TD version. Never trust online docs or this skill's tables — always verify. +2. **Parameter is read-only.** Some parameters are computed/locked. +3. **Wrong value type.** Menu parameters need integer index or exact string label. +4. **Parameter has an expression.** If `node.par.X.expr` is set, `.val` is ignored. Clear the expression first. + +**Discovery-based approach (preferred):** +```python +execute_python_script(script=""" +n = op('/project1/mynode') +pars = [(p.name, type(p.val).__name__, p.val) for p in n.pars() + if any(k in p.name.lower() for k in ['color', 'size', 'dat', 'font', 'alpha'])] +result = pars +""") +``` + +**Safe parameter setter pattern:** +```python +def safe_par(node, name, value): + p = getattr(node.par, name, None) + if p is not None: + p.val = value + return True + return False # param doesn't exist in this TD version +``` + +### Common parameter name gotchas + +| What you expect | Actual name | Notes | +|----------------|-------------|-------| +| `width` | `resolutionw` | TOP resolution width | +| `height` | `resolutionh` | TOP resolution height | +| `filepath` | `file` | File path parameter | +| `color` | `colorr`, `colorg`, `colorb`, `colora` | Separate RGBA components | +| `position_x` | `tx` | Translate X | +| `rotation` | `rz` | Rotate Z (2D rotation) | +| `scale` | `sx`, `sy` | Separate X/Y scale | +| `blend_mode` | `operand` | Composite TOP blend mode (integer) | +| `opacity` | `opacity` | On Level TOP (this one is correct!) | + +### Composite TOP operand values + +| Mode | Index | +|------|-------| +| Over | 0 | +| Under | 1 | +| Inside | 2 | +| Add | 3 | +| Subtract | 4 | +| Difference | 5 | +| Multiply | 18 | +| Screen | 27 | +| Maximum | 13 | +| Minimum | 14 | +| Average | 28 | + +## Connection/Wiring Issues + +### Connections not working + +**Causes:** +1. **Cross-family wiring.** TOPs can only connect to TOPs, CHOPs to CHOPs, etc. Use converter operators to bridge families. +2. **Wrong connector index.** Most operators have one output connector (index 0). Multi-output operators may need index 1, 2, etc. +3. **Node path wrong.** Verify paths are absolute and correctly spelled. + +**Verify connections:** +```python +execute_python_script(script=""" +node = op('/project1/level1') +result = { + 'inputs': [i.path if i else None for i in node.inputs], + 'outputs': [o.path if o else None for o in node.outputs] +} +""") +``` + +### Feedback loops causing errors + +**Symptom:** "Circular dependency" or infinite cook loop. + +**Fix:** Always use a Feedback TOP (or a Null TOP with a one-frame delay) to break the loop: +``` +A -> B -> Feedback(references B) -> A +``` +Never create A -> B -> A directly. + +## Performance Issues + +### Low FPS / choppy output + +**Common causes and fixes:** + +1. **Resolution too high.** Start at 1920x1080, only go higher if GPU handles it. +2. **Too many operators.** Each operator has GPU/CPU overhead. Consolidate where possible. +3. **Expensive shader.** GLSL TOPs with complex math per-pixel drain GPU. Profile with TD's Performance Monitor (F2). +4. **No GPU instancing.** Rendering 1000 separate geometry objects is much slower than 1 instanced geometry. +5. **Unnecessary cooks.** Operators that don't change frame-to-frame still recook if inputs change. Use Null TOPs to cache stable results. +6. **Large texture transfers.** TOP to CHOP and CHOP to TOP involve GPU-CPU memory transfers. Minimize these. + +**Performance Monitor:** +```python +execute_python_script(script="td.performanceMonitor = True") +# After testing: +execute_python_script(script="td.performanceMonitor = False") +``` + +### Memory growing over time + +**Causes:** +- Cache TOPs with high `length` value +- Feedback loops without brightness decay (values accumulate) +- Table DATs growing without clearing +- Movie File In loading many unique frames + +**Fix:** Always add slight decay in feedback loops (Level TOP with `opacity=0.98` or multiply blend). Clear tables periodically. + +## Export / Recording Issues + +### Movie File Out not recording + +**Checklist:** +1. Is the `record` parameter toggled on? `update_td_node_parameters(properties={"record": true})` +2. Is an input connected? The Movie File Out needs a TOP input. +3. Is the output path valid and writable? Check `file` parameter. +4. Is the codec available? H.264 (type 4) is most reliable. + +### Exported video is black + +**Causes:** +1. The TOP chain output is all black (brightness too low). +2. The input TOP has errors (check with `get_td_node_errors`). +3. Resolution mismatch — the output may be wrong resolution. + +**Debug:** Check the input TOP's actual pixel values: +```python +execute_python_script(script=""" +import numpy as np +top = op('/project1/out') +arr = top.numpyArray(delayed=True) +result = {'mean': float(arr.mean()), 'max': float(arr.max()), 'shape': list(arr.shape)} +""") +``` + +### .tox export losing connections + +**Note:** When saving a component as .tox, only the component and its internal children are saved. External connections (wires to operators outside the component) are lost. Design self-contained components. + +## Python Scripting Issues + +### execute_python_script returns empty result + +**Causes:** +1. The script used `exec()` semantics (multi-line) but didn't set `result`. +2. The last expression has no return value (e.g., `print()` returns None). + +**Fix:** Explicitly set `result`: +```python +execute_python_script(script=""" +nodes = op('/project1').findChildren(type=TOP) +result = len(nodes) # explicitly set return value +""") +``` + +### Script errors not clear + +**Check stderr in the response.** The MCP server captures both stdout and stderr from script execution. Error tracebacks appear in stderr. + +### Module not found in TD Python + +**Cause:** TD's Python environment may not have the module. TD bundles numpy, scipy, opencv, Pillow, and requests. Other packages need manual installation. + +**Check available packages:** +```python +execute_python_script(script=""" +import sys +result = [p for p in sys.path] +""") +``` + +## Common Workflow Pitfalls + +### Building before verifying connection + +Always call `get_td_info` first. If TD isn't running or the WebServer DAT isn't loaded, all subsequent tool calls will fail. + +### Not checking errors after building + +Always call `get_td_node_errors(nodePath="/project1")` after creating and wiring a network. Broken connections and missing references are silent until you check. + +### Creating too many operators in one go + +When building complex networks, create in logical groups: +1. Create all operators in a section +2. Wire that section +3. Verify with `get_td_node_errors` +4. Move to the next section + +Don't create 50 operators, wire them all, then discover something was wrong 30 operators ago. + +### Parameter expressions vs static values + +If you set `node.par.X.val = 5` but there's an expression on that parameter (`node.par.X.expr`), the expression wins. To use a static value, clear the expression first: +```python +execute_python_script(script=""" +op('/project1/noise1').par.seed.expr = '' # clear expression +op('/project1/noise1').par.seed.val = 42 # now static value works +""") +``` + +### Forgetting to start audio playback + +Audio File In CHOP won't produce data unless `play` is True and a valid `file` is set: +``` +update_td_node_parameters(nodePath="/project1/audio_in", + properties={"file": "/path/to/music.wav", "play": true}) +``` + +### GLSL shader compilation errors + +If a GLSL TOP shows errors after setting shader code: +1. Check the shader code in the Text DAT for syntax errors +2. Ensure the GLSL version is compatible (TD uses GLSL 3.30+) +3. Input sampler name must be `sTD2DInputs[0]` (not custom names) +4. Output must use `layout(location = 0) out vec4 fragColor` +5. UV coordinates come from `vUV.st` (not `gl_FragCoord`) diff --git a/skills/creative/touchdesigner/scripts/custom_api_handler.py b/skills/creative/touchdesigner/scripts/custom_api_handler.py new file mode 100644 index 000000000..fd3772a87 --- /dev/null +++ b/skills/creative/touchdesigner/scripts/custom_api_handler.py @@ -0,0 +1,140 @@ +""" +Custom API Handler for TouchDesigner WebServer DAT +=================================================== +Use this when mcp_webserver_base.tox fails to load its modules +(common — the .tox relies on relative paths to a modules/ folder +that often break during import). + +Paste into TD Textport or run via exec(open('...').read()): + Creates a WebServer DAT + Text DAT callback handler on port 9981. + Implements the core endpoints the MCP server expects. + +After running, test with: + curl http://127.0.0.1:9981/api/td/server/td +""" + +root = op('/project1') + +# Remove broken webserver if present +old = op('/project1/mcp_webserver_base') +if old and old.valid: + old.destroy() + +# Create WebServer DAT +ws = root.create(webserverDAT, 'api_server') +ws.par.port = 9981 +ws.par.active = True +ws.nodeX = -800; ws.nodeY = 500 + +# Create callback handler +cb = root.create(textDAT, 'api_handler') +cb.nodeX = -800; cb.nodeY = 400 +cb.text = r''' +import json, traceback, io, sys + +def onHTTPRequest(webServerDAT, request, response): + uri = request.get('uri', '') + method = request.get('method', 'GET') + response['statusCode'] = 200 + response['statusReason'] = 'OK' + response['headers'] = {'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*'} + + try: + # TD sends POST body as bytes in request['data'] + raw = request.get('data', request.get('body', '')) + if isinstance(raw, bytes): + raw = raw.decode('utf-8') + body = {} + if raw and isinstance(raw, str) and raw.strip(): + body = json.loads(raw) + pars = request.get('pars', {}) + + if uri == '/api/td/server/td': + response['data'] = json.dumps({ + 'version': str(app.version), + 'osName': sys.platform, + 'apiVersion': '1.4.3', + 'product': 'TouchDesigner' + }) + + elif uri == '/api/td/server/exec': + script = body.get('script', '') + old_stdout = sys.stdout + sys.stdout = buf = io.StringIO() + result_val = None + err_text = '' + try: + globs = {'op': op, 'ops': ops, 'me': webServerDAT, 'parent': parent, + 'project': project, 'td': td, 'result': None, + 'app': app, 'absTime': absTime} + lines = script.strip().split('\n') + if len(lines) == 1: + try: + result_val = eval(script, globs) + except SyntaxError: + exec(script, globs) + result_val = globs.get('result') + else: + exec(script, globs) + result_val = globs.get('result') + except Exception as e: + err_text = traceback.format_exc() + finally: + captured = buf.getvalue() + sys.stdout = old_stdout + response['data'] = json.dumps({ + 'result': _serialize(result_val), + 'stdout': captured, + 'stderr': err_text + }) + + elif uri == '/api/nodes': + pp = pars.get('parentPath', ['/project1'])[0] + p = op(pp) + nodes = [] + if p: + for c in p.children: + nodes.append({'name': c.name, 'path': c.path, + 'opType': c.OPType, 'family': c.family}) + response['data'] = json.dumps({'data': nodes}) + + elif uri == '/api/nodes/errors': + np = pars.get('nodePath', ['/project1'])[0] + n = op(np) + errors = [] + if n: + def _collect(node, depth=0): + if depth > 10: return + e = node.errors() + if e: + errors.append({'nodePath': node.path, 'nodeName': node.name, + 'opType': node.OPType, 'errors': str(e)}) + if hasattr(node, 'children'): + for c in node.children: _collect(c, depth+1) + _collect(n) + response['data'] = json.dumps({'data': errors, 'hasErrors': len(errors)>0, + 'errorCount': len(errors)}) + + else: + response['statusCode'] = 404 + response['data'] = json.dumps({'error': 'Unknown: ' + uri}) + + except Exception as e: + response['statusCode'] = 500 + response['data'] = json.dumps({'error': str(e), 'trace': traceback.format_exc()}) + + return response + +def _serialize(v): + if v is None: return None + if isinstance(v, (int, float, bool, str)): return v + if isinstance(v, (list, tuple)): return [_serialize(i) for i in v] + if isinstance(v, dict): return {str(k): _serialize(vv) for k, vv in v.items()} + return str(v) +''' + +# Point webserver to callback +ws.par.callbacks = cb.path + +print("Custom API server created on port 9981") +print("Test: curl http://127.0.0.1:9981/api/td/server/td") diff --git a/skills/creative/touchdesigner/scripts/setup.sh b/skills/creative/touchdesigner/scripts/setup.sh new file mode 100644 index 000000000..ce8b56870 --- /dev/null +++ b/skills/creative/touchdesigner/scripts/setup.sh @@ -0,0 +1,152 @@ +#!/usr/bin/env bash +# TouchDesigner MCP Setup Verification Script +# Checks all prerequisites and guides configuration + +set -euo pipefail + +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' + +pass() { echo -e " ${GREEN}✓${NC} $1"; } +fail() { echo -e " ${RED}✗${NC} $1"; } +warn() { echo -e " ${YELLOW}!${NC} $1"; } +info() { echo -e " ${BLUE}→${NC} $1"; } + +echo "" +echo "TouchDesigner MCP Setup Check" +echo "==============================" +echo "" + +ERRORS=0 + +# 1. Check Node.js +echo "1. Node.js" +if command -v node &>/dev/null; then + NODE_VER=$(node --version 2>/dev/null || echo "unknown") + MAJOR=$(echo "$NODE_VER" | sed 's/^v//' | cut -d. -f1) + if [ "$MAJOR" -ge 18 ] 2>/dev/null; then + pass "Node.js $NODE_VER (>= 18 required)" + else + fail "Node.js $NODE_VER (>= 18 required, please upgrade)" + ERRORS=$((ERRORS + 1)) + fi +else + fail "Node.js not found" + info "Install: https://nodejs.org/ or 'brew install node'" + ERRORS=$((ERRORS + 1)) +fi + +# 2. Check npm/npx +echo "2. npm/npx" +if command -v npx &>/dev/null; then + NPX_VER=$(npx --version 2>/dev/null || echo "unknown") + pass "npx $NPX_VER" +else + fail "npx not found (usually comes with Node.js)" + ERRORS=$((ERRORS + 1)) +fi + +# 3. Check MCP Python package +echo "3. MCP Python package" +HERMES_HOME="${HERMES_HOME:-$HOME/.hermes}" +VENV_PYTHON="" + +# Try to find the Hermes venv Python +if [ -f "$HERMES_HOME/hermes-agent/.venv/bin/python" ]; then + VENV_PYTHON="$HERMES_HOME/hermes-agent/.venv/bin/python" +elif [ -f "$HERMES_HOME/hermes-agent/venv/bin/python" ]; then + VENV_PYTHON="$HERMES_HOME/hermes-agent/venv/bin/python" +fi + +if [ -n "$VENV_PYTHON" ]; then + if $VENV_PYTHON -c "import mcp" 2>/dev/null; then + MCP_VER=$($VENV_PYTHON -c "import importlib.metadata; print(importlib.metadata.version('mcp'))" 2>/dev/null || echo "installed") + pass "mcp package ($MCP_VER) in Hermes venv" + else + fail "mcp package not installed in Hermes venv" + info "Install: $VENV_PYTHON -m pip install mcp" + ERRORS=$((ERRORS + 1)) + fi +else + warn "Could not find Hermes venv — check mcp package manually" +fi + +# 4. Check TouchDesigner +echo "4. TouchDesigner" +TD_FOUND=false + +# macOS +if [ -d "/Applications/TouchDesigner.app" ]; then + TD_FOUND=true + pass "TouchDesigner found at /Applications/TouchDesigner.app" +fi + +# Linux (common install locations) +if command -v TouchDesigner &>/dev/null; then + TD_FOUND=true + pass "TouchDesigner found in PATH" +fi + +if [ -d "$HOME/TouchDesigner" ]; then + TD_FOUND=true + pass "TouchDesigner found at ~/TouchDesigner" +fi + +if [ "$TD_FOUND" = false ]; then + warn "TouchDesigner not detected (may be installed elsewhere)" + info "Download from: https://derivative.ca/download" + info "Free Non-Commercial license available" +fi + +# 5. Check TD WebServer DAT reachability +echo "5. TouchDesigner WebServer DAT" +TD_URL="${TD_API_URL:-http://127.0.0.1:9981}" +if command -v curl &>/dev/null; then + HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --connect-timeout 3 "$TD_URL/api/td/server/td" 2>/dev/null || echo "000") + if [ "$HTTP_CODE" = "200" ]; then + TD_INFO=$(curl -s --connect-timeout 3 "$TD_URL/api/td/server/td" 2>/dev/null || echo "{}") + pass "TD WebServer DAT responding at $TD_URL" + info "Response: $TD_INFO" + elif [ "$HTTP_CODE" = "000" ]; then + warn "Cannot reach TD WebServer DAT at $TD_URL" + info "Make sure TouchDesigner is running with mcp_webserver_base.tox imported" + else + warn "TD WebServer DAT returned HTTP $HTTP_CODE at $TD_URL" + fi +else + warn "curl not found — cannot test TD connection" +fi + +# 6. Check Hermes config +echo "6. Hermes MCP config" +CONFIG_FILE="$HERMES_HOME/config.yaml" +if [ -f "$CONFIG_FILE" ]; then + if grep -q "touchdesigner" "$CONFIG_FILE" 2>/dev/null; then + pass "TouchDesigner MCP server configured in config.yaml" + else + warn "No 'touchdesigner' entry found in mcp_servers config" + info "Add a touchdesigner entry under mcp_servers: in $CONFIG_FILE" + info "See references/mcp-tools.md for the configuration block" + fi +else + warn "No Hermes config.yaml found at $CONFIG_FILE" +fi + +# Summary +echo "" +echo "==============================" +if [ $ERRORS -eq 0 ]; then + echo -e "${GREEN}All critical checks passed!${NC}" + echo "" + echo "Next steps:" + echo " 1. Open TouchDesigner and import mcp_webserver_base.tox" + echo " 2. Add the MCP server config to Hermes (see references/mcp-tools.md)" + echo " 3. Restart Hermes and test: 'Get TouchDesigner server info'" +else + echo -e "${RED}$ERRORS critical issue(s) found.${NC}" + echo "Fix the issues above, then re-run this script." +fi +echo ""