mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-04-25 00:51:20 +00:00
feat: add TouchDesigner integration skill
New skill: creative/touchdesigner — control a running TouchDesigner
instance via REST API. Build real-time visual networks programmatically.
Architecture:
Hermes Agent -> HTTP REST (curl) -> TD WebServer DAT -> TD Python env
Key features:
- Custom API handler (scripts/custom_api_handler.py) that creates a
self-contained WebServer DAT + callback in TD. More reliable than the
official mcp_webserver_base.tox which frequently fails module imports.
- Discovery-first workflow: never hardcode TD parameter names. Always
probe the running instance first since names change across versions.
- Persistent setup: save the TD project once with the API handler baked
in. TD auto-opens the last project on launch, so port 9981 is live
with zero manual steps after first-time setup.
- Works via curl in execute_code (no MCP dependency required).
- Optional MCP server config for touchdesigner-mcp-server npm package.
Skill structure (2823 lines total):
SKILL.md (209 lines) — setup, workflow, key rules, operator reference
references/pitfalls.md (276 lines) — 24 hard-won lessons
references/operators.md (239 lines) — all 6 operator families
references/network-patterns.md (589 lines) — audio-reactive, generative,
video processing, GLSL, instancing, live performance recipes
references/mcp-tools.md (501 lines) — 13 MCP tool schemas
references/python-api.md (443 lines) — TD Python scripting patterns
references/troubleshooting.md (274 lines) — connection diagnostics
scripts/custom_api_handler.py (140 lines) — REST API handler for TD
scripts/setup.sh (152 lines) — prerequisite checker
Tested on TouchDesigner 099 Non-Commercial (macOS/darwin).
This commit is contained in:
kshitijk4poor
Teknium
parent
c49a58a6d0
commit
7a5371b20d
9 changed files with 3277 additions and 0 deletions
278
skills/creative/touchdesigner/SKILL.md
Normal file
278
skills/creative/touchdesigner/SKILL.md
Normal file
|
|
@ -0,0 +1,278 @@
|
|||
---
|
||||
name: touchdesigner
|
||||
description: "Control a running TouchDesigner instance programmatically — create operators, set parameters, wire connections, execute Python, build real-time visuals. Covers: GLSL shaders, audio-reactive, generative art, video processing, instancing, and live performance."
|
||||
version: 3.0.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [TouchDesigner, MCP, creative-coding, real-time-visuals, generative-art, audio-reactive, VJ, installation, GLSL]
|
||||
related_skills: [native-mcp, ascii-video, manim-video, hermes-video]
|
||||
security:
|
||||
allow_network: true
|
||||
allow_install: true
|
||||
allow_config_write: true
|
||||
---
|
||||
|
||||
# TouchDesigner Integration
|
||||
|
||||
## Architecture
|
||||
|
||||
Hermes Agent -> HTTP REST (curl) -> TD WebServer DAT (port 9981) -> TD Python environment.
|
||||
|
||||
The agent controls a **running TouchDesigner instance** via a REST API on port 9981. It does NOT generate .toe files from scratch.
|
||||
|
||||
## First-Time Setup (one-time, persists across sessions)
|
||||
|
||||
### 1. Verify TD is running and check for existing API
|
||||
|
||||
```bash
|
||||
lsof -i :9981 -P -n | grep LISTEN # TD listening?
|
||||
curl -s --max-time 5 http://127.0.0.1:9981/api/td/server/td # API working?
|
||||
```
|
||||
|
||||
If HTTP 200 + JSON → skip to **Discovery**. Setup is already done.
|
||||
|
||||
### 2. If no API: deploy the custom handler
|
||||
|
||||
The user must paste ONE line into TD Textport (Alt+T / Dialogs > Textport and DATs):
|
||||
|
||||
```
|
||||
exec(open('PATH_TO_SKILL/scripts/custom_api_handler.py').read())
|
||||
```
|
||||
|
||||
Copy this to their clipboard with `pbcopy`. This creates a WebServer DAT + callback handler pair in `/project1` that implements the REST API. No external dependencies.
|
||||
|
||||
**Why not the official .tox?** The `mcp_webserver_base.tox` from 8beeeaaat/touchdesigner-mcp frequently fails to import its Python modules after drag-drop (relative path resolution issue). Our custom handler is self-contained and more reliable. See `references/pitfalls.md` #1-2.
|
||||
|
||||
### 3. Save the project to persist the API
|
||||
|
||||
After the handler is running, save the project so the API auto-starts on every future TD launch:
|
||||
|
||||
```python
|
||||
td_exec("project.save(os.path.expanduser('~/Documents/HermesAgent.toe'))")
|
||||
```
|
||||
|
||||
TD auto-opens the last saved project on launch. From now on, `open /Applications/TouchDesigner.app` → port 9981 is live → agent can connect immediately.
|
||||
|
||||
To launch TD with this project explicitly:
|
||||
```bash
|
||||
open /Applications/TouchDesigner.app ~/Documents/HermesAgent.toe
|
||||
```
|
||||
|
||||
### 4. Optional: Configure Hermes MCP
|
||||
|
||||
Add under `mcp_servers:` in the user's Hermes config:
|
||||
```yaml
|
||||
touchdesigner:
|
||||
command: npx
|
||||
args: ["-y", "touchdesigner-mcp-server@latest"]
|
||||
env:
|
||||
TD_API_URL: "http://127.0.0.1:9981"
|
||||
timeout: 120
|
||||
```
|
||||
|
||||
This is optional — the agent works fully via `curl` to the REST API using `execute_code`. MCP tools are a convenience layer.
|
||||
|
||||
## Talking to TD (the td_exec pattern)
|
||||
|
||||
All communication uses this pattern in `execute_code`:
|
||||
|
||||
```python
|
||||
import json, shlex
|
||||
from hermes_tools import terminal
|
||||
|
||||
API = "http://127.0.0.1:9981"
|
||||
def td_exec(script):
|
||||
payload = json.dumps({"script": script})
|
||||
cmd = f"curl -s --max-time 15 -X POST -H 'Content-Type: application/json' -d {shlex.quote(payload)} '{API}/api/td/server/exec'"
|
||||
r = terminal(cmd, timeout=20)
|
||||
return json.loads(r['output'])
|
||||
|
||||
# Returns: {"result": <value>, "stdout": "...", "stderr": "..."}
|
||||
```
|
||||
|
||||
For large GLSL shaders: write to a temp file, then `td_exec("op('...').text = open('/tmp/shader.glsl').read()")`.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 0: Discovery (MANDATORY — never skip)
|
||||
|
||||
**Never hardcode parameter names.** They change between TD versions. Run this first:
|
||||
|
||||
```python
|
||||
td_exec("""
|
||||
import sys
|
||||
info = {'version': str(app.version), 'platform': sys.platform}
|
||||
root = op('/project1')
|
||||
for name, optype in [('glslTOP', glslTOP), ('constantTOP', constantTOP),
|
||||
('blurTOP', blurTOP), ('textTOP', textTOP),
|
||||
('levelTOP', levelTOP), ('compositeTOP', compositeTOP),
|
||||
('transformTOP', transformTOP), ('feedbackTOP', feedbackTOP),
|
||||
('windowCOMP', windowCOMP)]:
|
||||
n = root.create(optype, '_d_' + name)
|
||||
kw = ['color','size','font','dat','alpha','opacity','resolution','text',
|
||||
'extend','operand','top','pixel','format','win','type']
|
||||
info[name] = [p.name for p in n.pars() if any(k in p.name.lower() for k in kw)]
|
||||
n.destroy()
|
||||
result = info
|
||||
""")
|
||||
```
|
||||
|
||||
Use the returned param names for ALL subsequent calls. Store them in your session context.
|
||||
|
||||
### Step 1: Clean + Build
|
||||
|
||||
Build the entire network in ONE `td_exec` call (batching avoids round-trip overhead and ensures TD advances frames between calls):
|
||||
|
||||
```python
|
||||
td_exec("""
|
||||
root = op('/project1')
|
||||
keep = {'api_server', 'api_handler'}
|
||||
for child in list(root.children): # snapshot before destroying
|
||||
if child.name not in keep and child.valid:
|
||||
child.destroy()
|
||||
|
||||
# Create nodes, set params (using discovered names), wire, verify
|
||||
...
|
||||
result = {'nodes': len(list(root.children)), 'errors': [...]}
|
||||
""")
|
||||
```
|
||||
|
||||
### Step 2: Wire connections
|
||||
|
||||
```python
|
||||
gl.outputConnectors[0].connect(comp.inputConnectors[0])
|
||||
```
|
||||
|
||||
### Step 3: Verify
|
||||
|
||||
```python
|
||||
for c in list(root.children):
|
||||
e = c.errors(); w = c.warnings()
|
||||
if e: print(c.name, 'ERR:', e)
|
||||
```
|
||||
|
||||
### Step 4: Display
|
||||
|
||||
```python
|
||||
win = root.create(windowCOMP, 'display')
|
||||
win.par.winop = out.path # discovered param name
|
||||
win.par.winw = 1280; win.par.winh = 720
|
||||
win.par.winopen.pulse()
|
||||
```
|
||||
|
||||
## Key Implementation Rules
|
||||
|
||||
**Always clean safely:** `list(root.children)` before iterating + `child.valid` check.
|
||||
|
||||
**GLSL time:** No `uTDCurrentTime` in TD 099. Feed time via 1x1 Constant TOP.
|
||||
**CRITICAL: must use `rgba32float` format** — the default 8-bit format clamps values to 0-1, so `absTime.seconds % 1000.0` becomes 1.0 and the shader appears frozen:
|
||||
```python
|
||||
t = root.create(constantTOP, 'time_driver')
|
||||
t.par.format = 'rgba32float' # ← REQUIRED or time is stuck at 1.0
|
||||
t.par.outputresolution = 'custom'
|
||||
t.par.resolutionw = 1
|
||||
t.par.resolutionh = 1
|
||||
t.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
t.par.colorg.expr = "int(absTime.seconds / 1000.0)"
|
||||
t.outputConnectors[0].connect(glsl.inputConnectors[0])
|
||||
# In GLSL: vec4 td = texture(sTD2DInputs[0], vec2(.5)); float t = td.r + td.g*1000.;
|
||||
```
|
||||
|
||||
**Feedback TOP:** Use `top` parameter reference (not direct input wire). The "Not enough sources" error resolves after first cook. The "Cook dependency loop" warning is expected.
|
||||
|
||||
**Resolution:** Non-Commercial caps at 1280×1280. Use `outputresolution = 'custom'`.
|
||||
|
||||
**Large shaders:** Write GLSL to `/tmp/file.glsl`, then `td_exec("op('shader').text = open('/tmp/file.glsl').read()")`.
|
||||
|
||||
**WebServer DAT quirk:** Response body goes in `response['data']` not `response['body']`. Request POST body comes as bytes in `request['data']`.
|
||||
|
||||
## Recording / Exporting Video
|
||||
|
||||
To capture TD output as video or image sequence for external use (e.g., ASCII video pipeline):
|
||||
|
||||
### Movie Recording (recommended)
|
||||
|
||||
```python
|
||||
# Put a Null TOP before the recorder (official best practice)
|
||||
rec = root.create(moviefileoutTOP, 'recorder')
|
||||
null_out.outputConnectors[0].connect(rec.inputConnectors[0])
|
||||
|
||||
rec.par.type = 'movie'
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
rec.par.videocodec = 'mjpa' # Motion JPEG — works on Non-Commercial
|
||||
|
||||
# Start/stop recording (par.record is a toggle, NOT .record() method)
|
||||
rec.par.record = True # start
|
||||
# ... wait ...
|
||||
rec.par.record = False # stop
|
||||
```
|
||||
|
||||
**H.264/H.265 require a Commercial license** — use `mjpa` (Motion JPEG) or `prores` on Non-Commercial. Extract frames afterward with ffmpeg if needed:
|
||||
```bash
|
||||
ffmpeg -i /tmp/output.mov -vframes 120 /tmp/frames/frame_%06d.png
|
||||
```
|
||||
|
||||
### Image Sequence Export
|
||||
|
||||
```python
|
||||
rec.par.type = 'imagesequence'
|
||||
rec.par.imagefiletype = 'png'
|
||||
rec.par.file.expr = "'/tmp/frames/out' + me.fileSuffix" # fileSuffix is REQUIRED
|
||||
rec.par.record = True
|
||||
```
|
||||
|
||||
### Pitfalls
|
||||
|
||||
- **Race condition:** When setting `par.file` and starting recording in the same script, use `run("...", delayFrames=2)` so the file path is applied before recording begins.
|
||||
- **TOP.save() is useless for animation:** Calling `op('null1').save(path)` in a loop or rapid API calls captures the same GPU texture every time — TD doesn't cook new frames between save calls. Always use MovieFileOut for animated output.
|
||||
- See `references/pitfalls.md` #25-27 for full details.
|
||||
|
||||
## Audio-Reactive GLSL (Proven Recipe)
|
||||
|
||||
Complete chain for music-driven visuals: AudioFileIn → AudioSpectrum → Math (boost) → Resample (256) → CHOP To TOP → GLSL TOP (spectrum sampled per-pixel). See `references/network-patterns.md` Pattern 3b for the full working recipe with shader code.
|
||||
|
||||
## Audio-Reactive Visuals
|
||||
|
||||
The most powerful TD workflow for the agent: play an audio file, analyze its spectrum, and drive a GLSL shader in real-time. The agent builds the entire signal chain programmatically.
|
||||
|
||||
**Signal chain:**
|
||||
```
|
||||
AudioFileIn CHOP → AudioSpectrum CHOP → Math CHOP (gain=5)
|
||||
→ Resample CHOP (256) → CHOP To TOP (spectrum texture)
|
||||
↓ (GLSL input 1)
|
||||
Constant TOP (rgba32float, time) → GLSL TOP → Null TOP → MovieFileOut
|
||||
(input 0)
|
||||
```
|
||||
|
||||
**Key technique:** The spectrum becomes a 256×1 texture. In GLSL, `texture(sTD2DInputs[1], vec2(x, 0.0)).r` samples frequency at position x (0=bass, 1=treble). This lets the shader react per-pixel to different frequency bands.
|
||||
|
||||
**Smoothing is critical:** Raw FFT jitters. Use `Math CHOP` gain to boost weak signal, then the GLSL shader's own temporal integration (via feedback or time-smoothed params) handles visual smoothing.
|
||||
|
||||
See `references/network-patterns.md` Pattern 9b for the complete build script + shader code.
|
||||
|
||||
## Operator Quick Reference
|
||||
|
||||
| Family | Color | Examples | Suffix |
|
||||
|--------|-------|----------|--------|
|
||||
| TOP | Purple | noiseTop, glslTop, compositeTop, levelTop, blurTop, textTop, nullTop, feedbackTop, renderTop | TOP |
|
||||
| CHOP | Green | audiofileinChop, audiospectrumChop, mathChop, lfoChop, constantChop | CHOP |
|
||||
| SOP | Blue | gridSop, sphereSop, transformSop, noiseSop | SOP |
|
||||
| DAT | White | textDat, tableDat, scriptDat, webserverDAT | DAT |
|
||||
| MAT | Yellow | phongMat, pbrMat, glslMat, constMat | MAT |
|
||||
| COMP | Gray | geometryComp, containerComp, cameraComp, lightComp, windowCOMP | COMP |
|
||||
|
||||
See `references/operators.md` for full catalog. See `references/network-patterns.md` for recipes.
|
||||
|
||||
## References
|
||||
|
||||
| File | What |
|
||||
|------|------|
|
||||
| `references/pitfalls.md` | **READ FIRST** — 31 hard-won lessons from real sessions |
|
||||
| `references/operators.md` | All operator families with params and use cases |
|
||||
| `references/network-patterns.md` | Recipes: audio-reactive, generative, video, GLSL, instancing |
|
||||
| `references/mcp-tools.md` | MCP tool schemas (optional — curl works without MCP) |
|
||||
| `references/python-api.md` | TD Python: op(), scripting, extensions |
|
||||
| `references/troubleshooting.md` | Connection diagnostics, param debugging, performance |
|
||||
| `scripts/custom_api_handler.py` | Self-contained REST API handler for TD WebServer DAT |
|
||||
501
skills/creative/touchdesigner/references/mcp-tools.md
Normal file
501
skills/creative/touchdesigner/references/mcp-tools.md
Normal file
|
|
@ -0,0 +1,501 @@
|
|||
# TouchDesigner MCP Tools Reference
|
||||
|
||||
Complete parameter schemas and usage examples for all 13 MCP tools from the 8beeeaaat/touchdesigner-mcp server.
|
||||
|
||||
## Hermes Configuration
|
||||
|
||||
Add a `touchdesigner` entry under the `mcp_servers` section of your Hermes config. Example YAML block:
|
||||
|
||||
```yaml
|
||||
# Under mcp_servers: in config.yaml
|
||||
mcp_servers:
|
||||
touchdesigner:
|
||||
command: npx
|
||||
args: ["-y", "touchdesigner-mcp-server@latest"]
|
||||
env:
|
||||
TD_API_URL: "http://127.0.0.1:9981"
|
||||
timeout: 120
|
||||
connect_timeout: 60
|
||||
```
|
||||
|
||||
For a locally built server, point `command` to `node` and `args` to the built server index.js path. Set `TD_API_URL` to the TouchDesigner WebServer DAT address (default port 9981).
|
||||
|
||||
For the documentation/knowledge server (no running TD needed), add a `td_docs` entry using `touchdesigner-mcp-server` as the npx package.
|
||||
|
||||
Tools are registered as `mcp_touchdesigner_<tool_name>` in Hermes.
|
||||
|
||||
**If MCP tools are not available as direct function calls** (common when the MCP server connects but Hermes doesn't expose them as callable tools), use the custom API handler directly via `curl` in `execute_code` or `terminal`:
|
||||
|
||||
```python
|
||||
import json, shlex
|
||||
from hermes_tools import terminal
|
||||
|
||||
def td_exec(script):
|
||||
"""Execute Python in TouchDesigner via the REST API."""
|
||||
escaped = json.dumps({"script": script})
|
||||
cmd = f"curl -s --max-time 15 -X POST -H 'Content-Type: application/json' -d {shlex.quote(escaped)} 'http://127.0.0.1:9981/api/td/server/exec'"
|
||||
r = terminal(cmd, timeout=20)
|
||||
return json.loads(r['output'])
|
||||
|
||||
# Example: list all nodes
|
||||
result = td_exec('result = [c.name for c in op("/project1").children]')
|
||||
print(result) # {"result": ["node1", "node2", ...], "stdout": "", "stderr": ""}
|
||||
```
|
||||
|
||||
This `td_exec` helper works with both the official .tox handler and the custom API handler from `scripts/custom_api_handler.py`.
|
||||
|
||||
Tools are registered as `mcp_touchdesigner_<tool_name>` in Hermes.
|
||||
|
||||
## Common Formatting Parameters
|
||||
|
||||
Most tools accept these optional formatting parameters:
|
||||
|
||||
| Parameter | Type | Values | Description |
|
||||
|-----------|------|--------|-------------|
|
||||
| `detailLevel` | string | `"minimal"`, `"summary"`, `"detailed"` | Response verbosity |
|
||||
| `responseFormat` | string | `"json"`, `"yaml"`, `"markdown"` | Output format |
|
||||
| `limit` | integer | 1-500 | Max items (on list-type tools only) |
|
||||
|
||||
These are client-side formatting — they control how the MCP server formats the response text, not what data TD returns.
|
||||
|
||||
---
|
||||
|
||||
## Tool 1: describe_td_tools
|
||||
|
||||
**Purpose:** Meta-tool — lists all available TouchDesigner MCP tools with descriptions and parameters.
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `filter` | string | No | Keyword to filter tools by name, description, or parameter |
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
|
||||
**Example:** Find tools related to node creation
|
||||
```
|
||||
describe_td_tools(filter="create")
|
||||
```
|
||||
|
||||
**Note:** This tool runs entirely in the MCP server — it does NOT contact TouchDesigner. Use it to discover what's available.
|
||||
|
||||
---
|
||||
|
||||
## Tool 2: get_td_info
|
||||
|
||||
**Purpose:** Get TouchDesigner server information (version, OS, build).
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
|
||||
**Example:** Check TD is running and get version
|
||||
```
|
||||
get_td_info()
|
||||
```
|
||||
|
||||
**Returns:** TD version, build number, OS name/version, MCP API version.
|
||||
|
||||
**Use this first** to verify the connection is working before building networks.
|
||||
|
||||
---
|
||||
|
||||
## Tool 3: execute_python_script
|
||||
|
||||
**Purpose:** Execute arbitrary Python code inside TouchDesigner's Python environment.
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `script` | string | **Yes** | Python code to execute |
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
|
||||
**Available globals in the script:**
|
||||
- `op` — find operators by path
|
||||
- `ops` — find multiple operators by pattern
|
||||
- `me` — the WebServer DAT running the script
|
||||
- `parent` — me.parent()
|
||||
- `project` — root project component
|
||||
- `td` — the full td module
|
||||
- `result` — set this to explicitly return a value
|
||||
|
||||
**Execution behavior:**
|
||||
- Single-line scripts: tries `eval()` first (returns value), falls back to `exec()`
|
||||
- Multi-line scripts: uses `exec()` always
|
||||
- stdout/stderr are captured and returned separately
|
||||
- If `result` is not set, tries to evaluate the last expression as the return value
|
||||
|
||||
**Examples:**
|
||||
|
||||
```python
|
||||
# Simple query
|
||||
execute_python_script(script="op('/project1/noise1').par.seed.val")
|
||||
# Returns: {"result": 42, "stdout": "", "stderr": ""}
|
||||
|
||||
# Multi-line script
|
||||
execute_python_script(script="""
|
||||
nodes = op('/project1').findChildren(type=TOP)
|
||||
result = [{'name': n.name, 'type': n.OPType} for n in nodes]
|
||||
""")
|
||||
|
||||
# Connect two operators
|
||||
execute_python_script(script="op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1'))")
|
||||
|
||||
# Create and configure in one script
|
||||
execute_python_script(script="""
|
||||
parent = op('/project1')
|
||||
n = parent.create(noiseTop, 'my_noise')
|
||||
n.par.seed.val = 42
|
||||
n.par.monochrome.val = True
|
||||
n.par.resolutionw.val = 1920
|
||||
n.par.resolutionh.val = 1080
|
||||
result = {'path': n.path, 'type': n.OPType}
|
||||
""")
|
||||
|
||||
# Batch wire a chain
|
||||
execute_python_script(script="""
|
||||
chain = ['noise1', 'level1', 'blur1', 'composite1', 'null_out']
|
||||
for i in range(len(chain) - 1):
|
||||
src = op(f'/project1/{chain[i]}')
|
||||
dst = op(f'/project1/{chain[i+1]}')
|
||||
if src and dst:
|
||||
src.outputConnectors[0].connect(dst)
|
||||
result = 'Wired chain: ' + ' -> '.join(chain)
|
||||
""")
|
||||
```
|
||||
|
||||
**When to use:** Wiring connections, complex logic, batch operations, querying state that other tools don't cover. This is the most powerful and flexible tool.
|
||||
|
||||
---
|
||||
|
||||
## Tool 4: create_td_node
|
||||
|
||||
**Purpose:** Create a new operator in TouchDesigner.
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `parentPath` | string | **Yes** | Path to parent (e.g., `/project1`) |
|
||||
| `nodeType` | string | **Yes** | Operator type (e.g., `noiseTop`, `mathChop`) |
|
||||
| `nodeName` | string | No | Custom name (auto-generated if omitted) |
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
|
||||
**Examples:**
|
||||
|
||||
```
|
||||
create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="bg_noise")
|
||||
create_td_node(parentPath="/project1", nodeType="compositeTop") # auto-named
|
||||
create_td_node(parentPath="/project1/audio_chain", nodeType="audiospectrumChop", nodeName="spectrum")
|
||||
```
|
||||
|
||||
**Returns:** Node summary with id, name, path, opType, and all default parameter values.
|
||||
|
||||
**Node type naming convention:** camelCase family suffix — `noiseTop`, `mathChop`, `gridSop`, `tableDat`, `phongMat`, `geometryComp`. See `references/operators.md` for the full list.
|
||||
|
||||
---
|
||||
|
||||
## Tool 5: delete_td_node
|
||||
|
||||
**Purpose:** Delete an existing operator.
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `nodePath` | string | **Yes** | Absolute path to node (e.g., `/project1/noise1`) |
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
delete_td_node(nodePath="/project1/noise1")
|
||||
```
|
||||
|
||||
**Returns:** Confirmation with the deleted node's summary (captured before deletion).
|
||||
|
||||
---
|
||||
|
||||
## Tool 6: get_td_nodes
|
||||
|
||||
**Purpose:** List operators under a path with optional filtering.
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `parentPath` | string | **Yes** | Parent path (e.g., `/project1`) |
|
||||
| `pattern` | string | No | Glob pattern for name filtering (default: `*`) |
|
||||
| `includeProperties` | boolean | No | Include full parameter values (default: false) |
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
| `limit` | integer | No | Max items (1-500) |
|
||||
|
||||
**Examples:**
|
||||
|
||||
```
|
||||
# List all direct children of /project1
|
||||
get_td_nodes(parentPath="/project1")
|
||||
|
||||
# Find all noise operators
|
||||
get_td_nodes(parentPath="/project1", pattern="noise*")
|
||||
|
||||
# Get full parameter details
|
||||
get_td_nodes(parentPath="/project1", pattern="*", includeProperties=true, limit=20)
|
||||
```
|
||||
|
||||
**Returns:** List of node summaries. With `includeProperties=false` (default): id, name, path, opType only. With `includeProperties=true`: full parameter values included.
|
||||
|
||||
---
|
||||
|
||||
## Tool 7: get_td_node_parameters
|
||||
|
||||
**Purpose:** Get detailed parameters of a specific node.
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `nodePath` | string | **Yes** | Node path (e.g., `/project1/noise1`) |
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
| `limit` | integer | No | Max parameters (1-500) |
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
get_td_node_parameters(nodePath="/project1/noise1")
|
||||
```
|
||||
|
||||
**Returns:** All parameter name-value pairs for the node. Use this to discover available parameters before calling update_td_node_parameters.
|
||||
|
||||
---
|
||||
|
||||
## Tool 8: get_td_node_errors
|
||||
|
||||
**Purpose:** Check for errors on a node and all its descendants (recursive).
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `nodePath` | string | **Yes** | Absolute path to inspect (e.g., `/project1`) |
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
| `limit` | integer | No | Max error items (1-500) |
|
||||
|
||||
**Examples:**
|
||||
|
||||
```
|
||||
# Check entire project for errors
|
||||
get_td_node_errors(nodePath="/project1")
|
||||
|
||||
# Check a specific chain
|
||||
get_td_node_errors(nodePath="/project1/audio_chain")
|
||||
```
|
||||
|
||||
**Returns:** Error count, hasErrors boolean, and list of errors each with nodePath, nodeName, opType, and error message.
|
||||
|
||||
**Always call this after building a network** to catch wiring mistakes, missing references, and configuration errors.
|
||||
|
||||
---
|
||||
|
||||
## Tool 9: update_td_node_parameters
|
||||
|
||||
**Purpose:** Update parameters on an existing node.
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `nodePath` | string | **Yes** | Path to node (e.g., `/project1/noise1`) |
|
||||
| `properties` | object | **Yes** | Key-value pairs to update (e.g., `{"seed": 42, "monochrome": true}`) |
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
|
||||
**Examples:**
|
||||
|
||||
```
|
||||
# Set noise parameters
|
||||
update_td_node_parameters(
|
||||
nodePath="/project1/noise1",
|
||||
properties={"seed": 42, "monochrome": false, "period": 4.0, "harmonics": 3,
|
||||
"resolutionw": 1920, "resolutionh": 1080}
|
||||
)
|
||||
|
||||
# Set a file path
|
||||
update_td_node_parameters(
|
||||
nodePath="/project1/moviefilein1",
|
||||
properties={"file": "/Users/me/Videos/clip.mp4", "play": true}
|
||||
)
|
||||
|
||||
# Set compositing mode
|
||||
update_td_node_parameters(
|
||||
nodePath="/project1/composite1",
|
||||
properties={"operand": 0} # 0=Over, 1=Under, 3=Add, 18=Multiply, 27=Screen
|
||||
)
|
||||
```
|
||||
|
||||
**Returns:** List of successfully updated properties and any that failed (with reasons). Raises error if zero properties were updated.
|
||||
|
||||
**Parameter value types:** Floats, ints, booleans, and strings are all accepted. For menu parameters, use either the string label or the integer index.
|
||||
|
||||
---
|
||||
|
||||
## Tool 10: exec_node_method
|
||||
|
||||
**Purpose:** Call a Python method directly on a specific node.
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `nodePath` | string | **Yes** | Path to node |
|
||||
| `method` | string | **Yes** | Method name to call |
|
||||
| `args` | array | No | Positional arguments (strings, numbers, booleans) |
|
||||
| `kwargs` | object | No | Keyword arguments |
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
|
||||
**Examples:**
|
||||
|
||||
```
|
||||
# Get all children of a component
|
||||
exec_node_method(nodePath="/project1", method="findChildren")
|
||||
|
||||
# Find specific children
|
||||
exec_node_method(nodePath="/project1", method="findChildren",
|
||||
kwargs={"name": "noise*", "depth": 1})
|
||||
|
||||
# Get node errors
|
||||
exec_node_method(nodePath="/project1/noise1", method="errors")
|
||||
|
||||
# Get node warnings
|
||||
exec_node_method(nodePath="/project1/noise1", method="warnings")
|
||||
|
||||
# Save a component as .tox
|
||||
exec_node_method(nodePath="/project1/myContainer", method="save",
|
||||
args=["/path/to/component.tox"])
|
||||
```
|
||||
|
||||
**Returns:** Processed return value of the method call. TD operators are serialized to their path strings, iterables to lists, etc.
|
||||
|
||||
---
|
||||
|
||||
## Tool 11: get_td_classes
|
||||
|
||||
**Purpose:** List available TouchDesigner Python classes and modules.
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
| `limit` | integer | No | Max items (default: 50) |
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
get_td_classes(limit=100)
|
||||
```
|
||||
|
||||
**Returns:** List of class/module names and their docstrings from the td module. Useful for discovering what's available in TD's Python environment.
|
||||
|
||||
---
|
||||
|
||||
## Tool 12: get_td_class_details
|
||||
|
||||
**Purpose:** Get methods and properties of a specific TD Python class.
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `className` | string | **Yes** | Class name (e.g., `noiseTop`, `OP`, `COMP`) |
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
| `limit` | integer | No | Max methods/properties (default: 30) |
|
||||
|
||||
**Examples:**
|
||||
|
||||
```
|
||||
# Inspect the noiseTop class
|
||||
get_td_class_details(className="noiseTop")
|
||||
|
||||
# Inspect the base OP class (all operators inherit from this)
|
||||
get_td_class_details(className="OP", limit=50)
|
||||
|
||||
# Inspect COMP (component) class
|
||||
get_td_class_details(className="COMP")
|
||||
```
|
||||
|
||||
**Returns:** Class name, type, description, methods (name + description + type), and properties (name + description + type).
|
||||
|
||||
---
|
||||
|
||||
## Tool 13: get_td_module_help
|
||||
|
||||
**Purpose:** Retrieve Python help() text for any TD module, class, or function.
|
||||
|
||||
**Parameters:**
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `moduleName` | string | **Yes** | Module/class name (e.g., `noiseCHOP`, `tdu`, `td.OP`) |
|
||||
| `detailLevel` | string | No | Response verbosity |
|
||||
| `responseFormat` | string | No | Output format |
|
||||
|
||||
**Examples:**
|
||||
|
||||
```
|
||||
# Get help for the noise CHOP class
|
||||
get_td_module_help(moduleName="noiseCHOP")
|
||||
|
||||
# Get help for the tdu utilities module
|
||||
get_td_module_help(moduleName="tdu")
|
||||
|
||||
# Dotted name resolution works
|
||||
get_td_module_help(moduleName="td.OP")
|
||||
```
|
||||
|
||||
**Returns:** Full Python help() text output, cleaned of backspace characters.
|
||||
|
||||
---
|
||||
|
||||
## Workflow: Building a Complete Network
|
||||
|
||||
Typical sequence of tool calls to build a project:
|
||||
|
||||
1. `get_td_info` — verify connection
|
||||
2. `get_td_nodes(parentPath="/project1")` — see what already exists
|
||||
3. `create_td_node` (multiple) — create all operators
|
||||
4. `update_td_node_parameters` (multiple) — configure each operator
|
||||
5. `execute_python_script` — wire all connections in one batch script
|
||||
6. `get_td_node_errors(nodePath="/project1")` — check for problems
|
||||
7. `get_td_node_parameters` — verify specific nodes if needed
|
||||
8. Iterate: adjust parameters, add operators, fix errors
|
||||
|
||||
## TD Documentation MCP Server Tools
|
||||
|
||||
The bottobot/touchdesigner-mcp-server provides 21 reference/knowledge tools (no running TD needed):
|
||||
|
||||
| Tool | Purpose |
|
||||
|------|---------|
|
||||
| `get_operator` | Get full documentation for a specific operator |
|
||||
| `search_operators` | Search operators by keyword |
|
||||
| `list_operators` | List all operators (filterable by family) |
|
||||
| `compare_operators` | Compare two operators side by side |
|
||||
| `get_operator_examples` | Get usage examples for an operator |
|
||||
| `suggest_workflow` | Get workflow suggestions for a task |
|
||||
| `get_tutorial` | Get a full TD tutorial |
|
||||
| `list_tutorials` | List available tutorials |
|
||||
| `search_tutorials` | Search tutorial content |
|
||||
| `get_python_api` | Get Python API class documentation |
|
||||
| `search_python_api` | Search Python API |
|
||||
| `list_python_classes` | List all documented Python classes |
|
||||
| `get_version_info` | Get TD version release notes |
|
||||
| `list_versions` | List all documented TD versions |
|
||||
| `get_experimental_techniques` | Get advanced technique guides (GLSL, ML, generative, etc.) |
|
||||
| `search_experimental` | Search experimental techniques |
|
||||
| `get_glsl_pattern` | Get GLSL code patterns (SDF, color, math utilities) |
|
||||
| `get_operator_connections` | Get common operator wiring patterns |
|
||||
| `get_network_template` | Get complete network templates with Python generation scripts |
|
||||
| `get_experimental_build` | Get experimental build info |
|
||||
| `list_experimental_builds` | List experimental builds |
|
||||
|
||||
This server contains 630 operator docs, 14 tutorials, 69 Python API classes, and 7 experimental technique categories with working code.
|
||||
914
skills/creative/touchdesigner/references/network-patterns.md
Normal file
914
skills/creative/touchdesigner/references/network-patterns.md
Normal file
|
|
@ -0,0 +1,914 @@
|
|||
# TouchDesigner Network Patterns
|
||||
|
||||
Complete network recipes for common creative coding tasks. Each pattern shows the operator chain, MCP tool calls to build it, and key parameter settings.
|
||||
|
||||
## Audio-Reactive Visuals
|
||||
|
||||
### Pattern 1: Audio Spectrum -> Noise Displacement
|
||||
|
||||
Audio drives noise parameters for organic, music-responsive textures.
|
||||
|
||||
```
|
||||
Audio File In CHOP -> Audio Spectrum CHOP -> Math CHOP (scale)
|
||||
|
|
||||
v (export to noise params)
|
||||
Noise TOP -> Level TOP -> Feedback TOP -> Composite TOP -> Null TOP (out)
|
||||
^ |
|
||||
|________________|
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. create_td_node(parentPath="/project1", nodeType="audiofileinChop", nodeName="audio_in")
|
||||
2. create_td_node(parentPath="/project1", nodeType="audiospectrumChop", nodeName="spectrum")
|
||||
3. create_td_node(parentPath="/project1", nodeType="mathChop", nodeName="spectrum_scale")
|
||||
4. create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="noise1")
|
||||
5. create_td_node(parentPath="/project1", nodeType="levelTop", nodeName="level1")
|
||||
6. create_td_node(parentPath="/project1", nodeType="feedbackTop", nodeName="feedback1")
|
||||
7. create_td_node(parentPath="/project1", nodeType="compositeTop", nodeName="comp1")
|
||||
8. create_td_node(parentPath="/project1", nodeType="nullTop", nodeName="out")
|
||||
|
||||
9. update_td_node_parameters(nodePath="/project1/audio_in",
|
||||
properties={"file": "/path/to/music.wav", "play": true})
|
||||
10. update_td_node_parameters(nodePath="/project1/spectrum",
|
||||
properties={"size": 512})
|
||||
11. update_td_node_parameters(nodePath="/project1/spectrum_scale",
|
||||
properties={"gain": 2.0, "postoff": 0.0})
|
||||
12. update_td_node_parameters(nodePath="/project1/noise1",
|
||||
properties={"type": 1, "monochrome": false, "resolutionw": 1920, "resolutionh": 1080,
|
||||
"period": 4.0, "harmonics": 3, "amp": 1.0})
|
||||
13. update_td_node_parameters(nodePath="/project1/level1",
|
||||
properties={"opacity": 0.95, "gamma1": 0.75})
|
||||
14. update_td_node_parameters(nodePath="/project1/feedback1",
|
||||
properties={"top": "/project1/comp1"})
|
||||
15. update_td_node_parameters(nodePath="/project1/comp1",
|
||||
properties={"operand": 0})
|
||||
|
||||
16. execute_python_script: """
|
||||
op('/project1/audio_in').outputConnectors[0].connect(op('/project1/spectrum'))
|
||||
op('/project1/spectrum').outputConnectors[0].connect(op('/project1/spectrum_scale'))
|
||||
op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1'))
|
||||
op('/project1/level1').outputConnectors[0].connect(op('/project1/comp1').inputConnectors[0])
|
||||
op('/project1/feedback1').outputConnectors[0].connect(op('/project1/comp1').inputConnectors[1])
|
||||
op('/project1/comp1').outputConnectors[0].connect(op('/project1/out'))
|
||||
"""
|
||||
|
||||
17. execute_python_script: """
|
||||
# Export spectrum values to drive noise parameters
|
||||
# This makes the noise react to audio frequencies
|
||||
op('/project1/noise1').par.seed.expr = "op('/project1/spectrum_scale')['chan1']"
|
||||
op('/project1/noise1').par.period.expr = "tdu.remap(op('/project1/spectrum_scale')['chan1'].eval(), 0, 1, 1, 8)"
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 2: Beat Detection -> Visual Pulses
|
||||
|
||||
Detect beats from audio and trigger visual events.
|
||||
|
||||
```
|
||||
Audio Device In CHOP -> Audio Spectrum CHOP -> Math CHOP (isolate bass)
|
||||
|
|
||||
Trigger CHOP (envelope)
|
||||
|
|
||||
[export to visual params]
|
||||
```
|
||||
|
||||
**Key parameter settings:**
|
||||
|
||||
```
|
||||
# Isolate bass frequencies (20-200 Hz)
|
||||
Math CHOP: chanop=1 (Add channels), range1low=0, range1high=10
|
||||
(first 10 FFT bins = bass frequencies with 512 FFT at 44100Hz)
|
||||
|
||||
# ADSR envelope on each beat
|
||||
Trigger CHOP: attack=0.02, peak=1.0, decay=0.3, sustain=0.0, release=0.1
|
||||
|
||||
# Export to visual: Scale, brightness, or color intensity
|
||||
execute_python_script: "op('/project1/level1').par.brightness1.expr = \"1.0 + op('/project1/trigger1')['chan1'] * 0.5\""
|
||||
```
|
||||
|
||||
### Pattern 3: Multi-Band Audio -> Multi-Layer Visuals
|
||||
|
||||
Split audio into frequency bands, drive different visual layers per band.
|
||||
|
||||
```
|
||||
Audio In -> Spectrum -> Audio Band EQ (3 bands: bass, mid, treble)
|
||||
|
|
||||
+---------+---------+
|
||||
| | |
|
||||
Bass Mids Treble
|
||||
| | |
|
||||
Noise TOP Circle TOP Text TOP
|
||||
(slow,dark) (mid,warm) (fast,bright)
|
||||
| | |
|
||||
+-----+----+----+----+
|
||||
| |
|
||||
Composite Composite
|
||||
|
|
||||
Out
|
||||
```
|
||||
|
||||
### Pattern 3b: Audio-Reactive GLSL Fractal (Proven td_exec Recipe)
|
||||
|
||||
Complete working recipe tested in TD 099. Plays an MP3, runs FFT, feeds spectrum as a texture into a GLSL shader where inner fractal reacts to bass, outer to treble.
|
||||
|
||||
**Network:**
|
||||
```
|
||||
AudioFileIn CHOP → AudioSpectrum CHOP → Math CHOP (boost) → Resample CHOP (256)
|
||||
↓
|
||||
CHOP To TOP (256x1 spectrum texture)
|
||||
↓
|
||||
Constant TOP (time, rgba32float) → GLSL TOP (input 0=time, input 1=spectrum) → Null → MovieFileOut
|
||||
↓
|
||||
AudioFileIn CHOP → Audio Device Out CHOP Record to .mov
|
||||
```
|
||||
|
||||
**Build via td_exec (one call per step for reliability):**
|
||||
|
||||
```python
|
||||
# Step 1: Audio chain
|
||||
td_exec("""
|
||||
root = op('/project1')
|
||||
audio = root.create(audiofileinCHOP, 'audio_in')
|
||||
audio.par.file = '/path/to/music.mp3'
|
||||
audio.par.playmode = 0 # Locked to timeline
|
||||
audio.par.volume = 0.5
|
||||
|
||||
spec = root.create(audiospectrumCHOP, 'spectrum')
|
||||
audio.outputConnectors[0].connect(spec.inputConnectors[0])
|
||||
|
||||
math_n = root.create(mathCHOP, 'math_norm')
|
||||
spec.outputConnectors[0].connect(math_n.inputConnectors[0])
|
||||
math_n.par.gain = 5 # boost signal
|
||||
|
||||
resamp = root.create(resampleCHOP, 'resample_spec')
|
||||
math_n.outputConnectors[0].connect(resamp.inputConnectors[0])
|
||||
resamp.par.timeslice = True
|
||||
resamp.par.rate = 256
|
||||
|
||||
chop2top = root.create(choptoTOP, 'spectrum_tex')
|
||||
resamp.outputConnectors[0].connect(chop2top.inputConnectors[0])
|
||||
|
||||
# Audio output (hear the music)
|
||||
aout = root.create(audiodeviceoutCHOP, 'audio_out')
|
||||
audio.outputConnectors[0].connect(aout.inputConnectors[0])
|
||||
result = 'audio chain ok'
|
||||
""")
|
||||
|
||||
# Step 2: Time driver (MUST be rgba32float — see pitfalls #12)
|
||||
td_exec("""
|
||||
root = op('/project1')
|
||||
td = root.create(constantTOP, 'time_driver')
|
||||
td.par.format = 'rgba32float'
|
||||
td.par.outputresolution = 'custom'
|
||||
td.par.resolutionw = 1
|
||||
td.par.resolutionh = 1
|
||||
td.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
td.par.colorg.expr = "int(absTime.seconds / 1000.0)"
|
||||
result = 'time ok'
|
||||
""")
|
||||
|
||||
# Step 3: GLSL shader (write to /tmp, load from file)
|
||||
td_exec("""
|
||||
root = op('/project1')
|
||||
glsl = root.create(glslTOP, 'audio_shader')
|
||||
glsl.par.outputresolution = 'custom'
|
||||
glsl.par.resolutionw = 1280
|
||||
glsl.par.resolutionh = 720
|
||||
|
||||
sd = root.create(textDAT, 'shader_code')
|
||||
sd.text = open('/tmp/my_shader.glsl').read()
|
||||
glsl.par.pixeldat = sd
|
||||
|
||||
# Wire: input 0 = time, input 1 = spectrum texture
|
||||
op('/project1/time_driver').outputConnectors[0].connect(glsl.inputConnectors[0])
|
||||
op('/project1/spectrum_tex').outputConnectors[0].connect(glsl.inputConnectors[1])
|
||||
result = 'glsl ok'
|
||||
""")
|
||||
|
||||
# Step 4: Output + recorder
|
||||
td_exec("""
|
||||
root = op('/project1')
|
||||
out = root.create(nullTOP, 'output')
|
||||
op('/project1/audio_shader').outputConnectors[0].connect(out.inputConnectors[0])
|
||||
|
||||
rec = root.create(moviefileoutTOP, 'recorder')
|
||||
out.outputConnectors[0].connect(rec.inputConnectors[0])
|
||||
rec.par.type = 'movie'
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
rec.par.videocodec = 'mjpa'
|
||||
result = 'output ok'
|
||||
""")
|
||||
```
|
||||
|
||||
**GLSL shader pattern (audio-reactive fractal):**
|
||||
```glsl
|
||||
out vec4 fragColor;
|
||||
|
||||
vec3 palette(float t) {
|
||||
vec3 a = vec3(0.5); vec3 b = vec3(0.5);
|
||||
vec3 c = vec3(1.0); vec3 d = vec3(0.263, 0.416, 0.557);
|
||||
return a + b * cos(6.28318 * (c * t + d));
|
||||
}
|
||||
|
||||
void main() {
|
||||
// Input 0 = time (1x1 rgba32float constant)
|
||||
// Input 1 = audio spectrum (256x1 CHOP To TOP)
|
||||
vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
float t = td.r + td.g * 1000.0;
|
||||
|
||||
vec2 res = uTDOutputInfo.res.zw;
|
||||
vec2 uv = (gl_FragCoord.xy * 2.0 - res) / min(res.x, res.y);
|
||||
vec2 uv0 = uv;
|
||||
vec3 finalColor = vec3(0.0);
|
||||
|
||||
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.0)).r;
|
||||
float mids = texture(sTD2DInputs[1], vec2(0.25, 0.0)).r;
|
||||
|
||||
for (float i = 0.0; i < 4.0; i++) {
|
||||
uv = fract(uv * (1.4 + bass * 0.3)) - 0.5;
|
||||
float d = length(uv) * exp(-length(uv0));
|
||||
|
||||
// Sample spectrum at distance: inner=bass, outer=treble
|
||||
float freq = texture(sTD2DInputs[1], vec2(clamp(d * 0.5, 0.0, 1.0), 0.0)).r;
|
||||
|
||||
vec3 col = palette(length(uv0) + i * 0.4 + t * 0.35);
|
||||
d = sin(d * (7.0 + bass * 4.0) + t * 1.5) / 8.0;
|
||||
d = abs(d);
|
||||
d = pow(0.012 / d, 1.2 + freq * 0.8 + bass * 0.5);
|
||||
finalColor += col * d;
|
||||
}
|
||||
|
||||
// Tone mapping
|
||||
finalColor = finalColor / (finalColor + vec3(1.0));
|
||||
fragColor = TDOutputSwizzle(vec4(finalColor, 1.0));
|
||||
}
|
||||
```
|
||||
|
||||
**Key insights from testing:**
|
||||
- `spectrum_tex` (CHOP To TOP) produces a 256x1 texture — x position = frequency
|
||||
- Sampling at `vec2(0.05, 0.0)` gets bass, `vec2(0.65, 0.0)` gets treble
|
||||
- Sampling based on pixel distance (`d * 0.5`) makes inner fractal react to bass, outer to treble
|
||||
- `bass * 0.3` in the `fract()` zoom makes the fractal breathe with kicks
|
||||
- Math CHOP gain of 5 is needed because raw spectrum values are very small
|
||||
|
||||
## Generative Art
|
||||
|
||||
### Pattern 4: Feedback Loop with Transform
|
||||
|
||||
Classic generative technique — texture evolves through recursive transformation.
|
||||
|
||||
```
|
||||
Noise TOP -> Composite TOP -> Level TOP -> Null TOP (out)
|
||||
^ |
|
||||
| v
|
||||
Transform TOP <- Feedback TOP
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="seed_noise")
|
||||
2. create_td_node(parentPath="/project1", nodeType="compositeTop", nodeName="mix")
|
||||
3. create_td_node(parentPath="/project1", nodeType="transformTop", nodeName="evolve")
|
||||
4. create_td_node(parentPath="/project1", nodeType="feedbackTop", nodeName="fb")
|
||||
5. create_td_node(parentPath="/project1", nodeType="levelTop", nodeName="color_correct")
|
||||
6. create_td_node(parentPath="/project1", nodeType="nullTop", nodeName="out")
|
||||
|
||||
7. update_td_node_parameters(nodePath="/project1/seed_noise",
|
||||
properties={"type": 1, "monochrome": false, "period": 2.0, "amp": 0.3,
|
||||
"resolutionw": 1920, "resolutionh": 1080})
|
||||
8. update_td_node_parameters(nodePath="/project1/mix",
|
||||
properties={"operand": 27}) # 27 = Screen blend
|
||||
9. update_td_node_parameters(nodePath="/project1/evolve",
|
||||
properties={"sx": 1.003, "sy": 1.003, "rz": 0.5, "extend": 2}) # slight zoom + rotate, repeat edges
|
||||
10. update_td_node_parameters(nodePath="/project1/fb",
|
||||
properties={"top": "/project1/mix"})
|
||||
11. update_td_node_parameters(nodePath="/project1/color_correct",
|
||||
properties={"opacity": 0.98, "gamma1": 0.85})
|
||||
|
||||
12. execute_python_script: """
|
||||
op('/project1/seed_noise').outputConnectors[0].connect(op('/project1/mix').inputConnectors[0])
|
||||
op('/project1/fb').outputConnectors[0].connect(op('/project1/evolve'))
|
||||
op('/project1/evolve').outputConnectors[0].connect(op('/project1/mix').inputConnectors[1])
|
||||
op('/project1/mix').outputConnectors[0].connect(op('/project1/color_correct'))
|
||||
op('/project1/color_correct').outputConnectors[0].connect(op('/project1/out'))
|
||||
"""
|
||||
```
|
||||
|
||||
**Variations:**
|
||||
- Change Transform: `rz` (rotation), `sx/sy` (zoom), `tx/ty` (drift)
|
||||
- Change Composite operand: Screen (glow), Add (bright), Multiply (dark)
|
||||
- Add HSV Adjust in the feedback loop for color evolution
|
||||
- Add Blur for dreamlike softness
|
||||
- Replace Noise with a GLSL TOP for custom seed patterns
|
||||
|
||||
### Pattern 5: Instancing (Particle-Like Systems)
|
||||
|
||||
Render thousands of copies of geometry, each with unique position/rotation/scale driven by CHOP data or DATs.
|
||||
|
||||
```
|
||||
Table DAT (instance data) -> DAT to CHOP -> Geometry COMP (instancing on) -> Render TOP
|
||||
+ Sphere SOP (template geometry)
|
||||
+ Constant MAT (material)
|
||||
+ Camera COMP
|
||||
+ Light COMP
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. create_td_node(parentPath="/project1", nodeType="tableDat", nodeName="instance_data")
|
||||
2. create_td_node(parentPath="/project1", nodeType="geometryComp", nodeName="geo1")
|
||||
3. create_td_node(parentPath="/project1/geo1", nodeType="sphereSop", nodeName="sphere")
|
||||
4. create_td_node(parentPath="/project1", nodeType="constMat", nodeName="mat1")
|
||||
5. create_td_node(parentPath="/project1", nodeType="cameraComp", nodeName="cam1")
|
||||
6. create_td_node(parentPath="/project1", nodeType="lightComp", nodeName="light1")
|
||||
7. create_td_node(parentPath="/project1", nodeType="renderTop", nodeName="render1")
|
||||
|
||||
8. execute_python_script: """
|
||||
import random, math
|
||||
dat = op('/project1/instance_data')
|
||||
dat.clear()
|
||||
dat.appendRow(['tx', 'ty', 'tz', 'sx', 'sy', 'sz', 'cr', 'cg', 'cb'])
|
||||
for i in range(500):
|
||||
angle = i * 0.1
|
||||
r = 2 + i * 0.01
|
||||
dat.appendRow([
|
||||
str(math.cos(angle) * r),
|
||||
str(math.sin(angle) * r),
|
||||
str((i - 250) * 0.02),
|
||||
'0.05', '0.05', '0.05',
|
||||
str(random.random()),
|
||||
str(random.random()),
|
||||
str(random.random())
|
||||
])
|
||||
"""
|
||||
|
||||
9. update_td_node_parameters(nodePath="/project1/geo1",
|
||||
properties={"instancing": true, "instancechop": "",
|
||||
"instancedat": "/project1/instance_data",
|
||||
"material": "/project1/mat1"})
|
||||
10. update_td_node_parameters(nodePath="/project1/render1",
|
||||
properties={"camera": "/project1/cam1", "geometry": "/project1/geo1",
|
||||
"light": "/project1/light1",
|
||||
"resolutionw": 1920, "resolutionh": 1080})
|
||||
11. update_td_node_parameters(nodePath="/project1/cam1",
|
||||
properties={"tz": 10})
|
||||
```
|
||||
|
||||
### Pattern 6: Reaction-Diffusion (GLSL)
|
||||
|
||||
Classic Gray-Scott reaction-diffusion system running on the GPU.
|
||||
|
||||
```
|
||||
Text DAT (GLSL code) -> GLSL TOP (resolution, dat reference) -> Feedback TOP
|
||||
^ |
|
||||
|_______________________________________|
|
||||
Level TOP (out)
|
||||
```
|
||||
|
||||
**Key GLSL code (write to Text DAT via execute_python_script):**
|
||||
|
||||
```glsl
|
||||
// Gray-Scott reaction-diffusion
|
||||
uniform float feed; // 0.037
|
||||
uniform float kill; // 0.06
|
||||
uniform float dA; // 1.0
|
||||
uniform float dB; // 0.5
|
||||
|
||||
layout(location = 0) out vec4 fragColor;
|
||||
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
vec2 texel = 1.0 / uTDOutputInfo.res.zw;
|
||||
|
||||
vec4 c = texture(sTD2DInputs[0], uv);
|
||||
float a = c.r;
|
||||
float b = c.g;
|
||||
|
||||
// Laplacian (9-point stencil)
|
||||
float lA = 0.0, lB = 0.0;
|
||||
for(int dx = -1; dx <= 1; dx++) {
|
||||
for(int dy = -1; dy <= 1; dy++) {
|
||||
float w = (dx == 0 && dy == 0) ? -1.0 : (abs(dx) + abs(dy) == 1 ? 0.2 : 0.05);
|
||||
vec4 s = texture(sTD2DInputs[0], uv + vec2(dx, dy) * texel);
|
||||
lA += s.r * w;
|
||||
lB += s.g * w;
|
||||
}
|
||||
}
|
||||
|
||||
float reaction = a * b * b;
|
||||
float newA = a + (dA * lA - reaction + feed * (1.0 - a));
|
||||
float newB = b + (dB * lB + reaction - (kill + feed) * b);
|
||||
|
||||
fragColor = vec4(clamp(newA, 0.0, 1.0), clamp(newB, 0.0, 1.0), 0.0, 1.0);
|
||||
}
|
||||
```
|
||||
|
||||
## Video Processing
|
||||
|
||||
### Pattern 7: Video Effects Chain
|
||||
|
||||
Apply a chain of effects to a video file.
|
||||
|
||||
```
|
||||
Movie File In TOP -> HSV Adjust TOP -> Level TOP -> Blur TOP -> Composite TOP -> Null TOP (out)
|
||||
^
|
||||
Text TOP ---+
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. create_td_node(parentPath="/project1", nodeType="moviefileinTop", nodeName="video_in")
|
||||
2. create_td_node(parentPath="/project1", nodeType="hsvadjustTop", nodeName="color")
|
||||
3. create_td_node(parentPath="/project1", nodeType="levelTop", nodeName="levels")
|
||||
4. create_td_node(parentPath="/project1", nodeType="blurTop", nodeName="blur")
|
||||
5. create_td_node(parentPath="/project1", nodeType="compositeTop", nodeName="overlay")
|
||||
6. create_td_node(parentPath="/project1", nodeType="textTop", nodeName="title")
|
||||
7. create_td_node(parentPath="/project1", nodeType="nullTop", nodeName="out")
|
||||
|
||||
8. update_td_node_parameters(nodePath="/project1/video_in",
|
||||
properties={"file": "/path/to/video.mp4", "play": true})
|
||||
9. update_td_node_parameters(nodePath="/project1/color",
|
||||
properties={"hueoffset": 0.1, "saturationmult": 1.3})
|
||||
10. update_td_node_parameters(nodePath="/project1/levels",
|
||||
properties={"brightness1": 1.1, "contrast": 1.2, "gamma1": 0.9})
|
||||
11. update_td_node_parameters(nodePath="/project1/blur",
|
||||
properties={"sizex": 2, "sizey": 2})
|
||||
12. update_td_node_parameters(nodePath="/project1/title",
|
||||
properties={"text": "My Video", "fontsizex": 48, "alignx": 1, "aligny": 1})
|
||||
|
||||
13. execute_python_script: """
|
||||
chain = ['video_in', 'color', 'levels', 'blur']
|
||||
for i in range(len(chain) - 1):
|
||||
op(f'/project1/{chain[i]}').outputConnectors[0].connect(op(f'/project1/{chain[i+1]}'))
|
||||
op('/project1/blur').outputConnectors[0].connect(op('/project1/overlay').inputConnectors[0])
|
||||
op('/project1/title').outputConnectors[0].connect(op('/project1/overlay').inputConnectors[1])
|
||||
op('/project1/overlay').outputConnectors[0].connect(op('/project1/out'))
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 8: Video Recording
|
||||
|
||||
Record the output to a file. **H.264/H.265 require a Commercial license** — use Motion JPEG (`mjpa`) on Non-Commercial.
|
||||
|
||||
```
|
||||
[any TOP chain] -> Null TOP -> Movie File Out TOP
|
||||
```
|
||||
|
||||
```python
|
||||
# Build via td_exec():
|
||||
root = op('/project1')
|
||||
|
||||
# Always put a Null TOP before the recorder
|
||||
null_out = root.op('out') # or create one
|
||||
rec = root.create(moviefileoutTOP, 'recorder')
|
||||
null_out.outputConnectors[0].connect(rec.inputConnectors[0])
|
||||
|
||||
rec.par.type = 'movie'
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
rec.par.videocodec = 'mjpa' # Motion JPEG — works on Non-Commercial
|
||||
|
||||
# Start recording (par.record is a toggle — .record() method may not exist)
|
||||
rec.par.record = True
|
||||
# ... let TD run for desired duration ...
|
||||
rec.par.record = False
|
||||
|
||||
# For image sequences:
|
||||
# rec.par.type = 'imagesequence'
|
||||
# rec.par.imagefiletype = 'png'
|
||||
# rec.par.file.expr = "'/tmp/frames/out' + me.fileSuffix" # fileSuffix REQUIRED
|
||||
```
|
||||
|
||||
**Pitfalls:**
|
||||
- Setting `par.file` + `par.record = True` in the same script may race — use `run("...", delayFrames=2)`
|
||||
- `TOP.save()` called rapidly always captures the same frame — use MovieFileOut for animation
|
||||
- See `pitfalls.md` #25-27 for full details
|
||||
|
||||
### Pattern 8b: TD → External Pipeline (e.g., ASCII Video)
|
||||
|
||||
Export TD visuals for use in another tool (ffmpeg, Python, ASCII art, etc.):
|
||||
|
||||
```python
|
||||
# 1. Record with MovieFileOut (MJPEG)
|
||||
rec.par.videocodec = 'mjpa'
|
||||
rec.par.record = True
|
||||
# ... wait N seconds ...
|
||||
rec.par.record = False
|
||||
|
||||
# 2. Extract frames with ffmpeg (outside TD)
|
||||
# ffmpeg -i /tmp/output.mov -vframes 120 /tmp/frames/frame_%06d.png
|
||||
|
||||
# 3. Load frames in Python for processing
|
||||
# from PIL import Image
|
||||
# img = Image.open('/tmp/frames/frame_000001.png')
|
||||
```
|
||||
|
||||
## Data Visualization
|
||||
|
||||
### Pattern 9: Table Data -> Bar Chart via Instancing
|
||||
|
||||
Visualize tabular data as a 3D bar chart.
|
||||
|
||||
```
|
||||
Table DAT (data) -> Script DAT (transform to instance format) -> DAT to CHOP
|
||||
|
|
||||
Box SOP -> Geometry COMP (instancing from CHOP) -> Render TOP -> Null TOP (out)
|
||||
+ PBR MAT
|
||||
+ Camera COMP
|
||||
+ Light COMP
|
||||
```
|
||||
|
||||
```python
|
||||
# Script DAT code to transform data to instance positions
|
||||
execute_python_script: """
|
||||
source = op('/project1/data_table')
|
||||
instance = op('/project1/instance_transform')
|
||||
instance.clear()
|
||||
instance.appendRow(['tx', 'ty', 'tz', 'sx', 'sy', 'sz', 'cr', 'cg', 'cb'])
|
||||
|
||||
for i in range(1, source.numRows):
|
||||
value = float(source[i, 'value'])
|
||||
name = source[i, 'name']
|
||||
instance.appendRow([
|
||||
str(i * 1.5), # x position (spread bars)
|
||||
str(value / 2), # y position (center bar vertically)
|
||||
'0', # z position
|
||||
'1', str(value), '1', # scale (height = data value)
|
||||
'0.2', '0.6', '1.0' # color (blue)
|
||||
])
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 9b: Audio-Reactive GLSL Fractal (Proven Recipe)
|
||||
|
||||
Audio spectrum drives a GLSL fractal shader directly via a spectrum texture input. Bass thickens inner fractal lines, mids twist rotation, highs light outer edges. Tested and working on TD 099 Non-Commercial.
|
||||
|
||||
```
|
||||
Audio File In CHOP → Audio Spectrum CHOP → Math CHOP (boost gain=5)
|
||||
→ Resample CHOP (256 samples) → CHOP To TOP (spectrum texture, 256x1)
|
||||
↓ (input 1)
|
||||
Constant TOP (rgba32float, time) → GLSL TOP (audio-reactive shader) → Null TOP
|
||||
(input 0) ↑
|
||||
Text DAT (shader code)
|
||||
```
|
||||
|
||||
**Build via td_exec (complete working script):**
|
||||
|
||||
```python
|
||||
td_exec("""
|
||||
import os
|
||||
root = op('/project1')
|
||||
|
||||
# Audio input
|
||||
audio = root.create(audiofileinCHOP, 'audio_in')
|
||||
audio.par.file = '/path/to/music.mp3'
|
||||
audio.par.playmode = 0 # Locked to timeline
|
||||
|
||||
# FFT analysis
|
||||
spectrum = root.create(audiospectrumCHOP, 'spectrum')
|
||||
audio.outputConnectors[0].connect(spectrum.inputConnectors[0])
|
||||
|
||||
# Normalize + boost
|
||||
math = root.create(mathCHOP, 'math_norm')
|
||||
spectrum.outputConnectors[0].connect(math.inputConnectors[0])
|
||||
math.par.gain = 5
|
||||
|
||||
# Resample to 256 bins for texture
|
||||
resample = root.create(resampleCHOP, 'resample_spec')
|
||||
math.outputConnectors[0].connect(resample.inputConnectors[0])
|
||||
resample.par.timeslice = True
|
||||
resample.par.rate = 256
|
||||
|
||||
# Spectrum → texture (256x1 image)
|
||||
# NOTE: choptoTOP has NO input connectors — use par.chop reference!
|
||||
spec_tex = root.create(choptoTOP, 'spectrum_tex')
|
||||
spec_tex.par.chop = resample
|
||||
|
||||
# Time driver (rgba32float to avoid 0-1 clamping!)
|
||||
time_drv = root.create(constantTOP, 'time_driver')
|
||||
time_drv.par.format = 'rgba32float'
|
||||
time_drv.par.outputresolution = 'custom'
|
||||
time_drv.par.resolutionw = 1
|
||||
time_drv.par.resolutionh = 1
|
||||
time_drv.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
time_drv.par.colorg.expr = "int(absTime.seconds / 1000.0)"
|
||||
|
||||
# GLSL shader
|
||||
glsl = root.create(glslTOP, 'audio_shader')
|
||||
glsl.par.outputresolution = 'custom'
|
||||
glsl.par.resolutionw = 1280; glsl.par.resolutionh = 720
|
||||
|
||||
shader_dat = root.create(textDAT, 'shader_code')
|
||||
shader_dat.text = open('/tmp/shader.glsl').read()
|
||||
glsl.par.pixeldat = shader_dat
|
||||
|
||||
# Wire: input 0=time, input 1=spectrum
|
||||
time_drv.outputConnectors[0].connect(glsl.inputConnectors[0])
|
||||
spec_tex.outputConnectors[0].connect(glsl.inputConnectors[1])
|
||||
|
||||
# Output + audio playback
|
||||
out = root.create(nullTOP, 'output')
|
||||
glsl.outputConnectors[0].connect(out.inputConnectors[0])
|
||||
audio_out = root.create(audiodeviceoutCHOP, 'audio_out')
|
||||
audio.outputConnectors[0].connect(audio_out.inputConnectors[0])
|
||||
|
||||
result = 'network built'
|
||||
""")
|
||||
```
|
||||
|
||||
**GLSL shader (reads spectrum from input 1 texture):**
|
||||
|
||||
```glsl
|
||||
out vec4 fragColor;
|
||||
|
||||
vec3 palette(float t) {
|
||||
vec3 a = vec3(0.5); vec3 b = vec3(0.5);
|
||||
vec3 c = vec3(1.0); vec3 d = vec3(0.263, 0.416, 0.557);
|
||||
return a + b * cos(6.28318 * (c * t + d));
|
||||
}
|
||||
|
||||
void main() {
|
||||
vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
float t = td.r + td.g * 1000.0;
|
||||
|
||||
vec2 res = uTDOutputInfo.res.zw;
|
||||
vec2 uv = (gl_FragCoord.xy * 2.0 - res) / min(res.x, res.y);
|
||||
vec2 uv0 = uv;
|
||||
vec3 finalColor = vec3(0.0);
|
||||
|
||||
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.0)).r;
|
||||
float mids = texture(sTD2DInputs[1], vec2(0.25, 0.0)).r;
|
||||
float highs = texture(sTD2DInputs[1], vec2(0.65, 0.0)).r;
|
||||
|
||||
float ca = cos(t * (0.15 + mids * 0.3));
|
||||
float sa = sin(t * (0.15 + mids * 0.3));
|
||||
uv = mat2(ca, -sa, sa, ca) * uv;
|
||||
|
||||
for (float i = 0.0; i < 4.0; i++) {
|
||||
uv = fract(uv * (1.4 + bass * 0.3)) - 0.5;
|
||||
float d = length(uv) * exp(-length(uv0));
|
||||
float freq = texture(sTD2DInputs[1], vec2(clamp(d*0.5, 0.0, 1.0), 0.0)).r;
|
||||
vec3 col = palette(length(uv0) + i * 0.4 + t * 0.35);
|
||||
d = sin(d * (7.0 + bass * 4.0) + t * 1.5) / 8.0;
|
||||
d = abs(d);
|
||||
d = pow(0.012 / d, 1.2 + freq * 0.8 + bass * 0.5);
|
||||
finalColor += col * d;
|
||||
}
|
||||
|
||||
float glow = (0.03 + bass * 0.05) / (length(uv0) + 0.03);
|
||||
finalColor += vec3(0.4, 0.1, 0.7) * glow * (0.6 + 0.4 * sin(t * 2.5));
|
||||
|
||||
float ring = abs(length(uv0) - 0.4 - mids * 0.3);
|
||||
finalColor += vec3(0.1, 0.6, 0.8) * (0.005 / ring) * (0.2 + highs * 0.5);
|
||||
|
||||
finalColor *= smoothstep(0.0, 1.0, 1.0 - dot(uv0*0.55, uv0*0.55));
|
||||
finalColor = finalColor / (finalColor + vec3(1.0));
|
||||
|
||||
fragColor = TDOutputSwizzle(vec4(finalColor, 1.0));
|
||||
}
|
||||
```
|
||||
|
||||
**How spectrum sampling drives the visual:**
|
||||
- `texture(sTD2DInputs[1], vec2(x, 0.0)).r` — x position = frequency (0=bass, 1=treble)
|
||||
- Inner fractal iterations sample lower x → react to bass
|
||||
- Outer iterations sample higher x → react to treble
|
||||
- `bass * 0.3` on `fract()` scale → fractal zoom pulses with bass
|
||||
- `bass * 4.0` on sin frequency → line density pulses with bass
|
||||
- `mids * 0.3` on rotation speed → spiral twists faster during vocal/mid sections
|
||||
- `highs * 0.5` on ring opacity → high-frequency sparkle on outer ring
|
||||
|
||||
**Recording the output:** Use MovieFileOut TOP with `mjpa` codec (H.264 requires Commercial license). See pitfalls #25-27.
|
||||
|
||||
## GLSL Shaders
|
||||
|
||||
### Pattern 10: Custom Fragment Shader
|
||||
|
||||
Write a custom visual effect as a GLSL fragment shader.
|
||||
|
||||
```
|
||||
Text DAT (shader code) -> GLSL TOP -> Level TOP -> Null TOP (out)
|
||||
+ optional input TOPs for texture sampling
|
||||
```
|
||||
|
||||
**Common GLSL uniforms available in TouchDesigner:**
|
||||
|
||||
```glsl
|
||||
// Automatically provided by TD
|
||||
uniform vec4 uTDOutputInfo; // .res.zw = resolution
|
||||
|
||||
// NOTE: uTDCurrentTime does NOT exist in TD 099!
|
||||
// Feed time via a 1x1 Constant TOP (format=rgba32float):
|
||||
// t.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
// t.par.colorg.expr = "int(absTime.seconds / 1000.0)"
|
||||
// Then read in GLSL:
|
||||
// vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
// float t = td.r + td.g * 1000.0;
|
||||
|
||||
// Input textures (from connected TOP inputs)
|
||||
uniform sampler2D sTD2DInputs[1]; // array of input samplers
|
||||
|
||||
// From vertex shader
|
||||
in vec3 vUV; // UV coordinates (0-1 range)
|
||||
```
|
||||
|
||||
**Example: Plasma shader (using time from input texture)**
|
||||
|
||||
```glsl
|
||||
layout(location = 0) out vec4 fragColor;
|
||||
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
// Read time from Constant TOP input 0 (rgba32float format)
|
||||
vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
float t = td.r + td.g * 1000.0;
|
||||
|
||||
float v1 = sin(uv.x * 10.0 + t);
|
||||
float v2 = sin(uv.y * 10.0 + t * 0.7);
|
||||
float v3 = sin((uv.x + uv.y) * 10.0 + t * 1.3);
|
||||
float v4 = sin(length(uv - 0.5) * 20.0 - t * 2.0);
|
||||
|
||||
float v = (v1 + v2 + v3 + v4) * 0.25;
|
||||
|
||||
vec3 color = vec3(
|
||||
sin(v * 3.14159 + 0.0) * 0.5 + 0.5,
|
||||
sin(v * 3.14159 + 2.094) * 0.5 + 0.5,
|
||||
sin(v * 3.14159 + 4.189) * 0.5 + 0.5
|
||||
);
|
||||
|
||||
fragColor = vec4(color, 1.0);
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 11: Multi-Pass GLSL (Ping-Pong)
|
||||
|
||||
For effects needing state across frames (particles, fluid, cellular automata), use GLSL Multi TOP with multiple passes or a Feedback TOP loop.
|
||||
|
||||
```
|
||||
GLSL Multi TOP (pass 0: simulation, pass 1: rendering)
|
||||
+ Text DAT (simulation shader)
|
||||
+ Text DAT (render shader)
|
||||
-> Level TOP -> Null TOP (out)
|
||||
^
|
||||
|__ Feedback TOP (feeds simulation state back)
|
||||
```
|
||||
|
||||
## Interactive Installations
|
||||
|
||||
### Pattern 12: Mouse/Touch -> Visual Response
|
||||
|
||||
```
|
||||
Mouse In CHOP -> Math CHOP (normalize to 0-1) -> [export to visual params]
|
||||
|
||||
# Or for touch/multi-touch:
|
||||
Multi Touch In DAT -> Script CHOP (parse touches) -> [export to visual params]
|
||||
```
|
||||
|
||||
```python
|
||||
# Normalize mouse position to 0-1 range
|
||||
execute_python_script: """
|
||||
op('/project1/noise1').par.offsetx.expr = "op('/project1/mouse_norm')['tx']"
|
||||
op('/project1/noise1').par.offsety.expr = "op('/project1/mouse_norm')['ty']"
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 13: OSC Control (from external software)
|
||||
|
||||
```
|
||||
OSC In CHOP (port 7000) -> Select CHOP (pick channels) -> [export to visual params]
|
||||
```
|
||||
|
||||
```
|
||||
1. create_td_node(parentPath="/project1", nodeType="oscinChop", nodeName="osc_in")
|
||||
2. update_td_node_parameters(nodePath="/project1/osc_in", properties={"port": 7000})
|
||||
|
||||
# OSC messages like /frequency 440 will appear as channel "frequency" with value 440
|
||||
# Export to any parameter:
|
||||
3. execute_python_script: "op('/project1/noise1').par.period.expr = \"op('/project1/osc_in')['frequency']\""
|
||||
```
|
||||
|
||||
### Pattern 14: MIDI Control (DJ/VJ)
|
||||
|
||||
```
|
||||
MIDI In CHOP (device) -> Select CHOP -> [export channels to visual params]
|
||||
```
|
||||
|
||||
Common MIDI mappings:
|
||||
- CC channels (knobs/faders): continuous 0-127, map to float params
|
||||
- Note On/Off: binary triggers, map to Trigger CHOP for envelopes
|
||||
- Velocity: intensity/brightness
|
||||
|
||||
## Live Performance
|
||||
|
||||
### Pattern 15: Multi-Source VJ Setup
|
||||
|
||||
```
|
||||
Source A (generative) ----+
|
||||
Source B (video) ---------+-- Switch/Cross TOP -- Level TOP -- Window COMP (output)
|
||||
Source C (camera) --------+
|
||||
^
|
||||
MIDI/OSC control selects active source and crossfade
|
||||
```
|
||||
|
||||
```python
|
||||
# MIDI CC1 controls which source is active (0-127 -> 0-2)
|
||||
execute_python_script: """
|
||||
op('/project1/switch1').par.index.expr = "int(op('/project1/midi_in')['cc1'] / 42)"
|
||||
"""
|
||||
|
||||
# MIDI CC2 controls crossfade between current and next
|
||||
execute_python_script: """
|
||||
op('/project1/cross1').par.cross.expr = "op('/project1/midi_in')['cc2'] / 127.0"
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 16: Projection Mapping
|
||||
|
||||
```
|
||||
Content TOPs ----+
|
||||
|
|
||||
Stoner TOP (UV mapping) -> Composite TOP -> Window COMP (projector output)
|
||||
or
|
||||
Kantan Mapper COMP (external .tox)
|
||||
```
|
||||
|
||||
For projection mapping, the key is:
|
||||
1. Create your visual content as standard TOPs
|
||||
2. Use Stoner TOP or a third-party mapping tool to UV-map content to physical surfaces
|
||||
3. Output via Window COMP to the projector
|
||||
|
||||
### Pattern 17: Cue System
|
||||
|
||||
```
|
||||
Table DAT (cue list: cue_number, scene_name, duration, transition_type)
|
||||
|
|
||||
Script CHOP (cue state: current_cue, progress, next_cue_trigger)
|
||||
|
|
||||
[export to Switch/Cross TOPs to transition between scenes]
|
||||
```
|
||||
|
||||
```python
|
||||
execute_python_script: """
|
||||
# Simple cue system
|
||||
cue_table = op('/project1/cue_list')
|
||||
cue_state = op('/project1/cue_state')
|
||||
|
||||
def advance_cue():
|
||||
current = int(cue_state.par.value0.val)
|
||||
next_cue = min(current + 1, cue_table.numRows - 1)
|
||||
cue_state.par.value0.val = next_cue
|
||||
|
||||
scene = cue_table[next_cue, 'scene']
|
||||
duration = float(cue_table[next_cue, 'duration'])
|
||||
|
||||
# Set crossfade target and duration
|
||||
op('/project1/cross1').par.cross.val = 0
|
||||
# Animate cross to 1.0 over duration seconds
|
||||
# (use a Timer CHOP or LFO CHOP for smooth animation)
|
||||
"""
|
||||
```
|
||||
|
||||
## Networking
|
||||
|
||||
### Pattern 18: OSC Server/Client
|
||||
|
||||
```
|
||||
# Sending OSC
|
||||
OSC Out CHOP -> (network) -> external application
|
||||
|
||||
# Receiving OSC
|
||||
(network) -> OSC In CHOP -> Select CHOP -> [use values]
|
||||
```
|
||||
|
||||
### Pattern 19: NDI Video Streaming
|
||||
|
||||
```
|
||||
# Send video over network
|
||||
[any TOP chain] -> NDI Out TOP (source name)
|
||||
|
||||
# Receive video from network
|
||||
NDI In TOP (select source) -> [process as normal TOP]
|
||||
```
|
||||
|
||||
### Pattern 20: WebSocket Communication
|
||||
|
||||
```
|
||||
WebSocket DAT -> Script DAT (parse JSON messages) -> [update visuals]
|
||||
```
|
||||
|
||||
```python
|
||||
execute_python_script: """
|
||||
ws = op('/project1/websocket1')
|
||||
ws.par.address = 'ws://localhost:8080'
|
||||
ws.par.active = True
|
||||
|
||||
# In a DAT Execute callback (Script DAT watching WebSocket DAT):
|
||||
# def onTableChange(dat):
|
||||
# import json
|
||||
# msg = json.loads(dat.text)
|
||||
# op('/project1/noise1').par.seed.val = msg.get('seed', 0)
|
||||
"""
|
||||
```
|
||||
239
skills/creative/touchdesigner/references/operators.md
Normal file
239
skills/creative/touchdesigner/references/operators.md
Normal file
|
|
@ -0,0 +1,239 @@
|
|||
# TouchDesigner Operator Reference
|
||||
|
||||
## Operator Families Overview
|
||||
|
||||
TouchDesigner has 6 operator families. Each family processes a specific data type and is color-coded in the UI. Operators can only connect to others of the SAME family (with cross-family converters as the bridge).
|
||||
|
||||
## TOPs — Texture Operators (Purple)
|
||||
|
||||
2D image/texture processing on the GPU. The workhorse of visual output.
|
||||
|
||||
### Generators (create images from nothing)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Noise TOP | `noiseTop` | `type` (0-6), `monochrome`, `seed`, `period`, `harmonics`, `exponent`, `amp`, `offset`, `resolutionw/h` | Procedural noise textures — Perlin, Simplex, Sparse, etc. Foundation of generative art. |
|
||||
| Constant TOP | `constantTop` | `colorr/g/b/a`, `resolutionw/h` | Solid color. Use as background or blend input. |
|
||||
| Text TOP | `textTop` | `text`, `fontsizex`, `fontfile`, `alignx/y`, `colorr/g/b` | Render text to texture. Supports multi-line, word wrap. |
|
||||
| Ramp TOP | `rampTop` | `type` (0=horizontal, 1=vertical, 2=radial, 3=circular), `phase`, `period` | Gradient textures for masking, color mapping. |
|
||||
| Circle TOP | `circleTop` | `radiusx/y`, `centerx/y`, `width` | Circles, rings, ellipses. |
|
||||
| Rectangle TOP | `rectangleTop` | `sizex/y`, `centerx/y`, `softness` | Rectangles with optional softness. |
|
||||
| GLSL TOP | `glslTop` | `dat` (points to shader DAT), `resolutionw/h`, `outputformat`, custom uniforms | Custom fragment shaders. Most powerful TOP for custom visuals. |
|
||||
| GLSL Multi TOP | `glslmultiTop` | `dat`, `numinputs`, `numoutputs`, `numcomputepasses` | Multi-pass GLSL with compute shaders. Advanced. |
|
||||
| Render TOP | `renderTop` | `camera`, `geometry`, `lights`, `resolutionw/h` | Renders 3D scenes (SOPs + MATs + Camera/Light COMPs). |
|
||||
|
||||
### Filters (modify a single input)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Level TOP | `levelTop` | `opacity`, `brightness1/2`, `gamma1/2`, `contrast`, `invert`, `blacklevel/whitelevel` | Brightness, contrast, gamma, levels. Essential color correction. |
|
||||
| Blur TOP | `blurTop` | `sizex/y`, `type` (0=Gaussian, 1=Box, 2=Bartlett) | Gaussian/box blur. |
|
||||
| Transform TOP | `transformTop` | `tx/ty`, `sx/sy`, `rz`, `pivotx/y`, `extend` (0=Hold, 1=Zero, 2=Repeat, 3=Mirror) | Translate, scale, rotate textures. |
|
||||
| HSV Adjust TOP | `hsvadjustTop` | `hueoffset`, `saturationmult`, `valuemult` | HSV color adjustments. |
|
||||
| Lookup TOP | `lookupTop` | (input: texture + lookup table) | Color remapping via lookup table texture. |
|
||||
| Edge TOP | `edgeTop` | `type` (0=Sobel, 1=Frei-Chen) | Edge detection. |
|
||||
| Displace TOP | `displaceTop` | `scalex/y` | Pixel displacement using a second input as displacement map. |
|
||||
| Flip TOP | `flipTop` | `flipx`, `flipy`, `flop` (diagonal) | Mirror/flip textures. |
|
||||
| Crop TOP | `cropTop` | `cropleft/right/top/bottom` | Crop region of texture. |
|
||||
| Resolution TOP | `resolutionTop` | `resolutionw/h`, `outputresolution` | Resize textures. |
|
||||
| Null TOP | `nullTop` | (none significant) | Pass-through. Use for organization, referencing, feedback delay. |
|
||||
| Cache TOP | `cacheTop` | `length`, `step` | Store N frames of history. Useful for trails, time effects. |
|
||||
|
||||
### Compositors (combine multiple inputs)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Composite TOP | `compositeTop` | `operand` (0-31: Over, Add, Multiply, Screen, etc.) | Blend two textures with standard compositing modes. |
|
||||
| Over TOP | `overTop` | (simple alpha compositing) | Layer with alpha. Simpler than Composite. |
|
||||
| Add TOP | `addTop` | (additive blend) | Additive blending. Great for glow, light effects. |
|
||||
| Multiply TOP | `multiplyTop` | (multiplicative blend) | Multiply blend. Good for masking, darkening. |
|
||||
| Switch TOP | `switchTop` | `index` (0-based) | Switch between multiple inputs by index. |
|
||||
| Cross TOP | `crossTop` | `cross` (0.0-1.0) | Crossfade between two inputs. |
|
||||
|
||||
### I/O (input/output)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Movie File In TOP | `moviefileinTop` | `file`, `speed`, `trim`, `index` | Load video files, image sequences. |
|
||||
| Movie File Out TOP | `moviefileoutTop` | `file`, `type` (codec), `record` (toggle) | Record/export video files. |
|
||||
| NDI In TOP | `ndiinTop` | `sourcename` | Receive NDI video streams. |
|
||||
| NDI Out TOP | `ndioutTop` | `sourcename` | Send NDI video streams. |
|
||||
| Syphon Spout In/Out TOP | `syphonspoutinTop` / `syphonspoutoutTop` | `servername` | Inter-app texture sharing. |
|
||||
| Video Device In TOP | `videodeviceinTop` | `device` | Webcam/capture card input. |
|
||||
| Feedback TOP | `feedbackTop` | `top` (path to the TOP to feed back) | One-frame delay feedback. Essential for recursive effects. |
|
||||
|
||||
### Converters
|
||||
|
||||
| Operator | Type Name | Direction | Use |
|
||||
|----------|-----------|-----------|-----|
|
||||
| CHOP to TOP | `choptopTop` | CHOP -> TOP | Visualize channel data as texture (waveform, spectrum display). |
|
||||
| TOP to CHOP | `topchopChop` | TOP -> CHOP | Sample texture pixels as channel data. |
|
||||
|
||||
## CHOPs — Channel Operators (Green)
|
||||
|
||||
Time-varying numeric data: audio, animation curves, sensor data, control signals.
|
||||
|
||||
### Generators
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Constant CHOP | `constantChop` | `name0/value0`, `name1/value1`... | Static named channels. Control panel for parameters. |
|
||||
| LFO CHOP | `lfoChop` | `frequency`, `type` (0=Sin, 1=Tri, 2=Square, 3=Ramp, 4=Pulse), `amp`, `offset`, `phase` | Low frequency oscillator. Animation driver. |
|
||||
| Noise CHOP | `noiseChop` | `type`, `roughness`, `period`, `amp`, `seed`, `channels` | Smooth random motion. Organic animation. |
|
||||
| Pattern CHOP | `patternChop` | `type` (0=Sine, 1=Triangle, ...), `length`, `cycles` | Generate waveform patterns. |
|
||||
| Timer CHOP | `timerChop` | `length`, `play`, `cue`, `cycles` | Countdown/count-up timer with cue points. |
|
||||
| Count CHOP | `countChop` | `threshold`, `limittype`, `limitmin/max` | Event counter with wrapping/clamping. |
|
||||
|
||||
### Audio
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Audio File In CHOP | `audiofileinChop` | `file`, `volume`, `play`, `speed`, `trim` | Play audio files. |
|
||||
| Audio Device In CHOP | `audiodeviceinChop` | `device`, `channels` | Live microphone/line input. |
|
||||
| Audio Spectrum CHOP | `audiospectrumChop` | `size` (FFT size), `outputformat` (0=Power, 1=Magnitude) | FFT frequency analysis. |
|
||||
| Audio Band EQ CHOP | `audiobandeqChop` | `bands`, `gaindb` per band | Frequency band isolation. |
|
||||
| Audio Device Out CHOP | `audiodeviceoutChop` | `device` | Audio playback output. |
|
||||
|
||||
### Math/Logic
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Math CHOP | `mathChop` | `preoff`, `gain`, `postoff`, `chanop` (0=Off, 1=Add, 2=Subtract, 3=Multiply...) | Math operations on channels. The Swiss army knife. |
|
||||
| Logic CHOP | `logicChop` | `preop` (0=Off, 1=AND, 2=OR, 3=XOR, 4=NAND), `convert` | Boolean logic on channels. |
|
||||
| Filter CHOP | `filterChop` | `type` (0=Low Pass, 1=Band Pass, 2=High Pass, 3=Notch), `cutofffreq`, `filterwidth` | Smooth, dampen, filter signals. |
|
||||
| Lag CHOP | `lagChop` | `lag1/2`, `overshoot1/2` | Smooth transitions with overshoot. |
|
||||
| Limit CHOP | `limitChop` | `type` (0=Clamp, 1=Loop, 2=ZigZag), `min/max` | Clamp or wrap channel values. |
|
||||
| Speed CHOP | `speedChop` | (none significant) | Integrate values (velocity to position, acceleration to velocity). |
|
||||
| Trigger CHOP | `triggerChop` | `attack`, `peak`, `decay`, `sustain`, `release` | ADSR envelope from trigger events. |
|
||||
| Select CHOP | `selectChop` | `chop` (path), `channames` | Reference channels from another CHOP. |
|
||||
| Merge CHOP | `mergeChop` | `align` (0=Extend, 1=Trim to First, 2=Trim to Shortest) | Combine channels from multiple CHOPs. |
|
||||
| Null CHOP | `nullChop` | (none significant) | Pass-through for organization and referencing. |
|
||||
|
||||
### Input Devices
|
||||
|
||||
| Operator | Type Name | Use |
|
||||
|----------|-----------|-----|
|
||||
| Mouse In CHOP | `mouseinChop` | Mouse position, buttons, wheel. |
|
||||
| Keyboard In CHOP | `keyboardinChop` | Keyboard key states. |
|
||||
| MIDI In CHOP | `midiinChop` | MIDI note/CC input. |
|
||||
| OSC In CHOP | `oscinChop` | OSC message input (network). |
|
||||
|
||||
## SOPs — Surface Operators (Blue)
|
||||
|
||||
3D geometry: points, polygons, NURBS, meshes.
|
||||
|
||||
### Generators
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Grid SOP | `gridSop` | `rows`, `cols`, `sizex/y`, `type` (0=Polygon, 1=Mesh, 2=NURBS) | Flat grid mesh. Foundation for displacement, instancing. |
|
||||
| Sphere SOP | `sphereSop` | `type`, `rows`, `cols`, `radius` | Sphere geometry. |
|
||||
| Box SOP | `boxSop` | `sizex/y/z` | Box geometry. |
|
||||
| Torus SOP | `torusSop` | `radiusx/y`, `rows`, `cols` | Donut shape. |
|
||||
| Circle SOP | `circleSop` | `type`, `radius`, `divs` | Circle/ring geometry. |
|
||||
| Line SOP | `lineSop` | `dist`, `points` | Line segments. |
|
||||
| Text SOP | `textSop` | `text`, `fontsizex`, `fontfile`, `extrude` | 3D text geometry. |
|
||||
|
||||
### Modifiers
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Transform SOP | `transformSop` | `tx/ty/tz`, `rx/ry/rz`, `sx/sy/sz` | Transform geometry (translate, rotate, scale). |
|
||||
| Noise SOP | `noiseSop` | `type`, `amp`, `period`, `roughness` | Deform geometry with noise. |
|
||||
| Sort SOP | `sortSop` | `ptsort`, `primsort` | Reorder points/primitives. |
|
||||
| Facet SOP | `facetSop` | `unique`, `consolidate`, `computenormals` | Normals, consolidation, unique points. |
|
||||
| Merge SOP | `mergeSop` | (none significant) | Combine multiple geometry inputs. |
|
||||
| Null SOP | `nullSop` | (none significant) | Pass-through. |
|
||||
|
||||
## DATs — Data Operators (White)
|
||||
|
||||
Text, tables, scripts, network data.
|
||||
|
||||
### Core
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Table DAT | `tableDat` | (edit content directly) | Spreadsheet-like data tables. |
|
||||
| Text DAT | `textDat` | (edit content directly) | Arbitrary text content. Shader code, configs, scripts. |
|
||||
| Script DAT | `scriptDat` | `language` (0=Python, 1=C++) | Custom callbacks and DAT processing. |
|
||||
| CHOP Execute DAT | `chopexecDat` | `chop` (path to watch), callbacks | Trigger Python on CHOP value changes. |
|
||||
| DAT Execute DAT | `datexecDat` | `dat` (path to watch) | Trigger Python on DAT content changes. |
|
||||
| Panel Execute DAT | `panelexecDat` | `panel` | Trigger Python on UI panel events. |
|
||||
|
||||
### I/O
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Web DAT | `webDat` | `url`, `fetchmethod` (0=GET, 1=POST) | HTTP requests. API integration. |
|
||||
| TCP/IP DAT | `tcpipDat` | `address`, `port`, `mode` | TCP networking. |
|
||||
| OSC In DAT | `oscinDat` | `port` | Receive OSC as text messages. |
|
||||
| Serial DAT | `serialDat` | `port`, `baudrate` | Serial port communication (Arduino, etc.). |
|
||||
| File In DAT | `fileinDat` | `file` | Read text files. |
|
||||
| File Out DAT | `fileoutDat` | `file`, `write` | Write text files. |
|
||||
|
||||
### Conversions
|
||||
|
||||
| Operator | Type Name | Direction | Use |
|
||||
|----------|-----------|-----------|-----|
|
||||
| DAT to CHOP | `dattochopChop` | DAT -> CHOP | Convert table data to channels. |
|
||||
| CHOP to DAT | `choptodatDat` | CHOP -> DAT | Convert channel data to table rows. |
|
||||
| SOP to DAT | `soptodatDat` | SOP -> DAT | Extract geometry data as table. |
|
||||
|
||||
## MATs — Material Operators (Yellow)
|
||||
|
||||
Materials for 3D rendering in Render TOP / Geometry COMP.
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Phong MAT | `phongMat` | `diff_colorr/g/b`, `spec_colorr/g/b`, `shininess`, `colormap`, `normalmap` | Classic Phong shading. Simple, fast. |
|
||||
| PBR MAT | `pbrMat` | `basecolorr/g/b`, `metallic`, `roughness`, `normalmap`, `emitcolorr/g/b` | Physically-based rendering. Realistic materials. |
|
||||
| GLSL MAT | `glslMat` | `dat` (shader DAT), custom uniforms | Custom vertex + fragment shaders for 3D. |
|
||||
| Constant MAT | `constMat` | `colorr/g/b`, `colormap` | Flat unlit color/texture. No shading. |
|
||||
| Point Sprite MAT | `pointspriteMat` | `colormap`, `scale` | Render points as camera-facing sprites. Great for particles. |
|
||||
| Wireframe MAT | `wireframeMat` | `colorr/g/b`, `width` | Wireframe rendering. |
|
||||
| Depth MAT | `depthMat` | `near`, `far` | Render depth buffer as grayscale. |
|
||||
|
||||
## COMPs — Component Operators (Gray)
|
||||
|
||||
Containers, 3D scene elements, UI components.
|
||||
|
||||
### 3D Scene
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Geometry COMP | `geometryComp` | `material` (path), `instancechop` (path), `instancing` (toggle) | Renders geometry with material. Instancing host. |
|
||||
| Camera COMP | `cameraComp` | `tx/ty/tz`, `rx/ry/rz`, `fov`, `near/far` | Camera for Render TOP. |
|
||||
| Light COMP | `lightComp` | `lighttype` (0=Point, 1=Directional, 2=Spot, 3=Cone), `dimmer`, `colorr/g/b` | Lighting for 3D scenes. |
|
||||
| Ambient Light COMP | `ambientlightComp` | `dimmer`, `colorr/g/b` | Ambient lighting. |
|
||||
| Environment Light COMP | `envlightComp` | `envmap` | Image-based lighting (IBL). |
|
||||
|
||||
### Containers
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Container COMP | `containerComp` | `w`, `h`, `bgcolor1/2/3` | UI container. Holds other COMPs for panel layouts. |
|
||||
| Base COMP | `baseComp` | (none significant) | Generic container. Networks-inside-networks. |
|
||||
| Replicator COMP | `replicatorComp` | `template`, `operatorsdat` | Clone a template operator N times from a table. |
|
||||
|
||||
### Utilities
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Window COMP | `windowComp` | `winw/h`, `winoffsetx/y`, `monitor`, `borders` | Output window for display/projection. |
|
||||
| Select COMP | `selectComp` | `rowcol`, `panel` | Select and display content from elsewhere. |
|
||||
| Engine COMP | `engineComp` | `tox`, `externaltox` | Load external .tox components. Sub-process isolation. |
|
||||
|
||||
## Cross-Family Converter Summary
|
||||
|
||||
| From | To | Operator | Type Name |
|
||||
|------|-----|----------|-----------|
|
||||
| CHOP | TOP | CHOP to TOP | `choptopTop` |
|
||||
| TOP | CHOP | TOP to CHOP | `topchopChop` |
|
||||
| DAT | CHOP | DAT to CHOP | `dattochopChop` |
|
||||
| CHOP | DAT | CHOP to DAT | `choptodatDat` |
|
||||
| SOP | CHOP | SOP to CHOP | `soptochopChop` |
|
||||
| CHOP | SOP | CHOP to SOP | `choptosopSop` |
|
||||
| SOP | DAT | SOP to DAT | `soptodatDat` |
|
||||
| DAT | SOP | DAT to SOP | `dattosopSop` |
|
||||
| SOP | TOP | (use Render TOP + Geometry COMP) | — |
|
||||
| TOP | SOP | TOP to SOP | `toptosopSop` |
|
||||
336
skills/creative/touchdesigner/references/pitfalls.md
Normal file
336
skills/creative/touchdesigner/references/pitfalls.md
Normal file
|
|
@ -0,0 +1,336 @@
|
|||
# TouchDesigner MCP — Pitfalls & Lessons Learned
|
||||
|
||||
Hard-won knowledge from real TD sessions. Read this before building anything.
|
||||
|
||||
## Setup & Connection
|
||||
|
||||
### 1. The .tox from the git repo is BROKEN
|
||||
|
||||
The `td/mcp_webserver_base.tox` in the `8beeeaaat/touchdesigner-mcp` git clone is **incomplete**. It's missing the `td_server` Python module (generated by `npm run gen:webserver` which requires Docker). Port 9981 opens, but every route returns 404.
|
||||
|
||||
**Always download the release zip:**
|
||||
```bash
|
||||
curl -L -o td.zip \
|
||||
"https://github.com/8beeeaaat/touchdesigner-mcp/releases/latest/download/touchdesigner-mcp-td.zip"
|
||||
unzip -o td.zip -d touchdesigner-mcp-td
|
||||
```
|
||||
|
||||
### 2. The release .tox also breaks (frequently)
|
||||
|
||||
Even the correct release .tox fails after drag-and-drop import because `import_modules.py` resolves `modules/` via `parent().par.externaltox.eval()` — a relative path that often goes wrong. Symptoms: port 9981 listens, all routes 404, TD Textport shows `[ERROR] Failed to setup modules`.
|
||||
|
||||
**The custom API handler (`scripts/custom_api_handler.py`) is more reliable.** It has zero external module dependencies — just a WebServer DAT + Text DAT callback. The skill's setup workflow should try the .tox first, test with `curl`, and auto-deploy the handler if 404.
|
||||
|
||||
### 3. You CANNOT automate the .tox import from outside TD
|
||||
|
||||
TD has no CLI flag to import a .tox. macOS blocks keystroke injection via System Events for security. The only way to get code into TD from outside is:
|
||||
- Have a WebServer DAT already running (chicken-and-egg)
|
||||
- AppleScript to open Textport + clipboard paste (fragile, not always reliable)
|
||||
- User manually drags the .tox or pastes a script
|
||||
|
||||
**Plan for one manual step** from the user (either .tox drag-drop or Textport paste). Make it as frictionless as possible: `open -R /path/to/file` to reveal in Finder.
|
||||
|
||||
### 4. The npm package name is `touchdesigner-mcp-server` (not `@anthropic/...`)
|
||||
|
||||
The Hermes config should use:
|
||||
```yaml
|
||||
command: npx
|
||||
args: ["-y", "touchdesigner-mcp-server@latest"]
|
||||
```
|
||||
|
||||
### 5. MCP tools may register but not be callable
|
||||
|
||||
Hermes may report "17 MCP tool(s) now available" but the tools aren't exposed as function calls. Use the REST API directly via `curl` in `execute_code` as a reliable fallback:
|
||||
```python
|
||||
def td_exec(script):
|
||||
escaped = json.dumps({"script": script})
|
||||
cmd = f"curl -s -X POST -H 'Content-Type: application/json' -d {shlex.quote(escaped)} 'http://127.0.0.1:9981/api/td/server/exec'"
|
||||
return json.loads(terminal(cmd)['output'])
|
||||
```
|
||||
|
||||
## TD WebServer DAT Quirks
|
||||
|
||||
### 6. Response body goes in `response['data']`, NOT `response['body']`
|
||||
|
||||
When writing custom WebServer DAT handlers, the response payload must be set on the `data` key:
|
||||
```python
|
||||
response['data'] = json.dumps({"result": 42}) # ✓ works
|
||||
response['body'] = json.dumps({"result": 42}) # ✗ ignored
|
||||
```
|
||||
|
||||
### 7. Request POST body comes as BYTES in `request['data']`
|
||||
|
||||
Not `request['body']`, and it's `bytes` not `str`:
|
||||
```python
|
||||
raw = request.get('data', b'')
|
||||
if isinstance(raw, bytes):
|
||||
raw = raw.decode('utf-8')
|
||||
body = json.loads(raw) if raw else {}
|
||||
```
|
||||
|
||||
### 8. Non-Commercial license caps resolution at 1280×1280
|
||||
|
||||
Setting `resolutionw=1920` silently clamps to 1280. Always check effective resolution after creation:
|
||||
```python
|
||||
n.cook(force=True)
|
||||
actual = str(n.width) + 'x' + str(n.height)
|
||||
```
|
||||
|
||||
## Parameter Names
|
||||
|
||||
### 9. NEVER hardcode parameter names — always discover
|
||||
|
||||
Parameter names change between TD versions. What works in 099 may not work in 098 or 2023.x. Always run discovery first:
|
||||
```python
|
||||
n = root.create(glslTOP, '_test')
|
||||
pars = [(p.name, type(p.val).__name__) for p in n.pars()]
|
||||
n.destroy()
|
||||
```
|
||||
|
||||
Known differences from docs/online references:
|
||||
| What docs say | TD 099 actual | Notes |
|
||||
|---------------|---------------|-------|
|
||||
| `dat` | `pixeldat` | GLSL TOP pixel shader DAT |
|
||||
| `colora` | `alpha` | Constant TOP alpha |
|
||||
| `sizex` / `sizey` | `size` | Blur TOP (single value) |
|
||||
| `fontr/g/b/a` | `fontcolorr/g/b/a` | Text TOP font color (r/g/b) |
|
||||
| `fontcolora` | `fontalpha` | Text TOP font alpha (NOT `fontcolora`) |
|
||||
| `bgcolora` | `bgalpha` | Text TOP bg alpha |
|
||||
| `value1name` | `vec0name` | GLSL TOP uniform name |
|
||||
|
||||
### 10. Use `safe_par()` pattern for cross-version compatibility
|
||||
|
||||
```python
|
||||
def safe_par(node, name, value):
|
||||
p = getattr(node.par, name, None)
|
||||
if p is not None:
|
||||
p.val = value
|
||||
return True
|
||||
return False
|
||||
```
|
||||
|
||||
### 11. `td.tdAttributeError` crashes the whole script
|
||||
|
||||
If you do `node.par.nonexistent = value`, TD raises `tdAttributeError` and **stops the entire script**. There's no way to catch it with try/except in some TD versions. Always check with `getattr` first or use `safe_par()`.
|
||||
|
||||
## GLSL Shaders
|
||||
|
||||
### 12. `uTDCurrentTime` does NOT exist in TD 099
|
||||
|
||||
The GLSL builtin for time was removed or never existed in some builds. Feed time via a 1×1 Constant TOP input. **CRITICAL: set format to `rgba32float`** — the default 8-bit format clamps values to 0-1, so `absTime.seconds % 1000.0` gets clamped and the GLSL shader sees a frozen time value of 1.0:
|
||||
```python
|
||||
t = root.create(constantTOP, 'time_driver')
|
||||
t.par.format = 'rgba32float' # ← REQUIRED! Without this, time is stuck at 1.0
|
||||
t.par.outputresolution = 'custom'
|
||||
t.par.resolutionw = 1
|
||||
t.par.resolutionh = 1
|
||||
t.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
t.par.colorg.expr = "int(absTime.seconds / 1000.0)"
|
||||
t.outputConnectors[0].connect(glsl.inputConnectors[0])
|
||||
```
|
||||
In GLSL:
|
||||
```glsl
|
||||
vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
float t = td.r + td.g * 1000.0;
|
||||
```
|
||||
|
||||
### 13. GLSL compile errors are silent in the API
|
||||
|
||||
The GLSL TOP shows a yellow warning triangle in the UI but `node.errors()` may return empty string. Check `node.warnings()` too, and create an Info DAT pointed at the GLSL TOP to read the actual compiler output.
|
||||
|
||||
### 14. TD GLSL uses `vUV.st` not `gl_FragCoord`
|
||||
|
||||
Standard GLSL patterns don't work. TD provides:
|
||||
- `vUV.st` — UV coordinates (0-1)
|
||||
- `uTDOutputInfo.res.zw` — resolution
|
||||
- `sTD2DInputs[0]` — input textures
|
||||
- `layout(location = 0) out vec4 fragColor` — output
|
||||
|
||||
## Node Management
|
||||
|
||||
### 15. Destroying nodes while iterating `root.children` causes `tdError`
|
||||
|
||||
The iterator is invalidated when a child is destroyed. Always snapshot first:
|
||||
```python
|
||||
kids = list(root.children) # snapshot
|
||||
for child in kids:
|
||||
if child.valid: # check — earlier destroys may cascade
|
||||
child.destroy()
|
||||
```
|
||||
|
||||
### 16. Feedback TOP: use `top` parameter, NOT direct input wire
|
||||
|
||||
In TD 099, the feedbackTOP's `top` parameter references which TOP to delay. **Do not also wire that TOP directly into the feedback's input** — this creates a real cook dependency loop (warning flood, potential crash). The "Not enough sources" error on feedbackTOP is benign and resolves after a few frames of playback.
|
||||
|
||||
Correct setup:
|
||||
```python
|
||||
fb = root.create(feedbackTOP, 'fb_delay')
|
||||
fb.par.top = comp.path # reference only — no wire to fb input
|
||||
fb.outputConnectors[0].connect(xf) # fb output -> transform -> fade -> comp
|
||||
```
|
||||
|
||||
The resulting "Cook dependency loop detected" **warning** on the transform/fade chain is expected and correct — that's what feedback loops do. It's informational, not an error.
|
||||
|
||||
### 16. GLSL TOP auto-creates companion nodes
|
||||
|
||||
Creating a `glslTOP` also creates `name_pixel` (Text DAT), `name_info` (Info DAT), and `name_compute` (Text DAT). These are visible in the network and count toward node totals. Don't be alarmed by "extra" nodes.
|
||||
|
||||
### 17. The default project root is `/project1`
|
||||
|
||||
New TD files start with `/project1` as the main container. System nodes live at `/`, `/ui`, `/sys`, `/local`, `/perform`. Don't create user nodes outside `/project1`.
|
||||
|
||||
### 18. `open -R` reveals the file but can't automate the drag
|
||||
|
||||
Use `open -R /path/to/file.tox` to open Finder highlighting the file. The user must then drag it into TD manually. No AppleScript workaround exists on modern macOS due to accessibility restrictions.
|
||||
|
||||
## Workflow
|
||||
|
||||
### 19. Always verify after building — errors are silent
|
||||
|
||||
Node errors and broken connections produce no output. Always check:
|
||||
```python
|
||||
for c in list(root.children):
|
||||
e = c.errors()
|
||||
w = c.warnings()
|
||||
if e: print(c.name, 'ERR:', e)
|
||||
if w: print(c.name, 'WARN:', w)
|
||||
```
|
||||
|
||||
### 20. Build in one big `execute_python_script` call, not many small ones
|
||||
|
||||
Each API round-trip adds latency. Bundle node creation + parameter setting + wiring into a single script that creates everything at once, then verify in one final call.
|
||||
|
||||
### 21. Window COMP param for display target is `winop` (not `top` or `window`)
|
||||
|
||||
To display output in a separate window:
|
||||
```python
|
||||
win = root.create(windowCOMP, 'display')
|
||||
win.par.winop = '/project1/logo_out' # ← this is the correct param
|
||||
win.par.winw = 1280; win.par.winh = 720
|
||||
win.par.winopen.pulse() # open the window
|
||||
```
|
||||
|
||||
### 22. Save the project to make API persistent across TD restarts
|
||||
|
||||
After deploying the custom API handler, save the project:
|
||||
```python
|
||||
td_exec("project.save(os.path.expanduser('~/Documents/HermesAgent.toe'))")
|
||||
```
|
||||
TD auto-opens the last saved project on launch. The API handler is now baked into the .toe file — next time TD opens, port 9981 is live with zero manual steps. To explicitly launch with this project: `open /Applications/TouchDesigner.app ~/Documents/HermesAgent.toe`
|
||||
|
||||
### 23. `sample()` returns frozen pixels when called from WebServer DAT callback
|
||||
|
||||
`out.sample(x, y)` called from inside the API handler's `exec()` returns pixels from a single cook snapshot. It does NOT update between multiple API calls in quick succession. To verify animation is working, either:
|
||||
- Compare samples with a 2+ second delay between separate `td_exec()` calls
|
||||
- Use `screencapture` on the display window
|
||||
- Check `absTime.seconds` is advancing and shader uses time correctly
|
||||
|
||||
### 22. `outputresolution` is a string menu, not an integer
|
||||
|
||||
### 25. MovieFileOut TOP: H.264/H.265 requires Commercial license
|
||||
|
||||
In Non-Commercial TD 099, encoding with H.264 or H.265 produces an error: "GPU Accelerated H.264/H.265 Encoding requires a Commercial license". Use Motion JPEG instead:
|
||||
```python
|
||||
rec = root.create(moviefileoutTOP, 'recorder')
|
||||
rec.par.type = 'movie'
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
rec.par.videocodec = 'mjpa' # Motion JPEG — works on Non-Commercial
|
||||
```
|
||||
|
||||
For image sequences, use `type = 'imagesequence'` and the file param **must** use `me.fileSuffix`:
|
||||
```python
|
||||
rec.par.type = 'imagesequence'
|
||||
rec.par.imagefiletype = 'png'
|
||||
rec.par.file.expr = "'/tmp/frames/out' + me.fileSuffix"
|
||||
```
|
||||
|
||||
### 26. MovieFileOut `.record()` method may not exist
|
||||
|
||||
In TD 099, there is no `.record()` method on moviefileoutTOP. Use the toggle parameter instead:
|
||||
```python
|
||||
rec.par.record = True # start recording
|
||||
rec.par.record = False # stop recording
|
||||
```
|
||||
|
||||
When setting the file path and starting recording in the same script, use `run()` with `delayFrames` to avoid a race condition where the old filename is used:
|
||||
```python
|
||||
rec.par.file = '/tmp/new_output.mov'
|
||||
run("op('/project1/recorder').par.record = True", delayFrames=2)
|
||||
```
|
||||
|
||||
### 27. TOP.save() captures same frame when called rapidly
|
||||
|
||||
`op('null1').save(path)` captures the current GPU texture at call time. When called multiple times in a single script (or rapid API calls), TD doesn't cook new frames between saves — all exported PNGs will be identical. To get unique frames, use the MovieFileOut TOP which records in real-time from TD's cook cycle.
|
||||
|
||||
### 28. AudioFileIn CHOP: cue before recording for sync
|
||||
|
||||
When recording audio-reactive visuals, always cue the audio to the start before beginning the recording. Otherwise the visuals are synced to wherever the audio happens to be in its playback:
|
||||
```python
|
||||
op('/project1/audio_in').par.cue.pulse() # reset to start
|
||||
run("op('/project1/recorder').par.record = True", delayFrames=3)
|
||||
```
|
||||
The audio plays via `playmode=0` (Locked to Timeline), so it stays in sync with TD's frame clock. Use `audiodeviceoutCHOP` to hear the audio during recording.
|
||||
|
||||
### 29. Audio Spectrum CHOP output is weak — boost with Math CHOP
|
||||
|
||||
The raw AudioSpectrum CHOP output has very small values (often 0.001-0.05 range). When fed directly to CHOP To TOP → GLSL, the shader barely reacts. Always insert a Math CHOP with `gain=5` (or higher) between the spectrum and the CHOP To TOP to get usable 0-1 range values in the shader.
|
||||
|
||||
### 30. CHOP To TOP texture size — Resample to 256 first
|
||||
|
||||
`choptoTOP` creates a texture where width = number of samples. An AudioSpectrum CHOP at 44100Hz has ~24000 samples — creating a 24000×1 texture is wasteful. Use a Resample CHOP set to 256 or 512 samples before the CHOP To TOP for an efficient spectrum texture.
|
||||
|
||||
### 31. CHOP To TOP has NO input connectors — use par.chop reference
|
||||
|
||||
`choptoTOP` does NOT have input connectors. `resample.outputConnectors[0].connect(chop_to_top.inputConnectors[0])` silently does nothing. Use the `chop` parameter instead:
|
||||
```python
|
||||
spec_tex = root.create(choptoTOP, 'spectrum_tex')
|
||||
spec_tex.par.chop = resample # ← correct: parameter reference
|
||||
# NOT: resample.outputConnectors[0].connect(spec_tex.inputConnectors[0]) # ← WRONG: no connectors
|
||||
```
|
||||
|
||||
### 22. `outputresolution` is a string menu, not an integer
|
||||
|
||||
The `outputresolution` param is a menu with string values:
|
||||
```
|
||||
menuNames: ['useinput','eighth','quarter','half','2x','4x','8x','fit','limit','custom','parpanel']
|
||||
```
|
||||
Always use the string form. Setting `outputresolution = 9` may silently fail.
|
||||
```python
|
||||
node.par.outputresolution = 'custom' # ✓ correct
|
||||
node.par.resolutionw = 1280; node.par.resolutionh = 720
|
||||
```
|
||||
Discover valid values: `list(node.par.outputresolution.menuNames)`
|
||||
|
||||
### 23. Large GLSL shaders break curl JSON escaping
|
||||
|
||||
GLSL code full of single/double quotes, backslashes, and special chars will corrupt the JSON payload when sent via `curl -d`. **Write the shader to a temp file and load it in TD:**
|
||||
```python
|
||||
# Agent side: write shader to /tmp/shader.glsl via write_file
|
||||
# TD side (via td_exec):
|
||||
sd = root.create(textDAT, 'shader_code')
|
||||
with open('/tmp/shader.glsl', 'r') as f:
|
||||
sd.text = f.read()
|
||||
```
|
||||
This avoids all escaping issues. The TD Python environment has full filesystem access.
|
||||
|
||||
### 24. TD crashes lose everything — the WebServer DAT must be re-deployed
|
||||
|
||||
If TD crashes (common with heavy GLSL or rapid-fire API calls), all nodes including the WebServer DAT are lost. On relaunch, port 9981 is dead. Recovery:
|
||||
1. Detect: `curl` returns exit code 7 (connection refused) or `lsof -i :9981` shows nothing
|
||||
2. Check: `pgrep TouchDesigner` to confirm TD is running
|
||||
3. Re-deploy: user must paste `exec(open('...custom_api_handler.py').read())` into Textport again
|
||||
4. Verify: poll port 9981 until API responds
|
||||
|
||||
The `td_exec()` helper should handle this gracefully:
|
||||
```python
|
||||
def td_exec(script):
|
||||
escaped = json.dumps({"script": script})
|
||||
cmd = f"curl -s --max-time 15 -X POST -H 'Content-Type: application/json' -d {shlex.quote(escaped)} 'http://127.0.0.1:9981/api/td/server/exec'"
|
||||
r = terminal(cmd, timeout=20)
|
||||
if r.get('exit_code') == 7:
|
||||
return {'error': 'TD not responding — WebServer DAT may need re-deploy'}
|
||||
try:
|
||||
return json.loads(r['output'])
|
||||
except:
|
||||
return {'error': 'Bad response', 'raw': r['output'][:200]}
|
||||
```
|
||||
443
skills/creative/touchdesigner/references/python-api.md
Normal file
443
skills/creative/touchdesigner/references/python-api.md
Normal file
|
|
@ -0,0 +1,443 @@
|
|||
# TouchDesigner Python API Reference
|
||||
|
||||
## The td Module
|
||||
|
||||
TouchDesigner's Python environment auto-imports the `td` module. All TD-specific classes, functions, and constants live here. Scripts inside TD (Script DATs, CHOP/DAT Execute callbacks, Extensions) have full access.
|
||||
|
||||
When using the MCP `execute_python_script` tool, these globals are pre-loaded:
|
||||
- `op` — shortcut for `td.op()`, finds operators by path
|
||||
- `ops` — shortcut for `td.ops()`, finds multiple operators by pattern
|
||||
- `me` — the operator running the script (not meaningful via MCP — will be the WebServer DAT)
|
||||
- `parent` — shortcut for `me.parent()`
|
||||
- `project` — the root project component
|
||||
- `td` — the full td module
|
||||
|
||||
## Finding Operators: op() and ops()
|
||||
|
||||
### op(path) — Find a single operator
|
||||
|
||||
```python
|
||||
# Absolute path (always works from MCP)
|
||||
node = op('/project1/noise1')
|
||||
|
||||
# Relative path (relative to current operator — only in Script DATs)
|
||||
node = op('noise1') # sibling
|
||||
node = op('../noise1') # parent's sibling
|
||||
|
||||
# Returns None if not found (does NOT raise)
|
||||
node = op('/project1/nonexistent') # None
|
||||
```
|
||||
|
||||
### ops(pattern) — Find multiple operators
|
||||
|
||||
```python
|
||||
# Glob patterns
|
||||
nodes = ops('/project1/noise*') # all nodes starting with "noise"
|
||||
nodes = ops('/project1/*') # all direct children
|
||||
nodes = ops('/project1/container1/*') # all children of container1
|
||||
|
||||
# Returns a tuple of operators (may be empty)
|
||||
for n in ops('/project1/*'):
|
||||
print(n.name, n.OPType)
|
||||
```
|
||||
|
||||
### Navigation from a node
|
||||
|
||||
```python
|
||||
node = op('/project1/noise1')
|
||||
|
||||
node.name # 'noise1'
|
||||
node.path # '/project1/noise1'
|
||||
node.OPType # 'noiseTop'
|
||||
node.type # <class 'noiseTop'>
|
||||
node.family # 'TOP'
|
||||
|
||||
# Parent / children
|
||||
node.parent() # the parent COMP
|
||||
node.parent().children # all siblings + self
|
||||
node.parent().findChildren(name='noise*') # filtered
|
||||
|
||||
# Type checking
|
||||
node.isTOP # True
|
||||
node.isCHOP # False
|
||||
node.isSOP # False
|
||||
node.isDAT # False
|
||||
node.isMAT # False
|
||||
node.isCOMP # False
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
Every operator has parameters accessed via the `.par` attribute.
|
||||
|
||||
### Reading parameters
|
||||
|
||||
```python
|
||||
node = op('/project1/noise1')
|
||||
|
||||
# Direct access
|
||||
node.par.seed.val # current evaluated value (may be an expression result)
|
||||
node.par.seed.eval() # same as .val
|
||||
node.par.seed.default # default value
|
||||
node.par.monochrome.val # boolean parameters: True/False
|
||||
|
||||
# List all parameters
|
||||
for p in node.pars():
|
||||
print(f"{p.name}: {p.val} (default: {p.default})")
|
||||
|
||||
# Filter by page (parameter group)
|
||||
for p in node.pars('Noise'): # page name
|
||||
print(f"{p.name}: {p.val}")
|
||||
```
|
||||
|
||||
### Setting parameters
|
||||
|
||||
```python
|
||||
# Direct value setting
|
||||
node.par.seed.val = 42
|
||||
node.par.monochrome.val = True
|
||||
node.par.resolutionw.val = 1920
|
||||
node.par.resolutionh.val = 1080
|
||||
|
||||
# String parameters
|
||||
op('/project1/text1').par.text.val = 'Hello World'
|
||||
|
||||
# File paths
|
||||
op('/project1/moviefilein1').par.file.val = '/path/to/video.mp4'
|
||||
|
||||
# Reference another operator (for "dat", "chop", "top" type parameters)
|
||||
op('/project1/glsl1').par.dat.val = '/project1/shader_code'
|
||||
```
|
||||
|
||||
### Parameter expressions
|
||||
|
||||
```python
|
||||
# Python expressions that evaluate dynamically
|
||||
node.par.seed.expr = "me.time.frame"
|
||||
node.par.tx.expr = "math.sin(me.time.seconds * 2)"
|
||||
|
||||
# Reference another parameter
|
||||
node.par.brightness1.expr = "op('/project1/constant1').par.value0.val"
|
||||
|
||||
# Export (one-way binding from CHOP to parameter)
|
||||
# This makes the parameter follow a CHOP channel value
|
||||
op('/project1/noise1').par.seed.val # can also be driven by exports
|
||||
```
|
||||
|
||||
### Parameter types
|
||||
|
||||
| Type | Python Type | Example |
|
||||
|------|------------|---------|
|
||||
| Float | `float` | `node.par.brightness1.val = 0.5` |
|
||||
| Int | `int` | `node.par.seed.val = 42` |
|
||||
| Toggle | `bool` | `node.par.monochrome.val = True` |
|
||||
| String | `str` | `node.par.text.val = 'hello'` |
|
||||
| Menu | `int` (index) or `str` (label) | `node.par.type.val = 'sine'` |
|
||||
| File | `str` (path) | `node.par.file.val = '/path/to/file'` |
|
||||
| OP reference | `str` (path) | `node.par.dat.val = '/project1/text1'` |
|
||||
| Color | separate r/g/b/a floats | `node.par.colorr.val = 1.0` |
|
||||
| XY/XYZ | separate x/y/z floats | `node.par.tx.val = 0.5` |
|
||||
|
||||
## Creating and Deleting Operators
|
||||
|
||||
```python
|
||||
# Create via parent component
|
||||
parent = op('/project1')
|
||||
new_node = parent.create(noiseTop) # using class reference
|
||||
new_node = parent.create(noiseTop, 'my_noise') # with custom name
|
||||
|
||||
# The MCP create_td_node tool handles this automatically:
|
||||
# create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="my_noise")
|
||||
|
||||
# Delete
|
||||
node = op('/project1/my_noise')
|
||||
node.destroy()
|
||||
|
||||
# Copy
|
||||
original = op('/project1/noise1')
|
||||
copy = parent.copy(original, name='noise1_copy')
|
||||
```
|
||||
|
||||
## Connections (Wiring Operators)
|
||||
|
||||
### Output to Input connections
|
||||
|
||||
```python
|
||||
# Connect noise1's output to level1's input
|
||||
op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1'))
|
||||
|
||||
# Connect to specific input index (for multi-input operators like Composite)
|
||||
op('/project1/noise1').outputConnectors[0].connect(op('/project1/composite1').inputConnectors[0])
|
||||
op('/project1/text1').outputConnectors[0].connect(op('/project1/composite1').inputConnectors[1])
|
||||
|
||||
# Disconnect all outputs
|
||||
op('/project1/noise1').outputConnectors[0].disconnect()
|
||||
|
||||
# Query connections
|
||||
node = op('/project1/level1')
|
||||
inputs = node.inputs # list of connected input operators
|
||||
outputs = node.outputs # list of connected output operators
|
||||
```
|
||||
|
||||
### Connection patterns for common setups
|
||||
|
||||
```python
|
||||
# Linear chain: A -> B -> C -> D
|
||||
ops_list = [op(f'/project1/{name}') for name in ['noise1', 'level1', 'blur1', 'null1']]
|
||||
for i in range(len(ops_list) - 1):
|
||||
ops_list[i].outputConnectors[0].connect(ops_list[i+1])
|
||||
|
||||
# Fan-out: A -> B, A -> C, A -> D
|
||||
source = op('/project1/noise1')
|
||||
for target_name in ['level1', 'composite1', 'transform1']:
|
||||
source.outputConnectors[0].connect(op(f'/project1/{target_name}'))
|
||||
|
||||
# Merge: A + B + C -> Composite
|
||||
comp = op('/project1/composite1')
|
||||
for i, source_name in enumerate(['noise1', 'text1', 'ramp1']):
|
||||
op(f'/project1/{source_name}').outputConnectors[0].connect(comp.inputConnectors[i])
|
||||
```
|
||||
|
||||
## DAT Content Manipulation
|
||||
|
||||
### Text DATs
|
||||
|
||||
```python
|
||||
dat = op('/project1/text1')
|
||||
|
||||
# Read
|
||||
content = dat.text # full text as string
|
||||
|
||||
# Write
|
||||
dat.text = "new content"
|
||||
dat.text = '''multi
|
||||
line
|
||||
content'''
|
||||
|
||||
# Append
|
||||
dat.text += "\nnew line"
|
||||
```
|
||||
|
||||
### Table DATs
|
||||
|
||||
```python
|
||||
dat = op('/project1/table1')
|
||||
|
||||
# Read cell
|
||||
val = dat[0, 0] # row 0, col 0
|
||||
val = dat[0, 'name'] # row 0, column named 'name'
|
||||
val = dat['key', 1] # row named 'key', col 1
|
||||
|
||||
# Write cell
|
||||
dat[0, 0] = 'value'
|
||||
|
||||
# Read row/col
|
||||
row = dat.row(0) # list of Cell objects
|
||||
col = dat.col('name') # list of Cell objects
|
||||
|
||||
# Dimensions
|
||||
rows = dat.numRows
|
||||
cols = dat.numCols
|
||||
|
||||
# Append row
|
||||
dat.appendRow(['col1_val', 'col2_val', 'col3_val'])
|
||||
|
||||
# Clear
|
||||
dat.clear()
|
||||
|
||||
# Set entire table
|
||||
dat.clear()
|
||||
dat.appendRow(['name', 'value', 'type'])
|
||||
dat.appendRow(['frequency', '440', 'float'])
|
||||
dat.appendRow(['amplitude', '0.8', 'float'])
|
||||
```
|
||||
|
||||
## Time and Animation
|
||||
|
||||
```python
|
||||
# Global time
|
||||
td.absTime.frame # absolute frame number (never resets)
|
||||
td.absTime.seconds # absolute seconds
|
||||
|
||||
# Timeline time (affected by play/pause/loop)
|
||||
me.time.frame # current frame on timeline
|
||||
me.time.seconds # current seconds on timeline
|
||||
me.time.rate # FPS setting
|
||||
|
||||
# Timeline control (via execute_python_script)
|
||||
project.play = True
|
||||
project.play = False
|
||||
project.frameRange = (1, 300) # set timeline range
|
||||
|
||||
# Cook frame (when operator was last computed)
|
||||
node.cookFrame
|
||||
node.cookTime
|
||||
```
|
||||
|
||||
## Extensions (Custom Python Classes on Components)
|
||||
|
||||
Extensions add custom Python methods and attributes to COMPs.
|
||||
|
||||
```python
|
||||
# Create extension on a Base COMP
|
||||
base = op('/project1/myBase')
|
||||
|
||||
# The extension class is defined in a Text DAT inside the COMP
|
||||
# Typically named 'ExtClass' with the extension code:
|
||||
|
||||
extension_code = '''
|
||||
class MyExtension:
|
||||
def __init__(self, ownerComp):
|
||||
self.ownerComp = ownerComp
|
||||
self.counter = 0
|
||||
|
||||
def Reset(self):
|
||||
self.counter = 0
|
||||
|
||||
def Increment(self):
|
||||
self.counter += 1
|
||||
return self.counter
|
||||
|
||||
@property
|
||||
def Count(self):
|
||||
return self.counter
|
||||
'''
|
||||
|
||||
# Write extension code to DAT inside the COMP
|
||||
op('/project1/myBase/extClass').text = extension_code
|
||||
|
||||
# Configure the extension on the COMP
|
||||
base.par.extension1 = 'extClass' # name of the DAT
|
||||
base.par.promoteextension1 = True # promote methods to parent
|
||||
|
||||
# Call extension methods
|
||||
base.Increment() # calls MyExtension.Increment()
|
||||
count = base.Count # accesses MyExtension.Count property
|
||||
base.Reset()
|
||||
```
|
||||
|
||||
## Useful Built-in Modules
|
||||
|
||||
### tdu — TouchDesigner Utilities
|
||||
|
||||
```python
|
||||
import tdu
|
||||
|
||||
# Dependency tracking (reactive values)
|
||||
dep = tdu.Dependency(initial_value)
|
||||
dep.val = new_value # triggers dependents to recook
|
||||
|
||||
# File path utilities
|
||||
tdu.expandPath('$HOME/Desktop/output.mov')
|
||||
|
||||
# Math
|
||||
tdu.clamp(value, min, max)
|
||||
tdu.remap(value, from_min, from_max, to_min, to_max)
|
||||
```
|
||||
|
||||
### TDFunctions
|
||||
|
||||
```python
|
||||
from TDFunctions import *
|
||||
|
||||
# Commonly used utilities
|
||||
clamp(value, low, high)
|
||||
remap(value, inLow, inHigh, outLow, outHigh)
|
||||
interp(value1, value2, t) # linear interpolation
|
||||
```
|
||||
|
||||
### TDStoreTools — Persistent Storage
|
||||
|
||||
```python
|
||||
from TDStoreTools import StorageManager
|
||||
|
||||
# Store data that survives project reload
|
||||
me.store('myKey', 'myValue')
|
||||
val = me.fetch('myKey', default='fallback')
|
||||
|
||||
# Storage dict
|
||||
me.storage['key'] = value
|
||||
```
|
||||
|
||||
## Common Patterns via execute_python_script
|
||||
|
||||
### Build a complete chain
|
||||
|
||||
```python
|
||||
# Create a complete audio-reactive noise chain
|
||||
parent = op('/project1')
|
||||
|
||||
# Create operators
|
||||
audio_in = parent.create(audiofileinChop, 'audio_in')
|
||||
spectrum = parent.create(audiospectrumChop, 'spectrum')
|
||||
chop_to_top = parent.create(choptopTop, 'chop_to_top')
|
||||
noise = parent.create(noiseTop, 'noise1')
|
||||
level = parent.create(levelTop, 'level1')
|
||||
null_out = parent.create(nullTop, 'out')
|
||||
|
||||
# Wire the chain
|
||||
audio_in.outputConnectors[0].connect(spectrum)
|
||||
spectrum.outputConnectors[0].connect(chop_to_top)
|
||||
noise.outputConnectors[0].connect(level)
|
||||
level.outputConnectors[0].connect(null_out)
|
||||
|
||||
# Set parameters
|
||||
audio_in.par.file = '/path/to/music.wav'
|
||||
audio_in.par.play = True
|
||||
spectrum.par.size = 512
|
||||
noise.par.type = 1 # Sparse
|
||||
noise.par.monochrome = False
|
||||
noise.par.resolutionw = 1920
|
||||
noise.par.resolutionh = 1080
|
||||
level.par.opacity = 0.8
|
||||
level.par.gamma1 = 0.7
|
||||
```
|
||||
|
||||
### Query network state
|
||||
|
||||
```python
|
||||
# Get all TOPs in the project
|
||||
tops = [c for c in op('/project1').findChildren(type=TOP)]
|
||||
for t in tops:
|
||||
print(f"{t.path}: {t.OPType} {'ERROR' if t.errors() else 'OK'}")
|
||||
|
||||
# Find all operators with errors
|
||||
def find_errors(parent_path='/project1'):
|
||||
parent = op(parent_path)
|
||||
errors = []
|
||||
for child in parent.findChildren(depth=-1):
|
||||
if child.errors():
|
||||
errors.append((child.path, child.errors()))
|
||||
return errors
|
||||
|
||||
result = find_errors()
|
||||
```
|
||||
|
||||
### Batch parameter changes
|
||||
|
||||
```python
|
||||
# Set parameters on multiple nodes at once
|
||||
settings = {
|
||||
'/project1/noise1': {'seed': 42, 'monochrome': False, 'resolutionw': 1920},
|
||||
'/project1/level1': {'brightness1': 1.2, 'gamma1': 0.8},
|
||||
'/project1/blur1': {'sizex': 5, 'sizey': 5},
|
||||
}
|
||||
|
||||
for path, params in settings.items():
|
||||
node = op(path)
|
||||
if node:
|
||||
for key, val in params.items():
|
||||
setattr(node.par, key, val)
|
||||
```
|
||||
|
||||
## Python Version and Packages
|
||||
|
||||
TouchDesigner bundles Python 3.11+ (as of TD 2024) with these pre-installed:
|
||||
- **numpy** — array operations, fast math
|
||||
- **scipy** — signal processing, FFT
|
||||
- **OpenCV** (cv2) — computer vision
|
||||
- **PIL/Pillow** — image processing
|
||||
- **requests** — HTTP client
|
||||
- **json**, **re**, **os**, **sys** — standard library
|
||||
|
||||
Custom packages can be installed to TD's Python site-packages directory. See TD documentation for the exact path per platform.
|
||||
274
skills/creative/touchdesigner/references/troubleshooting.md
Normal file
274
skills/creative/touchdesigner/references/troubleshooting.md
Normal file
|
|
@ -0,0 +1,274 @@
|
|||
# TouchDesigner Troubleshooting
|
||||
|
||||
> See `references/pitfalls.md` for the comprehensive lessons-learned list.
|
||||
|
||||
## Quick Connection Diagnostic
|
||||
|
||||
```bash
|
||||
lsof -i :9981 -P -n | grep LISTEN # Step 1: Is TD listening?
|
||||
curl -s http://127.0.0.1:9981/api/td/server/td # Step 2: API working?
|
||||
```
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
|---------|-------|-----|
|
||||
| Connection refused | No WebServer DAT | Deploy `scripts/custom_api_handler.py` in TD Textport |
|
||||
| HTTP 404 on all routes | .tox module import failed | Deploy custom handler (pitfalls #1-2) |
|
||||
| HTTP 200, empty body | Response in wrong key | Handler uses `response['data']` not `response['body']` (pitfalls #6) |
|
||||
| HTTP 200, JSON body | Working | Proceed to discovery |
|
||||
| MCP tools not callable | Normal — use curl instead | `td_exec()` pattern in SKILL.md works without MCP |
|
||||
|
||||
## Node Creation Issues
|
||||
|
||||
### "Node type not found" error
|
||||
|
||||
**Cause:** Wrong `nodeType` string in `create_td_node`.
|
||||
|
||||
**Fix:** Use camelCase with family suffix. Common mistakes:
|
||||
- Wrong: `NoiseTop`, `noise_top`, `NOISE TOP`, `Noise`
|
||||
- Right: `noiseTop`
|
||||
- Wrong: `AudioSpectrum`, `audio_spectrum_chop`
|
||||
- Right: `audiospectrumChop`
|
||||
|
||||
**Discovery method:** Use `get_td_classes` to see available types, or `execute_python_script` with `dir(td)` filtered for operator classes.
|
||||
|
||||
### Node created but not visible in TD
|
||||
|
||||
**Cause:** Node was created in a different container than expected, or TD viewport is looking at a different network.
|
||||
|
||||
**Fix:** Check `parentPath` — use absolute paths like `/project1`. Verify with `get_td_nodes(parentPath="/project1")`.
|
||||
|
||||
### Cannot create node inside a non-COMP
|
||||
|
||||
**Cause:** Only COMP operators (Container, Base, Geometry, etc.) can contain child operators. You cannot create nodes inside a TOP, CHOP, SOP, DAT, or MAT.
|
||||
|
||||
**Fix:** Create a Container COMP or Base COMP first, then create nodes inside it.
|
||||
|
||||
## Parameter Issues
|
||||
|
||||
### Parameter not updating
|
||||
|
||||
**Causes:**
|
||||
1. **Wrong parameter name.** TD parameter names change across versions. Run the discovery script (SKILL.md Step 0) or use `get_td_node_parameters` to discover exact names for your TD version. Never trust online docs or this skill's tables — always verify.
|
||||
2. **Parameter is read-only.** Some parameters are computed/locked.
|
||||
3. **Wrong value type.** Menu parameters need integer index or exact string label.
|
||||
4. **Parameter has an expression.** If `node.par.X.expr` is set, `.val` is ignored. Clear the expression first.
|
||||
|
||||
**Discovery-based approach (preferred):**
|
||||
```python
|
||||
execute_python_script(script="""
|
||||
n = op('/project1/mynode')
|
||||
pars = [(p.name, type(p.val).__name__, p.val) for p in n.pars()
|
||||
if any(k in p.name.lower() for k in ['color', 'size', 'dat', 'font', 'alpha'])]
|
||||
result = pars
|
||||
""")
|
||||
```
|
||||
|
||||
**Safe parameter setter pattern:**
|
||||
```python
|
||||
def safe_par(node, name, value):
|
||||
p = getattr(node.par, name, None)
|
||||
if p is not None:
|
||||
p.val = value
|
||||
return True
|
||||
return False # param doesn't exist in this TD version
|
||||
```
|
||||
|
||||
### Common parameter name gotchas
|
||||
|
||||
| What you expect | Actual name | Notes |
|
||||
|----------------|-------------|-------|
|
||||
| `width` | `resolutionw` | TOP resolution width |
|
||||
| `height` | `resolutionh` | TOP resolution height |
|
||||
| `filepath` | `file` | File path parameter |
|
||||
| `color` | `colorr`, `colorg`, `colorb`, `colora` | Separate RGBA components |
|
||||
| `position_x` | `tx` | Translate X |
|
||||
| `rotation` | `rz` | Rotate Z (2D rotation) |
|
||||
| `scale` | `sx`, `sy` | Separate X/Y scale |
|
||||
| `blend_mode` | `operand` | Composite TOP blend mode (integer) |
|
||||
| `opacity` | `opacity` | On Level TOP (this one is correct!) |
|
||||
|
||||
### Composite TOP operand values
|
||||
|
||||
| Mode | Index |
|
||||
|------|-------|
|
||||
| Over | 0 |
|
||||
| Under | 1 |
|
||||
| Inside | 2 |
|
||||
| Add | 3 |
|
||||
| Subtract | 4 |
|
||||
| Difference | 5 |
|
||||
| Multiply | 18 |
|
||||
| Screen | 27 |
|
||||
| Maximum | 13 |
|
||||
| Minimum | 14 |
|
||||
| Average | 28 |
|
||||
|
||||
## Connection/Wiring Issues
|
||||
|
||||
### Connections not working
|
||||
|
||||
**Causes:**
|
||||
1. **Cross-family wiring.** TOPs can only connect to TOPs, CHOPs to CHOPs, etc. Use converter operators to bridge families.
|
||||
2. **Wrong connector index.** Most operators have one output connector (index 0). Multi-output operators may need index 1, 2, etc.
|
||||
3. **Node path wrong.** Verify paths are absolute and correctly spelled.
|
||||
|
||||
**Verify connections:**
|
||||
```python
|
||||
execute_python_script(script="""
|
||||
node = op('/project1/level1')
|
||||
result = {
|
||||
'inputs': [i.path if i else None for i in node.inputs],
|
||||
'outputs': [o.path if o else None for o in node.outputs]
|
||||
}
|
||||
""")
|
||||
```
|
||||
|
||||
### Feedback loops causing errors
|
||||
|
||||
**Symptom:** "Circular dependency" or infinite cook loop.
|
||||
|
||||
**Fix:** Always use a Feedback TOP (or a Null TOP with a one-frame delay) to break the loop:
|
||||
```
|
||||
A -> B -> Feedback(references B) -> A
|
||||
```
|
||||
Never create A -> B -> A directly.
|
||||
|
||||
## Performance Issues
|
||||
|
||||
### Low FPS / choppy output
|
||||
|
||||
**Common causes and fixes:**
|
||||
|
||||
1. **Resolution too high.** Start at 1920x1080, only go higher if GPU handles it.
|
||||
2. **Too many operators.** Each operator has GPU/CPU overhead. Consolidate where possible.
|
||||
3. **Expensive shader.** GLSL TOPs with complex math per-pixel drain GPU. Profile with TD's Performance Monitor (F2).
|
||||
4. **No GPU instancing.** Rendering 1000 separate geometry objects is much slower than 1 instanced geometry.
|
||||
5. **Unnecessary cooks.** Operators that don't change frame-to-frame still recook if inputs change. Use Null TOPs to cache stable results.
|
||||
6. **Large texture transfers.** TOP to CHOP and CHOP to TOP involve GPU-CPU memory transfers. Minimize these.
|
||||
|
||||
**Performance Monitor:**
|
||||
```python
|
||||
execute_python_script(script="td.performanceMonitor = True")
|
||||
# After testing:
|
||||
execute_python_script(script="td.performanceMonitor = False")
|
||||
```
|
||||
|
||||
### Memory growing over time
|
||||
|
||||
**Causes:**
|
||||
- Cache TOPs with high `length` value
|
||||
- Feedback loops without brightness decay (values accumulate)
|
||||
- Table DATs growing without clearing
|
||||
- Movie File In loading many unique frames
|
||||
|
||||
**Fix:** Always add slight decay in feedback loops (Level TOP with `opacity=0.98` or multiply blend). Clear tables periodically.
|
||||
|
||||
## Export / Recording Issues
|
||||
|
||||
### Movie File Out not recording
|
||||
|
||||
**Checklist:**
|
||||
1. Is the `record` parameter toggled on? `update_td_node_parameters(properties={"record": true})`
|
||||
2. Is an input connected? The Movie File Out needs a TOP input.
|
||||
3. Is the output path valid and writable? Check `file` parameter.
|
||||
4. Is the codec available? H.264 (type 4) is most reliable.
|
||||
|
||||
### Exported video is black
|
||||
|
||||
**Causes:**
|
||||
1. The TOP chain output is all black (brightness too low).
|
||||
2. The input TOP has errors (check with `get_td_node_errors`).
|
||||
3. Resolution mismatch — the output may be wrong resolution.
|
||||
|
||||
**Debug:** Check the input TOP's actual pixel values:
|
||||
```python
|
||||
execute_python_script(script="""
|
||||
import numpy as np
|
||||
top = op('/project1/out')
|
||||
arr = top.numpyArray(delayed=True)
|
||||
result = {'mean': float(arr.mean()), 'max': float(arr.max()), 'shape': list(arr.shape)}
|
||||
""")
|
||||
```
|
||||
|
||||
### .tox export losing connections
|
||||
|
||||
**Note:** When saving a component as .tox, only the component and its internal children are saved. External connections (wires to operators outside the component) are lost. Design self-contained components.
|
||||
|
||||
## Python Scripting Issues
|
||||
|
||||
### execute_python_script returns empty result
|
||||
|
||||
**Causes:**
|
||||
1. The script used `exec()` semantics (multi-line) but didn't set `result`.
|
||||
2. The last expression has no return value (e.g., `print()` returns None).
|
||||
|
||||
**Fix:** Explicitly set `result`:
|
||||
```python
|
||||
execute_python_script(script="""
|
||||
nodes = op('/project1').findChildren(type=TOP)
|
||||
result = len(nodes) # explicitly set return value
|
||||
""")
|
||||
```
|
||||
|
||||
### Script errors not clear
|
||||
|
||||
**Check stderr in the response.** The MCP server captures both stdout and stderr from script execution. Error tracebacks appear in stderr.
|
||||
|
||||
### Module not found in TD Python
|
||||
|
||||
**Cause:** TD's Python environment may not have the module. TD bundles numpy, scipy, opencv, Pillow, and requests. Other packages need manual installation.
|
||||
|
||||
**Check available packages:**
|
||||
```python
|
||||
execute_python_script(script="""
|
||||
import sys
|
||||
result = [p for p in sys.path]
|
||||
""")
|
||||
```
|
||||
|
||||
## Common Workflow Pitfalls
|
||||
|
||||
### Building before verifying connection
|
||||
|
||||
Always call `get_td_info` first. If TD isn't running or the WebServer DAT isn't loaded, all subsequent tool calls will fail.
|
||||
|
||||
### Not checking errors after building
|
||||
|
||||
Always call `get_td_node_errors(nodePath="/project1")` after creating and wiring a network. Broken connections and missing references are silent until you check.
|
||||
|
||||
### Creating too many operators in one go
|
||||
|
||||
When building complex networks, create in logical groups:
|
||||
1. Create all operators in a section
|
||||
2. Wire that section
|
||||
3. Verify with `get_td_node_errors`
|
||||
4. Move to the next section
|
||||
|
||||
Don't create 50 operators, wire them all, then discover something was wrong 30 operators ago.
|
||||
|
||||
### Parameter expressions vs static values
|
||||
|
||||
If you set `node.par.X.val = 5` but there's an expression on that parameter (`node.par.X.expr`), the expression wins. To use a static value, clear the expression first:
|
||||
```python
|
||||
execute_python_script(script="""
|
||||
op('/project1/noise1').par.seed.expr = '' # clear expression
|
||||
op('/project1/noise1').par.seed.val = 42 # now static value works
|
||||
""")
|
||||
```
|
||||
|
||||
### Forgetting to start audio playback
|
||||
|
||||
Audio File In CHOP won't produce data unless `play` is True and a valid `file` is set:
|
||||
```
|
||||
update_td_node_parameters(nodePath="/project1/audio_in",
|
||||
properties={"file": "/path/to/music.wav", "play": true})
|
||||
```
|
||||
|
||||
### GLSL shader compilation errors
|
||||
|
||||
If a GLSL TOP shows errors after setting shader code:
|
||||
1. Check the shader code in the Text DAT for syntax errors
|
||||
2. Ensure the GLSL version is compatible (TD uses GLSL 3.30+)
|
||||
3. Input sampler name must be `sTD2DInputs[0]` (not custom names)
|
||||
4. Output must use `layout(location = 0) out vec4 fragColor`
|
||||
5. UV coordinates come from `vUV.st` (not `gl_FragCoord`)
|
||||
140
skills/creative/touchdesigner/scripts/custom_api_handler.py
Normal file
140
skills/creative/touchdesigner/scripts/custom_api_handler.py
Normal file
|
|
@ -0,0 +1,140 @@
|
|||
"""
|
||||
Custom API Handler for TouchDesigner WebServer DAT
|
||||
===================================================
|
||||
Use this when mcp_webserver_base.tox fails to load its modules
|
||||
(common — the .tox relies on relative paths to a modules/ folder
|
||||
that often break during import).
|
||||
|
||||
Paste into TD Textport or run via exec(open('...').read()):
|
||||
Creates a WebServer DAT + Text DAT callback handler on port 9981.
|
||||
Implements the core endpoints the MCP server expects.
|
||||
|
||||
After running, test with:
|
||||
curl http://127.0.0.1:9981/api/td/server/td
|
||||
"""
|
||||
|
||||
root = op('/project1')
|
||||
|
||||
# Remove broken webserver if present
|
||||
old = op('/project1/mcp_webserver_base')
|
||||
if old and old.valid:
|
||||
old.destroy()
|
||||
|
||||
# Create WebServer DAT
|
||||
ws = root.create(webserverDAT, 'api_server')
|
||||
ws.par.port = 9981
|
||||
ws.par.active = True
|
||||
ws.nodeX = -800; ws.nodeY = 500
|
||||
|
||||
# Create callback handler
|
||||
cb = root.create(textDAT, 'api_handler')
|
||||
cb.nodeX = -800; cb.nodeY = 400
|
||||
cb.text = r'''
|
||||
import json, traceback, io, sys
|
||||
|
||||
def onHTTPRequest(webServerDAT, request, response):
|
||||
uri = request.get('uri', '')
|
||||
method = request.get('method', 'GET')
|
||||
response['statusCode'] = 200
|
||||
response['statusReason'] = 'OK'
|
||||
response['headers'] = {'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*'}
|
||||
|
||||
try:
|
||||
# TD sends POST body as bytes in request['data']
|
||||
raw = request.get('data', request.get('body', ''))
|
||||
if isinstance(raw, bytes):
|
||||
raw = raw.decode('utf-8')
|
||||
body = {}
|
||||
if raw and isinstance(raw, str) and raw.strip():
|
||||
body = json.loads(raw)
|
||||
pars = request.get('pars', {})
|
||||
|
||||
if uri == '/api/td/server/td':
|
||||
response['data'] = json.dumps({
|
||||
'version': str(app.version),
|
||||
'osName': sys.platform,
|
||||
'apiVersion': '1.4.3',
|
||||
'product': 'TouchDesigner'
|
||||
})
|
||||
|
||||
elif uri == '/api/td/server/exec':
|
||||
script = body.get('script', '')
|
||||
old_stdout = sys.stdout
|
||||
sys.stdout = buf = io.StringIO()
|
||||
result_val = None
|
||||
err_text = ''
|
||||
try:
|
||||
globs = {'op': op, 'ops': ops, 'me': webServerDAT, 'parent': parent,
|
||||
'project': project, 'td': td, 'result': None,
|
||||
'app': app, 'absTime': absTime}
|
||||
lines = script.strip().split('\n')
|
||||
if len(lines) == 1:
|
||||
try:
|
||||
result_val = eval(script, globs)
|
||||
except SyntaxError:
|
||||
exec(script, globs)
|
||||
result_val = globs.get('result')
|
||||
else:
|
||||
exec(script, globs)
|
||||
result_val = globs.get('result')
|
||||
except Exception as e:
|
||||
err_text = traceback.format_exc()
|
||||
finally:
|
||||
captured = buf.getvalue()
|
||||
sys.stdout = old_stdout
|
||||
response['data'] = json.dumps({
|
||||
'result': _serialize(result_val),
|
||||
'stdout': captured,
|
||||
'stderr': err_text
|
||||
})
|
||||
|
||||
elif uri == '/api/nodes':
|
||||
pp = pars.get('parentPath', ['/project1'])[0]
|
||||
p = op(pp)
|
||||
nodes = []
|
||||
if p:
|
||||
for c in p.children:
|
||||
nodes.append({'name': c.name, 'path': c.path,
|
||||
'opType': c.OPType, 'family': c.family})
|
||||
response['data'] = json.dumps({'data': nodes})
|
||||
|
||||
elif uri == '/api/nodes/errors':
|
||||
np = pars.get('nodePath', ['/project1'])[0]
|
||||
n = op(np)
|
||||
errors = []
|
||||
if n:
|
||||
def _collect(node, depth=0):
|
||||
if depth > 10: return
|
||||
e = node.errors()
|
||||
if e:
|
||||
errors.append({'nodePath': node.path, 'nodeName': node.name,
|
||||
'opType': node.OPType, 'errors': str(e)})
|
||||
if hasattr(node, 'children'):
|
||||
for c in node.children: _collect(c, depth+1)
|
||||
_collect(n)
|
||||
response['data'] = json.dumps({'data': errors, 'hasErrors': len(errors)>0,
|
||||
'errorCount': len(errors)})
|
||||
|
||||
else:
|
||||
response['statusCode'] = 404
|
||||
response['data'] = json.dumps({'error': 'Unknown: ' + uri})
|
||||
|
||||
except Exception as e:
|
||||
response['statusCode'] = 500
|
||||
response['data'] = json.dumps({'error': str(e), 'trace': traceback.format_exc()})
|
||||
|
||||
return response
|
||||
|
||||
def _serialize(v):
|
||||
if v is None: return None
|
||||
if isinstance(v, (int, float, bool, str)): return v
|
||||
if isinstance(v, (list, tuple)): return [_serialize(i) for i in v]
|
||||
if isinstance(v, dict): return {str(k): _serialize(vv) for k, vv in v.items()}
|
||||
return str(v)
|
||||
'''
|
||||
|
||||
# Point webserver to callback
|
||||
ws.par.callbacks = cb.path
|
||||
|
||||
print("Custom API server created on port 9981")
|
||||
print("Test: curl http://127.0.0.1:9981/api/td/server/td")
|
||||
152
skills/creative/touchdesigner/scripts/setup.sh
Normal file
152
skills/creative/touchdesigner/scripts/setup.sh
Normal file
|
|
@ -0,0 +1,152 @@
|
|||
#!/usr/bin/env bash
|
||||
# TouchDesigner MCP Setup Verification Script
|
||||
# Checks all prerequisites and guides configuration
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
|
||||
pass() { echo -e " ${GREEN}✓${NC} $1"; }
|
||||
fail() { echo -e " ${RED}✗${NC} $1"; }
|
||||
warn() { echo -e " ${YELLOW}!${NC} $1"; }
|
||||
info() { echo -e " ${BLUE}→${NC} $1"; }
|
||||
|
||||
echo ""
|
||||
echo "TouchDesigner MCP Setup Check"
|
||||
echo "=============================="
|
||||
echo ""
|
||||
|
||||
ERRORS=0
|
||||
|
||||
# 1. Check Node.js
|
||||
echo "1. Node.js"
|
||||
if command -v node &>/dev/null; then
|
||||
NODE_VER=$(node --version 2>/dev/null || echo "unknown")
|
||||
MAJOR=$(echo "$NODE_VER" | sed 's/^v//' | cut -d. -f1)
|
||||
if [ "$MAJOR" -ge 18 ] 2>/dev/null; then
|
||||
pass "Node.js $NODE_VER (>= 18 required)"
|
||||
else
|
||||
fail "Node.js $NODE_VER (>= 18 required, please upgrade)"
|
||||
ERRORS=$((ERRORS + 1))
|
||||
fi
|
||||
else
|
||||
fail "Node.js not found"
|
||||
info "Install: https://nodejs.org/ or 'brew install node'"
|
||||
ERRORS=$((ERRORS + 1))
|
||||
fi
|
||||
|
||||
# 2. Check npm/npx
|
||||
echo "2. npm/npx"
|
||||
if command -v npx &>/dev/null; then
|
||||
NPX_VER=$(npx --version 2>/dev/null || echo "unknown")
|
||||
pass "npx $NPX_VER"
|
||||
else
|
||||
fail "npx not found (usually comes with Node.js)"
|
||||
ERRORS=$((ERRORS + 1))
|
||||
fi
|
||||
|
||||
# 3. Check MCP Python package
|
||||
echo "3. MCP Python package"
|
||||
HERMES_HOME="${HERMES_HOME:-$HOME/.hermes}"
|
||||
VENV_PYTHON=""
|
||||
|
||||
# Try to find the Hermes venv Python
|
||||
if [ -f "$HERMES_HOME/hermes-agent/.venv/bin/python" ]; then
|
||||
VENV_PYTHON="$HERMES_HOME/hermes-agent/.venv/bin/python"
|
||||
elif [ -f "$HERMES_HOME/hermes-agent/venv/bin/python" ]; then
|
||||
VENV_PYTHON="$HERMES_HOME/hermes-agent/venv/bin/python"
|
||||
fi
|
||||
|
||||
if [ -n "$VENV_PYTHON" ]; then
|
||||
if $VENV_PYTHON -c "import mcp" 2>/dev/null; then
|
||||
MCP_VER=$($VENV_PYTHON -c "import importlib.metadata; print(importlib.metadata.version('mcp'))" 2>/dev/null || echo "installed")
|
||||
pass "mcp package ($MCP_VER) in Hermes venv"
|
||||
else
|
||||
fail "mcp package not installed in Hermes venv"
|
||||
info "Install: $VENV_PYTHON -m pip install mcp"
|
||||
ERRORS=$((ERRORS + 1))
|
||||
fi
|
||||
else
|
||||
warn "Could not find Hermes venv — check mcp package manually"
|
||||
fi
|
||||
|
||||
# 4. Check TouchDesigner
|
||||
echo "4. TouchDesigner"
|
||||
TD_FOUND=false
|
||||
|
||||
# macOS
|
||||
if [ -d "/Applications/TouchDesigner.app" ]; then
|
||||
TD_FOUND=true
|
||||
pass "TouchDesigner found at /Applications/TouchDesigner.app"
|
||||
fi
|
||||
|
||||
# Linux (common install locations)
|
||||
if command -v TouchDesigner &>/dev/null; then
|
||||
TD_FOUND=true
|
||||
pass "TouchDesigner found in PATH"
|
||||
fi
|
||||
|
||||
if [ -d "$HOME/TouchDesigner" ]; then
|
||||
TD_FOUND=true
|
||||
pass "TouchDesigner found at ~/TouchDesigner"
|
||||
fi
|
||||
|
||||
if [ "$TD_FOUND" = false ]; then
|
||||
warn "TouchDesigner not detected (may be installed elsewhere)"
|
||||
info "Download from: https://derivative.ca/download"
|
||||
info "Free Non-Commercial license available"
|
||||
fi
|
||||
|
||||
# 5. Check TD WebServer DAT reachability
|
||||
echo "5. TouchDesigner WebServer DAT"
|
||||
TD_URL="${TD_API_URL:-http://127.0.0.1:9981}"
|
||||
if command -v curl &>/dev/null; then
|
||||
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --connect-timeout 3 "$TD_URL/api/td/server/td" 2>/dev/null || echo "000")
|
||||
if [ "$HTTP_CODE" = "200" ]; then
|
||||
TD_INFO=$(curl -s --connect-timeout 3 "$TD_URL/api/td/server/td" 2>/dev/null || echo "{}")
|
||||
pass "TD WebServer DAT responding at $TD_URL"
|
||||
info "Response: $TD_INFO"
|
||||
elif [ "$HTTP_CODE" = "000" ]; then
|
||||
warn "Cannot reach TD WebServer DAT at $TD_URL"
|
||||
info "Make sure TouchDesigner is running with mcp_webserver_base.tox imported"
|
||||
else
|
||||
warn "TD WebServer DAT returned HTTP $HTTP_CODE at $TD_URL"
|
||||
fi
|
||||
else
|
||||
warn "curl not found — cannot test TD connection"
|
||||
fi
|
||||
|
||||
# 6. Check Hermes config
|
||||
echo "6. Hermes MCP config"
|
||||
CONFIG_FILE="$HERMES_HOME/config.yaml"
|
||||
if [ -f "$CONFIG_FILE" ]; then
|
||||
if grep -q "touchdesigner" "$CONFIG_FILE" 2>/dev/null; then
|
||||
pass "TouchDesigner MCP server configured in config.yaml"
|
||||
else
|
||||
warn "No 'touchdesigner' entry found in mcp_servers config"
|
||||
info "Add a touchdesigner entry under mcp_servers: in $CONFIG_FILE"
|
||||
info "See references/mcp-tools.md for the configuration block"
|
||||
fi
|
||||
else
|
||||
warn "No Hermes config.yaml found at $CONFIG_FILE"
|
||||
fi
|
||||
|
||||
# Summary
|
||||
echo ""
|
||||
echo "=============================="
|
||||
if [ $ERRORS -eq 0 ]; then
|
||||
echo -e "${GREEN}All critical checks passed!${NC}"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo " 1. Open TouchDesigner and import mcp_webserver_base.tox"
|
||||
echo " 2. Add the MCP server config to Hermes (see references/mcp-tools.md)"
|
||||
echo " 3. Restart Hermes and test: 'Get TouchDesigner server info'"
|
||||
else
|
||||
echo -e "${RED}$ERRORS critical issue(s) found.${NC}"
|
||||
echo "Fix the issues above, then re-run this script."
|
||||
fi
|
||||
echo ""
|
||||
Loading…
Add table
Add a link
Reference in a new issue