mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-04-26 01:01:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

feat: rewrite TouchDesigner skill for twozero MCP (v2.0.0)

Browse source 
Major rewrite of the TouchDesigner skill:
- Replace custom API handler with twozero MCP (36 native tools)
- Add audio-reactive GLSL proven recipe (spectrum chain, pitfalls)
- Add recording checklist (FPS>0, non-black, audio cueing)
- Expand pitfalls: 38 entries from real sessions (was 20)
- Update network-patterns with MCP-native build scripts
- Rewrite mcp-tools reference for twozero v2.774+
- Update troubleshooting for MCP-based workflow
- Remove obsolete custom_api_handler.py
- Generalize Environment section for all users
- Remove session-specific Paired Skills section
- Bump version to 2.0.0
This commit is contained in:
kshitijk4poor 2026-04-17 21:50:35 +05:30 • committed by Teknium
parent 7a5371b20d
commit 6f27390fae
8 changed files with 1398 additions and 1420 deletions
Download patch file Download diff file Expand all files Collapse all files

622
skills/creative/touchdesigner/references/pitfalls.md
View file

@ -2,93 +2,16 @@
Hard-won knowledge from real TD sessions. Read this before building anything.
## Setup & Connection
### 1. The .tox from the git repo is BROKEN
The `td/mcp_webserver_base.tox` in the `8beeeaaat/touchdesigner-mcp` git clone is **incomplete**. It's missing the `td_server` Python module (generated by `npm run gen:webserver` which requires Docker). Port 9981 opens, but every route returns 404.
**Always download the release zip:**
```bash
curl -L -o td.zip \
"https://github.com/8beeeaaat/touchdesigner-mcp/releases/latest/download/touchdesigner-mcp-td.zip"
unzip -o td.zip -d touchdesigner-mcp-td
```
### 2. The release .tox also breaks (frequently)
Even the correct release .tox fails after drag-and-drop import because `import_modules.py` resolves `modules/` via `parent().par.externaltox.eval()` — a relative path that often goes wrong. Symptoms: port 9981 listens, all routes 404, TD Textport shows `[ERROR] Failed to setup modules`.
**The custom API handler (`scripts/custom_api_handler.py`) is more reliable.** It has zero external module dependencies — just a WebServer DAT + Text DAT callback. The skill's setup workflow should try the .tox first, test with `curl`, and auto-deploy the handler if 404.
### 3. You CANNOT automate the .tox import from outside TD
TD has no CLI flag to import a .tox. macOS blocks keystroke injection via System Events for security. The only way to get code into TD from outside is:
- Have a WebServer DAT already running (chicken-and-egg)
- AppleScript to open Textport + clipboard paste (fragile, not always reliable)
- User manually drags the .tox or pastes a script
**Plan for one manual step** from the user (either .tox drag-drop or Textport paste). Make it as frictionless as possible: `open -R /path/to/file` to reveal in Finder.
### 4. The npm package name is `touchdesigner-mcp-server` (not `@anthropic/...`)
The Hermes config should use:
```yaml
command: npx
args: ["-y", "touchdesigner-mcp-server@latest"]
```
### 5. MCP tools may register but not be callable
Hermes may report "17 MCP tool(s) now available" but the tools aren't exposed as function calls. Use the REST API directly via `curl` in `execute_code` as a reliable fallback:
```python
def td_exec(script):
escaped = json.dumps({"script": script})
cmd = f"curl -s -X POST -H 'Content-Type: application/json' -d {shlex.quote(escaped)} 'http://127.0.0.1:9981/api/td/server/exec'"
return json.loads(terminal(cmd)['output'])
```
## TD WebServer DAT Quirks
### 6. Response body goes in `response['data']`, NOT `response['body']`
When writing custom WebServer DAT handlers, the response payload must be set on the `data` key:
```python
response['data'] = json.dumps({"result": 42}) # ✓ works
response['body'] = json.dumps({"result": 42}) # ✗ ignored
```
### 7. Request POST body comes as BYTES in `request['data']`
Not `request['body']`, and it's `bytes` not `str`:
```python
raw = request.get('data', b'')
if isinstance(raw, bytes):
raw = raw.decode('utf-8')
body = json.loads(raw) if raw else {}
```
### 8. Non-Commercial license caps resolution at 1280×1280
Setting `resolutionw=1920` silently clamps to 1280. Always check effective resolution after creation:
```python
n.cook(force=True)
actual = str(n.width) + 'x' + str(n.height)
```
## Parameter Names
### 9. NEVER hardcode parameter names — always discover
### 1. NEVER hardcode parameter names — always discover
Parameter names change between TD versions. What works in 099 may not work in 098 or 2023.x. Always run discovery first:
```python
n = root.create(glslTOP, '_test')
pars = [(p.name, type(p.val).__name__) for p in n.pars()]
n.destroy()
```
Parameter names change between TD versions. What works in one build may not work in another. ALWAYS use td_get_par_info to discover actual names from TD.
Known differences from docs/online references:
| What docs say | TD 099 actual | Notes |
The agent's LLM training data contains WRONG parameter names. Do not trust them.
Known historical differences (may vary further — always verify):
| What docs/training say | Actual in some versions | Notes |
|---------------|---------------|-------|
| `dat` | `pixeldat` | GLSL TOP pixel shader DAT |
| `colora` | `alpha` | Constant TOP alpha |
@ -98,7 +21,15 @@ Known differences from docs/online references:
| `bgcolora` | `bgalpha` | Text TOP bg alpha |
| `value1name` | `vec0name` | GLSL TOP uniform name |
### 10. Use `safe_par()` pattern for cross-version compatibility
### 2. twozero td_execute_python response format
When calling `td_execute_python` via twozero MCP, successful responses return `(ok)` followed by FPS/error summary (e.g. `[fps 60.0/60] [0 err/0 warn]`), NOT the raw Python `result` dict. If you're parsing responses programmatically, check for the `(ok)` prefix — don't pattern-match on Python variable names from the script. Use `td_get_operator_info` or separate inspection calls to read back values.
### 3. When using td_set_operator_pars, param names must match exactly
Use td_get_par_info to discover them. The MCP tool validates parameter names and returns clear errors explaining what went wrong, unlike raw Python which crashes the whole script with tdAttributeError and stops execution. Always discover before setting.
### 3. Use `safe_par()` pattern for cross-version compatibility
```python
def safe_par(node, name, value):
@ -109,36 +40,65 @@ def safe_par(node, name, value):
return False
```
### 11. `td.tdAttributeError` crashes the whole script
### 4. `td.tdAttributeError` crashes the whole script — use defensive access
If you do `node.par.nonexistent = value`, TD raises `tdAttributeError` and **stops the entire script**. There's no way to catch it with try/except in some TD versions. Always check with `getattr` first or use `safe_par()`.
If you do `node.par.nonexistent = value`, TD raises `tdAttributeError` and stops the entire script. Prevention is better than catching:
- Use `op()` instead of `opex()` — `op()` returns None on failure, `opex()` raises
- Use `hasattr(node.par, 'name')` before accessing any parameter
- Use `getattr(node.par, 'name', None)` with a default
- Use the `safe_par()` pattern from pitfall #3
```python
# WRONG — crashes if param doesn't exist:
node.par.nonexistent = value
# CORRECT — defensive access:
if hasattr(node.par, 'nonexistent'):
node.par.nonexistent = value
```
### 5. `outputresolution` is a string menu, not an integer
```
menuNames: ['useinput','eighth','quarter','half','2x','4x','8x','fit','limit','custom','parpanel']
```
Always use the string form. Setting `outputresolution = 9` may silently fail.
```python
node.par.outputresolution = 'custom' # correct
node.par.resolutionw = 1280; node.par.resolutionh = 720
```
Discover valid values: `list(node.par.outputresolution.menuNames)`
## GLSL Shaders
### 12. `uTDCurrentTime` does NOT exist in TD 099
### 6. `uTDCurrentTime` does NOT exist in GLSL TOP
The GLSL builtin for time was removed or never existed in some builds. Feed time via a 1×1 Constant TOP input. **CRITICAL: set format to `rgba32float`** — the default 8-bit format clamps values to 0-1, so `absTime.seconds % 1000.0` gets clamped and the GLSL shader sees a frozen time value of 1.0:
There is NO built-in time uniform for GLSL TOPs. GLSL MAT has `uTDGeneral.seconds` but that's NOT available in GLSL TOP context.
**PRIMARY — GLSL TOP Vectors/Values page:**
```python
gl.par.value0name = 'uTime'
gl.par.value0.expr = "absTime.seconds"
# In GLSL: uniform float uTime;
```
**FALLBACK — Constant TOP texture (for complex time data):**
CRITICAL: set format to `rgba32float` — default 8-bit clamps to 0-1:
```python
t = root.create(constantTOP, 'time_driver')
t.par.format = 'rgba32float' # ← REQUIRED! Without this, time is stuck at 1.0
t.par.format = 'rgba32float'
t.par.outputresolution = 'custom'
t.par.resolutionw = 1
t.par.resolutionh = 1
t.par.resolutionw = 1; t.par.resolutionh = 1
t.par.colorr.expr = "absTime.seconds % 1000.0"
t.par.colorg.expr = "int(absTime.seconds / 1000.0)"
t.outputConnectors[0].connect(glsl.inputConnectors[0])
```
In GLSL:
```glsl
vec4 td = texture(sTD2DInputs[0], vec2(0.5));
float t = td.r + td.g * 1000.0;
```
### 13. GLSL compile errors are silent in the API
### 7. GLSL compile errors are silent in the API
The GLSL TOP shows a yellow warning triangle in the UI but `node.errors()` may return empty string. Check `node.warnings()` too, and create an Info DAT pointed at the GLSL TOP to read the actual compiler output.
### 14. TD GLSL uses `vUV.st` not `gl_FragCoord`
### 8. TD GLSL uses `vUV.st` not `gl_FragCoord` — and REQUIRES `TDOutputSwizzle()` on macOS
Standard GLSL patterns don't work. TD provides:
- `vUV.st` — UV coordinates (0-1)
@ -146,9 +106,26 @@ Standard GLSL patterns don't work. TD provides:
- `sTD2DInputs[0]` — input textures
- `layout(location = 0) out vec4 fragColor` — output
CRITICAL on macOS: Always wrap output with `TDOutputSwizzle()`:
```glsl
fragColor = TDOutputSwizzle(color);
```
TD uses GLSL 4.60 (Vulkan backend). GLSL 3.30 and earlier removed.
### 9. Large GLSL shaders — write to temp file
GLSL code with special characters can corrupt JSON payloads. Write the shader to a temp file and load it in TD:
```python
# Agent side: write shader to /tmp/shader.glsl via write_file
# TD side:
sd = root.create(textDAT, 'shader_code')
with open('/tmp/shader.glsl', 'r') as f:
sd.text = f.read()
```
## Node Management
### 15. Destroying nodes while iterating `root.children` causes `tdError`
### 10. Destroying nodes while iterating `root.children` causes `tdError`
The iterator is invalidated when a child is destroyed. Always snapshot first:
```python
@ -158,9 +135,34 @@ for child in kids:
child.destroy()
```
### 16. Feedback TOP: use `top` parameter, NOT direct input wire
### 10b. Split cleanup and creation into SEPARATE td_execute_python calls
In TD 099, the feedbackTOP's `top` parameter references which TOP to delay. **Do not also wire that TOP directly into the feedback's input** — this creates a real cook dependency loop (warning flood, potential crash). The "Not enough sources" error on feedbackTOP is benign and resolves after a few frames of playback.
Creating nodes with the same names you just destroyed in the SAME script causes "Invalid OP object" errors — even with `list()` snapshot. TD's internal references can go stale within one execution context.
**WRONG (single call):**
```python
# td_execute_python:
for c in list(root.children):
if c.valid and c.name.startswith('promo_'):
c.destroy()
# ... then create promo_audio, promo_shader etc. in same script → CRASHES
```
**CORRECT (two separate calls):**
```python
# Call 1: td_execute_python — clean only
for c in list(root.children):
if c.valid and c.name.startswith('promo_'):
c.destroy()
# Call 2: td_execute_python — build (separate MCP call)
audio = root.create(audiofileinCHOP, 'promo_audio')
# ... rest of build
```
### 11. Feedback TOP: use `top` parameter, NOT direct input wire
The feedbackTOP's `top` parameter references which TOP to delay. Do NOT also wire that TOP directly into the feedback's input — this creates a real cook dependency loop.
Correct setup:
```python
@ -169,23 +171,173 @@ fb.par.top = comp.path # reference only — no wire to fb input
fb.outputConnectors[0].connect(xf) # fb output -> transform -> fade -> comp
```
The resulting "Cook dependency loop detected" **warning** on the transform/fade chain is expected and correct — that's what feedback loops do. It's informational, not an error.
The "Cook dependency loop detected" warning on the transform/fade chain is expected.
### 16. GLSL TOP auto-creates companion nodes
### 12. GLSL TOP auto-creates companion nodes
Creating a `glslTOP` also creates `name_pixel` (Text DAT), `name_info` (Info DAT), and `name_compute` (Text DAT). These are visible in the network and count toward node totals. Don't be alarmed by "extra" nodes.
Creating a `glslTOP` also creates `name_pixel` (Text DAT), `name_info` (Info DAT), and `name_compute` (Text DAT). These are visible in the network. Don't be alarmed by "extra" nodes.
### 17. The default project root is `/project1`
### 13. The default project root is `/project1`
New TD files start with `/project1` as the main container. System nodes live at `/`, `/ui`, `/sys`, `/local`, `/perform`. Don't create user nodes outside `/project1`.
### 18. `open -R` reveals the file but can't automate the drag
### 14. Non-Commercial license caps resolution at 1280x1280
Use `open -R /path/to/file.tox` to open Finder highlighting the file. The user must then drag it into TD manually. No AppleScript workaround exists on modern macOS due to accessibility restrictions.
Setting `resolutionw=1920` silently clamps to 1280. Always check effective resolution after creation:
```python
n.cook(force=True)
actual = str(n.width) + 'x' + str(n.height)
```
## Recording & Codecs
### 15. MovieFileOut TOP: H.264/H.265/AV1 requires Commercial license
In Non-Commercial TD, these codecs produce an error. Recommended alternatives:
- `prores` — Apple ProRes, **best on macOS**, HW accelerated, NOT license-restricted. ~55MB/s at 1280x720 but lossless quality. **Use this as default on macOS.**
- `cineform` — GoPro Cineform, supports alpha
- `hap` — GPU-accelerated playback, large files
- `notchlc` — GPU-accelerated, good quality
- `mjpa` — Motion JPEG, legacy fallback (lossy, use only if ProRes unavailable)
For image sequences: `rec.par.type = 'imagesequence'`, `rec.par.imagefiletype = 'png'`
### 16. MovieFileOut `.record()` method may not exist
Use the toggle parameter instead:
```python
rec.par.record = True # start recording
rec.par.record = False # stop recording
```
When setting file path and starting recording in the same script, use delayFrames:
```python
rec.par.file = '/tmp/new_output.mov'
run("op('/project1/recorder').par.record = True", delayFrames=2)
```
### 17. TOP.save() captures same frame when called rapidly
Use MovieFileOut for real-time recording. Set `project.realTime = False` for frame-accurate output.
### 18. AudioFileIn CHOP: cue and recording sequence matters
The recording sequence must be done in exact order, or the recording will be empty, audio will start mid-file, or the file won't be written.
**Proven recording sequence:**
```python
# Step 1: Stop any existing recording
rec.par.record = False
# Step 2: Reset audio to beginning
audio.par.play = False
audio.par.cue = True
audio.par.cuepoint = 0 # may need cuepointunit=0 too
# Verify: audio.par.cue.eval() should be True
# Step 3: Set output file path
rec.par.file = '/tmp/output.mov'
# Step 4: Release cue + start playing + start recording (with frame delay)
audio.par.cue = False
audio.par.play = True
audio.par.playmode = 2 # Sequential — plays once through
run("op('/project1/recorder').par.record = True", delayFrames=3)
```
**Why each step matters:**
- `rec.par.record = False` first — if a previous recording is active, setting `par.file` may fail silently
- `audio.par.cue = True` + `cuepoint = 0` — guarantees audio starts from the beginning, otherwise the spectrum may be silent for the first few seconds
- `delayFrames=3` on the record start — setting `par.file` and `par.record = True` in the same script can race; the file path needs a frame to register before recording starts
- `playmode = 2` (Sequential) — plays the file once. Use `playmode = 0` (Locked to Timeline) if you want TD's timeline to control position
## TD Python API Patterns
### 19. COMP extension setup: ext0object format is CRITICAL
`ext0object` expects a CONSTANT string (NOT expression mode):
```python
comp.par.ext0object = "op('./myExtensionDat').module.MyClassName(me)"
```
NEVER set as just the DAT name. NEVER use ParMode.EXPRESSION. ALWAYS ensure the DAT has `par.language='python'`.
### 20. td.Panel is NOT subscriptable — use attribute access
```python
comp.panel.select # correct (attribute access, returns float)
comp.panel['select'] # WRONG — 'td.Panel' object is not subscriptable
```
### 21. ALWAYS use relative paths in script callbacks
In scriptTOP/CHOP/SOP/DAT callbacks, use paths relative to `scriptOp` or `me`:
```python
root = scriptOp.parent().parent()
dat = root.op('pixel_data')
```
NEVER hardcode absolute paths like `op('/project1/myComp/child')` — they break when containers are renamed or copied.
### 22. keyboardinCHOP channel names have 'k' prefix
Channel names are `kup`, `kdown`, `kleft`, `kright`, `ka`, `kb`, etc. — NOT `up`, `down`, `a`, `b`. Always verify with:
```python
channels = [c.name for c in op('/project1/keyboard1').chans()]
```
### 23. expressCHOP cook-only properties — false positive errors
`me.inputVal`, `me.chanIndex`, `me.sampleIndex` work ONLY in cook-context. Calling `par.expr0expr.eval()` from outside always raises an error — this is NOT a real operator error. Ignore these in error scans.
### 24. td.Vertex attributes — use index access not named attributes
In TD 2025.32, `td.Vertex` objects do NOT have `.x`, `.y`, `.z` attributes:
```python
# WRONG — crashes:
vertex.x, vertex.y, vertex.z
# CORRECT — index-based:
vertex.point.P[0], vertex.point.P[1], vertex.point.P[2]
# Or for SOP point positions:
pt = sop.points()[i]
pos = pt.P # use P[0], P[1], P[2]
```
## Audio
### 25. Audio Spectrum CHOP output is weak — boost it
Raw output is very small (0.001-0.05). Use built-in boost: `spectrum.par.highfrequencyboost = 3.0`
If still weak, add Math CHOP in Range mode: `fromrangehi=0.05, torangehi=1.0`
### 26. AudioSpectrum CHOP: timeslice and sample count are the #1 gotcha
AudioSpectrum at 44100Hz with `timeslice=False` outputs the ENTIRE audio file as samples (~24000+). CHOP-to-TOP then exceeds texture resolution max and warns/fails.
**Fix:** Keep `timeslice = True` (default) for real-time per-frame FFT. Set `fftsize` to control bin count (it's a STRING enum: `'256'` not `256`).
If the CHOP-to-TOP still gets too many samples, set `layout = 'rowscropped'` on the choptoTOP.
```python
spectrum.par.fftsize = '256' # STRING, not int — enum values
spectrum.par.timeslice = True # MUST be True for real-time audio reactivity
spectex.par.layout = 'rowscropped' # handles oversized CHOP inputs
```
**resampleCHOP has NO `numsamples` param.** It uses `rate`, `start`, `end`, `method`. Don't guess — always `td_get_par_info('resampleCHOP')` first.
### 27. CHOP To TOP has NO input connectors — use par.chop reference
```python
spec_tex = root.create(choptoTOP, 'spectrum_tex')
spec_tex.par.chop = resample # correct: parameter reference
# NOT: resample.outputConnectors[0].connect(spec_tex.inputConnectors[0]) # WRONG
```
## Workflow
### 19. Always verify after building — errors are silent
### 28. Always verify after building — errors are silent
Node errors and broken connections produce no output. Always check:
```python
@ -196,141 +348,161 @@ for c in list(root.children):
if w: print(c.name, 'WARN:', w)
```
### 20. Build in one big `execute_python_script` call, not many small ones
### 29. Window COMP param for display target is `winop`
Each API round-trip adds latency. Bundle node creation + parameter setting + wiring into a single script that creates everything at once, then verify in one final call.
### 21. Window COMP param for display target is `winop` (not `top` or `window`)
To display output in a separate window:
```python
win = root.create(windowCOMP, 'display')
win.par.winop = '/project1/logo_out' # ← this is the correct param
win.par.winop = '/project1/logo_out'
win.par.winw = 1280; win.par.winh = 720
win.par.winopen.pulse() # open the window
win.par.winopen.pulse()
```
### 22. Save the project to make API persistent across TD restarts
### 30. `sample()` returns frozen pixels in rapid calls
After deploying the custom API handler, save the project:
`out.sample(x, y)` returns pixels from a single cook snapshot. Compare samples with 2+ second delays, or use screencapture on the display window.
### 31. Audio-reactive GLSL: dual-layer sync pipeline
For audio-synced visuals, use BOTH layers for maximum effect:
**Layer 1 (TD-side, real-time):** AudioFileIn → AudioSpectrum(timeslice=True, fftsize='256') → Math(gain=5) → choptoTOP(par.chop=math, layout='rowscropped') → GLSL input. The shader samples `sTD2DInputs[1]` at different x positions for bass/mid/hi. Record the TD output with MovieFileOut.
**Layer 2 (Python-side, post-hoc):** scipy FFT on the SAME audio file → per-frame features (rms, bass, mid, hi, beat detection) → drive ASCII brightness, chromatic aberration, beat flashes during the render pass.
Both layers locked to the same audio file = visuals genuinely sync to the beat at two independent stages.
**Key gotcha:** AudioFileIn must be cued (`par.cue=True` → `par.cuepulse.pulse()`) then uncued (`par.cue=False`, `par.play=True`) before recording starts. Otherwise the spectrum is silent for the first few seconds.
### 32. twozero MCP: benchmark and prefer native tools
Benchmarked April 2026: twozero MCP with 36 native tools. The old curl/REST method (port 9981) had zero native tools.
**Always prefer native MCP tools over td_execute_python:**
- `td_create_operator` over `root.create()` scripts (handles viewport positioning)
- `td_set_operator_pars` over `node.par.X = Y` scripts (validates param names)
- `td_get_par_info` over temp-node discovery dance (instant, no cleanup)
- `td_get_errors` over manual `c.errors()` loops
- `td_get_focus` for context awareness (no equivalent in old method)
Only fall back to `td_execute_python` for multi-step logic (wiring chains, conditional builds, loops).
### 33. twozero td_execute_python response wrapping
twozero wraps `td_execute_python` responses with status info: `(ok)\n\n[fps 60.0/60] [0 err/0 warn]`. Your Python `result` variable value may not appear verbatim in the response text. If you need to check results programmatically, use `print()` statements in the script — they appear in the response. Don't rely on string-matching the `result` dict.
### 34. Audio-reactive chain: DO NOT use Lag CHOP or Filter CHOP for spectrum smoothing
The Derivative docs and tutorials suggest using Lag CHOP (lag1=0.2, lag2=0.5) to smooth raw FFT output before passing to a shader. **This does NOT work with AudioSpectrum → CHOP to TOP → GLSL.**
What happens: Lag CHOP operates in timeslice mode. A 256-sample spectrum input gets expanded to 1600-2400 samples. The Lag averaging drives all values to near-zero (~1e-06). The CHOP to TOP produces a 2400x2 texture instead of 256x2. The shader receives effectively zero audio data.
**The correct chain is: Spectrum(outlength=256) → Math(gain=10) → CHOPtoTOP → GLSL.** No CHOP smoothing at all. If you need smoothing, do it in the GLSL shader via temporal lerp with a feedback texture.
Verified values with audio playing:
- Without Lag CHOP: bass bins = 5.0-5.4, mid bins = 1.0-1.7 (strong, usable)
- With Lag CHOP: ALL bins = 0.000001-0.00004 (dead, zero audio reactivity)
### 35. AudioSpectrum Output Length: set manually to avoid CHOP to TOP overflow
AudioSpectrum in Visualization mode with FFT 8192 outputs 22,050 samples by default (1 per Hz, 0–22050). CHOP to TOP cannot handle this — you get "Number of samples exceeded texture resolution max".
Fix: `spectrum.par.outputmenu = 'setmanually'` and `spectrum.par.outlength = 256`. This gives 256 frequency bins — plenty for visual FFT.
DO NOT set `timeslice = False` as a workaround — that processes the entire audio file at once and produces even more samples.
### 36. GLSL spectrum texture from CHOP to TOP is 256x2 not 256x1
AudioSpectrum outputs 2 channels (stereo: chan1, chan2). CHOP to TOP with `dataformat='r'` creates a 256x2 texture — one row per channel. Sample the first channel at `y=0.25` (center of first row), NOT `y=0.5` (boundary between rows):
```glsl
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.25)).r; // correct
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.5)).r; // WRONG — samples between rows
```
### 37. FPS=0 doesn't mean ops aren't cooking — check play state
TD can show `fps:0` in `td_get_perf` while ops still cook and `TOP.save()` still produces valid screenshots. The two most common causes:
**a) Project is paused (playbar stopped).** TD's playbar can be toggled with spacebar. The `root` at `/` has no `.playbar` attribute (it's on the perform COMP). The easiest fix is sending a spacebar keypress via `td_input_execute`, though this tool can sometimes error. As a workaround, `TOP.save()` always works regardless of play state — use it to verify rendering is actually happening before spending time debugging FPS.
**b) Audio device CHOP blocking the main thread.** An `audiooutCHOP` with an active audio device can consume 300-400ms/s (2000%+ of frame budget), stalling the cook loop at FPS=0. Fix: keep the CHOP active but set `volume=0` to prevent the audio driver from blocking. Disabling it entirely (`active=False`) may also work but can prevent downstream audio processing CHOPs from cooking.
Diagnostic sequence when FPS=0:
1. `td_get_perf` — check if any op has extreme CPU/s
2. `TOP.save()` on the output — if it produces a valid image, the pipeline works, just not at real-time rate
3. Check for blocking CHOPs (audioout, audiodevin, etc.)
4. Toggle play state (spacebar, or check if absTime.seconds is advancing)
### 38. Recording while FPS=0 produces empty or near-empty files
This is the #1 cause of "I recorded for 30 seconds but got a 2-frame video." If TD's cook loop is stalled (FPS=0 or very low), MovieFileOut has nothing to record. Unlike `TOP.save()` which captures the last cooked frame regardless, MovieFileOut only writes frames that actually cook.
**Always verify FPS before starting a recording:**
```python
td_exec("project.save(os.path.expanduser('~/Documents/HermesAgent.toe'))")
# Check via td_get_perf first
# If FPS < 30, do NOT start recording — fix the performance issue first
# If FPS=0, the playbar is likely paused — see pitfall #37
```
TD auto-opens the last saved project on launch. The API handler is now baked into the .toe file — next time TD opens, port 9981 is live with zero manual steps. To explicitly launch with this project: `open /Applications/TouchDesigner.app ~/Documents/HermesAgent.toe`
### 23. `sample()` returns frozen pixels when called from WebServer DAT callback
Common causes of recording empty video:
- Playbar paused (FPS=0) — see pitfall #37
- Audio device CHOP blocking the main thread — see pitfall #37b
- Recording started before audio was cued — audio is silent, GLSL outputs black, MovieFileOut records black frames that look empty
- `par.file` set in the same script as `par.record = True` — see pitfall #18
`out.sample(x, y)` called from inside the API handler's `exec()` returns pixels from a single cook snapshot. It does NOT update between multiple API calls in quick succession. To verify animation is working, either:
- Compare samples with a 2+ second delay between separate `td_exec()` calls
- Use `screencapture` on the display window
- Check `absTime.seconds` is advancing and shader uses time correctly
### 39. GLSL shader produces black output — test before committing to a long render
### 22. `outputresolution` is a string menu, not an integer
New GLSL shaders can fail silently (see pitfall #7). Before recording a long take, always:
### 25. MovieFileOut TOP: H.264/H.265 requires Commercial license
1. **Write a minimal test shader first** that just outputs a solid color or pass-through:
```glsl
void main() {
vec2 uv = vUV.st;
fragColor = TDOutputSwizzle(vec4(uv, 0.0, 1.0));
}
```
In Non-Commercial TD 099, encoding with H.264 or H.265 produces an error: "GPU Accelerated H.264/H.265 Encoding requires a Commercial license". Use Motion JPEG instead:
2. **Verify the test renders correctly** via `td_get_screenshot` on the GLSL TOP's output.
3. **Swap in the real shader** and screenshot again immediately. If black, the shader has a compile error or logic issue.
4. **Only then start recording.** A 90-second ProRes recording is ~5GB. Recording black frames wastes disk and time.
Common causes of black GLSL output:
- Missing `TDOutputSwizzle()` on macOS (pitfall #8)
- Time uniform not connected — shader uses default 0.0, fractal stays at origin
- Spectrum texture not connected — audio values all 0.0, driving everything to black
- Integer division where float division was expected (`1/2 = 0` not `0.5`)
- `absTime.seconds % 1000.0` rolled over past 1000 and the modulo produces unexpected values
### 40. td_write_dat uses `text` parameter, NOT `content`
The MCP tool `td_write_dat` expects a `text` parameter for full replacement. Passing `content` returns an error: `"Provide either 'text' for full replace, or 'old_text'+'new_text' for patching"`.
If `td_write_dat` fails, fall back to `td_execute_python`:
```python
rec = root.create(moviefileoutTOP, 'recorder')
rec.par.type = 'movie'
rec.par.file = '/tmp/output.mov'
rec.par.videocodec = 'mjpa' # Motion JPEG — works on Non-Commercial
op("/project1/shader_code").text = shader_string
```
For image sequences, use `type = 'imagesequence'` and the file param **must** use `me.fileSuffix`:
### 41. td_execute_python does NOT return stdout or print() output
Despite what earlier versions of pitfall #33 stated, `print()` and `debug()` output from `td_execute_python` scripts does NOT appear in the MCP response. The response is always just `(ok)` + FPS/error summary. To read values back, use dedicated inspection tools (`td_get_operator_info`, `td_read_dat`, `td_read_chop`) instead of trying to print from within a script.
### 42. td_get_operator_info JSON is appended with `[fps X.X/X]` — breaks json.loads()
The response text from `td_get_operator_info` has `[fps 60.0/60]` appended after the JSON object. This causes `json.loads()` to fail with "Extra data" errors. Strip it before parsing:
```python
rec.par.type = 'imagesequence'
rec.par.imagefiletype = 'png'
rec.par.file.expr = "'/tmp/frames/out' + me.fileSuffix"
clean = response_text.rsplit('[fps', 1)[0]
data = json.loads(clean)
```
### 26. MovieFileOut `.record()` method may not exist
### 43. td_get_screenshot is asynchronous — returns `{"status": "pending"}`
In TD 099, there is no `.record()` method on moviefileoutTOP. Use the toggle parameter instead:
```python
rec.par.record = True # start recording
rec.par.record = False # stop recording
```
When setting the file path and starting recording in the same script, use `run()` with `delayFrames` to avoid a race condition where the old filename is used:
```python
rec.par.file = '/tmp/new_output.mov'
run("op('/project1/recorder').par.record = True", delayFrames=2)
```
### 27. TOP.save() captures same frame when called rapidly
`op('null1').save(path)` captures the current GPU texture at call time. When called multiple times in a single script (or rapid API calls), TD doesn't cook new frames between saves — all exported PNGs will be identical. To get unique frames, use the MovieFileOut TOP which records in real-time from TD's cook cycle.
### 28. AudioFileIn CHOP: cue before recording for sync
When recording audio-reactive visuals, always cue the audio to the start before beginning the recording. Otherwise the visuals are synced to wherever the audio happens to be in its playback:
```python
op('/project1/audio_in').par.cue.pulse() # reset to start
run("op('/project1/recorder').par.record = True", delayFrames=3)
```
The audio plays via `playmode=0` (Locked to Timeline), so it stays in sync with TD's frame clock. Use `audiodeviceoutCHOP` to hear the audio during recording.
### 29. Audio Spectrum CHOP output is weak — boost with Math CHOP
The raw AudioSpectrum CHOP output has very small values (often 0.001-0.05 range). When fed directly to CHOP To TOP → GLSL, the shader barely reacts. Always insert a Math CHOP with `gain=5` (or higher) between the spectrum and the CHOP To TOP to get usable 0-1 range values in the shader.
### 30. CHOP To TOP texture size — Resample to 256 first
`choptoTOP` creates a texture where width = number of samples. An AudioSpectrum CHOP at 44100Hz has ~24000 samples — creating a 24000×1 texture is wasteful. Use a Resample CHOP set to 256 or 512 samples before the CHOP To TOP for an efficient spectrum texture.
### 31. CHOP To TOP has NO input connectors — use par.chop reference
`choptoTOP` does NOT have input connectors. `resample.outputConnectors[0].connect(chop_to_top.inputConnectors[0])` silently does nothing. Use the `chop` parameter instead:
```python
spec_tex = root.create(choptoTOP, 'spectrum_tex')
spec_tex.par.chop = resample # ← correct: parameter reference
# NOT: resample.outputConnectors[0].connect(spec_tex.inputConnectors[0]) # ← WRONG: no connectors
```
### 22. `outputresolution` is a string menu, not an integer
The `outputresolution` param is a menu with string values:
```
menuNames: ['useinput','eighth','quarter','half','2x','4x','8x','fit','limit','custom','parpanel']
```
Always use the string form. Setting `outputresolution = 9` may silently fail.
```python
node.par.outputresolution = 'custom' # ✓ correct
node.par.resolutionw = 1280; node.par.resolutionh = 720
```
Discover valid values: `list(node.par.outputresolution.menuNames)`
### 23. Large GLSL shaders break curl JSON escaping
GLSL code full of single/double quotes, backslashes, and special chars will corrupt the JSON payload when sent via `curl -d`. **Write the shader to a temp file and load it in TD:**
```python
# Agent side: write shader to /tmp/shader.glsl via write_file
# TD side (via td_exec):
sd = root.create(textDAT, 'shader_code')
with open('/tmp/shader.glsl', 'r') as f:
sd.text = f.read()
```
This avoids all escaping issues. The TD Python environment has full filesystem access.
### 24. TD crashes lose everything — the WebServer DAT must be re-deployed
If TD crashes (common with heavy GLSL or rapid-fire API calls), all nodes including the WebServer DAT are lost. On relaunch, port 9981 is dead. Recovery:
1. Detect: `curl` returns exit code 7 (connection refused) or `lsof -i :9981` shows nothing
2. Check: `pgrep TouchDesigner` to confirm TD is running
3. Re-deploy: user must paste `exec(open('...custom_api_handler.py').read())` into Textport again
4. Verify: poll port 9981 until API responds
The `td_exec()` helper should handle this gracefully:
```python
def td_exec(script):
escaped = json.dumps({"script": script})
cmd = f"curl -s --max-time 15 -X POST -H 'Content-Type: application/json' -d {shlex.quote(escaped)} 'http://127.0.0.1:9981/api/td/server/exec'"
r = terminal(cmd, timeout=20)
if r.get('exit_code') == 7:
return {'error': 'TD not responding — WebServer DAT may need re-deploy'}
try:
return json.loads(r['output'])
except:
return {'error': 'Bad response', 'raw': r['output'][:200]}
Screenshots don't complete instantly. The tool returns `{"status": "pending", "requestId": "..."}` and the actual file appears later. Wait a few seconds before checking for the file. There is no callback or completion notification — poll the filesystem.
### 44. Recording duration is manual — no auto-stop at audio end
MovieFileOut records until `par.record = False` is set. If audio ends before you stop recording, the file keeps growing with repeated frames. Always stop recording promptly after the audio duration. For precision: set a timer on the agent side matching the audio length, then send `par.record = False`. Trim excess with ffmpeg as a safety net:
```bash
ffmpeg -i raw.mov -t 25 -c copy trimmed.mov
```