mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-05-03 02:11:48 +00:00
fix(skills/comfyui): bug fixes, cloud parity, expanded coverage, examples, tests
The audit of v4.1 surfaced ~70 issues across the five scripts and three
reference docs — most user-visible (silent file overwrites, status-error
misclassified as success, X-API-Key leaked to S3 on /api/view redirect,
Cloud endpoints that 404 because they were renamed). v5.0.0 fixes those
and fills the gaps that previously forced users to write their own glue
(WebSocket monitoring, batch/sweep, img2img upload helper, dep auto-fix,
log fetch, health check, example workflows).
Critical fixes
- run_workflow.py: poll_status now checks status_str==error BEFORE
completed:true, so a failed run no longer reports success
- run_workflow.py: download_output streams to disk via safe_path_join,
preserves server subfolder structure (no silent overwrites), and
retries with exponential backoff
- run_workflow.py: refuses to overwrite a link with a literal in
inject_params (would silently break wiring)
- _common.py: _StripSensitiveOnRedirectSession (subclasses
requests.Session.rebuild_auth) drops X-API-Key/Cookie on cross-host
redirects — fixes a real key-leak path through Cloud's signed-URL
download flow. Tested
- Cloud routing (verified live): /history → /history_v2,
/models/<f> → /experiment/models/<f>, plus folder aliases for the
unet ↔ diffusion_models and clip ↔ text_encoders rename
- check_deps.py: distinguishes 200/empty vs 404 folder_not_found vs
403 free-tier; emits concrete fix_command per missing dep
- extract_schema.py: prompt vs negative_prompt determined by tracing
KSampler.{positive,negative} connections (incl. through Reroute /
Primitive nodes) instead of meta-title heuristic; symmetric
duplicate-name resolution; cycle-safe trace_to_node
- hardware_check.py: multi-GPU pick-best, Apple variant detection,
Rosetta detection, WSL2, ROCm --json, disk-space check, optional
PyTorch probe; powershell preferred over deprecated wmic
- comfyui_setup.sh: prefers pipx → uvx → pip --user (with PEP-668
fallback); idempotent — skips relaunch if server already up;
configurable port/workspace; persistent log; SIGINT trap
New scripts
- run_batch.py — count or sweep (cartesian product), parallel up to
cloud tier limit
- ws_monitor.py — real-time WebSocket viewer; saves preview frames
- auto_fix_deps.py — runs comfy node install / model download for
whatever check_deps reports missing (with --dry-run)
- health_check.py — single command that runs the verification checklist
(comfy-cli + server + checkpoints + optional smoke test that cancels
itself to avoid burning compute)
- fetch_logs.py — pull traceback / status messages for a prompt_id
Coverage expansion
- Param patterns now cover Flux (BasicScheduler, BasicGuider,
RandomNoise, ModelSamplingFlux), SD3, Wan/Hunyuan/LTX video,
IPAdapter, rgthree, easy-use, AnimateDiff
- Embedding refs in CLIPTextEncode strings extracted as model deps
- ckpt_name / vae_name / lora_name / unet_name now controllable so
workflows can be retargeted per run
Examples
- workflows/{sd15,sdxl,flux_dev}_txt2img.json
- workflows/sdxl_{img2img,inpaint}.json
- workflows/upscale_4x.json
- workflows/{animatediff_video,wan_video_t2v}.json + README
Tests
- 117 tests (105 unit + 8 cloud integration + 4 cross-host security)
- Cloud tests auto-skip without COMFY_CLOUD_API_KEY; verified end-to-end
against live cloud API
Backwards compatibility
- All existing CLI flags continue to work; new behavior is opt-in
(--ws, --input-image, --randomize-seed, --flat-output, etc.)
This commit is contained in:
SHL0MS
Teknium
parent
7d48a16f14
commit
a7780fe05f
32 changed files with 6117 additions and 1372 deletions
|
|
@ -5,12 +5,16 @@ Docs: https://docs.comfy.org/comfy-cli/getting-started
|
|||
|
||||
## Installation
|
||||
|
||||
Order of preference:
|
||||
|
||||
```bash
|
||||
pip install comfy-cli
|
||||
# or
|
||||
uvx --from comfy-cli comfy --help
|
||||
pipx install comfy-cli # recommended (isolated env)
|
||||
uvx --from comfy-cli comfy --help # zero-install via uv
|
||||
pip install --user comfy-cli # fallback
|
||||
```
|
||||
|
||||
The skill's `comfyui_setup.sh` picks the best available method.
|
||||
|
||||
First run may prompt for analytics. Disable non-interactively:
|
||||
```bash
|
||||
comfy --skip-prompt tracking disable
|
||||
|
|
@ -32,9 +36,9 @@ Workspace resolution priority:
|
|||
3. `--here` (cwd)
|
||||
4. `comfy set-default` path
|
||||
5. Most recently used
|
||||
6. `~/comfy/ComfyUI` (Linux) or `~/Documents/comfy/ComfyUI` (macOS)
|
||||
6. `~/comfy/ComfyUI` (Linux) or `~/Documents/comfy/ComfyUI` (macOS/Win)
|
||||
|
||||
## Commands
|
||||
## Lifecycle Commands
|
||||
|
||||
### `comfy install`
|
||||
|
||||
|
|
@ -42,65 +46,53 @@ Download and install ComfyUI + ComfyUI-Manager.
|
|||
|
||||
```bash
|
||||
comfy install # interactive GPU selection
|
||||
comfy install --nvidia # NVIDIA (CUDA)
|
||||
comfy install --amd # AMD (ROCm)
|
||||
comfy install --nvidia
|
||||
comfy install --amd # ROCm (Linux)
|
||||
comfy install --m-series # Apple Silicon (MPS)
|
||||
comfy install --cpu # CPU only
|
||||
comfy install --fast-deps # use uv for faster deps
|
||||
comfy install --cpu # CPU only (slow)
|
||||
comfy install --fast-deps # use uv for deps
|
||||
comfy install --skip-manager # skip ComfyUI-Manager
|
||||
```
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--nvidia` | NVIDIA GPU |
|
||||
| `--amd` | AMD GPU (ROCm) |
|
||||
| `--m-series` | Apple Silicon |
|
||||
| `--cpu` | CPU only |
|
||||
| `--nvidia` / `--amd` / `--m-series` / `--cpu` | GPU type |
|
||||
| `--cuda-version` | 11.8, 12.1, 12.4, 12.6, 12.8, 12.9, 13.0 |
|
||||
| `--rocm-version` | 6.1, 6.2, 6.3, 7.0, 7.1 |
|
||||
| `--fast-deps` | Use uv for dependency resolution |
|
||||
| `--fast-deps` | uv-based dependency resolution |
|
||||
| `--skip-manager` | Don't install ComfyUI-Manager |
|
||||
| `--skip-torch-or-directml` | Skip PyTorch install |
|
||||
| `--version <ver>` | Specific ComfyUI version (e.g. `0.2.0`, `latest`, `nightly`) |
|
||||
| `--version <ver>` | `0.2.0`, `latest`, `nightly` |
|
||||
| `--commit <hash>` | Install specific commit |
|
||||
| `--pr "#1234"` | Install from a PR |
|
||||
| `--restore` | Restore deps for existing install |
|
||||
|
||||
Default location: `~/comfy/ComfyUI` (Linux), `~/Documents/comfy/ComfyUI` (macOS/Win).
|
||||
|
||||
### `comfy launch`
|
||||
|
||||
Start ComfyUI server.
|
||||
|
||||
```bash
|
||||
comfy launch # foreground on :8188
|
||||
comfy launch --background # background daemon
|
||||
comfy launch -- --listen 0.0.0.0 # listen on all interfaces
|
||||
comfy launch -- --port 8190 # custom port
|
||||
comfy launch -- --cpu # force CPU mode
|
||||
comfy launch # foreground :8188
|
||||
comfy launch --background # background daemon
|
||||
comfy launch -- --listen 0.0.0.0 # LAN-accessible
|
||||
comfy launch -- --port 8190 # custom port
|
||||
comfy launch -- --cpu # force CPU mode
|
||||
comfy launch -- --lowvram # 6 GB cards
|
||||
comfy launch --background -- --listen 0.0.0.0 --port 8190
|
||||
```
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--background` | Run as background daemon |
|
||||
| `--frontend-pr "#456"` | Test a frontend PR |
|
||||
| Extra args after `--` | Passed directly to ComfyUI's `main.py` |
|
||||
|
||||
Common extra args: `--listen`, `--port`, `--cpu`, `--lowvram`, `--novram`,
|
||||
`--fp16-vae`, `--force-fp32`.
|
||||
Common extra args after `--`: `--listen`, `--port`, `--cpu`, `--lowvram`,
|
||||
`--novram`, `--fp16-vae`, `--force-fp32`, `--disable-cuda-malloc`.
|
||||
|
||||
### `comfy stop`
|
||||
|
||||
Stop background ComfyUI instance.
|
||||
|
||||
```bash
|
||||
comfy stop
|
||||
```
|
||||
|
||||
### `comfy run`
|
||||
|
||||
Execute a raw workflow JSON file against a running server.
|
||||
Submit a raw workflow JSON to a running server. **Limited** — no parameter
|
||||
injection, no structured output download. For agents, use
|
||||
`scripts/run_workflow.py` instead.
|
||||
|
||||
```bash
|
||||
comfy run --workflow workflow_api.json
|
||||
|
|
@ -108,31 +100,15 @@ comfy run --workflow workflow_api.json --host 10.0.0.5 --port 8188
|
|||
comfy run --workflow workflow_api.json --timeout 300 --wait
|
||||
```
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--workflow` | Path to API-format workflow JSON (required) |
|
||||
| `--host` | Server hostname (default: 127.0.0.1) |
|
||||
| `--port` | Server port (default: 8188) |
|
||||
| `--timeout` | Seconds to wait (default: 30) |
|
||||
| `--wait/--no-wait` | Wait for completion (default: wait) |
|
||||
| `--verbose` | Show per-node execution details |
|
||||
|
||||
**Limitations:** No parameter injection, no structured output, no image download.
|
||||
For agent use, prefer `scripts/run_workflow.py` which adds those capabilities.
|
||||
|
||||
### `comfy which`
|
||||
|
||||
Show which ComfyUI workspace is currently targeted.
|
||||
|
||||
```bash
|
||||
comfy which
|
||||
comfy which # show targeted workspace
|
||||
comfy --recent which
|
||||
```
|
||||
|
||||
### `comfy set-default`
|
||||
|
||||
Set the default workspace path.
|
||||
|
||||
```bash
|
||||
comfy set-default /path/to/ComfyUI
|
||||
comfy set-default /path/to/ComfyUI --launch-extras="--listen 0.0.0.0"
|
||||
|
|
@ -140,8 +116,6 @@ comfy set-default /path/to/ComfyUI --launch-extras="--listen 0.0.0.0"
|
|||
|
||||
### `comfy update`
|
||||
|
||||
Update ComfyUI or custom nodes.
|
||||
|
||||
```bash
|
||||
comfy update # update ComfyUI core
|
||||
comfy node update all # update all custom nodes
|
||||
|
|
@ -151,33 +125,32 @@ comfy node update all # update all custom nodes
|
|||
|
||||
## `comfy node` — Custom Node Management
|
||||
|
||||
All node operations use ComfyUI-Manager (cm-cli) under the hood.
|
||||
All node operations use ComfyUI-Manager (`cm-cli`) under the hood.
|
||||
|
||||
```bash
|
||||
comfy node show installed # list installed nodes
|
||||
comfy node show enabled # list enabled nodes
|
||||
comfy node show all # all available nodes
|
||||
comfy node show installed # list installed
|
||||
comfy node show enabled # list enabled
|
||||
comfy node show all # all available in registry
|
||||
comfy node simple-show installed # compact list
|
||||
|
||||
comfy node install comfyui-impact-pack # install by name
|
||||
comfy node install <name> --uv-compile # with unified dep resolution (Manager v4.1+)
|
||||
comfy node uninstall <name> # remove
|
||||
comfy node update <name> # update one
|
||||
comfy node update all # update all
|
||||
comfy node enable <name> # enable disabled node
|
||||
comfy node disable <name> # disable without uninstalling
|
||||
comfy node fix <name> # fix broken dependencies
|
||||
comfy node install comfyui-impact-pack
|
||||
comfy node install <name> --uv-compile # ComfyUI-Manager v4.1+ unified resolver
|
||||
comfy node uninstall <name>
|
||||
comfy node update <name> | all
|
||||
comfy node enable <name>
|
||||
comfy node disable <name>
|
||||
comfy node fix <name> # fix broken deps
|
||||
|
||||
comfy node install-deps --workflow=workflow.json # install all deps a workflow needs
|
||||
comfy node deps-in-workflow --workflow=w.json --output=deps.json # extract dep list
|
||||
comfy node install-deps --workflow=workflow.json
|
||||
comfy node deps-in-workflow --workflow=w.json --output=deps.json
|
||||
|
||||
comfy node save-snapshot # save current state
|
||||
comfy node restore-snapshot <file> # restore from snapshot
|
||||
comfy node save-snapshot
|
||||
comfy node restore-snapshot <file>
|
||||
|
||||
comfy node bisect start # find culprit node (binary search)
|
||||
comfy node bisect good # current set is fine
|
||||
comfy node bisect bad # problem is in current set
|
||||
comfy node bisect reset # abort bisect
|
||||
comfy node bisect start # binary-search a culprit node
|
||||
comfy node bisect good
|
||||
comfy node bisect bad
|
||||
comfy node bisect reset
|
||||
```
|
||||
|
||||
### Dependency Resolution Options
|
||||
|
|
@ -188,21 +161,21 @@ comfy node bisect reset # abort bisect
|
|||
| `--uv-compile` | ComfyUI-Manager v4.1+ unified resolver (recommended) |
|
||||
| `--no-deps` | Skip dep installation |
|
||||
|
||||
Set uv-compile as default: `comfy manager uv-compile-default true`
|
||||
Make `uv-compile` default: `comfy manager uv-compile-default true`
|
||||
|
||||
---
|
||||
|
||||
## `comfy model` — Model Management
|
||||
|
||||
```bash
|
||||
comfy model list # list all downloaded models
|
||||
comfy model list --relative-path models/checkpoints # specific folder
|
||||
comfy model list
|
||||
comfy model list --relative-path models/checkpoints
|
||||
|
||||
comfy model download --url <URL> # download model
|
||||
comfy model download --url <URL>
|
||||
comfy model download --url <URL> --relative-path models/loras
|
||||
comfy model download --url <URL> --filename custom_name.safetensors
|
||||
|
||||
comfy model remove # interactive removal
|
||||
comfy model remove # interactive
|
||||
comfy model remove --relative-path models/checkpoints --model-names "model.safetensors"
|
||||
```
|
||||
|
||||
|
|
@ -210,24 +183,27 @@ comfy model remove --relative-path models/checkpoints --model-names "model.safet
|
|||
|--------|-------------|
|
||||
| `--url` | Download URL (CivitAI, HuggingFace, direct) |
|
||||
| `--relative-path` | Subdirectory under workspace (e.g. `models/checkpoints`) |
|
||||
| `--filename` | Custom filename to save as |
|
||||
| `--set-civitai-api-token` | Set CivitAI API token |
|
||||
| `--set-hf-api-token` | Set HuggingFace API token |
|
||||
| `--filename` | Custom save filename |
|
||||
| `--set-civitai-api-token` | Persist CivitAI token |
|
||||
| `--set-hf-api-token` | Persist HuggingFace token |
|
||||
| `--downloader` | `httpx` (default) or `aria2` |
|
||||
|
||||
Model directory structure:
|
||||
Standard model directories:
|
||||
```
|
||||
ComfyUI/models/
|
||||
├── checkpoints/ # Full model files (.safetensors, .ckpt)
|
||||
├── loras/ # LoRA adapters
|
||||
├── vae/ # VAE models
|
||||
├── controlnet/ # ControlNet models
|
||||
├── clip/ # CLIP text encoders
|
||||
├── clip_vision/ # CLIP vision encoders
|
||||
├── upscale_models/ # Upscaler models (ESRGAN, etc.)
|
||||
├── embeddings/ # Textual inversion embeddings
|
||||
├── unet/ # UNet models
|
||||
└── diffusion_models/ # Diffusion model files
|
||||
├── checkpoints/ # Full model files
|
||||
├── loras/ # LoRA adapters
|
||||
├── vae/ # VAE models
|
||||
├── controlnet/ # ControlNet models
|
||||
├── clip/ # CLIP / T5 text encoders
|
||||
├── clip_vision/ # CLIP vision encoders
|
||||
├── upscale_models/ # ESRGAN / SwinIR / etc.
|
||||
├── embeddings/ # Textual inversion embeddings
|
||||
├── unet/ # Standalone UNet weights
|
||||
├── diffusion_models/ # Flux / SD3 / Wan diffusion models
|
||||
├── animatediff_models/ # AnimateDiff motion modules
|
||||
├── ipadapter/ # IPAdapter weights
|
||||
└── style_models/ # Style adapters
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -235,12 +211,12 @@ ComfyUI/models/
|
|||
## `comfy manager` — ComfyUI-Manager Settings
|
||||
|
||||
```bash
|
||||
comfy manager disable # disable Manager completely
|
||||
comfy manager enable-gui # enable new GUI
|
||||
comfy manager disable-gui # disable GUI (API-only)
|
||||
comfy manager enable-legacy-gui # legacy GUI
|
||||
comfy manager disable # disable Manager completely
|
||||
comfy manager enable-gui # enable new GUI
|
||||
comfy manager disable-gui # API-only
|
||||
comfy manager enable-legacy-gui # legacy GUI
|
||||
comfy manager uv-compile-default true # make --uv-compile the default
|
||||
comfy manager clear # clear startup action
|
||||
comfy manager clear # clear startup action
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -248,21 +224,32 @@ comfy manager clear # clear startup action
|
|||
## `comfy pr-cache` — Frontend PR Cache
|
||||
|
||||
```bash
|
||||
comfy pr-cache list # list cached PR builds
|
||||
comfy pr-cache clean # clean all
|
||||
comfy pr-cache clean 456 # clean specific PR
|
||||
comfy pr-cache list
|
||||
comfy pr-cache clean
|
||||
comfy pr-cache clean 456
|
||||
```
|
||||
|
||||
Cache expires after 7 days; max 10 builds kept.
|
||||
Cache expires after 7 days; max 10 builds.
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
Config file location:
|
||||
- Linux: `~/.config/comfy-cli/config.ini`
|
||||
- macOS: `~/Library/Application Support/comfy-cli/config.ini`
|
||||
- Windows: `~/AppData/Local/comfy-cli/config.ini`
|
||||
| OS | Path |
|
||||
|----|------|
|
||||
| Linux | `~/.config/comfy-cli/config.ini` |
|
||||
| macOS | `~/Library/Application Support/comfy-cli/config.ini` |
|
||||
| Windows | `~/AppData/Local/comfy-cli/config.ini` |
|
||||
|
||||
Stores: default workspace, recent workspace, background server info, API tokens,
|
||||
manager GUI mode, launch extras.
|
||||
Stores: default workspace, recent workspace, background server PID, API
|
||||
tokens, manager GUI mode, launch extras.
|
||||
|
||||
## Discovery
|
||||
|
||||
Custom-node registry:
|
||||
- https://registry.comfy.org/
|
||||
|
||||
Model browsers:
|
||||
- https://huggingface.co/models
|
||||
- https://civitai.com (NSFW; requires API token for many)
|
||||
- https://comfyworkflows.com (community workflows)
|
||||
|
|
|
|||
|
|
@ -1,16 +1,50 @@
|
|||
# ComfyUI REST API Reference
|
||||
# ComfyUI REST + WebSocket API Reference
|
||||
|
||||
ComfyUI exposes a REST API + WebSocket for workflow execution and management.
|
||||
Same API surface for local servers and Comfy Cloud (with auth differences).
|
||||
ComfyUI exposes a REST + WebSocket interface for workflow execution and
|
||||
management. **The same surface is used locally and on Comfy Cloud, with
|
||||
auth/path differences.**
|
||||
|
||||
## Connection
|
||||
|
||||
| | Local | Cloud |
|
||||
| | Local ComfyUI | Comfy Cloud |
|
||||
|---|---|---|
|
||||
| Base URL | `http://127.0.0.1:8188` | `https://cloud.comfy.org` |
|
||||
| Auth | None (or bearer token) | `X-API-Key` header |
|
||||
| API path prefix | none (`/prompt`, `/view`, …) | `/api/...` (`/api/prompt`, `/api/view`, …) |
|
||||
| Auth | none (or bearer token if configured) | `X-API-Key` header |
|
||||
| WebSocket | `ws://host:port/ws?clientId={uuid}` | `wss://cloud.comfy.org/ws?clientId={uuid}&token={API_KEY}` |
|
||||
| Output download | Direct bytes from `/view` | 302 redirect → signed URL (use `curl -L`) |
|
||||
| `/api/view` response | direct bytes | 302 redirect → signed URL (use `curl -L`) |
|
||||
|
||||
The skill scripts route URLs automatically via `_common.resolve_url()`.
|
||||
|
||||
## Endpoint differences on Comfy Cloud
|
||||
|
||||
The cloud surface diverges from local ComfyUI in several ways. The skill
|
||||
scripts handle these transparently; document them here so anyone calling
|
||||
`curl` directly knows.
|
||||
|
||||
| Local path | Cloud path | Notes |
|
||||
|------------|-----------|-------|
|
||||
| `/system_stats` | `/api/system_stats` | Cloud version is **public** (no auth required) |
|
||||
| `/object_info` | `/api/object_info` | **Paid tier only** — free returns 403 |
|
||||
| `/queue` | `/api/queue` | Paid tier only |
|
||||
| `/userdata` | `/api/userdata` | Paid tier only |
|
||||
| `/prompt` (POST) | `/api/prompt` (POST) | Paid tier only |
|
||||
| `/upload/image` | `/api/upload/image` | Paid tier only; `subfolder` accepted but ignored |
|
||||
| `/upload/mask` | `/api/upload/mask` | Same as above |
|
||||
| `/view` | `/api/view` | Paid tier only; **returns 302** to signed URL |
|
||||
| `/history` | `/api/history_v2` | **Renamed**; old path returns 404 |
|
||||
| `/history/{id}` | `/api/history_v2/{id}` or `/api/jobs/{id}` | Both work; `/jobs` returns full job |
|
||||
| `/models` | `/api/experiment/models` | **Renamed** |
|
||||
| `/models/{folder}` | `/api/experiment/models/{folder}` | **Renamed**; response shape differs (see below) |
|
||||
|
||||
### Cloud model-list response shape
|
||||
|
||||
- **Local:** `["a.safetensors", "b.safetensors", …]` — flat list of strings.
|
||||
- **Cloud:** `[{"name": "a.safetensors", "pathIndex": 0}, …]` — list of objects.
|
||||
- **Cloud 404 with `code: "folder_not_found"`** — folder is empty or unknown,
|
||||
not an "endpoint missing" error. Distinguish by reading the body.
|
||||
|
||||
The skill helper `_common.parse_model_list()` normalizes both.
|
||||
|
||||
## Workflow Execution
|
||||
|
||||
|
|
@ -34,7 +68,8 @@ curl -X POST "https://cloud.comfy.org/api/prompt" \
|
|||
{"prompt_id": "abc-123-def", "number": 1, "node_errors": {}}
|
||||
```
|
||||
|
||||
If `node_errors` is non-empty, the workflow has validation errors (missing nodes, bad inputs).
|
||||
If `node_errors` is non-empty, the workflow has validation errors (missing
|
||||
nodes, bad inputs).
|
||||
|
||||
### Check Job Status (Cloud)
|
||||
|
||||
|
|
@ -43,133 +78,146 @@ curl -X GET "https://cloud.comfy.org/api/job/{prompt_id}/status" \
|
|||
-H "X-API-Key: $COMFY_CLOUD_API_KEY"
|
||||
```
|
||||
|
||||
| Status | Description |
|
||||
|--------|-------------|
|
||||
| `pending` | Queued, waiting to start |
|
||||
| `in_progress` | Currently executing |
|
||||
| `completed` | Finished successfully |
|
||||
| `failed` | Encountered an error |
|
||||
| `cancelled` | Cancelled by user |
|
||||
| Status | Description |
|
||||
| ------------- | ---------------------------------- |
|
||||
| `pending` | Job is queued and waiting to start |
|
||||
| `in_progress` | Job is currently executing |
|
||||
| `completed` | Job finished successfully |
|
||||
| `failed` | Job encountered an error |
|
||||
| `cancelled` | Job was cancelled by user |
|
||||
|
||||
### Job detail with outputs (Cloud)
|
||||
|
||||
```bash
|
||||
curl -X GET "https://cloud.comfy.org/api/jobs/{prompt_id}" \
|
||||
-H "X-API-Key: $COMFY_CLOUD_API_KEY"
|
||||
```
|
||||
|
||||
Response includes `outputs` keyed by node ID. Cloud uses `video` (singular)
|
||||
in the output structure; local uses `videos` (plural). The skill scripts
|
||||
accept both.
|
||||
|
||||
### Get History (Local)
|
||||
|
||||
```bash
|
||||
# All history
|
||||
curl -s "http://127.0.0.1:8188/history"
|
||||
|
||||
# Specific prompt
|
||||
curl -s "http://127.0.0.1:8188/history/{prompt_id}"
|
||||
curl -s "http://127.0.0.1:8188/history" # all
|
||||
curl -s "http://127.0.0.1:8188/history/{id}" # one prompt_id
|
||||
```
|
||||
|
||||
Response contains `outputs` keyed by node ID with file references.
|
||||
Local entry shape:
|
||||
```json
|
||||
{
|
||||
"<prompt_id>": {
|
||||
"prompt": [...],
|
||||
"outputs": {"<node_id>": {"images": [...]}},
|
||||
"status": {
|
||||
"status_str": "success" | "error",
|
||||
"completed": true | false,
|
||||
"messages": [["execution_start", {...}], ["execution_error", {...}], …]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Important:** when reading status, check `status_str == "error"` BEFORE
|
||||
checking `completed`, because both can be true for failed runs.
|
||||
|
||||
### Download Output
|
||||
|
||||
```bash
|
||||
# Local
|
||||
# Local (direct bytes)
|
||||
curl -s "http://127.0.0.1:8188/view?filename=ComfyUI_00001_.png&subfolder=&type=output" \
|
||||
-o output.png
|
||||
|
||||
# Cloud (follow redirect)
|
||||
curl -L "https://cloud.comfy.org/api/view?filename=ComfyUI_00001_.png&subfolder=&type=output" \
|
||||
# Cloud (302 → signed URL; -L follows; STRIP X-API-Key for the second hop)
|
||||
curl -L "https://cloud.comfy.org/api/view?filename=...&type=output" \
|
||||
-H "X-API-Key: $COMFY_CLOUD_API_KEY" \
|
||||
-o output.png
|
||||
```
|
||||
|
||||
---
|
||||
The skill's `run_workflow.py` strips `X-API-Key` automatically on the
|
||||
cross-host redirect, so the signed URL never sees your auth.
|
||||
|
||||
## WebSocket Monitoring
|
||||
|
||||
Connect to WebSocket for real-time execution progress.
|
||||
|
||||
### Connection
|
||||
Connect for real-time execution events.
|
||||
|
||||
```bash
|
||||
# Local
|
||||
wscat -c "ws://127.0.0.1:8188/ws?clientId=MY-UUID"
|
||||
|
||||
# Cloud
|
||||
wscat -c "wss://cloud.comfy.org/ws?clientId=MY-UUID&token=API_KEY"
|
||||
wscat -c "wss://cloud.comfy.org/ws?clientId=MY-UUID&token=$COMFY_CLOUD_API_KEY"
|
||||
```
|
||||
|
||||
### Message Types (JSON)
|
||||
**Note:** on Cloud the `clientId` is currently ignored — all messages for a
|
||||
user are broadcast to every connection. Filter messages client-side by
|
||||
`data.prompt_id`.
|
||||
|
||||
### JSON Message Types
|
||||
|
||||
| Type | When | Key Fields |
|
||||
|------|------|------------|
|
||||
| `status` | Queue change | `queue_remaining` |
|
||||
| `status` | Queue change | `status.exec_info.queue_remaining` |
|
||||
| `notification` | User-friendly status string | `value` |
|
||||
| `execution_start` | Workflow begins | `prompt_id` |
|
||||
| `executing` | Node running | `node` (ID), `prompt_id` |
|
||||
| `executing` | Node running (or end-of-run if `node` is null on local) | `node`, `prompt_id` |
|
||||
| `progress` | Sampling steps | `node`, `value`, `max` |
|
||||
| `executed` | Node output ready | `node`, `output` |
|
||||
| `execution_cached` | Nodes skipped | `nodes` (list of IDs) |
|
||||
| `progress_state` | Extended progress with per-node metadata | `nodes` (dict) |
|
||||
| `executed` | Node output ready | `node`, `output` (with `images`/`video`/etc.) |
|
||||
| `execution_cached` | Nodes skipped because of cache | `nodes` (list of IDs) |
|
||||
| `execution_success` | All done | `prompt_id` |
|
||||
| `execution_error` | Failure | `exception_type`, `exception_message`, `traceback` |
|
||||
| `execution_error` | Failure | `exception_type`, `exception_message`, `traceback`, `node_id` |
|
||||
| `execution_interrupted` | Cancelled | `prompt_id` |
|
||||
|
||||
When `executing` has `node: null`, the workflow is complete.
|
||||
### Binary Frames (Preview Images)
|
||||
|
||||
### Binary Messages (Preview Images)
|
||||
| Type code | Meaning |
|
||||
|-----------|---------|
|
||||
| `0x00000001` | `PREVIEW_IMAGE` — `[type:4][image_type:4][data]` (image_type 1=JPEG, 2=PNG) |
|
||||
| `0x00000003` | `TEXT` — `[type:4][nid_len:4][nid][text]` (UTF-8) |
|
||||
| `0x00000004` | `PREVIEW_IMAGE_WITH_METADATA` — `[type:4][meta_len:4][json][image_data]` |
|
||||
|
||||
Format: `[4B type][4B image_type: 1=JPEG, 2=PNG][image_data...]`
|
||||
|
||||
---
|
||||
`scripts/ws_monitor.py --previews <dir>` saves preview frames to disk.
|
||||
|
||||
## File Upload
|
||||
|
||||
### Upload Image
|
||||
|
||||
```bash
|
||||
# Image
|
||||
curl -X POST "http://127.0.0.1:8188/upload/image" \
|
||||
-F "image=@photo.png" \
|
||||
-F "type=input" \
|
||||
-F "overwrite=true"
|
||||
```
|
||||
-F "image=@photo.png" -F "type=input" -F "overwrite=true"
|
||||
# Returns: {"name": "photo.png", "subfolder": "", "type": "input"}
|
||||
|
||||
Response: `{"name": "photo.png", "subfolder": "", "type": "input"}`
|
||||
|
||||
### Upload Mask
|
||||
|
||||
```bash
|
||||
# Mask (linked to a previously uploaded image)
|
||||
curl -X POST "http://127.0.0.1:8188/upload/mask" \
|
||||
-F "image=@mask.png" \
|
||||
-F "type=input" \
|
||||
-F "image=@mask.png" -F "type=input" \
|
||||
-F 'original_ref={"filename":"photo.png","subfolder":"","type":"input"}'
|
||||
```
|
||||
|
||||
---
|
||||
Cloud equivalent: prepend `https://cloud.comfy.org/api` and add `-H "X-API-Key: $COMFY_CLOUD_API_KEY"`.
|
||||
|
||||
## Node & Model Discovery
|
||||
|
||||
### Object Info (All Nodes)
|
||||
|
||||
```bash
|
||||
# All node types and their input specs
|
||||
curl -s "http://127.0.0.1:8188/object_info" | python3 -m json.tool
|
||||
# Returns all node types with input/output definitions
|
||||
|
||||
# Specific node
|
||||
curl -s "http://127.0.0.1:8188/object_info/KSampler"
|
||||
# Returns info for one specific node type
|
||||
```
|
||||
|
||||
### Models by Folder
|
||||
|
||||
```bash
|
||||
# Models per folder (local)
|
||||
curl -s "http://127.0.0.1:8188/models/checkpoints"
|
||||
curl -s "http://127.0.0.1:8188/models/loras"
|
||||
curl -s "http://127.0.0.1:8188/models/vae"
|
||||
curl -s "http://127.0.0.1:8188/models/controlnet"
|
||||
curl -s "http://127.0.0.1:8188/models/clip"
|
||||
curl -s "http://127.0.0.1:8188/models/upscale_models"
|
||||
curl -s "http://127.0.0.1:8188/models/embeddings"
|
||||
|
||||
# Models per folder (cloud — note the experimental prefix)
|
||||
curl -s "https://cloud.comfy.org/api/experiment/models/checkpoints" \
|
||||
-H "X-API-Key: $COMFY_CLOUD_API_KEY"
|
||||
```
|
||||
|
||||
Returns arrays of filenames (relative to model folder).
|
||||
|
||||
---
|
||||
|
||||
## Queue Management
|
||||
|
||||
```bash
|
||||
# View queue (running + pending)
|
||||
# View queue
|
||||
curl -s "http://127.0.0.1:8188/queue"
|
||||
|
||||
# Clear all pending
|
||||
|
|
@ -177,21 +225,19 @@ curl -X POST "http://127.0.0.1:8188/queue" \
|
|||
-H "Content-Type: application/json" \
|
||||
-d '{"clear": true}'
|
||||
|
||||
# Delete specific items from queue
|
||||
# Delete specific items
|
||||
curl -X POST "http://127.0.0.1:8188/queue" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"delete": ["prompt_id_1", "prompt_id_2"]}'
|
||||
|
||||
# Cancel currently running job
|
||||
# Cancel currently-running job
|
||||
curl -X POST "http://127.0.0.1:8188/interrupt"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## System Management
|
||||
|
||||
```bash
|
||||
# System stats (VRAM, RAM, GPU, versions)
|
||||
# Stats (VRAM, RAM, GPU, ComfyUI version)
|
||||
curl -s "http://127.0.0.1:8188/system_stats"
|
||||
|
||||
# Free GPU memory
|
||||
|
|
@ -200,14 +246,13 @@ curl -X POST "http://127.0.0.1:8188/free" \
|
|||
-d '{"unload_models": true, "free_memory": true}'
|
||||
```
|
||||
|
||||
---
|
||||
## ComfyUI-Manager Endpoints (Optional)
|
||||
|
||||
## ComfyUI Manager Endpoints (Optional)
|
||||
|
||||
These require ComfyUI-Manager installed.
|
||||
These require ComfyUI-Manager installed. Useful for installing nodes/models
|
||||
via the API instead of `comfy-cli`.
|
||||
|
||||
```bash
|
||||
# Install custom node from git repo
|
||||
# Install a custom node from a git URL
|
||||
curl -X POST "http://127.0.0.1:8188/manager/queue/install" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"git_url": "https://github.com/user/comfyui-node.git"}'
|
||||
|
|
@ -221,8 +266,6 @@ curl -X POST "http://127.0.0.1:8188/manager/queue/install_model" \
|
|||
-d '{"url": "https://...", "path": "models/checkpoints", "filename": "model.safetensors"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## POST /prompt Payload Format
|
||||
|
||||
```json
|
||||
|
|
@ -246,11 +289,24 @@ curl -X POST "http://127.0.0.1:8188/manager/queue/install_model" \
|
|||
},
|
||||
"client_id": "unique-uuid-for-ws-filtering",
|
||||
"extra_data": {
|
||||
"api_key_comfy_org": "optional-partner-node-key"
|
||||
"api_key_comfy_org": "optional-PARTNER-NODE-key (NOT the cloud auth key)"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `prompt`: The workflow graph (API format)
|
||||
- `client_id`: UUID for WebSocket event filtering
|
||||
- `extra_data.api_key_comfy_org`: Required for paid partner nodes (Flux Pro, Ideogram, etc.)
|
||||
- `prompt`: workflow graph in API format
|
||||
- `client_id`: UUID — local server uses it to filter WebSocket events; cloud
|
||||
ignores it.
|
||||
- `extra_data.api_key_comfy_org`: ONLY required when the workflow uses
|
||||
partner nodes (Flux Pro, Ideogram, etc.). Don't conflate with `X-API-Key`.
|
||||
|
||||
## Error Categories (cloud `execution_error` `exception_type`)
|
||||
|
||||
| Type | Meaning |
|
||||
|------|---------|
|
||||
| `ValidationError` | Bad workflow / inputs (often nicer to surface from `node_errors`) |
|
||||
| `ModelDownloadError` | Required model not available |
|
||||
| `ImageDownloadError` | Failed to fetch input image from URL |
|
||||
| `OOMError` | Out of GPU memory |
|
||||
| `InsufficientFundsError` | Account balance too low (partner nodes) |
|
||||
| `InactiveSubscriptionError` | Subscription not active |
|
||||
|
|
|
|||
|
|
@ -1,10 +1,12 @@
|
|||
# ComfyUI Workflow JSON Format
|
||||
|
||||
## Two Formats
|
||||
## Two Formats — Only API Format Is Executable
|
||||
|
||||
ComfyUI uses two workflow formats. **Only API format works for programmatic execution.**
|
||||
**API format** is required for `/api/prompt` and every script in this skill.
|
||||
The web UI also produces an "editor format" used for visual editing, which
|
||||
**cannot** be submitted directly.
|
||||
|
||||
### API Format (what we use)
|
||||
### API Format
|
||||
|
||||
Top-level keys are string node IDs. Each node has `class_type` and `inputs`:
|
||||
|
||||
|
|
@ -28,191 +30,197 @@ Top-level keys are string node IDs. Each node has `class_type` and `inputs`:
|
|||
},
|
||||
"4": {
|
||||
"class_type": "CheckpointLoaderSimple",
|
||||
"inputs": {
|
||||
"ckpt_name": "v1-5-pruned-emaonly.safetensors"
|
||||
}
|
||||
},
|
||||
"5": {
|
||||
"class_type": "EmptyLatentImage",
|
||||
"inputs": {"width": 512, "height": 512, "batch_size": 1}
|
||||
},
|
||||
"6": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {
|
||||
"text": "a beautiful cat",
|
||||
"clip": ["4", 1]
|
||||
}
|
||||
},
|
||||
"7": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {
|
||||
"text": "bad quality, ugly",
|
||||
"clip": ["4", 1]
|
||||
}
|
||||
},
|
||||
"9": {
|
||||
"class_type": "SaveImage",
|
||||
"inputs": {
|
||||
"filename_prefix": "ComfyUI",
|
||||
"images": ["8", 0]
|
||||
}
|
||||
"inputs": {"ckpt_name": "v1-5-pruned-emaonly.safetensors"}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**How to detect:** Top-level keys are numeric strings, each value has `class_type`.
|
||||
**Detection:** every top-level value has `class_type`. The skill's
|
||||
`_common.is_api_format()` does this check.
|
||||
|
||||
### Editor Format (not directly executable)
|
||||
|
||||
Has `nodes[]` and `links[]` arrays — the visual graph data from the ComfyUI web editor.
|
||||
This is what "Save" produces. For API use, export with "Save (API Format)" instead.
|
||||
Has `nodes[]` and `links[]` arrays — the visual graph. To convert: open in
|
||||
ComfyUI's web UI and use **Workflow → Export (API)** (newer UI) or the
|
||||
"Save (API Format)" button (older UI).
|
||||
|
||||
**How to detect:** Top-level has `"nodes"` and `"links"` keys.
|
||||
**Detection:** top-level has `"nodes"` and `"links"` keys.
|
||||
|
||||
---
|
||||
## Inputs: Literals vs Links
|
||||
|
||||
## Input Connections
|
||||
```json
|
||||
"inputs": {
|
||||
"text": "a cat", // literal — modifiable
|
||||
"seed": 42, // literal — modifiable
|
||||
"clip": ["4", 1] // link — wiring; do NOT overwrite
|
||||
}
|
||||
```
|
||||
|
||||
Inputs can be:
|
||||
- **Literal values**: `"text": "a cat"`, `"seed": 42`, `"width": 512`
|
||||
- **Links to other nodes**: `["node_id", output_index]` — e.g., `["4", 0]` means
|
||||
output slot 0 of node "4"
|
||||
|
||||
Only literal values can be modified by parameter injection. Linked inputs are wiring.
|
||||
|
||||
---
|
||||
Links are length-2 arrays of `[upstream_node_id, output_slot]`. The skill's
|
||||
parameter injector refuses to overwrite a link with a literal (logs a
|
||||
warning and skips).
|
||||
|
||||
## Common Node Types and Their Controllable Parameters
|
||||
|
||||
The full catalog lives in `scripts/_common.py` (`PARAM_PATTERNS` and
|
||||
`MODEL_LOADERS`). Highlights:
|
||||
|
||||
### Text Prompts
|
||||
|
||||
| Node Class | Key Fields |
|
||||
|------------|-----------|
|
||||
| `CLIPTextEncode` | `text` (the prompt string) |
|
||||
|------------|------------|
|
||||
| `CLIPTextEncode` | `text` |
|
||||
| `CLIPTextEncodeSDXL` | `text_g`, `text_l`, `width`, `height` |
|
||||
| `CLIPTextEncodeFlux` | `clip_l`, `t5xxl`, `guidance` |
|
||||
|
||||
Usually: positive prompt → one CLIPTextEncode, negative prompt → another.
|
||||
Distinguish by checking the `_meta.title` field or by tracing which feeds into
|
||||
positive vs negative inputs of the sampler.
|
||||
To distinguish positive from negative the skill traces `KSampler.negative`
|
||||
back through Reroute / Primitive nodes to the source CLIPTextEncode. Falls
|
||||
back to `_meta.title` heuristics ("negative", "neg", "anti").
|
||||
|
||||
### Sampling
|
||||
|
||||
| Node Class | Key Fields |
|
||||
|------------|-----------|
|
||||
|------------|------------|
|
||||
| `KSampler` | `seed`, `steps`, `cfg`, `sampler_name`, `scheduler`, `denoise` |
|
||||
| `KSamplerAdvanced` | `noise_seed`, `steps`, `cfg`, `sampler_name`, `scheduler`, `start_at_step`, `end_at_step` |
|
||||
| `SamplerCustom` | `cfg`, `sampler`, `sigmas` |
|
||||
| `KSamplerAdvanced` | `noise_seed`, `steps`, `cfg`, `start_at_step`, `end_at_step` |
|
||||
| `SamplerCustom` | `noise_seed`, `cfg`, `sampler`, `sigmas` |
|
||||
| `SamplerCustomAdvanced` | `noise_seed` (via RandomNoise input) |
|
||||
| `RandomNoise` | `noise_seed` |
|
||||
| `BasicScheduler` | `steps`, `scheduler`, `denoise` |
|
||||
| `KSamplerSelect` | `sampler_name` |
|
||||
| `BasicGuider` / `CFGGuider` | `cfg` |
|
||||
| `ModelSamplingFlux` | `max_shift`, `base_shift`, `width`, `height` |
|
||||
| `SDTurboScheduler` | `steps`, `denoise` |
|
||||
|
||||
### Image Dimensions
|
||||
### Latent / Dimensions
|
||||
|
||||
| Node Class | Key Fields |
|
||||
|------------|-----------|
|
||||
|------------|------------|
|
||||
| `EmptyLatentImage` | `width`, `height`, `batch_size` |
|
||||
| `LatentUpscale` | `width`, `height`, `upscale_method` |
|
||||
| `EmptySD3LatentImage` | `width`, `height`, `batch_size` |
|
||||
| `EmptyHunyuanLatentVideo` | `width`, `height`, `length`, `batch_size` |
|
||||
| `EmptyMochiLatentVideo` | `width`, `height`, `length`, `batch_size` |
|
||||
| `EmptyLTXVLatentVideo` | `width`, `height`, `length`, `batch_size` |
|
||||
|
||||
### Model Loading
|
||||
|
||||
| Node Class | Key Fields | Model Folder |
|
||||
|------------|-----------|-------------|
|
||||
| Node Class | Key Fields | Folder |
|
||||
|------------|------------|--------|
|
||||
| `CheckpointLoaderSimple` | `ckpt_name` | `checkpoints` |
|
||||
| `LoraLoader` | `lora_name`, `strength_model`, `strength_clip` | `loras` |
|
||||
| `LoraLoaderModelOnly` | `lora_name`, `strength_model` | `loras` |
|
||||
| `VAELoader` | `vae_name` | `vae` |
|
||||
| `ControlNetLoader` | `control_net_name` | `controlnet` |
|
||||
| `CLIPLoader` | `clip_name` | `clip` |
|
||||
| `DualCLIPLoader` | `clip_name1`, `clip_name2` | `clip` |
|
||||
| `TripleCLIPLoader` | `clip_name1/2/3` | `clip` |
|
||||
| `UNETLoader` | `unet_name` | `unet` |
|
||||
| `DiffusionModelLoader` | `model_name` | `diffusion_models` |
|
||||
| `UpscaleModelLoader` | `model_name` | `upscale_models` |
|
||||
| `IPAdapterModelLoader` | `ipadapter_file` | `ipadapter` |
|
||||
| `ADE_AnimateDiffLoaderWithContext` | `model_name`, `motion_scale` | `animatediff_models` |
|
||||
|
||||
### Image Input/Output
|
||||
|
||||
| Node Class | Key Fields |
|
||||
|------------|-----------|
|
||||
| `LoadImage` | `image` (filename on server, after upload) |
|
||||
| `LoadImageMask` | `image`, `channel` |
|
||||
|------------|------------|
|
||||
| `LoadImage` | `image` (server-side filename, after upload) |
|
||||
| `LoadImageMask` | `image`, `channel` (`red` / `green` / `blue` / `alpha`) |
|
||||
| `VAEEncode` / `VAEDecode` | (no controllable fields) |
|
||||
| `VAEEncodeForInpaint` | `grow_mask_by` |
|
||||
| `SaveImage` | `filename_prefix` |
|
||||
| `PreviewImage` | (no controllable fields, just previews) |
|
||||
| `VHS_VideoCombine` | `frame_rate`, `format`, `filename_prefix`, `loop_count`, `pingpong` |
|
||||
|
||||
### ControlNet
|
||||
|
||||
| Node Class | Key Fields |
|
||||
|------------|-----------|
|
||||
|------------|------------|
|
||||
| `ControlNetApply` | `strength` |
|
||||
| `ControlNetApplyAdvanced` | `strength`, `start_percent`, `end_percent` |
|
||||
|
||||
### Video (AnimateDiff)
|
||||
### IPAdapter (community pack `comfyui_ipadapter_plus`)
|
||||
|
||||
| Node Class | Key Fields |
|
||||
|------------|-----------|
|
||||
| `ADE_AnimateDiffLoaderWithContext` | `model_name`, `motion_scale` |
|
||||
| `VHS_VideoCombine` | `frame_rate`, `format`, `filename_prefix` |
|
||||
|------------|------------|
|
||||
| `IPAdapterAdvanced` | `weight`, `start_at`, `end_at` |
|
||||
| `IPAdapter` | `weight` |
|
||||
|
||||
---
|
||||
### Embeddings (referenced inside prompt strings)
|
||||
|
||||
ComfyUI scans prompt text for `embedding:NAME` syntax. The skill's
|
||||
`_common.iter_embedding_refs()` extracts these as model dependencies.
|
||||
|
||||
```text
|
||||
"a beautiful cat, embedding:goodvibes:1.2, embedding:art-style"
|
||||
```
|
||||
|
||||
`extract_schema.py` and `check_deps.py` surface these in
|
||||
`embedding_dependencies` / `missing_embeddings`.
|
||||
|
||||
## Parameter Injection Pattern
|
||||
|
||||
To modify a workflow programmatically:
|
||||
|
||||
```python
|
||||
import json, copy
|
||||
|
||||
with open("workflow_api.json") as f:
|
||||
workflow = json.load(f)
|
||||
|
||||
# Deep copy to avoid mutating original
|
||||
wf = copy.deepcopy(workflow)
|
||||
|
||||
# Inject parameters by node ID + field name
|
||||
wf["6"]["inputs"]["text"] = "a beautiful sunset" # positive prompt
|
||||
wf["7"]["inputs"]["text"] = "ugly, blurry" # negative prompt
|
||||
wf["3"]["inputs"]["seed"] = 42 # seed
|
||||
wf["3"]["inputs"]["steps"] = 30 # steps
|
||||
wf["5"]["inputs"]["width"] = 1024 # width
|
||||
wf["5"]["inputs"]["height"] = 1024 # height
|
||||
wf["6"]["inputs"]["text"] = "a beautiful sunset"
|
||||
wf["7"]["inputs"]["text"] = "ugly, blurry"
|
||||
wf["3"]["inputs"]["seed"] = 42
|
||||
wf["3"]["inputs"]["steps"] = 30
|
||||
wf["5"]["inputs"]["width"] = 1024
|
||||
wf["5"]["inputs"]["height"] = 1024
|
||||
```
|
||||
|
||||
The `scripts/extract_schema.py` in this skill automates discovering which
|
||||
node IDs and fields correspond to which user-facing parameters.
|
||||
|
||||
---
|
||||
`scripts/extract_schema.py` automates discovering which node IDs/fields
|
||||
correspond to which user-facing parameters. It returns a `parameters` dict
|
||||
that `run_workflow.py` reads to inject values from `--args`.
|
||||
|
||||
## Identifying Controllable Parameters (Heuristics)
|
||||
|
||||
When analyzing an unknown workflow, these patterns identify user-facing params:
|
||||
For unknown workflows:
|
||||
|
||||
1. **Prompt text**: Any `CLIPTextEncode` → `text` field. Title/meta usually
|
||||
indicates positive vs negative.
|
||||
|
||||
2. **Seed**: Any `KSampler` / `KSamplerAdvanced` → `seed` / `noise_seed`.
|
||||
Randomizable — set to different values for variations.
|
||||
|
||||
3. **Dimensions**: `EmptyLatentImage` → `width`, `height`. Common: 512, 768,
|
||||
1024 (must be multiples of 8).
|
||||
|
||||
4. **Steps**: `KSampler` → `steps`. More = higher quality + slower. 20-50 typical.
|
||||
|
||||
5. **CFG scale**: `KSampler` → `cfg`. How closely to follow prompt. 5-15 typical.
|
||||
|
||||
6. **Model/checkpoint**: `CheckpointLoaderSimple` → `ckpt_name`. Must match an
|
||||
installed model filename exactly.
|
||||
|
||||
7. **LoRA**: `LoraLoader` → `lora_name`, `strength_model`. Adapter name + weight.
|
||||
|
||||
8. **Images for img2img**: `LoadImage` → `image`. Filename on server after upload.
|
||||
|
||||
9. **Denoise strength**: `KSampler` → `denoise`. 0.0-1.0. Lower = closer to input
|
||||
image. Only relevant for img2img.
|
||||
|
||||
---
|
||||
1. **Prompt text** — any `CLIPTextEncode.text`. Use connection tracing back
|
||||
from `KSampler.positive` / `.negative` to disambiguate (don't trust
|
||||
meta-title alone).
|
||||
2. **Seed** — `KSampler.seed` / `KSamplerAdvanced.noise_seed` / `RandomNoise.noise_seed`.
|
||||
3. **Dimensions** — `Empty*LatentImage.width/height` (must be multiples of 8).
|
||||
4. **Steps / CFG** — `KSampler.steps`, `KSampler.cfg`. Steps 20–50 typical.
|
||||
CFG 5–15 typical (Flux uses guidance, not CFG).
|
||||
5. **Model / checkpoint** — `CheckpointLoaderSimple.ckpt_name`. Filename must
|
||||
match an installed file *exactly*.
|
||||
6. **LoRA** — `LoraLoader.lora_name`, `.strength_model`.
|
||||
7. **Images for img2img / inpaint** — `LoadImage.image`. Server-side filename
|
||||
after upload.
|
||||
8. **Denoise** — `KSampler.denoise`. 0.0–1.0; 1.0 = ignore input image,
|
||||
0.0 = pass through. Sweet spot for img2img: 0.4–0.7.
|
||||
|
||||
## Output Nodes
|
||||
|
||||
Output is produced by these node types:
|
||||
Output is produced by these node types. The skill's `OUTPUT_NODES` set
|
||||
extends to common community packs.
|
||||
|
||||
| Node | Output Key | Content |
|
||||
|------|-----------|---------|
|
||||
| `SaveImage` | `images` | List of `{filename, subfolder, type}` |
|
||||
| `VHS_VideoCombine` | `gifs` or `videos` | Video file references |
|
||||
| `SaveAudio` | `audio` | Audio file references |
|
||||
| `PreviewImage` | `images` | Temporary preview (not saved) |
|
||||
| `VHS_VideoCombine` | `gifs` (older) or `videos`/`video` (newer cloud) | Video file refs |
|
||||
| `SaveAudio` | `audio` | Audio file refs |
|
||||
| `SaveAnimatedWEBP` / `SaveAnimatedPNG` | `images` | Animated images |
|
||||
| `Save3D` | `3d` | 3D asset refs |
|
||||
|
||||
After execution, fetch outputs from `/history/{prompt_id}` → `outputs` → `{node_id}`.
|
||||
After execution, fetch outputs from `/history/{prompt_id}` (local) or
|
||||
`/api/jobs/{prompt_id}` (cloud) → `outputs` → `{node_id}` → `{key}`.
|
||||
|
||||
## Wrapper Variants
|
||||
|
||||
Some saved JSON files wrap the workflow under a `"prompt"` key (matching
|
||||
the `/api/prompt` payload shape). The skill's `_common.unwrap_workflow()`
|
||||
handles this — pass any of:
|
||||
|
||||
- raw API format: `{"3": {...}, "4": {...}}`
|
||||
- wrapped: `{"prompt": {"3": {...}}, "client_id": "..."}`
|
||||
|
||||
It rejects editor format with a clear error and a re-export instruction.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue