mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-07-31 19:16:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

fix(voice): reconcile NeuTTS backbone/codec GPU device strings

Browse source 
llama_cpp's GGUF backbone loader only enables n_gpu_layers for the
literal device string "gpu"; torch's codec only accepts "cuda". A
single --device value passed straight through can't satisfy both,
so config.yaml's documented tts.neutts.device: cuda silently ran the
backbone on CPU while the codec ran on GPU.

Map cuda -> gpu for the backbone only; leave the codec on the
torch-native string. Measured 24.9s -> 13.1s per synthesis call on
an RTX 5070 Ti (neutts-air-q4-gguf) with this fix.
This commit is contained in:
William Reed 2026-07-20 10:53:05 -04:00 • committed by Teknium
parent 555d4e10ad
commit 946ed96785
1 changed files with 8 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

10
tools/neutts_synth.py
View file

@ -59,6 +59,12 @@ def main():
parser.add_argument("--device", default="cpu", help="Device (cpu/cuda/mps)")
args = parser.parse_args()
# llama_cpp (backbone) offloads to GPU only for the literal string "gpu";
# torch (codec) only accepts "cuda". A single --device value can't satisfy
# both — "cuda" silently no-ops on the backbone, leaving it on CPU.
backbone_device = "gpu" if args.device == "cuda" else args.device
codec_device = args.device
# Validate inputs
ref_audio = Path(args.ref_audio).expanduser()
ref_text_path = Path(args.ref_text).expanduser()
@ -80,9 +86,9 @@ def main():
tts = NeuTTS(
backbone_repo=args.model,
backbone_device=args.device,
backbone_device=backbone_device,
codec_repo="neuphonic/neucodec",
codec_device=args.device,
codec_device=codec_device,
)
ref_codes = tts.encode_reference(str(ref_audio))
wav = tts.infer(args.text, ref_codes, ref_text)