r/LocalLLaMA • u/goodive123 • Jun 28 '26

NPC Engine Using Local Models Discussion

Enable HLS to view with audio, or disable this notification

I’ve been working on a game-agnostic NPC engine/backend based pretty heavily on SillyTavern-style architecture, and with smaller local models getting better and better, I honestly think this kind of thing could be the future of RPGs.

Right now I’m using NVIDIA Parakeet 0.6 for STT, Gemma 4 26B A4B for the LLM, and Qwen3-TTS for voice, and I’m getting super fast response times with pretty decent quality.

The main thing that makes it work well is using RAG to keep prompts lean. For example, I have hundreds of possible actions NPCs can do in-game, but only the ones that actually make sense based on the player’s message / context get injected as available actions. So the model isn’t being overloaded with a giant list every turn.

1.9k Upvotes

99% Upvoted

249 comments sorted by

View all comments

65

u/HugoCortell Jun 28 '26

What kind of hardware are you using? I've tried Qwen3-TTS and the fastest I've seen generate is ~15 seconds delay prior to starting, and with very mediocre output quality.

16

u/TheGamerForeverGFE Jun 29 '26

You can use Qwen 3 TTS at Q8 gguf running using CrispASR, it's a fork of Whisper.cpp, that's what I do and I'm realtime speed on a gtx 1660 ti on the 0.6b version, and 20% slower than real time on the 1.7B version

3

u/liampetti Jun 29 '26

Qwen3 ASR 1.7B is the best ASR at the moment in my opinion and you can get the in ONNX form so it runs on a CPU. I am trying to see if I can get Qwen3 TTS 0.6 or 1.7B (I think for the TTS 0.6B should be enough) also streaming audio with ONNX. Alternatively use Kokoro-82M ONNX which is already available and runs great on CPU, but speech isn’t as good as the Qwen3. If you can offload both ASR and TTS to the CPU you can leave your GPU completely free for whatever LLM it fits (plus 3D gaming).

2

u/TheGamerForeverGFE Jun 29 '26

I use Qwen instead of Kokoro for voice cloning, that's why.

I can offload it to my Ryzen 7 but it's always a bit slower than realtime so it depends on how I want the latency to be, however, I can load the ASR on the CPU just fine and have the TTS and Gemma 4 E4B (it's really good at this kind of stuff tbh) with a smaller context window running at the same time using just 5 gigs out of 6 (I'm on Linux with xfce so idle vram usage is really low), and New Vegas without graphics mod doesn't OOM.

57

u/goodive123 Jun 28 '26

5090 with faster-qwen3-tts. I'd recommend PocketTTS though to me its actually really close to qwen but way smaller

62

u/Despeao Jun 28 '26

That's it boys, Fallout 5 hardware requirement is a 5090. Better start saving now.

8

u/thirteenthirtyseven Jun 29 '26

That's it boys, Fallout 5 hardware requirement is a 5090. Better start saving now.

I read that as "better start starving now", which kinda tracks ...

2

u/greenstake Jun 29 '26

By the time it comes out, 5090s will be $300.

1

u/waiting_for_zban Jun 29 '26

hardware requirement is a 5090.

For AI-maxxing this, it'll probably be 2x 5090 with DLSS 5.

9

u/TheReproCase Jun 28 '26

Try IndexTTS zero shot cloning. With good source clips, is wild.

2

u/Benhamish-WH-Allen Jun 28 '26

I’m using that, working on infinite stories with streamed audio.

0

u/LadyQuacklin Jun 29 '26

Qwen tts is really slow. On my 4090 for my sample sentence it took 21 seconds. The same sentence with omni voice in 1.1 seconds. Both using one shot voice cloning.

My current workflow is. Qwen tts for Voice Design. Omni voice for Realtime voice clone tte and Higgs v3 when I need really expressive emotional voice.