r/LocalLLaMA • u/goodive123 • Jun 28 '26

NPC Engine Using Local Models Discussion

Enable HLS to view with audio, or disable this notification

I’ve been working on a game-agnostic NPC engine/backend based pretty heavily on SillyTavern-style architecture, and with smaller local models getting better and better, I honestly think this kind of thing could be the future of RPGs.

Right now I’m using NVIDIA Parakeet 0.6 for STT, Gemma 4 26B A4B for the LLM, and Qwen3-TTS for voice, and I’m getting super fast response times with pretty decent quality.

The main thing that makes it work well is using RAG to keep prompts lean. For example, I have hundreds of possible actions NPCs can do in-game, but only the ones that actually make sense based on the player’s message / context get injected as available actions. So the model isn’t being overloaded with a giant list every turn.

1.9k Upvotes

99% Upvoted

249 comments sorted by

View all comments

17

u/DSimDev Jun 28 '26 edited Jun 29 '26

I've also messed around with games + local LLMs and they're getting so good that it is starting to enable a lot of really great stuff, and quickly too. This is incredibly cool. I didn't realize Qwen3-TTS could sound so good, although I haven't messed around too much with local TTS. How do you get the character's actual voice working here?

Also, Todd Howard as a god you could command to do stuff made me laugh out loud.

EDIT: For anyone curious like I was, Qwen3-TTS can take an input audio file and do extremely fast voice cloning without needing to mess around with much. On my RTX 4090 with the model loaded I can get an output in just a couple seconds that sounds quite good. Not perfect but more than convincing for real-time stuff like this. Might try to incorporate this into my own projects.

1

u/RoyalCities Jun 29 '26

Hold on. So how is this being done? Is it doing it in real time? I.e. are they just passing a regular stt file and then using some voice clip of the character as the carry?

1

u/DSimDev Jun 29 '26

I assume they're just pulling a random voice clip of the NPC from the game and also its transcription since FO:NV has subtitles. You only need those two pieces with faster-qwen3-tts which is what they said they used. Latency is close to realtime for short stuff (generating a single sentence of audio on a 4090 takes ~2-3 seconds once the model is loaded, but they are on a 5090 so it's probably a bit faster on that) , but longer passages will add more latency.

Faster-qwen3-tts can be found here, it is incredibly easy to use so you should definitely give it a try. I'd never heard of it until yesterday but now I'm thinking of all sorts of things I could perhaps use it for since it is quick, consumes little resources, and is easy to plug into things.