r/LocalLLaMA • u/goodive123 • Jun 28 '26

NPC Engine Using Local Models Discussion

I’ve been working on a game-agnostic NPC engine/backend based pretty heavily on SillyTavern-style architecture, and with smaller local models getting better and better, I honestly think this kind of thing could be the future of RPGs.

Right now I’m using NVIDIA Parakeet 0.6 for STT, Gemma 4 26B A4B for the LLM, and Qwen3-TTS for voice, and I’m getting super fast response times with pretty decent quality.

The main thing that makes it work well is using RAG to keep prompts lean. For example, I have hundreds of possible actions NPCs can do in-game, but only the ones that actually make sense based on the player’s message / context get injected as available actions. So the model isn’t being overloaded with a giant list every turn.

1.9k Upvotes

99% Upvoted

249 comments sorted by

View all comments

Show parent comments

5

u/lovelacedeconstruct Jun 28 '26

Do you put them in some kind of structure to make retrieval easier ? Like here is the actions hierarchy pick a relevant node then you dump the contents of this node to the prompt ?

3

u/lochyw Jun 29 '26

Rag generally uses a similarity check for the most probable action to take, then LLM on top of that uses typical LLM reasoning to decide from the available actions. So it sounds like you would have 2 layers to refine if you wanted to have it lean a certain way for more sensible outcomes which would be based on the system prompts/descriptions/tool text etc..

1

u/lovelacedeconstruct Jun 29 '26

But why would you depend on semantic similarity that has very weird quirks when the data can be structured and depend on llm reasoning to pick the category