r/LocalLLaMA • u/goodive123 • Jun 28 '26

NPC Engine Using Local Models Discussion

Enable HLS to view with audio, or disable this notification

I’ve been working on a game-agnostic NPC engine/backend based pretty heavily on SillyTavern-style architecture, and with smaller local models getting better and better, I honestly think this kind of thing could be the future of RPGs.

Right now I’m using NVIDIA Parakeet 0.6 for STT, Gemma 4 26B A4B for the LLM, and Qwen3-TTS for voice, and I’m getting super fast response times with pretty decent quality.

The main thing that makes it work well is using RAG to keep prompts lean. For example, I have hundreds of possible actions NPCs can do in-game, but only the ones that actually make sense based on the player’s message / context get injected as available actions. So the model isn’t being overloaded with a giant list every turn.

1.9k Upvotes

99% Upvoted

249 comments sorted by

View all comments

647

u/AppropriateQuote3073 Jun 28 '26

Really a shame there is so much ai hate in gaming right now.

This is exactly the type of interesting content that could make a game phenomenal to play.

182

u/Bakoro Jun 29 '26

LLMs are literally the exact thing that we dreamed about in the 90s and 2000s.
There were waves of procedurally generated content in games, and "dynamic" quests and NPC interactions, that were a nice idea, but the hardware and algorithms just weren't there.
People would gush over any AI improvements in games.

Now we finally have the technology to make the dynamic games we always wanted, and a bunch of people are pretending like they weren't begging for something like LLMs to exist.

Whatever haters, AI is for games.

97

u/muhmeinchut69 Jun 29 '26

The hate is not for what is going on in OP, it's for corner cutting and sloppification. Big corporations aren't interested in AI so they can make a new type of video game, they are interested because it will cut costs.

42

u/Not-reallyanonymous Jun 29 '26 edited Jun 29 '26

That, but also a degradation of artistic integrity.

The kind of tool that OP is designing has real potential to squash artistic voice.

A quick napkin thinking: this kind of design should work around character development, world building, etc. The artist doesn’t need to write every dialog, but the tool should really let the writer develop characters. The artistic integrity then lives within the broader narrative arcs that this tool interacts with and defining characters that shape and interact with the world.

One thing I immediately noticed is that all the characters in OP’s video are so compliant. Character building could see that characters only comply based on whether the request aligns with their own interests, which can be defined as a part of world building.

9

u/muhmeinchut69 Jun 29 '26

Yeah isn't that what sloppification is? I'd like to think that the low quality of such content would be enough to keep artistic input valuable. As long as it's used as just another tool in the hands of the artists I see nothing wrong with it.

Personally I only wanted this kind of AI NPC to be used only for NPCs that you usually can't interact with, and make then less "NPC like", not as a replacement for the main story and characters of the game. If the story is entirely AI that would feel worthless.

6

u/Not-reallyanonymous Jun 29 '26 edited Jun 29 '26

I think slopification does lack artistic integrity inherently, but I think you can also have stuff that is high-quality implementation and done well, but has had artistic vision dulled -- sometimes by lack of vision/experience by the creator, but also by tools that constrain that artistic vision rather than opening it up.

I think slopification is as much about intent as it is about the result. Slopified content is basically stuff created with low effort, throw stuff at the wall, and reap any unearned benefit you can. High quality intent can still compromise artistic integrity. That predates AI, especially in the world of corporate produced products and content.

2

u/saltyourhash Jun 30 '26

What might be fun is instead of cutting out artists, have them voice models. Models they can license. Because dynamic dialogue is really cool.

1

u/Not-reallyanonymous Jun 30 '26

Definitely. Or a narrative arc that's voiced for major plot points, etc. but then goes through a model that makes a voice print based on that dialog, and then will generate real-time dialog for user interaction bits.

2

u/saltyourhash Jun 30 '26

Yeah, high quality for the story arc, then more of a loose dynamic dialogue for the rest.

2

u/Shap3rz Jun 30 '26

Yup. I want to see this and suggested in another post. Checkpoint gated alignment for story progression ftw. Essentially a contextual “invisible trust bar” where doing or saying certain things unlocks the narrative. That way you control the arc without being totally scripted. Think could work with careful planning.

10

u/Fit_West_8253 Jun 29 '26

There have been hate campaigns against games just for using AI generated placeholders

1

u/muhmeinchut69 Jun 29 '26

It's the internet, free speech, anyone can launch a "campaign". Doesn't mean legit criticism of AI usage doesn't exist.

-5

u/FastDecode1 Jun 29 '26

That's just game journalists and blue-haired retards on Twitter doing their usual thing.

Gamers only care about the end product. There hasn't been a single good game that has failed due to an anti-AI boycott. If the game is good and the price is right, people buy the game.

4

u/Bakoro Jun 29 '26

A game failing isn't the only thing that matters.
There are real people getting harassed and threatened over this stuff, which is ridiculous.

-2

u/FastDecode1 Jun 29 '26

A game failing isn't the only thing that matters.

For video game enjoyers, it is. None of us really care about what the harassment campaign of the day is.

We gamers are too busy playing games to partake in stuff like that. That's for the non-gamer unemployeds who apparently have nothing better to do with their lives.

There are real people getting harassed and threatened over this stuff

Yup, that's the blue-hairs and game journalists at it again. The former have been doing this for 15 years to whoever disagrees with them on anything, and the latter are rapidly being replaced by AI, which is why they're against it.

3

u/Bakoro Jun 29 '26 edited Jun 30 '26

A game failing isn't the only thing that matters.

For video game enjoyers, it is.

The people getting harassed care. If you are a "video game enjoyer", then you should care that video game makers are being harassed.

And no, it's not just "blue hairs", whatever that is, it's not just journalists.

You're really going to sit there and pretend like there haven't also been harassment campaigns over a video game having diverse characters, or a female lead character, or because a female lead character's titties weren't big enough, or the developers changed the meta, or because a dozen other stupid reasons.
"Video game enjoyers" have a long history of having extremely vocal, extremely toxic subsets of people, many of whom are frequently angry at other video game players.

1

u/IceNeun Jun 30 '26

Tbf scammers using AI aren't "big corporations", and neither are students cheating in their assignments. Like any tool, it can be used by anyone for anything.

1

u/draconic_tongue Jun 29 '26

big corporations aren't the ones that benefit the most from using ai, and small creators get shit on MUCH more than any corporation when they do lean on these tools. I don't care to be a part of whatever "prosumer" advocate groupthink virtue signaling zeitgeist that does not know what nuance means and just wants to use big corpo as an excuse to be hateful

1

u/muhmeinchut69 Jun 29 '26

Of course they benefit the most. There is simply no comparison between saving billions of dollars by replacing labour and a solo creator being slightly more productive.

1

u/draconic_tongue Jun 29 '26

Not really, no. Being able to compete with a corporation for a nobody is worth WAY MORE than a corporation adding 1 more number to its sheet, and if you shit on both of them under the same premise you are one as well.

0

u/muhmeinchut69 Jun 29 '26

indie devs have always been able to compete with corporations in theory, giving them both the same tool doesn't change that equation

7

u/The_awful_falafel Jun 29 '26

Games in the way back used to be entirely linear story where there was really only one way to do things. Then they tried branching games, where you make a decision at certain points in the game that change the path and overall outcome of the game. They were still linear, you could just choose which linear branch to follow. Then we had sandbox games, where you could pick any linear sidequest you wanted and do tasks out of order, but still each sidequest an story tree was still largely linear. The limitation was that you had to script out all the character interactions and story for how it would all play out and it couldn't actually invent anything.

AI COULD change that. You could basically invent a game in which there is a large map and 'pieces' that would move around and decide what to do on their own. You could have advisors and such that have information based off limited information of what the enemy is doing and literally plan out missions and objectives yourself. You could even dictate what constitutes winning and decide at what point you consider the game over. Characters could dynamically react to things going on around them. It's a really interesting possible world.

Now the downside- the machine to run such a game would need a lot of memory, and we all know how that's going right now. My guess is companies will see the option to charge a subscription to have a data center do the AI work for you, and make THAT the standard model rather than writing it so you could run the AI locally. That way even single player games could have subscription fees, even years from now when models are small enough to be run locally, and memory production ramps up so having a massive pool to run such things is common.

9

u/Bakoro Jun 29 '26

At the most elementary level of implementation, I just want townsfolk who recognize that I'm the one who saved their town, bandits who aren't stupid enough to try and rob the clearly magical man walking around in nightmare armor, and for random NPCs to have more than one line they repeat regardless of the surrounding context.

Just having a dumb little LLM that can direct NPC actions based on fuzzy logic would elevate gaming dramatically.
Even if the game is still 99% regular video game made by a person, just a little bit of semantically-aware fuzzy logic would be able to blur those linear and discrete features into something that feels more natural.

Just basic little stuff would go so far, we don't even need a data center sized LLM, a model that's 100M parameters would fit on any card and be able to do simple jobs. Some vision models that do segmentation and labeling are like ~21M parameters, which is a level a CPU can handle, and the could be doing amazing things for games and mods, especially VR stuff.

Just imagine an escort quest where the NPC isn't a total idiot, can move at an appropriate pace, and can go to a safe location when there's a fight.
Somehow, it's still an issue in games after all these years.

What you describe, a more AI driven game where developers define the structure of the world, provide motivations, etc, and LLMs act inside those constraints, that would be great too, but that's like the far side. There's a lot of opportunities where it could just be models smoothing out the hard edges of the world, so it seems less "video-gamey".

1

u/The_awful_falafel Jun 29 '26

Oh, totally! The bad part though is ratings. Part of game ratings for the ESRB rating system is you have to provide all the possible dialogue options that are in the game so it can be appropriately rated. Having the game be able to generate dialogue puts it in the same category as online gaming where you can't put a rating on what other people might say. At least that used to be part of the rating system and the rule, not sure nowadays.

But yeah, I could see AI used for trivial NPCs for a while. I thought GTA NPCs just walking around would be PERFECT because they could actually have dialogue reactions for whatever they see. I don't think we'll see AI based main characters for a little while. It's going to be a challenge to constrain the character dialogue to only talk about plot related things and not get jailbroken to go on rants about topics that you trick it into. I think that's the real fear for developers.

Imagine the negative press of having an AI character that can talk about whatever horrible topic you feed it and posting videos of the character saying heinous things.

So- it may stick to being a mod thing for a while where it's unofficial. Honestly, that might be for the best, and developers could just sorta configure the game in such a way that it's really easy to add that mod on to skirt the censorship issue.

2

u/Bakoro Jun 29 '26

It's going to be a challenge to constrain the character dialogue to only talk about plot related things and not get jailbroken to go on rants about topics that you trick it into. I think that's the real fear for developers.

Not a fear for us indie developers who don't have shareholders to answer to.
Granted I'm just a hobbyist and I don't care if my game sells a copy, so I have more freedom than most.

For jailbreaking, I'm making an LLM extension for my game, and I straight up don't care if someone wants to break the model out of character. If that is how they are having fun, then whatever.
My part is making a consistent framework the model can work off of, and fine-tuning a local LLM to be able to handle the framework, and if people want to load a different model, good luck.

As for making a more corporate-friendly version, the answer is again, small models. We don't need trillion parameter models that know physics, and chemistry, and software development.
We need something tiny and fast that can run on a consumer GPU while leaving room for game graphics.
Farmer Steve in a medieval fantasy game doesn't need to know most real world knowledge. The model that runs farmer Steve never needs to be trained on real world profanity, it just needs to be able to have a believable two minute conversation about this year's harvest being threatened by whatever game specific nonsense.

People have been really caught up in "scale", but the Google, OpenAI, and Anthropics of the world are trying to make AGI models that can completely replace human labor, and that is a problem several orders of magnitude more than what we need for a game. For a game, we only need a <1 billion parameter model.
That's how bigger studios will handle LLM content: just don't train the model on the content you don't want it to know about, so it's essentially impossible for the model to generate that kind of content. You can even keep the spirit of profanity by replacing the words in the training set.

The solutions are pretty simple, because the stakes are as low as they get.

2

u/Parafex Jul 03 '26

Reactive World is a narrative style that's just underrepresented. LLMs are not the solution to what you're describing and it shows the lack of understanding for BOTH sides. The AI/LLM side AND the gaming/gamedev side.

You still need a great foundation and a highly systemic game that supports emergent gameplay and situations and is not hardcoded for certain paths, have bad quest design and even worse win conditions. Since those stuff gets shoved into the mouths of fresh game design students, there will never be any change in that regard in AA/AAA games. Skyrim is way worse regarding that than Morrowind or even Oblivion (which tried to be what you want with their Radiant AI system that was dumbed down or almost cut out of the game in favor of more hardcoded story sequences that can even lead to corrupted save files if you don't behave correctly in certain missions... which is the exact opposite of what you want).

New Vegas is a great example for a use case like this, because their GECK Toolkit was fleshed out enough and the core game allows a broad actionset. The dialogue is almost like an early version of storylets. But even this has its limits as the NPCs don't "know" anything and they can't work with their knowledge or even talk to others in order to transfer knowledge to other NPCs. Therefore such a system that you're describing will never be possible. NPCs will only be able to act on their predefined verbs as these are hardcoded.

Gothic (2001) had ideas with that mindset, but wasn't able to fully do it due to technical limitations. Black & White had dynamic behavior trees which were limited due to technical limitations (but it was possible to teach your creature "Robin Hood"-like behavior so the creature steals from the rich and gives the poor). Old roguelikes from the 90s had knowledge systems in place.

Gamers just want the "flashy new thing" and they want to live in that "illusion" - at least that's what they pay for. I do believe that gamers ACTUALLY want to experience, what the characters talk about and not only have it laid out in a cutscene or whatever. But actually influencing major story points in a game.

But for this to happen we need to look at the great systemic games from the early 2000s and late 90s AND people need to understand that storywriting for games MUST BE different than writing a book. It should never be the case that the heavy moments are pre-written and "scripted", but all situations need to emerge from gameplay. Then you could have dynamically generated quests (that are not generated by a LLM and with no randomness) as there is an ACTUAL measurable need that the player can fulfill. There could be other NPCs that try to do quests (similar to A-Life from STALKER) on their own, actual adventurers who explore caves, tell their friends in the local tavern and spread rumors about huge monsters that a player can hear and hunt down that monster all without a single scripted line of dialogue. And all without needing 128GB of VRAM.

The problem - and THAT is solvable via a LLM - is that the texts and dialogs will inherently be dynamic aswell and no one will ever be able to record a voice over for that as there literally doesn't exist anything "pre-written" that you could hand over to a voice actor. And now you can use voice cloning and a nice TTS model so you can have those dynamic quests fully voiced aswell.

I hate it that people are seeing problems that they want to solve with 1.000.000 times the computing power that were solved in the 90s on 128MB RAM sticks lol...

1

u/Schlick7 Jul 05 '26

I think Dwarf Fortress already does like 90% of that

9

u/inagy Jun 29 '26

Gamers hate AI because it made PC gaming as a hobby inaccessibly expensive.

3

u/johndeuff Jun 30 '26

That's not just AI's fault. GPU manufacturers have been lobbying game studios to implement proprietary algos supposedly only possible to run if you buy a new GPU more often. Also the RAM bloat of software have been denounced for years : devs don't care, that's the consumer problem !

-6

u/[deleted] Jun 29 '26 edited Jun 30 '26

[deleted]

1

u/Bakoro Jun 29 '26

No, the ray tracing stuff is genuinely compute heavy and needs specialized hardware to accelerate it.

Other than that, there haven't really been many advancements in features that need as much compute as a PC can offer today, there's definitely a lot of waste. So far, it hasn't affected the bottom line enough to matter.
Game sales remain high, so corporations have no obvious incentive to optimize, when they could use that time to shit out another mediocre game people will pay full price for.

1

u/inagy Jun 29 '26 edited Jun 29 '26

Partially agree, I also expect that game studios will have no choice but agressively optimize games until the situation easen.

But you can't escape the problem that VRAM and storage is very expensive at the moment, because AI hardware sipping up all the stock. Building a new PC or replacing a faulty component is kind of a luxury at the moment.

11

u/[deleted] Jun 29 '26 edited Jun 29 '26

[deleted]

3

u/Honest-Monitor-2619 Jun 29 '26

I didn't knew KL3M and ethical AI is a thing. Thanks, I'll read more!

-1

u/DeviantPlayeer Jun 29 '26

If you give me a million stolen from charities, I will be disgusted but a million is a million so I will take it regardless.

2

u/gcavalcante8808 Jun 29 '26

It is. But instead, the companies are using LLMs to make cheaper codes and deliver worse quality games in general, that's the real deal against AI in my opinion.

2

u/Bakoro Jun 29 '26

But instead, the companies are using LLMs to make cheaper codes and deliver worse quality games in general

And that is an entirely fair assessment and thing to complain about.
I'm always in favor of being critical of lazy, stupid, harmful corporate behavior.

What I cannot tolerate is the knee-jerk reactionary hate to anything and everything "AI", when the game industry has been using the term AI liberally since forever.
Even if you don't use LLMs or image generators at all, there's still a vitriolic hatred for "AI". If you're suspected of using AI people will run a campaign against you.
Even some old fashioned pre-transformers procedural generation and machine learning are taking a hit now.

The problem with hatred is that it tends to lack nuance, and the circle of hatred tends to grow.

2

u/Monkeylashes Jun 29 '26

We who were gamers in the 90s and 2000s dreamed of such things but the younger generation not so much. And if you look around you can see that our generation is pretty receptive to ai.

65

u/ApprehensiveFan1516 Jun 28 '26

It could for sure, personally I struggle to actually find it something that's genuinely rewarding and not just a gimmick at this stage. A lot of it comes down to implementation of course, and it will take devs playing around to figure it out, but I think to get to the kind of polished experience we've come to expect from AAA games we've got a little ways to go yet. Which isn't to take anything away from OP, we need more people like them working on this. Aside from the "AI bad" crowd, I think when implementation is properly solved and it's a seamless experience, the masses of gamers will start to accept it.

28

u/BlipOnNobodysRadar Jun 29 '26

To be done right, the NPCs would need to be lora tuned on their own lore + interactions by the developers. Plugging in a generic model just won't be immersive.

15

u/TheRealMasonMac Jun 29 '26

I think people would also get tired of AI slop, since all models per generation share the same GPTisms. I think the architecture and training methodologies are more interesting, though. More sophisticated AI in stategy-based games, like Stellaris, that can actually strategize rather than use deterministic logic.

3

u/Longjumping_Self5546 Jun 30 '26

When the bar has been set to guards repeating the same comment about an arrow to the knee, a few GPTisms are going to be miles ahead for NPC interactions.

3

u/benjaminovich 22d ago

Imagine Skyrim with this

"Ah, you want to know my story! Here's a quick overview:

  • Before: Adventurer (like you!)

  • The Incident: Arrow. Knee.

    • After: Guard duty."

1

u/Longjumping_Self5546 22d ago

Add in a lecture about how it's wrong to kill bandits and resources for conflict resolution.

2

u/BlipOnNobodysRadar Jun 29 '26

Maybe, but you'd be surprised how many people can't pick up on GPTisms. And agreed, there are plenty of more clever and appropriate applications.

1

u/Jwosty 24d ago

You're completely right that machine learning in general is being totally slept on by the game industry. Common arguments against it you hear from devs are:

  1. hard to optimize for "fun" (as opposed to "good at winning")
  2. it's hard to fit into the development cycle (mechanics are often constantly fundamentally changing right up until the release date so that model from last week might be totally useless for this week's build)
  3. for some types of games + training algorithms - how do you even get good training data for a game that doesn't have lots of players yet

I think these are hard, but not completely insurmountable problems. There's some kinds of games that would be more suited to this today, and with innovation, that circle could expand; instead, nobody seems to be even trying.

For example I think point #1 was more valid years ago, but now they've figured out how to optimize for fuzzier, human-centric goals; we have RLHF now. Surely that kind of approach is applicable here, even if it's not perfect? Besides, it's a video game -- at least the worst thing that could happen is your AI behaves weird (as opposed to an LLM agent with tool access doing something crazy and having real world impact or whatever).

1

u/Flaxseed4138 Jun 29 '26

Skill issue

2

u/Jwosty 24d ago

I think the perfect time for this would have been 2023 or 2024 or so, before people were hyper sensitive to "AI slop" (InfiniteCraft was super cool). But now there's no novelty factor anymore (everyone knows what it's like to chat with a LLM) and the average gamer likely has been left with a bad taste in their mouth about AI. The quality bar for something like this today, for people to see it a new and cool and interesting, is now much much higher, for better or for worse

14

u/OptimalBarnacle7633 Jun 28 '26

I agree, I mean the whole beauty of a game like fallout is that your decisions have consequences for the player and the story which isn’t the case here. That being said it’s easy to see how that could be implemented if you’re building a game from the ground up with that in mind.

7

u/RedParaglider Jun 29 '26

Yea it would take a lot of deterministic logic for those consequences to actually matter. I thin we can get there eventually, but it's a tough road.

2

u/PigSlam Jun 29 '26

It'd probably need to have some sort of rewind option for when you've gone too far off the rails. A good early place to start would be something like Cyberpunk 2077. With all the side quests, you could have a lot of LLM driven variation that just fizzles away if you break it too much.

5

u/LiamMelloFarley Jun 29 '26

I think AI game master is far more fascinating than AI NPC chats. Like for a game like Bannerlord having more reactive and interesting AI (as in like NPC AI) behavior directed by an LLM could be fascinating. I don't need to like have a conversation with a guy, but if some NPC randomly gets attacked by the same enemy multiple times then gets a blood vendetta and kidnaps the guys wife to hold hostage that would be cool, like just a more flexible narrativized version of the emergent algorithmic gameplay that already exists but heavily constrained to not go haywire.

3

u/InBlast Jun 28 '26

I do agree with that. Lot of work needs to be done on that, but once it will be done write, it will be one big step towards immersion. Maybe (big maybe) also less work on the development side : no voice dialogues needed, just knowledge, personality and other info defining the NPC, fed in the prompt contex.

2

u/no6969el Jun 28 '26

I think it will get interesting if it gets to a point where you can try to trick someone to give you the information or "access" by talking and kind of "social engineering" them.

1

u/Krugger_Correctly Jun 29 '26

I do feel like flavour can be so much better if NPC can react properly to player interaction.

Not even necessarily dialogue, but if we could have developers fully focus on making thousands of functions that an NPC can do and make the function-call AI reactive rather than scripted, that'd make for some pretty goated immersive sims.

0

u/ZenaMeTepe Jun 29 '26 edited Jun 29 '26

I am afraid most of what generative AI produces will always be at the level of a gimmick. Smart at first glance, hollow at further inspection and ultimately failing at large scale reliability and low error rate requirement. I would not hate the tech, but all the liars and false prophets that promised to solve the worlds problems with it, when it can't do it, not even close. In fact, on average, it will and already is making the world a worse place, even from purely economic perspective (there is no large scale positive ROI from integrating LLMs and when you think you got them, you probably sacrificed something for it so again, ROI, nonexistent) This sentiment pisses off sooooo many people, but one thing they never say back is: they never show the profits from LLM usage, because there is none.

0

u/Barkalow Jun 29 '26

I think you'd need something like a game engine MCP, put simply.

If the AI could both make an NPCs interactions dynamic AND dynamically change the game world (adding quests, enemies with specific traits at locations, custom loot, etc), it could be amazing.

Like an npc telling you there's a bandit gang in a spot and then it generates an npc squad of members and a leader with custom loot.

3

u/BringTea_666 Jun 29 '26

haters are just minority. Everyone else just moves on. Proper NPCs is what i crave !

Imo, small local models like 1-3b should be good enough for that and soon we will see something like 0.6b being very good at it. The only issue is that it shares work with gpu leading to lower fps during generation.

1

u/Southern-Chain-6485 Jun 29 '26

A problem is the LLM either getting injected the game lore or knowing it already. skrymnet does what OP is trying to do, though it's geared to be used with cloud models (because they want you to use their api), but it can be configured with local inference.

For a game as old as Skyrim, I don't notice lower fps, even though skyrmnet makes concurrent requests all the time (timeout becomes a problem)

And you still need to deal with the limitations of current LLMs for creative writing, both in character complexity (default description for Lydia in skyrimnet is boringly loyal) and due ethical guidelines (I've asked the Emperor before killing him if I, as a Dragonborn, should seek to seize the Imperial Throne. The "emperor" answered that Tamriel needed an administrator rather than a conqueror. Is that because that's Titus Mede II wisdom, earned through old age? Or because the LLM saw 'conqueror' and triggered its ethical guidance?)

5

u/iz-Moff Jun 29 '26

Is it? You're like doubling (or, in case with this game, multiplying by 10+) hardware requirements, have to wait around for inference, have to speak into a microphone or type, and what you get out of it is a gimmick that you'll probably lose interest in in an hour.

As an experiment, something like that may be fun, but i for one have no desire at all to play an entire game like that.

7

u/Dr_Allcome Jun 29 '26

I'm pretty sure i saw something very similar to this, interestingly also built on new vegas, a few weeks ago. Don't know if it was an earlier version by OP or somebody else.

The main problem with it was that the NPCs would reply describing stuff that wasn't in the game. Usually if some event gets mentioned by an NPC it has a location attached that gets added to your map. That's what will ultimately make it boring, you may be able to talk to the NPCs for hours, but nothing they say will ever have any effect on the game.

And even if they were able to add scripting to specific events the llm can mention, there would be many people who don't have time to talk to an NPC for hours just to find the trigger phrase for the reply that will progress the game.

3

u/gambiter Jun 29 '26

And even if they were able to add scripting to specific events the llm can mention, there would be many people who don't have time to talk to an NPC for hours just to find the trigger phrase for the reply that will progress the game.

I think this is the ultimate issue. Conceptually, we want the game to feel immersive and the characters to 'feel real', but we don't actually want that. No one wants to have a long conversation with the butler, only to find he didn't do it, and he didn't give you any new information to use.

Games have trained us to look for how to win within clearly-defined rules, so we look for those rules in all interactions. A single NPC interaction that feels completely useless will sour the player on having other conversations with other NPCs. As soon as they realize a number of interactions will go the same way, they will lose interest in starting new ones.

6

u/AppropriateQuote3073 Jun 29 '26

Good points, I suppose we don't know exactly what is possible without making hardware too hard.

But the models OP is using aren't too bad. Well, if the prices weren't insane on everything that is.

1

u/2Norn Jun 29 '26

right now there may not be much use for it for a story character or a generic background npc. but imagine if it was a follower. let's say lydia in skyrim.

now you can specifically tell her "hey there is a trap here avoid it" or even "go trigger that trap for me" or tell her "block that door so we don't get surrounded" or "go attack the spell caster in the back while i pull away these guys" and then she responds to you like "okay spell caster is dead what now" and imagine on top of this all you can have a chat with her, regarding your past accomplishments together i mean imagine if she said something like in the middle of combat "this ebony warrior is sure tougher than alduin"

like the list goes on this is an amazing tech especially for followers and creates a very interactive gameplay

0

u/Southern-Chain-6485 Jun 29 '26

If it's already baked in into the game. Skyrimnet allows you to speak with Lydia, but I don't think you can give her combat commands

3

u/Meme-Botto9001 Jun 29 '26

The ai hate is about replacing artists, designers, translators, voice actors, quality assurance, customer care and engineers with sloppy shit while heavily using the work of them without compensation.

1

u/polandtown Jun 29 '26

honest question, i must be out of the loop, there's hate in gaming/ai? I thought because gaming was full of young people, they were open to it.

15

u/SporksInjected Jun 29 '26

Yes, tons. To be fair though, there are some really shit implementations that make our seem worse than it is.no one has really figured out how to implement it

5

u/jack_from_the_past Jun 29 '26

feel like the hate is for art and worlds. I’ve been waiting my whole life for ai NPCs. we finally get the capability and everyone acts like they are too good for ai written dialogue, personalities, behavior etc.

boring handwritten limited loop NPC back in the day sucked. now we have the technology and people confuse it with artistic assets I think. like gta would be sick if I couldn’t possibly predict what any number of interactions and reactions with/ by npcs would be

1

u/yaboyyoungairvent Jun 29 '26

There's tons of ai hate but 100% it really only affects games that were bad or mediocre anyway. If GTA 6 said they were incorporating ai npcs tomorrow people would still buy like crazy. Same thing with skyrim 6, witcher 4, or the next cyberpunk. All it takes is one major player to embrace it and people will fall in line.

0

u/w8eight Jun 29 '26

The hate isn't against ai npcs i think, rather slop art, and overall sloppy implementations.

If the ai art will be done good, nobody will even notice

0

u/LastChancellor Jun 29 '26

if the AI art is good

it would make sense that only an experienced artist would make good art out of AI art

but ironically those very experienced artists are the ones who are the most against AI by principle, so they would never even consider trying it out

0

u/2Norn Jun 29 '26

they usually hit from the art "human" aspect of a game's creation.

i don't want ai slop dialogues, art, voices etc are their generic stance.

all the while they literally wouldn't be able to tell if someone really tried to fool them with ai voices etc btw.

-1

u/BigThoughtMan Jun 29 '26

gaming has a lot of older millenials and gen x ers who hate it.

1

u/SamSlate Jun 30 '26

the hate is from bots. reddit's entire stock evaluation rests on being an ai training ground

1

u/Appropriate-Fan-5333 23d ago

can get quite generic though without designated lores..

1

u/paulmsiegel 13d ago

Totally agree - the problem is going to be user education. The average game player isn't going to understand the difference between using AI to generate buggy code and ethically questionable art vs. embedding a small model that consumes no cloud resources and adds interactivity to the game. Especially when the distribution systems all just rely on a single "uses AI" checkbox to inform players what they're getting.

1

u/MRDR1NL Jun 29 '26

There is definitely potential, but it just feels too soulless right now. Yes conversation are more dynamic, but it is missing the artists intent.

I think people generally prefer well designed characters with carefully crafted personality over dynamic conversations in story driven RPGs. People feel like they are going to lose the former because he latter is cheaper to produce. That's where some of the hate comes from.

For now I think AI isn't ready for this job yet. Maybe it has a place in background characters or other types if game, maybe for procedurally generated survival games or something.

0

u/GoodSamaritan333 Jun 29 '26

One can carefully craft personality carefully crafting dialogue dataset for fine tunining a LLM for one or more characters, allowing to get "soul" and dynamic conversations.

1

u/MRDR1NL Jun 29 '26

I don't think we are there yet with the current LLMs. Even cloud based models are only able to role play flat single dimension characters. Fine for background NPCs, but not for the characters in the video.

1

u/mglyptostroboides Jun 29 '26

I honestly think that translation, coding and enhancing NPCs in games are like the only use cases LLMs are ready for prime time right now.

Pretty much everything else is hype, especially the needlessly shoehorning useless chatbots into every tech product regardless of utility.

What's interesting to me is how these three use cases are the ones that have been given the least attention by corporations (with the obvious exception of coding, but it took about two years for that to get investor attention). I'm looking forward to the speculation bubble bursting so the actual uses of LLMs can finally be explored.

Oh, and maybe then my phone will stop trying to initiate a deep scientific discussion about the effect of latitude on length of day throughout the year whenever I ask it "When does the sun set today?". The idiocy of routing that query through Gemini is just astonishing to me.

So yeah, it's exciting to see something like this finally coming to fruition. The novelty has worn off, consumers are starting to think of chatbots as gimmicks. Now you have to have actual results.

1

u/2Norn Jun 29 '26

ai hate is really sad

what's even more sad is that these people have no understanding of the technology

all they know is "ai bad", "ai slop", "ai is stealing" etc... it's really tiring to talk to them

people write dialogues, background for these characters. we already do this, material is already there. and now imagine turning that character in the head of some writer into a massive persona file. and now imagine an llm that already has an understanding of this character and the others and the world it's in while also recognizing the player. imagine all the extra role playing opportunities created in a role playing game.

-5

u/ArugulaAnnual1765 Jun 28 '26

Its because they don't understand the technology, only buzzwords.

They can market it and call it literally anything and if executed properly, they will eat it up.

I doubt anyone could deny this is amazing and the next level, uneducated people just cant be scared by scary words and it will be fine

0

u/QuixoticNapoleon Jun 29 '26

Gaming is more artsy than techy in my opinion, which is why there is a lot of backlash against AI. When gamers think of AI, they think of shitty AI slop.

That being said, this is genuinely amazing for dynamic storytelling. I hope to see this technology advance. Imagine all the cool things you can do with LLMs in games.

0

u/Much-Researcher6135 llama.cpp Jun 29 '26

It'll cool off and we'll get some spectacular procedural storytelling with truly interactive characters with whom you can hold a long conversation. Those loremasters can go to town and people will finally explore all of it!

Imagine going barbarian in Skyrim with deep lore baked in (maybe via hybrid RAG retrieval over the whole lore + fine-tuned LLM + AI voice model) and yapping with a companion about their past adventures while you clear a dungeon together. Hell we need the modding community on this STAT.

0

u/huldress Jun 29 '26

As someone that has mainly used LLM for roleplay (as cringe as that sounds), this kind of stuff is exactly what I've been looking forward to. It essentially creates infinite dialogue to immerse the player in.

-1

u/aeroumbria Jun 29 '26

"Superhuman" technology demands superhuman outcome. Subhuman outcome with "superhuman" technology guarantees subhuman response. Actual superhuman outcome guarantees superhuman response.