Imagine a game like “the sims” where you can adjust how autonomous the sims you control are. I could see Ai being used to control that.
Or having an elder scroll game were you just respond however you want and the npc adapts to it.
First of all, I’m going to replace AI with LLM, since that’s probably what you meant.
There are 2 distinct questions asked in this post:
- Why not use LLMs to provide different levels of automation? (Like, manual, medium, auto)
Answer: you don’t need LLMs for that. You can just code it in like any other feature. It’s not particularly hard, game developers know how to do it since they are used to programming automation for NPCs.
- Why not use LLMs to procedurally generate NPC dialogue?
Answer: games are primarily a form of art. NPC dialogues are written with a purpose. Different characters have different personalities. Some dialogues are meant to drive the plot. Other dialogues are meant to teach the player how to play. Others are meant to show the player things that they may have missed, or things that are interesting.
Procedural dialogues removes all the control from artists. They would all be generic npc n#473, with the “personality” of the LLM, maybe slightly varied if the developer writes a different prompt for each character.
Procedural dialogues would have the same issues as procedural world generation or photorealistic graphics, it would just not be interesting.
There is a practically infinite amount of Minecraft worlds, yet they all feel the same way. The thing that differentiates a Minecraft world from another is that which the player has built. The only part of the world that wasn’t procedurally generated.
There is a great amount of photorealistic games. And they all look very similar. You may only distinguish one from another by looking at their handcrafted worlds or their handcrafted characters. But not by staring at a wall. You can stare at a wall in non-photoreslistic games and know what game it is.
So if you put procedurally generated dialogues, no one will read them, since you’ll be bored by the time you read the same thing being said by 5 different NPCs from 5 different games.
AI in games (using code for entities to make non-player decisions) is about being good enough, cheap enough. It’s just like how games determine their physics. The existence of large scale “black box” AI like OpenAI does not reflect what’s good or cheap. It can’t play chess. You think it’s going to understand The Sims and make reasonable choices in that system?
They’ve already created well tuned system to give your Sims in obtaining their needs. It leads to you having to manage the chaos, and that’s what the fun is. To better hone that is to have the AI play the game for you. And even that, if efficiency of play is the goal, is better done by TASbot and machine learning.
That generic black box style of AI like popular LLMS is like creating a hammer. Now everyone is treating every problem like nails. AI decisions making in games is like washing windows; don’t use a hammer.
The problem is that “AI” is a poorly defined, very vague, and widely used term. Most people here have assumed you meant LLMs because everyone pitches those as ways to solve everything. “Oh, irer up an agent, give it instructions, and let it make requests that are context dependent”. Then, like everyone says here, that usually turns into people testing boundaries and breaking your game. So that makes it both “not good enough” and “not cheap enough”.
Now, look at AI with the term “machine learning” in mind and it’s different. Games like ARC Raiders use machine learning to teach NPCs movement behavior, and to train AI voices like Siri so they can’t add things without further paying people. They think that up-front investment is worthwhile. But those are both far cries from “uploading it to Claude or ChatGPT and see what happens”. Especially when you would have to teach that black box AI your system anyway, for it to use it. And you’re already doing that with current “good enough, cheap enough” bespoke methods, for much cheaper, and they’re good enough.
Have you ever tried to run an LLM locally? It makes CPU usage go way up, uses a lot of RAM, etc. It would tank game performance and/or require beefier PCs.
Games already have had AI for a long time, but the kind of AI you’re talking about would be far more computationally expensive than what they currently use.
There are asbolutely “lite” local systems that could do things like make particular NPC behavior in certain environments far more interesting and unpredictable, without harming performance, which is what everyone I’ve ever played games with would want over the same scripted paths and triggered behaviors.
I don’t think OP is asking for AI implementation that gives you an alternate path into the enemy base by romancing one of the guards and having a life together before you steal their keys off the dresser one morning and then then break into the facility as they send you child-support notices. We’re talking about maybe enemies have more options than crouching behind a red barrel or resuming their patrol when the shooting stops.
Ha. You are thinking of running games locally? Cute.
I have heard of some people experimenting with it, e.g in a stardew valley mod to allow you to have actual conversations with the characters.
While that sounds like fun. It also sounds like something that is fun for 10 minutes.
Agreed, I don’t think the NPCs are the best thing about the game. What I like best about SDV is that it’s essentially an industrial collectathon game.
If we could get complex conversations in games (e.g an adventure game where you’re not limited to 3 conversation options) I think it would be awesome. Might mean we waste more time playing the game, though.
i have watched some videos where its used on skyrim npcs, it seems to work surprisingly well.
Because fuck AI, that’s why.
Yah but OP is talking about AI in gaming, not generative AI that makes propaganda out of stolen art or turns your family photos into porn, there’s a big difference. You’ve been playing with AI in gaming for years, and probably have complained about it because it always sucks. Enemies walk predictable patterns, they see you kill twenty of their friends and then resume their patrol and say “it must have been the wind.”
Lets not mindlessly rage at terms without understanding the context, then you’re just becoming MAGA.
You would have to design the game around an LLM, not just drop one into existing games.
It might be cute for the guards in Skyrim to have unique dialogue, until one of them denies the Holocaust or says feminism is cancer.
I tried this as a side project and it’s such a pain in the ass to get the bots to actually behave like they’re in a world and not be overly eager losers.
You have to do so much prompting to get them to behave and, as others who have to work with these full time know, prompting can only go so far.
They’re not as autonomous and general usage as companies want to make you think they are.
Ah, but then you get to stab the Nazi in the head.
There actually is a semi-working system for Skyrim/Fallout https://art-from-the-machine.github.io/Mantella/
Also not all LLMs are Nazi machines, I almost exclusively use abilerated models and I’ve never once had it go on a nazi tirade.
Though I mostly use it for Linux/code or random home assistant projects, not as a conversation.
What value would it add to the game?
- LLMs are computationally expensive
- Replacing voice actors with AI means making dialogue worse
- Replacing writers with AI means making the story worse
At the end of the day AI is mostly a marketing term for LLMs and LLMs just aren’t that useful in most games, they just average out a dataset to autocomplete a response, that autocompletion is worse than what a human would have written.
We saw with procedurally generated worlds that it takes a lot of effort to prune what is generated to make the game interesting.
There are particular subgenres of games and applications where LLMs might be useful though.
I am a fan of using LLMs specifically to imitate the VAs on demand to pronounce character names. They’re generally good enough that a single word can blend in, and you have a couple minutes during the opening cutscene to run the computation. Just having all of the characters never say the custom player name and instead address them in the second person or with a title is a bit jarring
We do use AI in videogames, and have for multiple decades (with varying levels of sophistication).
Indeed! Seems like people have forgotten that AI is not just LLMs.
I think the Where Winds Meet tried this, right? The NPCs ended up saying anachronistic things and making travel itineraries for Beijing or something.
Do they? I’ve talked to several NPCs, never happened to me. At most, they get completely confused on what you are saying. Eg, one kid thought he was rich enough to buy a house. Trying to tell him he’s not and he thinks I took his money (and started crying, but also became friends?). In another a guqin player wondered if anyone could tell how sad she was from her playing. Instead, we’re keeping secrets? (No idea how that came about).
And before anyone points out, I dropped the game due to quests requiring MC drinking alcohol (can’t stand games like that. Just a me issue). Sad because I loved the everything else too :(
one kid thought he was rich enough to buy a house. Trying to tell him he’s not and he thinks I took his money (and started crying, but also became friends?).
I don’t know about confused, have you ever talked to a toddler?
I haven’t actually played it (wont play any game that used or uses LLM software), so I can only tell you what I’ve read.
Shame, it looked interesting
Have you ever talked with an AI? It sucks.
I’ve talked to them often. So I don’t bore my family with my wild ideas lol
Those wild ideas would be good for someone
It would need to be cloud-based, or otherwise require a lot of RAM
People have tried this a bit and it doesn’t work well. Remember that most games have some sort of plot which needs to move forward without deviating too far and this is not easy to manage with AI. AI systems are predictive text tuned up, so they tend to wander in the conversation and this can be disastrous for something like a video game.
The world is there to support the illusion but also to direct the player to game material. An AI agent going off on a tengent about some random thing that kind of fits the world could lead to users running around wasting their time and being frustrated.
Add to that the risk of the AI system stepping into awful places like reproducing Nazi ideology and it is a nightmare for developeds. Imagine getting your game rated when it can randomly start telling your character not to worry about saving those people over there because their skin tone is darker and that makes them less than human.
Now as a tool for building scripts quickly? Maybe, but it does produce slop now and if that will change I cannot predict when. Maybe it could be used as part of the process but I think it is so toxic now I would not bet on it. I also think it should be labeled as the use of AI comes with moral issues around the environmental impact and theft of content from other people. If a game has AI generated content I won’t be playing it, and I am not alone. Just the push back from audiences could be enough to discourage the use of AI systems.
Now on the other hand using a neural network design for making character behaviours more believable, for example using a series of needs and having the algorithm decide what to do next and so on, that could be cool, but we have that already and it isn’t considered AI.
Thank you that explains alot.
Unfortunately now all I can’t think of is “great sir knight the queen has been captured, and much like hitler she has done nothing wrong!”
Player “I will save the queen and… wait hang on what was that about hitler?”
Npc: I’m just saying if you look at the geo-political climate of the 1930s-
Player: I’m just going to find the dragon thank you.
I’d love to see it being used by enemies so they’re challenging without cheating, though.
I’d love to see it being used by enemies so they’re challenging without cheating, though.
This is a different sort of problem that’s outside the scope of generative AI. Making a computer opponent that can kick a human player’s ass is technology we’ve had since Deep Blue beat Garry Kasparov in 1997.
The problem isn’t actually making a computer that’s challenging, that’s been solved. The problem is that it won’t be any fun for the human if the computer is actually allowed to go all out, if Kasparov couldn’t win in 97 then you sure as hell aren’t winning today. But it also won’t be any fun if you nerf it too badly, low level chess bots are weird. The sweet spot isn’t just a matter of difficulty either, the nearly unsolveable part is getting it to play in a way that feels like a realistic human opponent.
And that’s just from a turn-based game, kinda the closest thing to a level playing field humans were ever gonna get. For any game played in real time, the computer is able to treat it like it’s being played at 60 turns per second. Is it “cheating” for the computer to have perfect reflexes, but otherwise still be following the rules of the game perfectly? How would you even try to take this away from the computer to make it see games the way humans do?
Generative AI doesn’t have any kind of solution for any of this. ChatGPT famously can’t play chess, at all. It’s a different type of AI that really can’t have any useful application here.
I’d love to see it being used by enemies so they’re challenging without cheating, though.
Check out Sony’s work with GT Sophy
Like those teddy bears taken off the shelves. Ai is not nearly as good as the marketing hype says. Eventually, but that isn’t happening soon.
The curiosity is killing me, do you remember what the highly upvoted comment you replied to said?
Someone put a connection to an LLM on a Teddy Bear so kids could have natural conversations with the toy. It started making sexual innuendos and creepy political commentaries and suggestions to children almost right away.
There is a game called Whispers from the Star which uses an LLM to run the script. It’s pretty much a fancy choose your own adventure book. It’s pretty shit.
It would suck.
