Well that is the neat part, at least for in-game dialogue, hallucinations wouldn’t be a problem at all lol.
It will be if half of what is hallucinates is quests that the rest of the game fails to actually implement. NPC: “Yes, you’re absolutely right, I misidrected you to a ‘Satin Forrest’ that you spent two hours wandering around trying to find. I actually meant to say that the Dryad Queen is waiting to be rescued in the ‘Cashmere Forrest’ that I totally, 100% guarantee exists for real this time.”
what an LLM could provide on top of that is “understanding” and reasoning
No. Just no. “Understanding” and “reasoning” are not things that LLMs can do. They can decide what word (or phrase or part-of-a-word or whatever) statistically follows the usual pattern given the training data. It’s not the magic you make it out to be.
See this…
Guarantee that took huge amounts of some combination of babysitting, editing, fraud, and/or other things that make it completely unsuitable for generating reliably-sensical cutscenes that dynamically respond to in-game events that the devs never accounted specifically for.
people do not see potential
That’s exactly the problem. It’s all empty promises of something “just around the corner”. And that’s all that’s driving the bubble. Fantasies about a super-unrealistic futures in which some completely vague and hypothetical advancement makes it actually work for something useful. I don’t believe the technologies we have now will ever fulfill those super-unrealistic promises. And as I said, if something fulfills those promises one day, I doubt the inner workings of whatever does will resemble an LLM or Stable Diffusion or whatever. If we want something that can generate game content on the fly, we’re very much barking up the wrong tree trying to somehow make LLMs do that.
it’s clear what could be possible.
That’s just straight up self-deception.
Instead of just generative AI, you render a skeleton animated character but it uses the last step of the AI output and skins that, and it looks as good as AI generated.
You haven’t seen a lot of those “fail” videos where some Twitch streamer’s beauty filter fails and flickers, have you?
I’m not 100% sure this is possible though, but pretty sure.
I’m not 100% sure this is impossible though, but pretty sure.
On AGI I make no predictions.
Me neither except that a) LLMs aren’t it and b) I think anything that could pull off the kind of dynamically-responsive realtime game content generation that you’re talking about would pretty much have to meet most reasonable definitions of “AGI”. And I doubt anything short of that in a game would be a net benefit to immersiveness. (*Maybe if the LLM had super limited scope somehow… like generating short stories that you’d find in random books around the game like the books you find in the Elder Scrolls games or something, but even that seems super iffy.) Mind you, I probably would still opt out of that if it was a thing. (For the same reason as I’d never touch anything with loot boxes – I don’t trust game studios not to take advantage of me to try to addict me to their products.)
The first video in the search results in the first link here has “it got weird fast” in the title and NPCs are breaking the third wall and responding strangely to things in general. (The lady at the orphanage mentioned the player “looking through a flickering screen.” I didn’t watch much further than that.)
The second is likely heavily curated and variables very strictly controlled. The characters are perfectly centered in frame and standing mostly still aside from idle animations. The landscapes are… well, they’re inherently still life. There’s no actual gameplay showcased, and there’s no indication from the video just how many of the video creator’s attempts had major issues like too many fingers or were scrapped in favor of the ones that turned out better. The art style switches strangely between characters, with, for instance, the first one looking much more photographic and the second mimicking more what you’d get from a better-quality 3D render. And many of the characters feel very uncanny valley. (The third one, for instance. I haven’t watched that video much further than that either.)
So, yeah, if those videos are “testing it to see if it works”, the answer is a resounding “no”. There’s nothing workable there, and no indication LLMs and Stable Diffusion are a step in the right direction, despite what Nvidia might tell you.
I mean, do you watch those videos and think to yourself “this is it – this is the destination – we’ve done it”?
I mean, there’s no there there, and no reason to think there ever will be beyond the fact that humans seem pretty intent on developing AGI. Clearly what AI can do now isn’t any kind of boon. So what’s keeping people high on hype if not empty hopes for a breakthrough in the near future?
That’s the opposite of what I said. The kind of real-time dynamic responding to things you’re talking about requires reasoning and intelligence, but LLMs possess neither.