GameNGen: Neural model game engine that "runs" Doom

4 hours ago, RataUnderground said:

Like everything else AI has produced, a bunch of technically impressive junk designed for marketing with absolutely no real utility.
Oh, gamechanger, so you need a finished game with thousands of hours of gameplay at your disposal to produce a Dadaist recreation of something that resembles the original game?

That's where I disagree genuinely. AI helped a lot to recreate the sounding of analog stuff like tube guitar amplifiers and even complex chains sounding really close, as close as that the results almost subtract one from another, it's a great gift for all guitarists. As soon as performance of computers increase we can increase the model complexity and get even more close results which is already almost unnecessary. The biggest milestone here is that plugin called "Neural Amp Modeler", the quality of which is sufficient to make you forget about purchasing those 5k$ amps and transfer them like few KBs files. It can't model super boosted amps due to an insufficient model complexity as it will put most computers on the knees but what it can do perfectly is to model unboosted amps (without additional boosting stomps) and later boosting them digitally which it also does perfectly well. So it still replaces those 5k$ amps anyway. So AI is not useless at all, I think it's only the technology infancy.

how does it work? did they train it on keyboard inputs and footage?

21 hours ago, janiform said:

Human raters are only slightly better than random chance at distinguishing short clips of the game from clips of the simulation.

This is hilarious. "People can't tell the difference if we flash it quickly enough!"

1 hour ago, Darkcrafter07 said:

That's where I disagree genuinely. AI helped a lot to recreate the sounding of analog stuff like tube guitar amplifiers and even complex chains sounding really close, as close as that the results almost subtract one from another, it's a great gift for all guitarists. As soon as performance of computers increase we can increase the model complexity and get even more close results which is already almost unnecessary. The biggest milestone here is that plugin called "Neural Amp Modeler", the quality of which is sufficient to make you forget about purchasing those 5k$ amps and transfer them like few KBs files. It can't model super boosted amps due to an insufficient model complexity as it will put most computers on the knees but what it can do perfectly is to model unboosted amps (without additional boosting stomps) and later boosting them digitally which it also does perfectly well. So it still replaces those 5k$ amps anyway. So AI is not useless at all, I think it's only the technology infancy.

I happen to work for one of the top amp modeling companies in the world... zero MLM used here and we have continued to push the accuracy boundary. AI can do a lot and is definitely a "competitor." But how exactly would you say it has improved modeling? 

6 hours ago, Gez said:

without any object persistence because AI still haven't learned that trick yet.

The object-persistence is actually what the white paper focuses on more than anything else. Generating the visuals for the game isn't anything particularly new: anyone with a 4000 series nVidia card can turn on frame-generation in many modern games and it does basically that already (albeit trained on just a few frames rather than hours of footage).

This white paper is about trying to extend the learning model for the generative AI to work over much longer periods. It's not perfect at all (which they freely admit to in the paper), but it's an impressive step up from anything I've seen before. 

As this is Google Research, this is the kind of tech that I could see making it's way eventually into Waymo or drone tech to help with navigation, even aside from the theoretical use-case of one day making a game engine out of it.

56 minutes ago, magicsofa said:

 

This is hilarious. "People can't tell the difference if we flash it quickly enough!"

 

I happen to work for one of the top amp modeling companies in the world... zero MLM used here and we have continued to push the accuracy boundary. AI can do a lot and is definitely a "competitor." But how exactly would you say it has improved modeling? 

I didn't say "improved modeling" but the fact that you can never make your conventional modeling to subtract from the reference thing implies it? Certainly a black box aprroach but would you care if it gave you excatly what you wanted? Some other ppl used neural networks to speed up their solvers for white box aprroach.

God in heaven, the shifting/morphing/oily motion of ai graphics makes me nauseous. Throw all this stuff in the sea, it's a madman's nightmare.

2 hours ago, Gifty said:

God in heaven, the shifting/morphing/oily motion of ai graphics makes me nauseous. Throw all this stuff in the sea, it's a madman's nightmare.

You should see the output of similar models from less than a year ago. This paper used doom purely to showcase the progress being made, creating even a somewhat cohesive 3d environment with next frame prediction is an insane accomplishment and is a good first step for allowing ai to predict things in the real world which is likely the real purpose of this paper. A self driving car equipped with a similar model would be able to "look forward" a few seconds to see what may become a potential hazard, a person about to step into the street for example.

And who knows? In a few years this might have practical uses for gaming, instead of invisible walls the game could just start filling in the blanks with ai once you leave the map.

Genuinely worthless. No one needs this. 

Its attempt to mimic Doom is laughable, and a horrendous waste of electricity that only serves to stroke the egos of AI bros. It's a worse emulation of Doom than Demos (which we have.. and are stupidly easy to use, and recreate inputs perfectly). And to top it off.. why do we need an AI recreation of a video of someone playing Doom...?

We have that already. actually we have that in abundance. Regulars in this forum like LadyMistDragon and The Lamp post videos of their playthroughs of new levels... *daily*.... DAILY!!!

Wishing everyone who worked on this to touch grass.

Environments that are familiar yet offputting for so many different reasons, creating a sense of uncanny versions of your favorite games and maps...

Honestly, not a bad premise for a horror game. Just as Tom Hall envisioned, surely /s

"The power consumption argument doesn't work because you use power all the time" yeah, for actual purposes, and this adds more power on top of whatever comparitively small amount of power I'm using to play nuts.wad

There is a difference between using 2 weeks of power of an average household to generate weird unappealing doom gameplay and an average household using two weeks of power.

18 minutes ago, HiMyNameIsChair said:

Genuinely worthless. No one needs this. 

 

Its attempt to mimic Doom is laughable, and a horrendous waste of electricity that only serves to stroke the egos of AI bros. It's a worse emulation of Doom than Demos (which we have.. and are stupidly easy to use, and recreate inputs perfectly). And to top it off.. why do we need an AI recreation of a video of someone playing Doom...?

 

We have that already. actually we have that in abundance. Regulars in this forum like LadyMistDragon and The Lamp post videos of their playthroughs of new levels... *daily*.... DAILY!!!

 

Wishing everyone who worked on this to touch grass.

 

 

I mean again... The point is not to play doom it's to showcase their research, there may not be any real uses for it for many years to come but that's the same for 99% of all research papers.

8 hours ago, Major Arlene said:

it shouldn't take the same or more amount of wattage to run Doom at 20 FPS on a single TPU (best guess here as I can't find the specs for the v5s as of now) as it does for a laptop running Cyberpunk 2077.

...according to whose standards? Yours?

4 hours ago, hexagonopus said:

I'll be honest, the negativity in this thread somewhat baffles me.

 

The poor quality of the footage: I think this speaks for itself. Decals appearing and disappearing, ephemeral enemies, nonsensical gameplay, etc. In my opinion, this stuff - the hallucinatory, baffling stuff - is the only real artistic appeal of AI projects, so I don't mind it and personally tend to embrace it for what it is. But on an "objective" level, it is a poor emulation of real Doom gameplay, and I can't argue against that.

Sure, but people act shocked that an early emerging technology isn't already perfect and flawless. Like... no, duh...?

The thing doesn't even have object permanence. L.

God, I discussed this in a Discord server earlier:

It's bad. It's barely a proof of concept and yet AI techbro shits are parading it around, hyping it up like you can just make a whole new game by typing things in, like the guy who fucked over the 3DO port.

Basic gameplay mechanics barely work. Something as simple as a damaging floor doesn't work. For god's sake, it's just "subtract 10 health every second" and it fails at that. The clip of E1M3 on the website ends with the player falling into a pit, suddenly losing 20 health, then another 10, and then just not taking any more damage. Things behind midtexes turn into unidentifiable slop in that video too.

Monsters phase through reality. E2M2's demo video shows that off really well. 2 imps appear, one gets shot and the other phases into existence, enters its pre-attack phase, and then decides to cha-cha slide to the left. One of the blood particles turns into an imp corpse.

Also, where the hell are all of the other enemies? Where's the shotgunners? Lost souls? Cacodemons? It's also very weird how none of the demos show a level being completed either.

It's barely feature-complete and yet they're saying it can "interactively simulate" Doom? By that logic, one of these is a driving simulator.

It's also very obvious that they're playing it really slowly to try and give it the best opportunity and it still fails. This shit would fall apart the moment a normal player tried it.

It is, in no way, going to be any kind of future of game development beyond breathless clickbait articles, dead-end grifting attempts and scams. Just like every other Big Techbro Hype Word, like NFTs.

4 minutes ago, maxmanium said:

Sure, but people act shocked that an early emerging technology isn't already perfect and flawless. Like... no, duh...?

I think what you aren't taking into account is that this generative approach to a game "engine" is going to result in so much behavior inconsistency, no matter how evolved it gets. This is not something that will take the place of real programming and is made for talentless hacks that don't want to learn to actually do anything.

Nobody's "shocked". If anything, its shortcomings are very unsurprising, given the nature of how generative AI tech works.

1 minute ago, OpenRift said:

I think what you aren't taking into account is that this generative approach to a game "engine" is going to result in so much behavior inconsistency, no matter how evolved it gets. This is not something that will take the place of real programming and is made for talentless hacks that don't want to learn to actually do anything.

 

Nobody's "shocked". If anything, its shortcomings are very unsurprising, given the nature of how generative AI tech works.

Maybe you're right, but it feels silly to write it off this early. Also, I don't mean to imply that there aren't any bad actors that are just like "ermagerd, programmers are obsolete now," but I mean, come on. I feel like there are plenty of reasonable people who like this stuff too. At least to me, it is incredibly interesting and, well, incredible, that a computer can do this kind of stuff. Not just the Doom thing but image generation and whatnot as well. Does the Doom thing serve a practical purpose right now? No, but that's why we research and develop these things. It's entirely possible that you're right and it won't be useful, but we'll never know if we don't experiment with it. Prior to this decade I never even thought image generation would be possible in my lifetime to the extent it is now. I just don't think I'm in the wrong for thinking this is exciting and not some sort of travesty.

This continues to support the fact that AI has never produced anything I would have gone out of my way to see, and I don't believe it will ever do that. It's a load of horseshit that'll probably be thrown aside by the techbros once they find a more lucrative buzzword before it even gets the chance.

22 minutes ago, HiMyNameIsChair said:

 

...cool. i don't fucking care lmao.

 

this shit is a dead end. The bubble is literally about to pop because investors finally realize this shit won't make money because AI bros are using it for the stupidest shit imaginable instead of practical uses for the technology, none of which involves art, game development, music, writing, or film.

 

i genuinely, cannot stress enough I don't fucking care. it's an abomination, I hate it, and I want it to stay 1000 feet away from my favorite game.

 

also, still a fucking waste of electricity for research that will never Pinocchio its way into practical use or help development for actual practical use down the road. 

The focus here is not game dev or art it's more generally just allowing ai to predict into the near future for any given image input, i don't understand how you're so confident that this research will never have a practical use when you don't even understand how model architectures can be adapted to different use cases, for all you and i know this research could be critical to developing general reasoning abilities for ai. I don't think that's likely but there are some obvious and very practical uses for this tech if it's developed further.

You raise a good point, Rata.

Even something as basic as Google Image Search has become flooded with AI garbage. Youtube is flooded with videos of AI voiceovers reading generated scripts about how giants are actually real. Spotify has been hit with generic AI-generated music that sounds like something you would hear in an advertisement... I don't want that shit in my games, the things I make and play for relaxation and leisure.

AI is, just like NFTs and the Blockchain before it, another techbro buzzword, heralded as the future by people who only see short-term profits. They don't care about the way they're making things worse, only the money.

If it bleeds...

...we can kill it.

8 minutes ago, giwake said:

Even something as basic as Google Image Search has become flooded with AI garbage.

100% this. I tend to use Google Images to look for locales to use as mapping references or just inspiration in general.

A while back I went looking for photos or artwork of libraries and temples and I had to filter out specific sites because I kept getting unusable AI slop in the results lol. AI has actively made looking for references harder lol.

I think the worst inefficiency, which I pointed out on twitter, is the actual steps you need to take for this to work.

• 1) make a game engine
• 2) Make a game for that engine
• 3) train the neural network on this game
• 4) ship the neural network instead of the game you just made

Game engines are not mathematically quantifiable like video and images are. They’re complex with well defined rules. People invent new engines all the time, and you won’t be able to make anything original with this model if your video game has literally never existed before.

Put it this way: At some point in the past, there’s a distinct before and after games like Minecraft, Hovertank  3D, Super Mario, Metroid, etc. You literally could not get an original game out of these models, when time and time again people come up with literal things that have never existed before.

6 minutes ago, HiMyNameIsChair said:

100% this. I tend to use Google Images to look for locales to use as mapping references or just inspiration in general.

 

A while back I went looking for photos or artwork of libraries and temples and I had to filter out specific sites because I kept getting unusable AI slop in the results lol. AI has actively made looking for references harder lol. 

God, I was just trying to find a picture of a borzoi dog looking at a fisheye lens and almost all of the image results are AI-generated nightmare dogs that look like they have some kind of horrible illness.

5 minutes ago, kevansevans said:

Put it this way: At some point in the past, there’s a distinct before and after games like Minecraft, Hovertank  3D, Super Mario, Metroid, etc. You literally could not get an original game out of these models, when time and time again people come up with literal things that have never existed before. 

That's the thing with AI art. It's never going to generate anything unique or interesting. AI art always makes the most generic, soulless, middle-of-the-road crap imaginable because of how it works. It just grabs a bunch of images and averages them out. Good luck making something like Cruelty Squad or Anthology of The Killer or 10 Beautiful Postcards, which all have some kind of fucked up non-traditional gameplay, with AI.

54 minutes ago, HiMyNameIsChair said:

 

 I can guarantee you the people making actual use of this tech, (like in the medical field it's being used to identify patients who are at a higher risk for cancer... which they were already doing before this monstrosity got published lol) are not going to benefit from a shoddily put-together "recreation" of Doom. I'm sure there are far more competent people already working on exactly that. 

 

Mainly because AI bros, and the people funding this crap do not fucking care about that, and that is not where the focus has been over the last few years. They are vultures. grimy capitalistic rats who only care about perpetuating the hype cycle of the next new tech fad, like Crypto, NFTs, and the Blockchain. That is all they care about.

 

again: cannot stress enough I. Do. Not. Fucking. Care.

 

i do not care about what this research could potentially be used for, because I do not trust the people who fund this crap to not think of the worst possible use for it, which has been the prevailing trend.

These ARE the competent people working on that. I am not expert in healthcare so i don't know what's actually possible, but imagine being able to take an mri of a heart with a certain condition and from that predicting it's movement and possible future health problems. Yeah there are plenty of grifters acting like this is gonna revolutionize gaming but that's because they're dumb, it doesn't change anything about the value of the research.

Edit: Ok i can't post anymore but i want to respond to the people replying to me. Yeah i came on here because i just wanted to see what people were saying about this tech because i think it's cool, i made an account to respond to the people shitting on it, sue me.

No i don't know that the model can be used for anything in medicine, just like you don't know that it can't, hell the researchers don't know the use cases. I was making the point that 1. this model and the associated research and architectures stemming from it could have loads of potential use cases, we don't know what's in the future but you've already decided that there are no practical use cases. 2. This research is helping advance the field of ai in general, the research that is making diagnosis of medical conditions possible stems from researchers trying to improve image classification ai, when that tech was in it's infancy it was similarly useless, but if they'd listened to the naysayers it would've never gotten to it's current state.

If you genuinely don't think this ai can ever do anything useful then why do you care? Just ignore it and let idiots invest in something that will never go anywhere

"I Bring Scientists, YOU Bring A Rock Star!"
"In Forty-Eight Hours, I'LL Be Accepting YOUR Apologies."
"Spared No Expense!"
"I Don't Believe it! I Don't Believe It. You're Meant To Come Down Here And Defend Me Against These Characters, And The Only One I've Got On MY Side Is The Blood-Sucking Lawyer!"
"I Really Hate That Man."
"Creation Is An Act Of Sheer Will! Next Time It'll Be Flawless!"

"You Were Right And I Was Wrong. There! Did You Ever Expect To Hear Me Say Such A Thing?"

1 minute ago, LKD said:

These ARE the competent people working on that.

yeah i fucking doubt that lmao. Also you don't fucking know that, how can you claim that??? again: the use of this tech in the medical field has been done long before this got shat out, with findings from a study out of Harvard Medical School and the University of Copenhagen... published more than a year ago lol.

2 minutes ago, LKD said:

I am not expert in healthcare so i don't know what's actually possible,

so you admit you don't know any more about this than me lol. cool. awesome.

41 minutes ago, giwake said:

Also - joined 1 hour ago? Just made an account to defend shitty AI slop. Incredible.

Actually let's cycle back to this: who the fuck are you? why are you here? do you even play Doom? Are you literally just here because a buddy linked this thread in discord or w/e so you could defend some AI slop?? if so... that's sad lol.