Generalisable how? The model completely hallucinates invalid input, it's not even high quality and required CSGO to work. What's the output you expect from this and what alternatives are there?
None of those questions are relevant are they? I get the impression you've already decided this isnt good enough, which is basically agreeing with everyone else. No one is talking about what it's capable of today. Read the thread again. We're imagining the great probability a few permutations later this thing will basically be The Matrix.
It did not require CSGO, that was simply one of their examples. The very first video in the link shows a bunch of classic Arati games, and even the video which is showing CSGO is captioned "DIAMOND's diffusion world model can also be trained to simulate 3D environments, such as CounterStrike: Global Offensive (CSGO)" — I draw your attention to "such as" being used rather than "only".
And I thought I was fairly explicit about video data, but just in case that's ambiguous: the stuff you record with your phone camera set to video mode, synchronised with the accelerometer data instead of player keyboard inputs.
As for output, with the model as it currently stands, I'd expect a 24h training video at 60fps to be "photorealisic and with similar weird hallucinations". Which is still interesting, even without combining this with a control net like Stable Diffusion can do.
You do the same thing at a larger scale, and instead of video game footage you use a few million hours of remote controlled drone input in the real world.
To answer your question directly, the benefit is that we could make something different from counter strike.
You see, there are these things called "proof of concept"s that are meant to not be a product, but instead show off capabilities.
Counterstrike is an example, meant to show off complex capabilities. It is not meant to show how the useful thing of these models is to literally recreate counterstrike.
Which capabilities are being shown off here? The ability to take an already existing world-model and take lots of compute to have a worse, less correct model?
The capability to have mostly working, real time generation of images that represent a world model.
If that capability is possible, then it could be possible to take 100 examples of seperate world models that exist, and then combine those world models together in interesting ways.
Combining together world models is an obvious next step (IE, not showed off in this proof of concept. But it is a logical/plausible future capability).
Having multiple world models combined together in new and interesting ways, is almost like creating an entirely new world model, even though thats not exactly the same.
Gosh I’m so excited for AAA games to get _even more garbage_ once this gets good enough for investor-driven studios to start slinging out AI driven, hyper-derivative, low-effort slop with even less effort.
We all thought Sports-Game $CURRENT_YEAR or CoD/MW MicroTransaction-Garbage was bottom of the barrel, a whole new world of fresh hell awaits us!
You have it all wrong. It's not going to be AAA studios doing this. Instead, it will be randos who have a cool/fun concept that they want to try out with a couple friends.
Sure, most will be bad. But there might be some gems in there that go viral.
Making it easier for regular people to experiment with games is a good thing.
Or even better, what if this allows people to throw something together to see if a game mechanic is fun, and once it's tested out, then they can make the game for "real".
Allowing quicker experimentation times is also a good thing.
You'd have to have the rights to do the training wouldn't you ? Or does that mean I should close off all of my creations legally just so you can't use them ?
I don't have a problem with you using this with a Camera and the real world or your creations ; but I do have a problem when people are able to use someone's work and use these blend and call them original.
It's just as I would have taken a 3d model static mesh, apply some blend on it and call it my own.
> you'd have to have the rights to do the training wouldn't you ?
Thats undecided by the courts. Everyone is training using other people's data right now though, and very few companies are even being sued (let alone have finished the multi-year process to actually be punished for it).
> Or does that mean I should close off all of my creations
When you put something out there, you should expect that other people are going to use those creations. Almost certainly in many ways that you don't approve over.
Don't release something publicly if you don't want other people to use it.
> but I do have a problem
Your anger isn't particularly useful. What are you going to do about it? This particular proof of concept model was made with 1 single GPU running for 2 weeks. You can't stop that.
> It's just as I would have taken a 3d model static mesh, apply some blend on it and call it my own.
Something that I am sure many people are doing all the time, already. Transforming and using other people's content is as old as the internet. AI does little to change that.
You misunderstand that I'm also a consumer and that I look upon these things. If I see an someone bring up some AI stuff and call themselves an artist I know how I look at them.
