Oh, nice. Third party implementations of Nanite playback.
Nanite is a very clever representation of graphics meshes. They're directed acyclic graphs rather than trees. Repetition is a link, not a copy. It's recursive; meshes can share submeshes, which in turn can share submeshes, all the way down. It's also set up for within-mesh level of detail support, so the submeshes drop out when they're small enough.
So you can have repetitive content of very large size with a finite amount of data and fast rendering times. The insight is that there are only so many pixels on screen, so there's an upper bound on rendering work really needed.
There's a really good SIGGRAPH video on this from someone at Epic.
Current GPU designs are a mismatch for Nanite, Some new hardware operations are needed to do more of this in the GPU, where it belongs. Whether that will happen, with NVidia distracted by the AI market, is a good question.
The scene needs a lot of instancing for this to pay off. Unreal Engine demos show such things as a hall of identical statues. If each statue was different, Nanite would help far less. So it works best for projects where a limited number of objects are reused to create large areas of content. That's the case for most AAA titles. Watch a video of Cyberpunk 2077, and look for railings and trash heaps. You'll see the same ones over and over in totally different contexts.
Making a nanite mesh is complicated, with a lot of internal offsets for linking, and so far only Unreal Engine's editor does it. With playback now open source, someone will probably do that.
Those internal offsets in the format present an attack surface which probably can be exploited with carefully crafted bad content, like hostile Microsoft Word .doc files.
> Repetition is a link, not a copy. It's recursive; meshes can share submeshes, which in turn can share submeshes, all the way down.
While it does construct a DAG to perform the graph cut, the final data set on disk is just a flat list of clusters for consideration, along with their cutoffs for inclusion/rejection. There seems to be a considerable misunderstanding of what the DAG is used for, and how it's constructed. It's constructed dynamically based on the vertex data, and doesn't have anything to do with how the artist constructed submeshes and things, nor does "repetition become a link".
> The scene needs a lot of instancing for this to pay off. Unreal Engine demos show such things as a hall of identical statues. If each statue was different, Nanite would help far less.
What makes you say this? The graph cut is different for each instance of the object, so they can't use traditional instancing, and I don't even see how it could help.
It may not be based on what the mesh's creator considered repetition, but repetition is encoded within the mesh. Not sure if the mesh builder discovers some of the repetition itself.
I'm not seeing what you claim to be seeing in that demo video. I see a per-triangle debug view, and a per-cluster debug view. None of that is showing repetition.
Not necessarily. Nanite compresses meshes (including in-memory) _very_ heavily, and _also_ streams in only the visible mesh data.
In general, I wouldn't think of Nanite as "one thing". It's a combination of many, many different techniques that add up into some really good technology.
I don't want to estimate storage space right now, but meshes can be stored very efficiently. For example, I think UE uses an optimization where vertex positions are heavily quantized to just a few bits within the meshlet's bounding box. Index buffers can be constructed to share the same vertices across LOD levels. Shading normals can be quantized quite a bit before shading artifacts become noticeable - if you even need them anymore at that triangle density.
If your triangles are at or below the size of a texel, texture values could even be looked up offline and stored in the vertex attributes directly rather than keeping the UV coordinates around, but that may not be a win.
In a past life (2000's) I was doing some dev stuff on Ps3, trying to figure out so decent uses for Cell's mass of compute and working around RSX's limited memory bandwidth while having the luxury of Blu-ray storage to burn through.
One such thing I did get a fair way into was something like Nanite - I called it compressive meshing. It is the typical case of misguided engineering hubris at work.
The initial work looked promising but the further into the problem I get the more complicated the entire thing become. Having to construct the entire asset generation pipeline was just way beyond what I could manage in the time frame that would look anything decent and not blow out the memory required.
I did manage to get something that vague resembled large scale meshes being rendered in a staggered level of detail but it ran SLOW and looked like rubbish unless you hammered the GPU to get sub-pixel accuracy. It was a fun experiment but it was far too much for the hardware and too big of a task to take on as a single programmer.
When Epic showed off Nantine... wow they did what I never could in a fashion way beyond even my best vision! It is one of those technologies that when it came along really was a true solution rather than just hype. Yes there are limits as with anything on that scale but it is one of the technical jewels of the modern graphics world. I have said that if Epic was public traded company I would considered putting in a sizable amount of money just based on Nanite tech alone.
Keep in mind, it took Epic a long time to get it sorted. I think I saw the primary creator say it took him a decade of research and work to come to the initial implementation of Nanite that shipped in Unreal.
And it's crazy to think that during that decade, GPU functionality was changing pretty rapidly. So, you really are aiming for future GPUs that don't exist yet as you develop the tech.
Of course, the trajectory of GPU advancements is somewhat predictable, and settled down a little bit relative to the not-too-distant past. Perhaps some luck involved, too (:
There's also this short high-level intro (2.5 min) that I thought was decent: "What is virtualized micropolygon geometry? An explainer on Nanite" (https://www.youtube.com/watch?v=-50MJf7hyOw)
I read through the papers and my impression was that the biggest gains were from quantised coordinates and dynamic LOD for small patches instead of the entire mesh.
The logic behind nanite as I understood it was to keep the mesh accuracy at roughly 1 pixel precision. So for example, a low detail mesh can be used with coordinates rounded to just 10 bits (or whatever) if the resulting error is only about half a pixel when perspective projected onto the screen.
I vaguely remember the quantisation pulling double duty: not only does it reduce the data storage size it also helps the LOD generation because it snaps vertices to the same locations in space. The duplicates can then be eliminated.
> Making a nanite mesh is complicated, with a lot of internal offsets for linking, and so far only Unreal Engine's editor does it.
meshoptimizer [1] is an OSS implementation of meshlet generation, which is what most people think of when they think of "Nanite's algorithm". Bevy, mentioned in a sibling reply, uses meshoptimizer as the generation tool.
(Strictly speaking, "Nanite" is a brand name that encompasses a large collection of techniques, including meshlets, software rasterization, streaming geometry, etc. For clarity, when discussing these concepts outside of the context of the Unreal Engine specifically, I prefer to refer to individual techniques instead of the "Nanite" brand. They're really separate, even though they complement one another. For example, software rasterization can be profitably used without meshlets if your triangles are really small. Streaming geometry can be useful even if you aren't using meshlets. And so on.)
Small correction: meshoptimizer only does the grouping triangles -> meshlets part, and the mesh simplification. Actually building the DAG, grouping clusters together, etc is handled by Bevy code (I'm the author, happy to answer questions).
That said I do know zeux was interested in experimenting with Nanite-like DAGs directly in meshoptimizer, so maybe a future version of the library will have an end-to-end API.
That's not what this is though. It's an implementation of the techniques/technology used in Nanite. It doesn't load data from Unreal Engine's editor. One of the mentioned goals:
Simplicity. We start with an OBJ file and everything is done
in the app. No magic pre-processing steps, Blender exports, etc.
You set the breakpoint at loadObjFile() and F10 your way till
the first frame finishes.
>Current GPU designs are a mismatch for Nanite, Some new hardware operations are needed to do more of this in the GPU, where it belongs. Whether that will happen, with NVidia distracted by the AI market, is a good question.
Unreal 5 was only released in 2022, and we have been iterating the Nanite idea since then. With Unreal 5.5 and more AAA Gaming titles coming in and we can take what we learned and put into hardware. Not to mention the lead time is 3-4 years down the road. Even if Nvidia decided to make one in 2023 it would have been at least 2026 before we see any GPU acceleration.
Nanite is a very clever representation of graphics meshes. They're directed acyclic graphs rather than trees. Repetition is a link, not a copy. It's recursive; meshes can share submeshes, which in turn can share submeshes, all the way down. It's also set up for within-mesh level of detail support, so the submeshes drop out when they're small enough. So you can have repetitive content of very large size with a finite amount of data and fast rendering times. The insight is that there are only so many pixels on screen, so there's an upper bound on rendering work really needed.
There's a really good SIGGRAPH video on this from someone at Epic.
Current GPU designs are a mismatch for Nanite, Some new hardware operations are needed to do more of this in the GPU, where it belongs. Whether that will happen, with NVidia distracted by the AI market, is a good question.
The scene needs a lot of instancing for this to pay off. Unreal Engine demos show such things as a hall of identical statues. If each statue was different, Nanite would help far less. So it works best for projects where a limited number of objects are reused to create large areas of content. That's the case for most AAA titles. Watch a video of Cyberpunk 2077, and look for railings and trash heaps. You'll see the same ones over and over in totally different contexts.
Making a nanite mesh is complicated, with a lot of internal offsets for linking, and so far only Unreal Engine's editor does it. With playback now open source, someone will probably do that.
Those internal offsets in the format present an attack surface which probably can be exploited with carefully crafted bad content, like hostile Microsoft Word .doc files.