Support NeoGAF

[Buggy Loop]

Because reconstructing the output of a physically based calculation is "basically the same" as predicting how something should look based on a collection of hints and a very confident neural network.

Ray Reconstruction starts with an actual lighting simulation, meanwhile DLSS 5 predicts a plausible result learned from training data,

Ray reconstruction uses generative AI to know how it should look. It recognizes patterns for effects like global illumination, reflections, and ambient occlusion.

DLSS 5 has access to the exact same information and can use generative AI to know how it should look. It's exactly at the same place in the pipeline.

rather than solving the physical process that produced the result.

It is working with the produced result from all the pipeline as RR does, what you think its doing? Re-doing the whole frame?

Which it couldn`t because it does have, at best, incomplete knowledge about the actual physical situation and the light stransport, even with the G buffer.

Repeat after me : The exact same inputs as ray reconstruction. RR uses the light accumulation sampling (in G-buffer, or other buffers for multi-light bounce but regardless, in screen space)

The comparison with a deferred renderer is equally ridiculous because a deferred renderer does not just say "this pixel looks like skin, therefore add skin lighting."

Uh yea? As with anything that is not baked and most of the pipeline beyond the first rasterized games, there's algorithms exactly for that. How do you think fragment shading works for almost all detailed effects that makes games look like they do nowadays? Ambient occlusion, again, how do you think that happens? An algorithm looks at the scene does depth sampling from G-buffer data and looks for pixels around it to determine the shape, will say oh here's a tight corner or an object touching another surface with different normals, well let's apply shadow creases. It's not an "artist".

Subsurface scattering from Nvidia/Epic/Unity has been screen space forever, the alternative is too costly in performances for little to no gains.

Here have fun, even a browser version

Der Schmale - Subsurface Scattering for Skin Rendering

Screen-space Subsurface Scattering for Skin Rendering

www.derschmale.com

You talk like games are pixel arts and that DLSS is gonna "guess" something, when the entirety of the pipeline has been "engineered" to cheat their way to the solution since the dawn of graphic rendering. A man-made algorithm probing slowly a scene taking many many iterations to fumble in the dark and detect the scene shapes, then algorithm says yes, right there, you detected a tight corner, apply shadowing... or an AI looking at the scene and understanding that there's walls there touching, should have a shadow, what's the problem here? Games are not pixel art. Neither cases an artist placed the shadowing.

Is your whole argument based on "math" good and AI bad? Because then the whole DLSS suite basically gets wiped. Remove AI upscaling too, goddamn, it knows how to fix images rather than use algorithms like TAA. Ray reconstruction would fall back to traditional denoising filters. Neural cache radiance PT too, nuh uh! Go back to Monte Carlo!

All your sermons literally just boil down to...."i´m sure Nvidia`s model will hallucinate correctly"

My sermons? Dude you went into all DLSS 5 threads to say it's an instagram filter as if its post processing effect and that there's an issue because its screen space. Probably because screen space = screen space reflections = bad right? Almost the entirety of a frame's rendering time is in screen space among oh so many effects.

Hallucinate?



Trusting one of the most prominent research scientist in the field that participated in almost all the latest nvidia technologies of the past years, such as ReSTIR GI, and is now director of applied deep learning research

Or Cumulus "screen space is an issue"

 

[Cumulus]

Ray reconstruction uses generative AI to know how it should look. It recognizes patterns for effects like global illumination, reflections, and ambient occlusion.

DLSS 5 has access to the exact same information and can use generative AI to know how it should look. It's exactly at the same place in the pipeline.

They are not solving the same issues in any way, the similarity your are dragging into the room kicking and screaming literally starts and ends with the pipeline position. "Same place in the pipeline" does not mean "same information" or "same task."

It is working with the produced result from all the pipeline as RR does, what you think its doing? Re-doing the whole frame?
Repeat after me : The exact same inputs as ray reconstruction. RR uses the light accumulation sampling (in G-buffer, or other buffers for multi-light bounce but regardless, in screen space)

Repeat after me: having some of the same buffers does not mean having the same input.
Ray Reconstruction is not simply looking at a G-buffer and deciding how the scene should look. Its key input is the output of a ray-traced lighting calculation: actual sampled radiance from rays that have been traced through the scene. Those samples contain information about the result of light transport — reflections, indirect lighting, shadows, visibility, etc. — but with noise and missing samples.
A G-buffer does not contain light accumulation. A G-buffer contains surface properties: depth, normals, albedo, roughness, material data, and so on. It describes what surfaces are there, not how light traveled through the scene to produce the final illumination.

Uh yea? As with anything that is not baked and most of the pipeline beyond the first rasterized games, there's algorithms exactly for that. How do you think fragment shading works for almost all detailed effects that makes games look like they do nowadays? Ambient occlusion, again, how do you think that happens? An algorithm looks at the scene does depth sampling from G-buffer data and looks for pixels around it to determine the shape, will say oh here's a tight corner or an object touching another surface with different normals, well let's apply shadow creases. It's not an "artist".

SSAO isn't an algorithm that says, "this probably looks like a corner." It samples explicit scene data—depth and normals—from the G-buffer and applies a deterministic mathematical model. Given the same inputs, it will always produce the same output. It's not making an educated guess based on training; it's executing a defined algorithm.

You talk like games are pixel arts and that DLSS is gonna "guess" something, when the entirety of the pipeline has been "engineered" to cheat their way to the solution since the dawn of graphic rendering. A man-made algorithm probing slowly a scene taking many many iterations to fumble in the dark and detect the scene shapes, then algorithm says yes, right there, you detected a tight corner, apply shadowing... or an AI looking at the scene and understanding that there's walls there touching, should have a shadow, what's the problem here? Games are not pixel art. Neither cases an artist placed the shadowing.

You're mixing up two completely different things.
Yes, real-time rendering has always relied on approximations. SSAO, SSR, shadow maps, voxel GI, clustered lighting, you name it...they're all compromises. But they're still explicit algorithms designed to approximate a specific physical phenomenon. Their behavior is defined by code and mathematics, not by what a neural network learned from a dataset. The network doesn't understand the scene in the way a renderer does. It maps inputs to outputs based on learned statistical relationships and we`ve seen time and time again what that means concerning stability and continuity. We´ve already seen the types of instability issues we`ll have to expect.....

Is your whole argument based on "math" good and AI bad? Because then the whole DLSS suite basically gets wiped. Remove AI upscaling too, goddamn, it knows how to fix images rather than use algorithms like TAA. Ray reconstruction would fall back to traditional denoising filters. Neural cache radiance PT too, nuh uh! Go back to Monte Carlo!

Nice strawman. I never said "AI bad." DLSS Super Resolution, Frame Generation, Ray Reconstruction, Neural Radiance Cache, there are enough valid use cases.
Just don't pretend fundamentally different techniques are "basically the same."

Ray Reconstruction reconstructs a physically simulated signal from insufficient ray samples. Traditional denoisers try to do the same thing with hand-written filters; RR does it with a learned model. That's a perfectly viable application of AI but still vastly different from profound scene understanding and enhancement like DLSS5 promises. The amount of room for let`s call it "statistical guesswork" is incomparable.

My sermons? Dude you went into all DLSS 5 threads to say it's an instagram filter as if its post processing effect and that there's an issue because its screen space. Probably because screen space = screen space reflections = bad right? Almost the entirety of a frame's rendering time is in screen space among oh so many effects.

More strawman bullshit....
My point was never that DLSS 5 is inferior because it operates in screen space. My point is that screen-space renderer data does not magically give the network complete knowledge of the scene or the physical process that produced the image which however would be necessary for consistent accurate results at the scope this framework promises.
You're also conflating being in the rendering pipeline with performing the renderer's job.
A deferred renderer uses screen-space buffers to evaluate explicitly defined shading equations. DLSS 5 uses screen-space buffers to feed a learned model that predicts a plausible appearance. Those are not the same operation just because both consume G-buffer-like inputs. Your generalisation and comparisons are just utter nonsense.
So no, this isn't "screen space bad." It's physics-based computation vs. learned inference. Those are fundamentally different approaches, even if they operate on similar renderer data.

Hallucinate?

"Hallucinate" as in: the network did not trace the light path, evaluate the BRDF, or calculate the scattering event — it inferred what the result should look like. and that is simply the way it is.


Your argument for DLSS 5 ultimately relies on trusting that the model will always correctly infer the missing information. The problem is that the renderer does not necessarily provide enough information at that stage of the pipeline to uniquely determine the physically correct result, especially for effects that depend on full scene geometry, material behavior, or light transport.
The scope is the issue here. Maybe it will boil down to partial object enhancement in the end or similar, but what they`ve shown earlier, the "whole scene" rework, that is simply not supported by the data this thing has access to.

 

[viveks86]

Is this shit good? give it to me straight papito

Short answer: No.

But like everything, it's not that simple.

Looooong answer:

Mayyyy be several years in the future when it is fully integrated into the rendering pipeline, it could be great... It's currently the DLSS 1 of AI image enhancement. The final objective is good. Ambitious even. But the approach is fundamentally flawed. There will be a shit ton of hype before release. Eventually people will get angry that it is a bunch of smoke and mirrors and false advertisement. Once the dust settles, the underlying tech, which holds promise, will be extended deep into the rendering pipeline. The big question is, by the time that happens, other non-generative AI techniques would be competing for the very same result. So would it even be relevant, especially if it is not platform agnostic by nature? Only time will tell. Color me skeptical.

I think Buggy Loop has misunderstood the presentation a little bit. It does not have access to what RR has access to. RR runs immediately after the ray tracing phase after the core rendering phase. DLSS 5 still runs after all that. It works on final pixels, either before or after upscaling. Not sure when. It would make sense to do it before upscaling for performance reasons.

It can still have access to the g-buffer. For whatever reason, they were communicating this incorrectly during the announcement that it only does pixel data and motion vectors. But having access to the g-buffer makes sense. That's how they are able to preserve surface normals, albedo, DoF etc. You can never touch material or lighting with just pixels while not knowing what materials are even in the frame. SSAO and SSR have access to the g-buffer too. That's how it even knows what angle to reflect or draw shadows. AC Black Flag uses it to generate screenspace shadows for mid to far distance. But guess what happens when you pan the camera on those distant shadows?

The g-buffer only maintains what's in the current frame. It doesn't have a 3d representation of the world outside. If there is a street lamp off screen and that lamp is bouncing light onto a store that is in the frame, the g-buffer can't tell if that's light from a lamp, an overhead helicopter or the Sun. DLSS 5 will just have to take a guess. It will most likely assume it's the Sun and change the apparent scene lighting as a result. Or it will think it's "bad data" and ignore it. Either way it's messing up the scene and removing important details.

But in addition to the g-buffer, RR has actual data from ray tracing feeding into it. Not just the g-buffer. It has historical data from previous frames too. G-buffer is still a 2D representation of the final frame, just broken down by layers so you have additional data of the material, depth and position of object in worldspace. It has no concept of offscreen/occluded info. That stuff is sitting outside the gbuffer in VRAM.

In terms of available input data, this is no different than how SSAO and SSR already work. It's applying a layer of AI inference to determine what screenspace effects to add (or mask) where at what intensity. All the cons of screenspace techniques will be visible. It was visible in every DLSS 5 footage shown where any significant motion occurs. There is no way around it until they go back to the drawing board and build something that has the full 3D context of the scene. But the moment they do that, the memory and performance overhead will be 100x. That's why I'm skeptical about this going anywhere. It's so ambitious that it has no gradual path to evolve.

Is it some cheap screenspace filter? Hell no. It's the culmination of years of research and complex AI models. But it is still operating in screenspace. And celebrating it as the future of rendering while everybody else, including other teams within Nvidia, work tirelessly to solve these complex equations in more practical ways is a bit sad to see. We will just have to watch this play out. And my bets are on this thing flopping hard and being relegated to modding communities messing around with it for fun and screenshot threads, as opposed to any sustained adoption on the dev/design side.

 

[ResetEraVetVIP]

Not quite done glazing Yotei. Call it art, tech, whatever you want. There's something very satisfying about the visuals that keeps me coming back, despite the areas of ugliness scattered throughout the game.

This. I think its one of the best looking games ever made, artistically not technically.

 

[viveks86]

They are not solving the same issues in any way, the similarity your are dragging into the room kicking and screaming literally starts and ends with the pipeline position. "Same place in the pipeline" does not mean "same information" or "same task."

Repeat after me: having some of the same buffers does not mean having the same input.
Ray Reconstruction is not simply looking at a G-buffer and deciding how the scene should look. Its key input is the output of a ray-traced lighting calculation: actual sampled radiance from rays that have been traced through the scene. Those samples contain information about the result of light transport — reflections, indirect lighting, shadows, visibility, etc. — but with noise and missing samples.
A G-buffer does not contain light accumulation. A G-buffer contains surface properties: depth, normals, albedo, roughness, material data, and so on. It describes what surfaces are there, not how light traveled through the scene to produce the final illumination.

SSAO isn't an algorithm that says, "this probably looks like a corner." It samples explicit scene data—depth and normals—from the G-buffer and applies a deterministic mathematical model. Given the same inputs, it will always produce the same output. It's not making an educated guess based on training; it's executing a defined algorithm.


You're mixing up two completely different things.
Yes, real-time rendering has always relied on approximations. SSAO, SSR, shadow maps, voxel GI, clustered lighting, you name it...they're all compromises. But they're still explicit algorithms designed to approximate a specific physical phenomenon. Their behavior is defined by code and mathematics, not by what a neural network learned from a dataset. The network doesn't understand the scene in the way a renderer does. It maps inputs to outputs based on learned statistical relationships and we`ve seen time and time again what that means concerning stability and continuity. We´ve already seen the types of instability issues we`ll have to expect.....



Nice strawman. I never said "AI bad." DLSS Super Resolution, Frame Generation, Ray Reconstruction, Neural Radiance Cache, there are enough valid use cases.
Just don't pretend fundamentally different techniques are "basically the same."

Ray Reconstruction reconstructs a physically simulated signal from insufficient ray samples. Traditional denoisers try to do the same thing with hand-written filters; RR does it with a learned model. That's a perfectly viable application of AI but still vastly different from profound scene understanding and enhancement like DLSS5 promises. The amount of room for let`s call it "statistical guesswork" is incomparable.


More strawman bullshit....
My point was never that DLSS 5 is inferior because it operates in screen space. My point is that screen-space renderer data does not magically give the network complete knowledge of the scene or the physical process that produced the image which however would be necessary for consistent accurate results at the scope this framework promises.
You're also conflating being in the rendering pipeline with performing the renderer's job.
A deferred renderer uses screen-space buffers to evaluate explicitly defined shading equations. DLSS 5 uses screen-space buffers to feed a learned model that predicts a plausible appearance. Those are not the same operation just because both consume G-buffer-like inputs. Your generalisation and comparisons are just utter nonsense.
So no, this isn't "screen space bad." It's physics-based computation vs. learned inference. Those are fundamentally different approaches, even if they operate on similar renderer data.



"Hallucinate" as in: the network did not trace the light path, evaluate the BRDF, or calculate the scattering event — it inferred what the result should look like. and that is simply the way it is.


Your argument for DLSS 5 ultimately relies on trusting that the model will always correctly infer the missing information. The problem is that the renderer does not necessarily provide enough information at that stage of the pipeline to uniquely determine the physically correct result, especially for effects that depend on full scene geometry, material behavior, or light transport.
The scope is the issue here. Maybe it will boil down to partial object enhancement in the end or similar, but what they`ve shown earlier, the "whole scene" rework, that is simply not supported by the data this thing has access to.

Glad you guys moved on to a real debate. Agree with all of this.

 

[kevboard]

More strawman bullshit....
My point was never that DLSS 5 is inferior because it operates in screen space. My point is that screen-space renderer data does not magically give the network complete knowledge of the scene or the physical process that produced the image which however would be necessary for consistent accurate results at the scope this framework promises.

also of note is that it doesn't even have the same information as typical screen space based effects.
SSR traces into the scene and gets depth buffer information. so when you usually talk about something working in screen space, you are typically talking about the depth buffer + normal maps and materials information.

DLSS5 literally works with the pure 2D color output of the framebuffer (minus the HUD) and motion vectors for "temporal stability"... that's it.

I bet going forward they will almost entire focus on showing faces, with the rest of the scene masked out, because realistically speaking, that's the only thing DLSS5 could work somewhat ok-ish in. and even then it will probably fuck up the light direction and stuff like that, and add a studio light effect to the face due to the training data skewing towards professionally taken photos

 

[Ironman_Bad]

Blood Message looks sweet as hell if you told me that was Naughty Dog, or Sony Santa Monica id believe it

 

[GooseMan69]

I understand it's apples and oranges, but it blows my mind that The Odyssey was made on a $250 million budget, while mid ass games had a similar or even higher budget. I saw it yesterday and it blew my mind. They're out there filming scenes on the actual fucking Mediterranean Sea, there's action scenes with 200 extras on screen, almost everything is done with practical effects, and seemingly every A list actor from the last decade or two makes an appearance. And it cost roughly the same as Dragon Age Veilguard lmao

I don't feel bad anymore. Genuine skill issue. Stop making shit.

 

[viveks86]

also of note is that it doesn't even have the same information as typical screen space based effects.
SSR traces into the scene and gets depth buffer information. so when you usually talk about something working in screen space, you are typically talking about the depth buffer + normal maps and materials information.

DLSS5 literally works with the pure 2D color output of the framebuffer (minus the HUD) and motion vectors for "temporal stability"... that's it.

This may not be entirely true. In their new presentation, they do say they are using albedos, surface normals and "some of the lighting information" whatever the hell that means, from internal buffers. They are all stuff typically available in the G-buffer.

So in addition to 2D color data and motion vectors, it does have some additional input to maintain model stability. It's a contradiction to their previous statements to Daniel Owen, but what they claim is all we have to go on, so I think we should recalibrate.

I still don't think it's remotely sufficient for what they are trying to achieve, but it's… something

 

[Buggy Loop]

Short answer: No.

But like everything, it's not that simple.

Looooong answer:

Mayyyy be several years in the future when it is fully integrated into the rendering pipeline, it could be great... It's currently the DLSS 1 of AI image enhancement. The final objective is good. Ambitious even. But the approach is fundamentally flawed. There will be a shit ton of hype before release. Eventually people will get angry that it is a bunch of smoke and mirrors and false advertisement. Once the dust settles, the underlying tech, which holds promise, will be extended deep into the rendering pipeline. The big question is, by the time that happens, other non-generative AI techniques would be competing for the very same result. So would it even be relevant, especially if it is not platform agnostic by nature? Only time will tell. Color me skeptical.

I think Buggy Loop has misunderstood the presentation a little bit. It does not have access to what RR has access to. RR runs immediately after the ray tracing phase after the core rendering phase. DLSS 5 still runs after all that. It works on final pixels, either before or after upscaling. Not sure when. It would make sense to do it before upscaling for performance reasons.

Streamline/docs/ProgrammingGuideDLSS_RR.md at main · NVIDIA-RTX/Streamline

Streamline Integration Framework. Contribute to NVIDIA-RTX/Streamline development by creating an account on GitHub.

github.com

4.0 TAG ALL REQUIRED RESOURCES
4.1 Buffers to tag
DLSS-RR requires the following buffers:
4.1.1 Diffuse Albedo → G-buffer
4.1.2 Specular Albedo → G-buffer
4.1.3 Normals → G-buffer
4.1.4 Roughness → G-buffer
4.1.5 Color Input → G-buffer
4.1.6 Motion Vectors → G-buffer or a dedicated motion vector buffer, depends on engine. Regardless, it has access to motion vectors.
4.1.7 Depth Buffer → G-buffer
4.1.8 Specular Motion Vector Reflections → G-buffer or dedicated rendering pass
4.1.9 Specular Hit Distance → G-buffer, or dedicated buffer for RT and specular radiance data, still in screen space.
4.1.10 Transparency Overlay Optional only → Frame buffer, holding the final ready-to-display (just a sign where RR is)
.. removing guides
4.1.14 Output

Basically this but even that G-buffer is outdated by modern engine standards, it keeps adding and adding to it.

Ray reconstruction happens at the end of the pipeline. It's basically concurrently with upscaling. On the rendering thread they are at the same location. DLSS will be injected into it.

[h4][/h4]

It can still have access to the g-buffer. For whatever reason, they were communicating this incorrectly during the announcement that it only does pixel data and motion vectors.

Thus my post.

It's in-pipeline as much as ray reconstruction is.

Doing a dual GPU setup with one rendering and one inferring, was really early prototype proof of concept and they probably said things that are true for that particular demo and not even the intended release parameters.

But having access to the g-buffer makes sense. That's how they are able to preserve surface normals, albedo, DoF etc. You can never touch material or lighting with just pixels while not knowing what materials are even in the frame.

It would just read the G-buffer for material based on coordinates. In a deferred shading pipeline, it's a non issue, it stores material attributes there. Albedo/diffuse, normals, roughness/metallic, depth. There's not much more needed, an engine works on all these for PBR materials. If you feel fancy you can even dedicated a channel to material IDs. In fact I doubt any modern engines worth a damn would skip on material IDs.
An example of snowdrop engine detecting material with RR so that RR didn't hallucinate the stones at the bottom of the water on the surface of the water as it is an opaque-surface and confuses RR.

SSAO and SSR have access to the g-buffer too. That's how it even knows what angle to reflect or draw shadows. AC Black Flag uses it to generate screenspace shadows for mid to far distance. But guess what happens when you pan the camera on those distant shadows?

The g-buffer only maintains what's in the current frame. It doesn't have a 3d representation of the world outside.

As is basically all shading's work. After rasterization, that's pretty much it for 3d world space. It's time to fucking fill all those pixels.

If there is a street lamp off screen and that lamp is bouncing light onto a store that is in the frame, the g-buffer can't tell if that's light from a lamp, an overhead helicopter or the Sun. DLSS 5 will just have to take a guess.

? How do you think deferred lighting works? Light sources don't ever have to be in front of you for all the shaders to calculate how each individual pixels should be in a gradient of light to dark depending on their location compared to the light source coordinates and color. You guys are confusing screen space principles which is everything in pipeline after rasterization and screen space reflections defects I feel like. Really not the same thing.

It will most likely assume it's the Sun and change the apparent scene lighting as a result. Or it will think it's "bad data" and ignore it. Either way it's messing up the scene and removing important details.

Why wouldn't it have access to the light vectors to know where light sources are like any lightning pass and much like any ReSTIR or Megalight, make a pass to determine which are priorities for processing and which to offload to the deferred lighting? More importantly, it has access to the light!

The entire past 2 decades of deferred rendering and hybrids worked exactly the same parameters. I don't know why this is a new puzzle or something.

But in addition to the g-buffer, RR has actual data from ray tracing feeding into it. Not just the g-buffer. It has historical data from previous frames too. G-buffer is still a 2D representation of the final frame, just broken down by layers so you have additional data of the material, depth and position of object in worldspace. It has no concept of offscreen/occluded info. That stuff is sitting outside the gbuffer in VRAM.

I think you have the wrong perception of where RTXDI happens. See the pipeline I included in the screenshot above but more importantly ReSTIR DI, GI, PT, are all screen space light sampling / re-sampling techniques.

At siggraph they said they have access to lighting information. Period. Be it the light pass sampling stored in the G-buffer's light accumulation channels or a dedicated buffer because it's too big, it doesn't matter. It feeds on that. Ray tracing is not feeding into ray reconstruction by some maginal channel, it's buffer!

In terms of available input data, this is no different than how SSAO and SSR already work.

And fragment shaders, most modern lighting passes, ray marching, bump mapping, shadows, specular highlights, translucency... basically every fragment shader. Cel shaders too.

It's applying a layer of AI inference to determine what screenspace effects to add (or mask) where at what intensity. All the cons of screenspace techniques will be visible.

That's like saying that shaders will have the screen space techniques defects. All lighting passes from ReSTIR GI and so on.

Screen-space is where most of what you see on screen happen. Screen space effects based on algorithms to cheat their way into an effect and their limitations are not a limitation of screen space. Motherfucking ReSTIR PT works in screen space.

It was visible in every DLSS 5 footage shown where any significant motion occurs.

The march footage? Cmon son. A proof of concept from two GPUs with limited communication. Inference was not even done in the same pipelines. This is not how they'll implement it. Watch the siggraph 2026.

There is no way around it until they go back to the drawing board and build something that has the full 3D context of the scene.

Which ray reconstruction nor any modern lighting passes don't either

What do you want to do with the with "full 3D context" such as vertexes? I feel like I'm taking crazy pills because the concept of real 3D is completely gone way early in the rendering. Because as soon as those are transformed to a 2D triangles and then rasterization determines exactly which pixels cover the surface area of that triangle, you're in *shock* screen space.

 

[SlimySnake]

They definitely upgraded the foliage and assets in this game. Wood and foliage has never looked better.

 

[viveks86]

Ray reconstruction happens at the end of the pipeline. It's basically concurrently with upscaling. On the rendering thread they are at the same location. DLSS will be injected into it.

[h4][/h4]

Sure. My point isn't that RR has to run before upscaling. In theory, you can do them concurrently, before or after. It's why AMD's RR is currently decoupled. Concurrent just happens to be the most robust version. And FSR 5 will likely go that route too. So you are right about that. But my point is that DLSS 5 can never run before RR. You simply cannot feed generative AI such noisy data. The model is trained to work on the final composited image as primary input and, apparently, select info from the g-buffer as control parameters. At that stage in the pipeline, as far as DLSS 5 is concerned, all the world space traced data (represented by the noisy image) is already consumed. The lighting and shading stage is complete. They reiterated several times, even on the new presentation, that the model is a compact one step diffusion model entirely reliant upon pixel space data. I don't know why you would even bring up light vectors when I'm talking about bounce lighting/GI. Are you expecting DLSS 5 to do a second lighting and shading pass by evaluating all the scene light sources (onscreen and offscreen) again? lol. If so, you have the wrong idea of what the model even does and are projecting your hopes and dreams on it. Having some light accumulation, radiance data etc in the buffer does nothing to tell you where the light sources actually are, which is what you need to generate new lighting/shadowing. You think all the megalights info (100s or even thousands) in E-day is sitting in the g-buffer? You will only have the final accumulation and that isn't going to tell you where the light should be bouncing further for DLSS to accurately generate additional lighting/shadows

It's in-pipeline as much as ray reconstruction is.

Did I say it is out of the pipeline? I only said RR has more data to work with ie the noisy ray traced input, and that it would have to run before DLSS 5. That noisy ray traced input came from performing ray traced direct and/or indirect lighting or PT. Once that has been denoised and composited with the rasterized primary, any attempt to change that lighting again in the frame, such as adding shadows is not based on world space. To base it again on world space, you will have to re-evaluate light sources using the same methods that were already used during the lighting/shading phase, making it redundant and costly. And like you pointed out, given RR is concurrent with upscaling, DLSS5 will have to run on the upscaled 4k image. It's why they could easily run the same solution on a dual gpu setup during development. Use one gpu as the traditional pipeline and then feed the upscaled image to DLSS 5. This is exactly how it will run come launch, except on 1 gpu with the requisite performance optimizations, instead of 2. I never doubted they will pull that part off. But don't expect some foundational change in workflow within months of production launch.

But I'm not even sure why you are fixated on RR in a conversation about DLSS 5. The goal of RR is simply to denoise an image. Not infer shadows or lighting that didn't even exist in the input data. RR is non-generative. DLSS 5 is generative (albeit constrained by some gbuffer parameters). Different use cases and goals. Is your argument that if RR is awesome, DLSS 5 will be awesome too, because there is access to a g-buffer? If the Apple I have is tasty, is my orange tasty too, because they grew on the same soil? G-buffer isn't some silver bullet that solves all rendering problems. It's a fraction of scene data focused entirely on what is needed for the current frame.

Doing a dual GPU setup with one rendering and one inferring, was really early prototype proof of concept and they probably said things that are true for that particular demo and not even the intended release parameters.

I have no qualms with this. I never complained about the dual gpu situation. If they say they can pull it off on a single gpu by release, I'll take their word for it. But I will say that nothing about what was shown at SIGGRAPH indicates any change in approach. The g-buffer is simply more detail we are learning about, and in all likelihood was already being used. Without that, stability of a generative model at realtime speeds would have been utter shit. Like they stated themselves, there would have been no incentive for the model to even maintain the same face or textures over multiple frames. The G-buffer IS the constraining factor. Maintaining albedo, normals and depth would prevent the model from going wild. It makes perfect sense.

ReSTIR DI, GI, PT, are all screen space light sampling / re-sampling techniques.

I feel like I'm taking crazy pills because the concept of real 3D is completely gone way early in the rendering.

No it's not. I think your entire understanding of the rendering paradigm is stuck at deferred rendering. Hence, all the non sequitur defense of screenspace techniques. All hardware ray tracing happens on 3d data. The BVH has actual 3d triangle geometry. In the not so distant future, the BVH will become the single source for all 3D triangle data in VRAM. Animations will occur by manipulating this data within the BVH so that there is no need for triangle duplication or lower resolution proxies. All RT and PT will continue being traced against this 3D "context". That is quite definitively NOT screenspace. In the context of hardware RT, Screenspace sampling is simply an origin point to trace a ray. One sample per pixel means you shoot out one ray for each screen pixel. But the ray itself will bounce and be evaluated across the scene, which may or may not fall within screenspace. It's the reason Raytracing is always an objectively better technique than its screenspace counterpart. The part that holds RT back is denoising issues, performance, BVH proxies etc. but the tracing itself can never be matched because it is, by nature, operating in 3D space. This claim alone suggests you are posting slides on RT without a true understanding of RT. Step into the brave new world. I would urge you to study it deeper.

Watch the siggraph 2026.

I did. Multiple times. Literally nothing new. The only thing new are the slider features hinted earlier being demonstrated now. Everything else behaves exactly like the first demo. And there is barely any motion at all to point out issues. They might as well have been screenshots. Like I said, smoke and mirrors.

 

[Toots]

to all the belligerent wall of texters that invaded the thread as of late :

 

[Gonzito]

to all the belligerent wall of texters that invaded the thread as of late :

People in this thread be like

 

[Cumulus]

to all the belligerent wall of texters that invaded the thread as of late :

 

[Trilobit]

People in this thread be like

Me looking for some cool pics in this thread:

 

[Gonzito]

Man say whatever you want about Ubisoft, but Anvil Engine is fucking insane

 

[Toots]

This one looks absolutely crazy, did you use any fx, or is it just as is ?

 

[Gonzito]

This one looks absolutely crazy, did you use any fx, or is it just as is ?

Nothing, the only thing I am using is Renodx which improves the look of HDR with better colours and tonemapping

 

[Gubaldo]

I would like to have Far Cry 3 Remake on Latest Anvil engine

 

[RJMacready73]

I would like to have Far Cry 3 Remake on Latest Anvil engine

I must be the only one wanting the original remade, it had the perfect setting, i love its Island of Dr Moreau vibe and would be a day 1 purchase for me

 

[amigastar]

I would like to have Far Cry 3 Remake on Latest Anvil engine

Actually not a bad idea. I would be interested.

 

[kevboard]

This may not be entirely true. In their new presentation, they do say they are using albedos, surface normals and "some of the lighting information" whatever the hell that means, from internal buffers. They are all stuff typically available in the G-buffer.

So in addition to 2D color data and motion vectors, it does have some additional input to maintain model stability. It's a contradiction to their previous statements to Daniel Owen, but what they claim is all we have to go on, so I think we should recalibrate.

I still don't think it's remotely sufficient for what they are trying to achieve, but it's… something

that's weird. either they did add that in the months since the announcement or they fucked up big time with their messaging before.

...or they try to be vague now and just mean that the AI "understands" the materials due to the training data.

but yes, that would at least keep it from thinking a shower curtain is a tree bark and shit like that, but not much more.
(I use this example because I saw an RE1 AI upscaled mod pack on PC a few weeks ago where the AI turned the shower curtains into a tree )

 

[setoman]

Short : Yes

DLSS 5 adding subsurface scattering to a character's skin is "in-pipeline" as much as the same process would be for unreal engine except an AI thought of it, as it thought it should look like that, rather than the dev omitting it or other limits of the scene. Just like ray reconstruction uses AI to fill in the gaps of how the shadows should look like based on a very noisy lighting result from RT.

It's basically RR++ but now looks at shading too.

As a Day 1 DLSS 5 supporter, this shit looks even better than before.

Give me Path traced RE10 with photo real zombies and gore on a 6080

You people are being hoodwinked and deceived.

DLSS 5 is NOT photorealistic its a bad looking IG filter. Nvidia is so good at lying that the two examples they showed u is,
a high poly character where they turned off almost all of the effects of that character that would come pre-built in UE5, including its subsurface scattering.

Now name me a AAA game or even a photorealistic based game that doesn't have SSS? none, zero, zip.

Here is a very old real-time UE4 rendering here just to show you how diabolical Nvidia are.

So Nvidia goes and disables it, then turns on DLSS 5 and goes see, voila SSS. Look how amazing it looks.
What they are implanting in your brain is, You DIDN'T have SSS before but with DLSS 5, now you do!
When every AAA / photoreal game ships with SSS.

They did it again with the kitchen cutting board, They disabled all the shadows. The image is completely flat.
Then they slide on DLSS 5 and say, look at how amazing those shadows are. What they are not telling you is that they disabled all the shadows.

What game comes with disabled shadows? The funny thing is similars scenes in AW2 path tracing for example the dinner table look way better than this so called DLSS 5 filter when u compare all the rendering element. How is that possible? someone explain.

Not just that, be this is the exact same path-traced NVIDIA Omniverse Ramen Shop using the exact same models .
DLSS 5 is supposed to be the photoreal toggle yet the the graphic rendering on the models in the original demo look 100x better than the so called DLSS 5 filter.
Its not even close. And yes the demo was pathtraced offline. But thats the point DLSS 5 is not giving you photoreal graphics. Its a lie. Its giving you a IG filter that look WORSE than other existing games that are aiming for photoreal graphics.

People naively say, well this is just like DLSS 1-4 uprender. What ppl are closing their eyes to is that DLSS aim was to get the uprender to look exactly like the 4k output of the game engine render with identical pixel quality. This is why when even one pixel is off you get DF videos pointing it out.

This is why DLSS was adopted en-mass.

DLSS 5 isn't trying to match what the game is rendering/outputting at a high settings/quality, its not like you are turning off SSS and a bunch of rendering features like reflection, shadows, gi, translucency, etc and then turning on DLSS 5 and DLSS 5 is producing output that is identical to what the rendering output would look for that particular game engine.

NO DLSS 5 isn't doing that, so DLSS 5 is NOT like DLSS.

its giving you an IG filter not what the game rendering engine would look like. That's why the output is bad.

If DLSS5 actually functioned like the original aim of DLSS, it would produce the Ramen demo with precise pixel quality.

Its not, its a slop machine.

 

[Gonzito]

Well papitos, I gifted myself the premium edition of Halo for my birthday, expect many screenshots (without that crappy Chromatic Aberration)

 

[viveks86]

that's weird. either they did add that in the months since the announcement or they fucked up big time with their messaging before.

...or they try to be vague now and just mean that the AI "understands" the materials due to the training data.

but yes, that would at least keep it from thinking a shower curtain is a tree bark and shit like that, but not much more.
(I use this example because I saw an RE1 AI upscaled mod pack on PC a few weeks ago where the AI turned the shower curtains into a tree )

It's the messaging. The model would hallucinate like crazy without it because, unlike 8 second video generation, they can't afford to have the memory of a goldfish. Here's the actual exchange with Daniel Owen that is relevant to this topic:

And their answer:

It's a generic non-answer that makes the model look less competent. They could have simply said they are able to stay consistent by feeding in attributes available in the g-buffer, including but not limited to motion vectors, along with the pixel color data. "Analyzing a single frame" is too vague. The g-buffer is also representative of that single frame. Explicitly mentioning it would have helped improve the perception of this model for anyone with basic technical knowledge of rendering. Instead, they dug the hole even further. I think the person responding simply did not have exposure to the plumbing and was parroting marketing talking points.

Puzzling indeed. He even explicitly asks about 3D depth and they avoided the question. Would be crazy if the model doesn't use depth. It probably does that too. If not now, that seems like a much needed improvement down the line.

But like you said, none of this is sufficient for it to surpass results of PT. By changing any lighting or shadows, they have undone the hard work PT has put in. Wasted precious resources. No matter how much they claim they are "building on PT", it's a flat out lie. The fact that they have a slider that changes ambient occlusion/shadow intensity is proof of that. What exactly is causing more shadows that PT hasn't calculated already? It's just made up nonsense based on what the model thinks it should be. It's doing the AI equivalent of, "Oh this area feels like it needs more shadows based on all my training of God knows what. Let me impress by darkening it a bit". It's either that or they have actually slapped on SSAO in the name of "structural intensity".

Here's a clear example of smoke and mirrors. Does the first image look path traced? It's missing basic shadows and GI (see left side of first mushroom) without DLSS5. They've simply removed the items in focus - like the bacon, from the BVH so that they can make DLSS5 look better. You don't need a fancy AI solution to pull that off. You don't even need PT. Just RT shadows and RTAO will do. The DLSS5 shadows actually look incorrect, it's making the bacon appear to float. A properly path traced version would be somewhere in between the 2.

EDIT: Look at S setoman 's reference images above. Absolute madness that they would try to pull this shit off at Siggraph. Presentations there are supposed to be so honest and peer reviewed that it shouldn't even warrant cynicism. It's a sacred space where actual science and truth is all that matters. It's shameful what they are trying to get away with, if I'm being honest. Hoodwinking indeed.

 

[littlezoo]

I think FH6 is definitely underrated.
yes, GT7 car models are bit more detailed in Photo Mode, but considering the scale of the game, the collision physics, the level of environmental detail, the visual effects, and everything else\FH6 is without a doubt the best looking racing game right now.

 

[SlimySnake]

I think FH6 is definitely underrated.
yes, GT7 car models are bit more detailed in Photo Mode, but considering the scale of the game, the collision physics, the level of environmental detail, the visual effects, and everything else\FH6 is without a doubt the best looking racing game right now.

looks good but too clean and gamey. if it was going for a more photorealistic look, i could excuse the clean visuals but its fairly last gen looking in terms of assets and lighting. still a looker though.

Going to check it out when i sub for Halo in a week.

Well papitos, I gifted myself the premium edition of Halo for my birthday, expect many screenshots (without that crappy Chromatic Aberration)

Happy Birthday!

 

[Buggy Loop]

Spoiler: The anti-Vick method ;)

Sure. My point isn't that RR has to run before upscaling. In theory, you can do them concurrently, before or after. It's why AMD's RR is currently decoupled. Concurrent just happens to be the most robust version. And FSR 5 will likely go that route too. So you are right about that. But my point is that DLSS 5 can never run before RR. You simply cannot feed generative AI such noisy data. The model is trained to work on the final composited image as primary input and, apparently, select info from the g-buffer as control parameters. At that stage in the pipeline, as far as DLSS 5 is concerned, all the world space traced data (represented by the noisy image) is already consumed. The lighting and shading stage is complete. They reiterated several times, even on the new presentation, that the model is a compact one step diffusion model entirely reliant upon pixel space data. I don't know why you would even bring up light vectors when I'm talking about bounce lighting/GI. Are you expecting DLSS 5 to do a second lighting and shading pass by evaluating all the scene light sources (onscreen and offscreen) again? lol. If so, you have the wrong idea of what the model even does and are projecting your hopes and dreams on it.

You're the one that alluded it wouldn't know where to look at for lighting or understand the scene, not me. I don't think it needs to look at the coordinates of lights just like RR doesn't need to. If it came to be that they want to add some occlusion near hair and they want to be sure its coherent with lights? There's a ton of tricks to be done in pipeline to access that information.

Spatial awareness of light sources is basic basic stuff

Having some light accumulation, radiance data etc in the buffer does nothing to tell you where the light sources actually are, which is what you need to generate new lighting/shadowing. You think all the megalights info (100s or even thousands) in E-day is sitting in the g-buffer? You will only have the final accumulation and that isn't going to tell you where the light should be bouncing further for DLSS to accurately generate additional lighting/shadows

I did say that light accumulation can have its dedicated buffer for more complex scenes did I not?
Reading the buffer, any buffer and spatial awareness of light sources, again, basics of rendering since the past 2 decades, just that you need to feed AI the information.

Did I say it is out of the pipeline?

No, a ton of peoples on this forum did before and I recall conversations between you and me on that, I said in my @ post way before hey vivek, here's confirmation it's in pipeline.

This means it can iterate easily by adding parameters as they see fit to improve on it down the line, if there's a need. The whole "its a post processing instagram filter" crowd are dead.

I only said RR has more data to work with ie the noisy ray traced input, and that it would have to run before DLSS 5. That noisy ray traced input came from performing ray traced direct and/or indirect lighting or PT. Once that has been denoised and composited with the rasterized primary, any attempt to change that lighting again in the frame, such as adding shadows is not based on world space. To base it again on world space, you will have to re-evaluate light sources using the same methods that were already used during the lighting/shading phase, making it redundant and costly.

It's no different than all deferred lighting for the past decade that had no access to world space. AI will just have more accuracy on how it should look. It can edit pixel by pixel as it sees fit the final pass before post processing.

And like you pointed out, given RR is concurrent with upscaling, DLSS5 will have to run on the upscaled 4k image. It's why they could easily run the same solution on a dual gpu setup during development. Use one gpu as the traditional pipeline and then feed the upscaled image to DLSS 5. This is exactly how it will run come launch, except on 1 gpu with the requisite performance optimizations, instead of 2. I never doubted they will pull that part off. But don't expect some foundational change in workflow within months of production launch.

I'm not expecting foundational changes, DLSS 5 sits right where it need to be in pipeline for what they plan to achieve.

But I'm not even sure why you are fixated on RR in a conversation about DLSS 5. The goal of RR is simply to denoise an image. Not infer shadows or lighting that didn't even exist in the input data. RR is non-generative. DLSS 5 is generative (albeit constrained by some gbuffer parameters). Different use cases and goals. Is your argument that if RR is awesome, DLSS 5 will be awesome too, because there is access to a g-buffer? If the Apple I have is tasty, is my orange tasty too, because they grew on the same soil? G-buffer isn't some silver bullet that solves all rendering problems. It's a fraction of scene data focused entirely on what is needed for the current frame.

I'm saying that there's nothing that RR has access to that DLSS 5 wouldn't if needed be.

No it's not. I think your entire understanding of the rendering paradigm is stuck at deferred rendering.

Yes it is.

• ReSTIR DI, short for Reservoir SpatioTemporal Importance Resampling for Direct Illumination, is a screen space light sampling technique used for illuminating primary surfaces during path tracing. It cheaply selects several lights from an initial distribution, such as a uniform or power-based one, then selects one light of that group based on a more expensive distribution, such as one including geometry, BRDF, and visibility terms, and uses that one for illumination. These samples are then resampled over time and across neighboring pixels to increase effective sample count.
• ReGIR, short for Reservoir-based Grid Importance Sampling, is a world space light sampling technique that can be used for illuminating primary or subsequent surfaces. It samples lights from an initial distribution, such as a uniform or power-based one, for each grid cell. Other algorithms, such as ReSTIR DI and ReSTIR GI, can sample lights from this grid if the surface they are illuminating falls in a grid cell.
• ReSTIR GI, short for Reservoir SpatioTemporal Importance Resampling for Global Illumination, is a screen space light sampling technique used for illuminating secondary surfaces during path tracing. ReSTIR GI requires an initial light sampling technique be implemented by the path tracer for the secondary surface. In the full-sample project, this method is the initial sampling pass from ReSTIR DI, although any technique may be used. This secondary surface sample is then resampled across other pixels in space and time.
• ReSTIR PT, short for Reservoir SpatioTemporal Importance Resampling for Path Tracing, is a screen space path resampling technique used for illuminating secondary surfaces. ReSTIR PT requires an integration of the RTXDI_PathTracingContext into a path tracer that calculates indirect lighting starting from the primary hit.

Because 99% of games that even output any kind of fidelity out there are deferred rendering. It seems most peoples don't even know how that works. RTXDI of course followed the modern pipeline. Only ReGIR is not.

Hence, all the non sequitur defense of screenspace techniques. All hardware ray tracing happens on 3d data. The BVH has actual 3d triangle geometry.
In the not so distant future, the BVH will become the single source for all 3D triangle data in VRAM. Animations will occur by manipulating this data within the BVH so that there is no need for triangle duplication or lower resolution proxies. All RT and PT will continue being traced against this 3D "context". That is quite definitively NOT screenspace.

After the excursion in hardware RT, you basically identified the intersection currently requiring shader evaluation. Again, that shader change from the a hit of the ray on a triangle, changing its brightness, its color, will transfer to per-pixel screen-space in the rendering pipeline.. Just like deferred rendering where after the light pass is done, it updates corresponding pixels. How else is this done you think?

An AI thinking that it should change the values of X,Y amounts of pixels for photorealism is not gonna have "screen space defects" any more than any shader manipulation since the dawn of pixel shaders. including ray tracing, path tracing.

How it thought it should look like and what it can use for coherency with the info given is another topic and up to debate, can it have something wrong? I mean you can have a simulation taking AI that will be ~95% accurate of a brute force physics simulation while taking 1/1000th the time to get the result. Is that a defect? How about everything neural rendering coming up?

I'm not sure peoples understand how advanced AI has become

Estimating a video and extracting back the parameters much like a game pipeline does, you would ask peoples on the street and they wouldn't know its "fake". A game has a much better ground truth than video, and it's not gonna toy around to move the lighting outside of the ground truth like they're doing in the video, nor move the camera around to see what's beyond what it worked with in the viewport, but as showcased, AI aware of what can be behind the apple from just a 2D picture and is close to ground truth with convincing results. Are peoples gonna blink when its that close to ground truth? (path traced GT).

I think its really fucking cool tech.

 

[SlimySnake]

God bless Ubisoft. Yes it's the artists making everything look pretty, but would it look pretty if the programmers didnt upgrade the water? or add support for RT reflections? or upgrade their lighting so everything at the side of the cliffs just feels grounded despite not getting direct sunlight?

Despite all of their issues this gen, Ubisoft has delivered with every single next gen game.

 

[YeulEmeralda]

Nvidia turn all characters into cute girls
"unable to comply authorities notified"

 

[Cumulus]

Spatial awareness of light sources is basic basic stuff

I stopped reading right there...

Determining off-screen light sources isn't a basic problem; it's fundamentally underdetermined and literally one of the hardest issues in inverse rendering. "Learned lighting" is not the same as actual awareness...

Jesus.....

 

[Buggy Loop]

I stopped reading right there...

Determining off-screen light sources isn't a basic problem; it's fundamentally underdetermined and literally one of the hardest issues in inverse rendering. "Learned lighting" is not the same as actual awareness...

Jesus.....

what_year_is_this?.gif

If your engine is not keeping up with coordinate spaces, you have a fucking big problem coming up. Jesus indeed

I said spatial awareness, i.e. coordinates, of light sources, available in any struct following the game rendering. AI sees a scene, looks at light accumulation, if it thinks there's missing details one place, can corroborate by looking at the light struct. A simple sanity check to make sure its not adding details that should not be there. Not asking it to do a light pass to evaluate the light touching a surface or blocking. If it was blocked the light mapping would be dark, that process was done.

in the end, seems AI does not even need that for near ground truth path tracing as shown in the diffuser demo

 

[Cumulus]

what_year_is_this?.gif

If your engine is not keeping up with coordinate spaces, you have a fucking big problem coming up. Jesus indeed

I said spatial awareness, i.e. coordinates, of light sources, available in any struct following the game rendering. AI sees a scene, looks at light accumulation, if it thinks there's missing details one place, can corroborate by looking at the light struct. A simple sanity check to make sure its not adding details that should not be there. Not asking it to do a light pass to evaluate the light touching a surface or blocking. If it was blocked the light mapping would be dark, that process was done.

in the end, seems AI does not even need that for near ground truth path tracing as shown in the diffuser demo

Utter nonsense with lights outside the screen space...

 

[Gonzito]

I am tired boss

 

[SlimySnake]

I am tired boss

yeah, twitter is a safe haven for morons.

The funny thing is that ray tracing is being blamed for lack of physics when they were ditched at the start of the PS3 era 11 years before ray tracing was even revealed let alone widely adopted some 3 years ago. If anything ray tracing has allowed destructible environments to come back because now the shadows and gi can be calculated in realtime after assets are destroyed. Black Flag shows this really well. Sadly other games dont because its just not a priority for most devs. Never has been.

Just the last couple of days on twitter, i had to tell myself mid reply not to interact with these low IQ idiots. One guy was trashing the new Konami baseball game saying it looks worse than MLB The Show based off a shitty low res twitter video. I typically have high res gifs ready for that game, but i was like nah, why bother.

Then just today, this tweet of the Mafia Remake went viral saying they shouldnt have switched to the new engine and the embedded video showed the Mafia Remake in a rainy night where you cant even make out any detail. No one seems to understand lighting and asset quality on twitter.

As for the reflections. I think this is where DF and other journos have just failed the gaming population. When covering RT reflections, the first thing they shouldve compared it to was Half Life 2 and explained why we cant use it anymore without killing the framerate. but no, they just compared it to screenspace reflections from the PS4 era.

 

[Buggy Loop]

Utter nonsense with lights outside the screen space...

Holy shit

You're ignoring decades of rendering because you can't wrap your head around screen space. I'm wasting my time here.

Tell me how deferred rendering can have hundreds of light sources if you really think that lights are simply not an easy access anytime in the render pipeline? or they're gone from screen space because screen space = bad? Light 3D bounding spheres would just have to intersect with the camera's view frustrum and it'll calculate it.

LearnOpenGL - Deferred Shading

Learn OpenGL . com provides good and clear modern 3.3+ OpenGL tutorials with clear examples. A great resource to learn modern OpenGL aimed at beginners.

learnopengl.com

Here, an hint :



This is almost all of the past 2 decades of graphic rendering when its not using RT. Older technique than most people son this forum it seems. Jesus...

 

[Cumulus]

. I'm wasting my time here.

What I was just thinking.

If only light coordinates alone were sufficient, we wouldn't need to solve the rendering equation in the first place. .....
You're arguing against a point no one ever made while mixing up light enumeration and transport and you don't even realize that while playing professor, lol.

You should take a few steps back and think long and hard about why we're using RT nowadays as your understanding really seems to be stuck at deferred rendering.


Anyways,I'm not wasting my time any further with your nonsense.

 

[viveks86]

Spoiler: The anti-Vick method ;)

You're the one that alluded it wouldn't know where to look at for lighting or understand the scene, not me. I don't think it needs to look at the coordinates of lights just like RR doesn't need to. If it came to be that they want to add some occlusion near hair and they want to be sure its coherent with lights? There's a ton of tricks to be done in pipeline to access that information.

Spatial awareness of light sources is basic basic stuff



I did say that light accumulation can have its dedicated buffer for more complex scenes did I not?
Reading the buffer, any buffer and spatial awareness of light sources, again, basics of rendering since the past 2 decades, just that you need to feed AI the information.



No, a ton of peoples on this forum did before and I recall conversations between you and me on that, I said in my @ post way before hey vivek, here's confirmation it's in pipeline.

This means it can iterate easily by adding parameters as they see fit to improve on it down the line, if there's a need. The whole "its a post processing instagram filter" crowd are dead.



It's no different than all deferred lighting for the past decade that had no access to world space. AI will just have more accuracy on how it should look. It can edit pixel by pixel as it sees fit the final pass before post processing.



I'm not expecting foundational changes, DLSS 5 sits right where it need to be in pipeline for what they plan to achieve.



I'm saying that there's nothing that RR has access to that DLSS 5 wouldn't if needed be.



Yes it is.



Because 99% of games that even output any kind of fidelity out there are deferred rendering. It seems most peoples don't even know how that works. RTXDI of course followed the modern pipeline. Only ReGIR is not.



After the excursion in hardware RT, you basically identified the intersection currently requiring shader evaluation. Again, that shader change from the a hit of the ray on a triangle, changing its brightness, its color, will transfer to per-pixel screen-space in the rendering pipeline.. Just like deferred rendering where after the light pass is done, it updates corresponding pixels. How else is this done you think?

An AI thinking that it should change the values of X,Y amounts of pixels for photorealism is not gonna have "screen space defects" any more than any shader manipulation since the dawn of pixel shaders. including ray tracing, path tracing.

How it thought it should look like and what it can use for coherency with the info given is another topic and up to debate, can it have something wrong? I mean you can have a simulation taking AI that will be ~95% accurate of a brute force physics simulation while taking 1/1000th the time to get the result. Is that a defect? How about everything neural rendering coming up?

I'm not sure peoples understand how advanced AI has become

Estimating a video and extracting back the parameters much like a game pipeline does, you would ask peoples on the street and they wouldn't know its "fake". A game has a much better ground truth than video, and it's not gonna toy around to move the lighting outside of the ground truth like they're doing in the video, nor move the camera around to see what's beyond what it worked with in the viewport, but as showcased, AI aware of what can be behind the apple from just a 2D picture and is close to ground truth with convincing results. Are peoples gonna blink when its that close to ground truth? (path traced GT).

I think its really fucking cool tech.

Spoiler: Fine, I'll tag my wall too

You're the one that alluded it wouldn't know where to look at for lighting or understand the scene, not me.

You missed the keyword on what I said. Bounce lighting. After light bounces off a surface, RTGI/PT simulates light scatter for further bounces. My example was an off screen street light bouncing light off the ground to a store nearby. It is indirect scattered light. When light scatters in multiple directions, you can't be maintaining light vectors in buffers anymore. You could have several light sources contributing to the light of an individual pixel. Identifying light source at that point by just looking at final light accumulation is not a trivial task. Combine that with something like megalights where there could be hundreds of light sources contributing to a pixel and there would be no such thing as a per pixel light vector that you can store in a buffer. The only way to go back to identifying the light source is by ray tracing. And if you don't know the light source, there is no way to handle variations. Go back and look at how many light variations happen in the e-day demo. They move, turn on/off, flicker or change colors throughout the scene. If those light sources are offscreen (say your character has his back to it all), there is no way to deduce any of that from the g-buffer or any buffer for that matter. The AI will have no clue why there are blues, yellows and greens bouncing all over the place or what the origin is, because it doesn't know they are coming from flickering televisions stacked on each other offscreen. So what it does to all that lighting is completely up in the air.

Spatial awareness of light sources is basic basic stuff.

I did say that light accumulation can have its dedicated buffer for more complex scenes did I not?
Reading the buffer, any buffer and spatial awareness of light sources, again, basics of rendering since the past 2 decades, just that you need to feed AI the information.

Only in cases of direct lighting of a limited and fixed number of lights. That is not what I'm talking about. See above. No buffer will be sufficient for a "many lights" problem. It's why solutions like megalights or ReSTIR came to existence even though deferred rendering existed for decades. Without ray tracing, this can never be handled at scale.

I recall conversations between you and me on that, I said in my @ post way before hey vivek, here's confirmation it's in pipeline.

Yeah I recall too and I appreciate the tagging. And I'm telling you the gbuffer would have always been part of the equation. It just wasn't explicitly revealed to us. It doesn't change anything though. Like I said, my primary contention isn't about where DLSS 5 is in the pipeline. They can and should fit it wherever it makes sense. It is about the fundamental technique being used that will hold it back, and the disingenuous marketing around it.

Yes it is.

Dude, that's what I'm trying to explain to you. A screen space light sampling technique is not the same as a screen space only technique.

SSAO, SSR, Ray Reconstruction are all screen space only techniques. The g-buffer represents screenspace only as well. All the ray tracing techniques you highlighted use the screenspace to initiate the sampling. The tracing goes towards the entire scene as presented in the BVH. I don't know how else to explain this to you. Why do you think ray tracing exists at all if all that is needed is screenspace data?

. Just like deferred rendering where after the light pass is done, it updates corresponding pixels. How else is this done you think?

Of course. I never said otherwise. Ultimately all rendering outputs to screenspace. The point of contention is what information is used to output those pixels. Screenspace only techniques are limited to the g-buffer, which stores 2D pixel data and supplemetary data for the 2d pixels, like motion, depth, normals etc. RT techniques on the other hand, have access to the BVH ie 3d geometry.

Almost all rendering is meant for screenspace as there is no point rendering anything that is not on the screen. But when having debates like this, we need to differentiate between techniques that are screenspace only (does not or cannot use data outside of screenspace) and techniques that go beyond.

RR does not have to interpret anything outside screenspace. It does not have to infer lighting outside of what is already in the frame that it is currently denoising. Screenspace only is sufficient for RR. But it is insufficient for any system that is trying to introduce lighting/shadowing, such as DLSS 5.

I'm not a screenspace hater, mind you. But the right technique needs to be used for the right use case. Lighting/shadowing is the wrong use case for screen space only techniques. It's good enough for subtle effects like subsurface scattering. It's perfectly fine for RR. But not for adding lighting and shadows that don't already exist via RT/PT.

The reason SSAO and SSR have so many issues is not because they are bad techniques per se. They have issues because they are screenspace only. Occluded and offscreen data simply don't exist for them. DLSS 5 will have this issue as well. And no amount of fancy AI can fill in those gaps if it doesn't have access to the 3D scene, like the BVH. That's the fundamental flaw. Fixing this flaw requires a complete redesign. Which takes us back to my first response to you, so I'll not repeat myself.

And as you know, I'm not an AI hater either. So i'll refrain from commenting on the progress AI is making in this field. All that is not relevant to our specific debate and I'm just as excited as you are for the progress in other specific areas.

I will reiterate. Just because you saw the word "screenspace" for the description of RT techniques, do not club them with screenspace only techniques like SSAO or SSR. Apples and Oranges. Study how the actual ray traversal and intersections work. None of that is limited to the screenspace. You will see the ReGIR is not labeled screenspace for this reason. It is meant to calculate lighting for probe grids in the world ie world space. Not for final pixels on screen ie screenspace.

 

[Abysse]

 

[ChiefDada]

I am tired boss

Boy did he make sure to keep those reflected assets in screen space didn't he lol.

 

[T4keD0wN]

Boy did he make sure to keep those reflected assets in screen space didn't he lol.

Never played it, but isnt water planar in HL2?

Funny how they mention performance impact...

 

[alloush]

Just the last couple of days on twitter, i had to tell myself mid reply not to interact with these low IQ idiots.

Don't. As much as I love Twitter as an overall platform (it's the only social media platform I use) I try my absolute best not to engage.

A few days ago I engaged with this dude who I follow, who has a decent number of followers. I left him a serious, and actual comment under one of his posts about futbol trying to have a civilized conversation. I was told to go fuck myself followed by a kissing emoji. My genuine comment got like 19 likes and his comment telling me to go fuck myself with a kissing emoji got over 50.

Being the trash talker that I am, my ego couldn't let it go. I wiped the floor with him and his followers who showered me with comments as well till I got reported, then proceeded to delete my posts and promised I wouldn't engage with low-lifers anymore. I rarely engage with people on Twitter anyway, but recently been feeling like engaging, but now I know who to engage with and who to ignore, a quick glance at their profile helps!!

 

[SlimySnake]

new look. actual ingame cutscene model this time.

 

[viveks86]

Never played it, but isnt water planar in HL2?

Funny how they mention performance impact...

Indeed. Planar for all water. It's basically a second camera rendering everything again and projecting all geometry back in the scene. Has no impact on performance whatsoever. None. Nada. R* should take notes and use it in GTA6 so we can all play the game at 15 fps. Maximum graphics!

new look. actual ingame cutscene model this time.

Well, we all complained the men looked like super models and only the women looked like old hags. So Insomniac has answered our prayers. Equal treatment for them all!

 

[Luipadre]

new look. actual ingame cutscene model this time.

5 years for this game. I bet it wont be longer than 10 hours. Modern AAA gaming...

 

[Abysse]

new look. actual ingame cutscene model this time.

He has low testosterone.

 

[T4keD0wN]

Indeed. Planar for all water. It's basically a second camera rendering everything again and projecting all geometry back in the scene. Has no impact on performance whatsoever. None. Nada. R* should take notes and use it in GTA6 so we can all play the game at 15 fps. Maximum graphics!

Imagine flat water in GTA6.

 

[viveks86]

Imagine flat water in GTA6.

Paging GymWolf to imagine it. RIP

 

[SlimySnake]

Boy did he make sure to keep those reflected assets in screen space didn't he lol.

its not screenspace. they are basically rendering the world twice. ND still does it for their mirrors. James Bond did it recently as well. Its extremely expensive to do it for bodies of water which take up far more space than small bathroom mirrors.