So Standard/Native > Checkerboarding > Upscaling if I'm looking at the pictures right?
Rainbow Six Siege. Quantum Break also used a form of it IIRC.
Incorrect, Rainbow Six, Killzone, Quantum Break they each use different techniques....And none of them are similar to checkerboarding and none of them are upscaling in the sense that we commonly referred to as.It's more that it's a relatively new idea than anything. Killzone Shadowfall and Rainbow Six Siege both use it, I believe.
Siege's MSAA reconstruction doesn't have a temporal element though does it? I mean as far as I know it doesn't use data from previous frame for the reconstruction of current frame but instead uses data from MSAA samples. Siege's technique is more similar to how the PS3 R&C games achieved 720P than Kilzone or QB imo.How is what Rainbow Six Siege does significantly different functionally to what is described in the OP?
The way I see it, using 2xMSAA samples is an implementation technique for checkerboarding.
Also, all of these techniques have some temporal reconstruction component, so I don't see your point w.r.t. Quantum Break.
It would be interesting if the GPU manufacturers were able to do something similar to DSR and implement a more universal checkerboard rendering style into games.
Siege's MSAA reconstruction doesn't have a temporal element though does it? I mean as far as I know it doesn't use data from previous frame for the reconstruction of current frame but instead uses data from MSAA samples.
Please read the presentation I linked earlier in this thread. It absolutely does.Siege's MSAA reconstruction doesn't have a temporal element though does it?
not exactly the same reduction in rendering cost, since you have to check against the old buffer by reading the velocity buffer and then probably sample the neighborhood to reject further false pixels, too.
Yes, upscaling is far, far cheaper per pixel than reprojection. My point with the "same cost" comment was comparing the entire pipeline of the two examples: 1800p upscaled versus 2160c.Well, it's usually not at the same cost, reconstruction techniques tend to be significantly more expensive than just running the game at a lower res.
But we're talking about very fine details indeed--by definition no larger than a single pixel. Even summed across an area, the effect will be perceptually infinitesimal. On a 4K display, my 10x10 examples would occupy about a thousandth of a percent of the area! And even then, most of the CBR pixels are identical to native rendering.Nice illustrations. Gives a good view of the finer details lost with checkerboard rendering.
All resolutions are achievable using CBR. (Technically, the resolutions have to be even, since the workload is split in half.) There is no "finished frame" that CBR is applied to, like there is with upscaling. CBR makes the finished frame.Any chance you can add something in to explain what resolutions are derivable using CBR?
I think that last part, while true, is misleading. Most people will read that as "1800c is the same as 1273p on a 4K display". But that is definitely not the case; the CBR frame will be miles sharper and more accurate.When games want/need to render even fewer pixels, many of them seem to do checkerboarding and then traditional upscaling on top of that.
E.g. rendering at 1800c and then upscaling to 2160p. (which is actually shading roughly as many samples as 1273p)
I feel like that is "wobbly, simplified language" leading to an inaccurate interpretation.The latter is how the false statement "CBR renders half as many pixels as native" got started. In fact, it renders exactly as many pixels as native. Just half of them are rendered in a quicker, less accurate way.
I obviously disagree. My point may be clearer if phrased as a question: if CBR only renders half the pixels, where do the other half come from?I feel like that is "wobbly, simplified language" leading to an inaccurate interpretation.
What you're describing here is basically exactly what's described in the OP, except alternating lines instead of a checkerboard pattern.Killzone Shadowfall: Uses temporal reprojection (only for MP). It first renders an image where only 960 of the vertical lines out of the 1920 in a 1080P image are rendered in an alternating fashion. And then for the subsequent frame it renders the other 960...effectively making it a vertically interlaced image. But after rendering the 2nd frame it uses the data obtained from the first frame and runs it through an approximation algorithm to generate the remaining 960 vertical pixels. There is zero upscaling here and you actually get a native 1080P image. The downside is that due to the nature of it relying on an approximation algorithm there is artifacting when in motion, this artifacting looks like interlacing/dithering. When still the image is a flawless 1080P image.
Well, sort of. I'm pretty sure the game still falls back to spatial upscaling for when low-confidence is detected; areas with high-frequency geometry discontinuities along the horizontal axis often look very much like they're rendered horizontally half-res, for instance.There is zero upscaling here
"The other half comes from a combination of reprojection of previously rendered pixels, and interpolation of the currently rendered ones[, depending on the reliability of the reprojection]."I also believe both statements are accurate. If you disagree, what response to my question would you propose instead?
Your suggestion is precise, and in the context of a conversation about the technical process is exactly what I'd say."The other half comes from a combination of reprojection of previously rendered pixels, and interpolation of the currently rendered ones[, depending on the reliability of the reprojection]."
That's still just a one line sentence (even with the optional part). I don't think everything needs to be broken down to a single verb, especially one that already has a pretty well-defined and distinct meaning in this context.
It's true that how CBR works is the focus of my post, but it's not the sole focus. I'm also trying to give something of the flavor of how it compares to the usual approach. So while I'm massively simplifying the standard process--more so than CBR--I'm very reluctant to reduce it further still. What you've proposed is simply three frames: 1.Meshes 2.Rendering 3.UI, which teeters toward "here a miracle occurs" in the middle there.It seems odd to separate texturing, shading and lighting when they are really all part of the same shading process. ...Given that you're already simplifying the meshes section so much, it seems fine to combine the issue of pixel colouring into one part.
I'd probably remove the part about particles - it seems a little redundant.
My thinking was that a triangle with pixel center coverage but not majority pixel coverage should be a low percentage of cases. But thinking through the possibilities, it may be more common than I initially surmised...common enough to make my usage misleading at least.In regards to rasterisation, the majority vote comment is incorrect. In the most basic scheme, a triangle will contribute a pixel if the triangle contains the middle of the pixel. Whether that ends up being the pixel you see most typically depends on whether another triangle contributes a pixel which is nearer the camera, in which case that will be used instead.
For some reason I had a brain fart and forgot that checkerboarding is actually based on the KZ Shadowfall's reprojection...just with a different pattern.What you're describing here is basically exactly what's described in the OP, except alternating lines instead of a checkerboard pattern.
Well, sort of. I'm pretty sure the game still falls back to spatial upscaling for when low-confidence is detected; areas with high-frequency geometry discontinuities along the horizontal axis often look very much like they're rendered horizontally half-res, for instance.
Thanks I'll have a look at it when I can, although looking at the slide you posted and the one above yours I guess it does use temporal component.Please read the presentation I linked earlier in this thread. It absolutely does.
Here's one slide:
It's extremely close to what is discussed in the OP (with more details).
It's more random-ish. Less likely to miss/badly-alias patterned scene content due to sample pattern.One question while we are at it, is there any advantage to using a checkerboard pattern instead of alternating vertical lines or vice versa? Just curious over what actually led to them deciding on that pattern.
How frequent the distinction is relevant depends on things like triangle/object thickness. If only one edge of a triangle exists in a pixel, then majority-coverage and center-coverage are equivalent. Otherwise, no:My thinking was that a triangle with pixel center coverage but not majority pixel coverage should be a low percentage of cases. But thinking through the possibilities, it may be more common than I initially surmised...common enough to make my usage misleading at least.
In addition to what HTupolev said, checkerboard also means any individual artifact will only be a single pixel. Though this could result in a sawtooth or--no surprise--a checker error, those may still be less perceptible than longer lines (such as Shadow Fall sometimes suffers).One question while we are at it, is there any advantage to using a checkerboard pattern instead of alternating vertical lines or vice versa? Just curious over what actually led to them deciding on that pattern.
What's the overhead like in the ID buffer and vector tracking of the pixels (steps 3 & 4)?
The only data about this I know of is in the Rainbow 6 Siege presentation that Durante linked in the thread. Their implementation does differ from current versions, so isn't a perfect guide. But they said their CBR took 1.4ms, versus 9.4 to 11.4ms without. (More advanced versions like the one I present in the OP likely don't save quite so much.)Using a time to newly rendered pixel as a baseline, what is the reproduction cost? 10%? 5%?
Actually, majority coverage can still occur when two edges (or even three) are within the pixel, provided only one edge is fully within:How frequent the distinction is relevant depends on things like triangle/object thickness. If only one edge of a triangle exists in a pixel, then majority-coverage and center-coverage are equivalent. Otherwise, no:
Not that I'm aware of. PC is the only platform where all approaches are available on the same piece of software, and there's only a handful of such titles. It's why I had to construct my examples from scratch.Is there some sort of video comparison of full, CB, and upscaling for a real scene available that could show the differences and synthesize everything?
Absolutely. Durante and I differ on how to label that second path, but you've captured what's going on.Is it fair to say that checkerboarding provides 50% newly rendered pixels each frame plus a 50% mixture of previously-rendered-and-reprojected pixels along with some portion of scaled pixels to determine the final frame.
I get it (almost) from console/PC output point of view,
but does the image we actually see on our TV screen not undergo further processing by the TV itself, eg. some approximation of pixels based on motion, or indeed upscaling a potentially checkerboarded 1440p/1800p to 2160p?
I didn't say they couldn't occur, just that they aren't equivalent questions. When only one edge exists within a pixel, whether a triangle "covers the majority of a pixel" and whether it "covers the center of a pixel" are exactly the same question. With multiple edges in the pixel, we can construct cases (such as the one I posted) where the first is false and the second is true.Actually, majority coverage can still occur when two edges (or even three) are within the pixel, provided only one edge is fully within:
This, also - what about the (unfortunately) large percentage of people who haven't disabled overscan on their TV's?
Does the fact that the TV zooms in on the image by 5% fuck all this up?
*Don't ask me why people don't take the 2.8 seconds to disable overscan on their TV's settings to get a perfect pixel mapped image* I wonder that myself.
Okay, understood now, and definitely correct.I didn't say they couldn't occur, just that they aren't equivalent questions. When only one edge exists within a pixel, whether a triangle "covers the majority of a pixel" and whether it "covers the center of a pixel" are exactly the same question. With multiple edges in the pixel, we can construct cases (such as the one I posted) where the first is false and the second is true.
It's not really like interlacing. There are a couple major ways they're different, not just the pattern they use:So, what I'm gathering is that it's kind of like interlacing, in concept? Is there a trade off when fast motion is concerned, then?
Again, I'd avoid the term "interlacing", which is misleading. But your general description is correct: every other pixel is rendered normally, and this checker is used for both the current frame, and as a source for the reprojected pixels in the next frame.So, at a really easy to understand level, checkerboarding is almost like a modern equivalent of interlacing, however rather than draw half the lines in one pass and then other half in the next this new method
Am I along the right lines here or am I well out?
- Renders half the pixels for the next frame (1,3,5,9,etc) in the first pass
- Uses the pixel data for 2,4,6,8,etc from the previous frame for the remaining pixels in the second pass
So, then, the difference between this and interlacing (pattern aside), is that this holds half the pixels from the previous frame to use for the next and renders the full frame, whereas interlacing just draws half the lines per frame. I can see how this would eliminate the dimness of old interlacing techniques, but wouldn't really fast motion still cause some artefacts, as the holdover pixels wouldn't be the right colors? In fact, wouldn't ghosting be even more prominent in this case, since the holdover pixels are fully lit?Okay, understood now, and definitely correct.
It's not really like interlacing. There are a couple major ways they're different, not just the pattern they use:
- Interlacing sends half the pixels to the display, then the other half. CBR sends full frames only; the viewer can never detect missing data.
- Interlacing costs exactly the same as normal rendering to create a full frame. CBR costs less than that.
Speed of motion isn't really a problem, and won't necessarily create artifacts even at high velocity. What undermines CBR is unpredictable motion, like a tumbling object or something that speeds up and slows down erratically.
Again, I'd avoid the term "interlacing", which is misleading. But your general description is correct: every other pixel is rendered normally, and this checker is used for both the current frame, and as a source for the reprojected pixels in the next frame.
The general answer is that reprojection can produce errors, especially artifacts; CBR is not as accurate as standard rendering. However, it's much closer than you're thinking. This is for two reasons.So, then, the difference between this and interlacing (pattern aside), is that this holds half the pixels from the previous frame to use for the next and renders the full frame, whereas interlacing just draws half the lines per frame. I can see how this would eliminate the dimness of old interlacing techniques, but wouldn't really fast motion still cause some artefacts, as the holdover pixels wouldn't be the right colors? In fact, wouldn't ghosting be even more prominent in this case, since the holdover pixels are fully lit?
Speed of motion isn't really a problem, and won't necessarily create artifacts even at high velocity. What undermines CBR is unpredictable motion, like a tumbling object or something that speeds up and slows down erratically.
I'm asking myself if CBR can't also be used for speeding up some global
illumination? For, let's say we cover the sphere (over which we will
integrate) with a set of well chosen points (not random) and compute the
illumination over half of these points (in a checkerboard fashion) while
backward integrating the other half from a previous frame, like in classic
CBR, yet all done on the sphere, then we would only need half the rays saving
about half the computation. Anyone?
I am tragically underqualified to answer this, but isn't this already kind of done for realtime GI? Voxelization (as in SVOGI) seems at some level to be the same kind of lower-precision approach, treating chunks instead of points. But I could be very, very far off here.I'm asking myself if CBR can't also be used for speeding up some global
illumination? For, let's say we cover the sphere (over which we will
integrate) with a set of well chosen points (not random) and compute the
illumination over half of these points (in a checkerboard fashion) while
backward integrating the other half from a previous frame, like in classic
CBR, yet all done on the sphere, then we would only need half the rays saving
about half the computation. Anyone?
Yes, my understanding is that temporal accumulation methods in general seem to have started with shadow improvements. Here's a paper from 2007 about an early implementation. Over the years, temporal data has been added to other effects, such as AA. From what I know, Guerrilla Games with Killzone: Shadow Fall were the first to use it for scene geometry, Ubisoft Montreal improved it with Rainbow 6 Siege, and various Sony teams (based on ICE Team? SN Systems?) have also continued refining it.Didn't insomniac also used temporal data to render shadow maps?
Seems like it
Cascaded shadow map scrolling
Pretty much this, yeah, also great work OP!So Standard/Native > Checkerboarding > Upscaling if I'm looking at the pictures right?
Yes, my understanding is that temporal accumulation methods in general seem to have started with shadow improvements. Here's a paper from 2007 about an early implementation. Over the years, temporal data has been added to other effects, such as AA. From what I know, Guerrilla Games with Killzone: Shadow Fall were the first to use it for scene geometry, Ubisoft Montreal improved it with Rainbow 6 Siege, and various Sony teams (based on ICE Team? SN Systems?) have also continued refining it.
Is checkerboard exclusive to PS4 Pro or can PC, Xbone and Switch use the same or do they need to implement it differently?
Is checkerboard exclusive to PS4 Pro or can PC, Xbone and Switch use the same or do they need to implement it differently?
It's hardware. Here's a direct quote from Mark Cerny:What i think is new is the geometry index buffer not sure if the Ps4 Pro has built in hardware that generates the buffer or Sony just provides an optimized shader/pipeline for developers to use.
Mark Cerny said:Our solution to this long-standing problem in computer graphics is the ID buffer. It's like a super-stencil. It's a separate buffer written by custom hardware that contains the object ID.
I'm hardly a great source due to lack of expertise, but I think Watch_Dogs 2 is also using checkerboard on Xbox One. However, DF believes these artifacts are due to HRAA, so I could be wrong.The basic techniques can certainly be used elsewhere, and indeed Rainbow Six Siege appears to use the technique across all platforms.
The ID buffer isn't just for artifact elimination at edges. It pushes better accuracy of all reprojection, since the relative positions of identified polygons are known.The only aspect of the PS4 Pro we have explicit public hints about is the use of an ID buffer with some hardware support to reduce or eliminate artifacting at object edges when a game is designed to take advantage of it.
It's hardware. The ID buffer isn't just for artifact elimination at edges. It pushes better accuracy of all reprojection, since the relative positions of identified polygons are known.
The ID buffer isn't just for artifact elimination at edges. It pushes better accuracy of all reprojection, since the relative positions of identified polygons are known.
Incorrect, Rainbow Six, Killzone, Quantum Break they each use different techniques....And none of them are similar to checkerboarding and none of them are upscaling in the sense that we commonly referred to as.
But I definitely disagree that "render" has a distinct meaning that disqualifies my more conversational option. The word is already used for a vastly variable array of methods. Forward, deferred, raytracing, signed distance fields--why is this panoply all properly "rendering" but reprojection isn't?
Of course there's got to be far, far more detailed application tools available to developers, but for the public the closest thing to a technical source has been that official talk in mid-October. It's likely that Cerny went into some specifics, but most invited news outlets did a whole lot of summary and paraphrasing, so details are sparse on the ground. The Eurogamer report I linked is the most in-depth I'm aware of, with Richard Leadbetter thankfully including plenty of direct quotes. Here's just a little bit more about the ID buffer from Cerny:Is there a definitive source with more information on the nature of the ID buffer and its recommended usage?
Mark Cerny said:As a result of the ID buffer, you can now know where the edges of objects and triangles are and track them from frame to frame, because you can use the same ID from frame to frame.
Richard Leadbetter said:It's all hardware based, written at the same time as the Z buffer, with no pixel shader invocation required and it operates at the same resolution as the Z buffer.
So the same ID buffer excels at reprojection and is likely why the Pro is able to make such a huge impact on the IQ of VR games compared to standard PS4?
My takeaway is that the ID buffer doesn't replace motion vector reprojection. Rather, it enhances the process by being a very strong "reality check" on reprojection results, giving better confidence levels across the board. As the OP diagram explains, higher confidence means less blending.Interesting. Are you saying that it completely replaces the per-pixel motion vector used in earlier checkerboard rendering implementations?
So if I'm understanding correctly, your definition is something like this:Of course, this gets messy when you throw in modern temporal AA solutions, but even then, the base for the pixel information in a natively rendered image is uniquely sourced each frame ("rendered")....
Rendering is the process of obtaining a pixel value for display,
[U]using only information from the current frame,
[I]unless we're talking about accumulation methods, in which case at least some data has to come from the current frame[/I][/U].
Rendering is the process of obtaining a pixel value for display.
They are using more present information (vector motion, confidence) than upscaled pixels, but the base for the pixel information is from the previous frame.
Rendering is the process of obtaining a pixel value for display,
[U]using only information from the current frame,
[I]unless we're talking about accumulation methods, in which case the initial pixel value has to come from the current frame[/I][/U].
Nah, it's different, yet can be used for SVOGI as well.I am tragically underqualified to answer this, but isn't this already kind of done for realtime GI? Voxelization (as in SVOGI) seems at some level to be the same kind of lower-precision approach, treating chunks instead of points. But I could be very, very far off here. ...