Let's Talk TAAU (Temporal Anti-Alias Upscaling)

[rainless]

For starters... I know absolutely nothing about it. But according to some leak rumors on the PS5 Pro... even the latest version of DLSS requires per-game training while the PS5 Pro will not.

PS5 Pro Was an Open Secret at Gamescom 2024

Wccftech can confirm that PS5 Pro was an open secret at Gamescom 2024, having also heard that PS5 Pro specs have been shared with developers.

wccftech.com

PSSR is an ML-enhanced version of Temporal Anti-Aliasing Upscaling (TAAU) that requires inputs similar to NVIDIA
DLSS or AMD FSR and fully supports High Dynamic Range (HDR) pipelines. It also does not require per-game training like the latest version of DLSS.

How does this per-game training work? And can DLSS evolve to not require it as well? Or is this something unique to TAAU?

 

[Kenrou]

"...the latest version of DLSS." - do they mean only the latest version, aka 3.7.20 that came out on the 19th? Further info would also be appreciated, been using it on Cyberpunk, very nice so far.

 

[rainless]

"...the latest version of DLSS." - do they mean only the latest version, aka 3.7.20 that came out on the 19th? Further info would also be appreciated, been using it on Cyberpunk, very nice so far.

You have all the info that I have. I both quoted the article... and posted a LINK to the article that I quoted. That's all I know.

 

[Kenrou]

You have all the info that I have. I both quoted the article... and posted a LINK to the article that I quoted. That's all I know.

Chill, I meant from the links, this looks like the type of half-baked stuff I come up with on most of my links

EDIT: Also for future posters that know more than us

 

[rainless]

Chill, I meant from the links, this looks like the type of half-baked stuff I come up with on most of my links

EDIT: Also for future posters that know more than us

I'm ALWAYS chill... Earthdog used to call me "Mr. Laidback"....

Yeah I have no clue what they're talking about. Everybody's trying to cash-in on AI technobabble... but I think the "boom" is gonna go "bust" imminently if it hasn't already.

All this nonsense about "AI's gonna change the world!"

...so far it's only enabled people to make fake movie trailers realllly fast...

As a post-production special effects producer... people still like a real human at the helm of their expensive film projects.

 

[Dolk]

Pre-training and no training are just two different inputs to the same machine learning algorithm. All machine learning algorithms are based off "training data", but the application of that algorithm can be biased to include further details or no details. That's the simple break down of ML TAA vs DLSS v3.

DLSS v1 and FSR use no training to produce their image while DLSS v3 uses training to produce the image. I guess Sony wants their own version of a ML TAA so that it can be more compatible with their hardware and an easier drop in for their developer tool chain. Probably a good thing, but AMD should be just as easy but since AMD has been slow to deliver better FSR features, I guess Sony got tired of waiting is trying to move quicker.

 

[rainless]

Pre-training and no training are just two different inputs to the same machine learning algorithm. All machine learning algorithms are based off "training data", but the application of that algorithm can be biased to include further details or no details. That's the simple break down of ML TAA vs DLSS v3.

DLSS v1 and FSR use no training to produce their image while DLSS v3 uses training to produce the image. I guess Sony wants their own version of a ML TAA so that it can be more compatible with their hardware and an easier drop in for their developer tool chain. Probably a good thing, but AMD should be just as easy but since AMD has been slow to deliver better FSR features, I guess Sony got tired of waiting is trying to move quicker.

Where the hell have YOU been??! The man with all the answers!

We need like twenty more of ya!

 

[mackerel]

DLSS v1 and FSR use no training to produce their image while DLSS v3 uses training to produce the image.

Maybe my memory is off since it was a long time ago, but I thought DLSS 1 did require each game to be specifically trained. DLSS 2 and later did not require specific per-game training.

Training is only needed for AI based methods, so no current version of FSR needs it. XeSS would.

 

[Dolk]

Let me go do a refresher too, but my understanding is that they all require their initial training (ML Algo creation first requires initial training data). But the newest ML TAAs allow runtime learning to be applied to the ML Algo so that you get a better image quality.

Edit:

I did some quick reading and I believe I'm still correct. FSR, DLSSv1/2/3, XeSS and DLAA all require training back at an HQ and have their applied algorithms adjusted based on the game you are playing. All this knowledge is kept in drivers. Some of these require targeted game training to work better, others can use arbitrary data to get their results. DLSSv3 and DLAA can adjust their training data on the fly based on what data is being presented.

So this PSSR will probably be a clone of FSR3/DLSS2 as it will require no specific game training but will probably have more in game learning and adjusting similar to DLSSv3/DLAA. (Speculation)

 

[Evilsizer]

i am pre-training to be a body builder, but i never leave my desk. Can we pretend we talked about it but don't?

 

[rainless]

Let me go do a refresher too, but my understanding is that they all require their initial training (ML Algo creation first requires initial training data). But the newest ML TAAs allow runtime learning to be applied to the ML Algo so that you get a better image quality.

Edit:

I did some quick reading and I believe I'm still correct. FSR, DLSSv1/2/3, XeSS and DLAA all require training back at an HQ and have their applied algorithms adjusted based on the game you are playing. All this knowledge is kept in drivers. Some of these require targeted game training to work better, others can use arbitrary data to get their results. DLSSv3 and DLAA can adjust their training data on the fly based on what data is being presented.

So this PSSR will probably be a clone of FSR3/DLSS2 as it will require no specific game training but will probably have more in game learning and adjusting similar to DLSSv3/DLAA. (Speculation)

Okay so this PSSR is just Sony's version of DLSS3. And FSR3/DLSS/PSSR are all TAAs? That clears it up nicely.

Like I was saying companies LOVE to just throw word salad at you to try to convince you that something in there is DEFINITELY worth you spending another thousand bucks.

Now it makes a lot more sense.

THANK YOU.

 

[Dolk]

Yep you got it.

Also you never know. Big tech companies will try out big bets because if they pay off, those returns are MASSIVE. Yet Sony hasn't been doing well in big bets recently (looking at you Concord).

 

[]-[itman]

Let me go do a refresher too, but my understanding is that they all require their initial training (ML Algo creation first requires initial training data). But the newest ML TAAs allow runtime learning to be applied to the ML Algo so that you get a better image quality.

Edit:

I did some quick reading and I believe I'm still correct. FSR, DLSSv1/2/3, XeSS and DLAA all require training back at an HQ and have their applied algorithms adjusted based on the game you are playing. All this knowledge is kept in drivers. Some of these require targeted game training to work better, others can use arbitrary data to get their results. DLSSv3 and DLAA can adjust their training data on the fly based on what data is being presented.

So this PSSR will probably be a clone of FSR3/DLSS2 as it will require no specific game training but will probably have more in game learning and adjusting similar to DLSSv3/DLAA. (Speculation)

DLSS v1 required training on a per game basis, DLSS v2 and later do not. I'm sure they are constantly updating the training with new games which can improve the visual fidelity, but it is not required that DLSS be trained on a specific game's assets after DLSS v1. DLSS v2 completely changed the way DLSS is done, they honestly shouldn't even be called the same thing but the branding was working so NV stuck with it.

This is also why people can mod DLSS into games now that don't officially support it and FSRR can be added to any game that supports DLSS. You just need to make sure the motion vectors are available in the game, as well as a couple other minor things, and you can get DLSS working.