GPU Testing Procedure 2024 – It’s all about that Ray Tracing and DLSS/FSR

With new video cards set to release by the end of the calendar year (2024), we’ve updated our testing suite to new games and benchmarks to bring you performance metrics from various titles using the latest technologies from AMD, Intel, and Nvidia.

This article will serve as a reference for those who would like to compare results with our reviews, whether you are writing an article for the site or just testing the performance of your new GPU. The plan is to update the article as changes in our testing procedures occur. On to the gritty details…

Test Platform

(Updated: 3/2024)

Our test systems consist of mainstream parts since most people will use them. This allows our results to be relevant to a wider audience and allows that audience to repeat our tests to compare with their systems. The required parts and clocks are as follows:

Synthetic Benchmarks

(Updated: 3/2024)

Synthetic benchmarks are very consistent and easily repeatable, making them some of the best tests for direct card-to-card comparisons. We have dropped support for Unigine Heaven and Valley and will continue with 3Dmark Fire Strike Extreme and Time Spy, along with Port Royal, where applicable.

UL 3DMark Benchmark Link

3DMark – Fire Strike Extreme (DX11 benchmark)
3DMark – Time Spy (DX12 benchmark)
3DMark – Speedway (DX12 Ultimate benchmark)
3DMark – Port Royal (Real-time ray tracing benchmark)

• Run at default settings (you do not need to run the demo)

3DM FSE

3DM PR

3DM SW

3DM TS

Unigine Superposition

For 2024, we’ve increased the difficulty in this benchmark, moving up a more stressful “4K Optimized “setting from the 1080p High we used since it was introduced in 2020.

Game Benchmarks

(Updated: 3/2024)

We have updated our gaming suite to remove some of the older titles. Now, all of the benchmarks are “canned” tests, making them as consistent as possible and more synthetic-like than typical gameplay. While this may not be the most accurate reflection of in-game performance, most are in the ballpark. The idea here is to show the relative performance and relative performance difference between the tested cards.

We test with the still most common (according to Steam statistics) resolution of 1920 x 1080, 2560 x 1440, and 3840 x 2160 using the canned/default Ultra settings and any manufacturer-specific features disabled/minimized where possible. Ray tracing and DLSS/FSR are tested at 2560 x 1440 and 3840 x 2160 with DLSS/FSR set to balanced and ray tracing to high or ultra.

Below, we’ve provided several screenshots of the video and graphics settings.

Assassin’s Creed: Mirage

AC:M should be set up as follows using the integrated benchmark:

• Ultra High Preset
• Resolution scale to 100%
• Adaptive Quality = OFF
• Upsample Type = TAA

…with DLSS/FSR

• DLSS, or FSR
• Upsample Quality to Balanced

Assasin’s Creed: Mirage Settings

Assasin’s Creed: Mirage Settings

Assasin’s Creed: Mirage Settings

Assasin’s Creed: Mirage Settings

Assasin’s Creed: Mirage Settings

F1 2023

We use the integrated benchmark using the Las Vegas track with Clear/Dry (default) settings:

…without RT:

• Ultra High Detail preset
• RT Shadows/Reflections/AOccl/Trans Reflections/DDGI = Off (for non-RT runs)
• Anti Alaising – TAA Only (for non RT runs)

…with ray tracing:

• Ray tracing – RT Shadows/Reflections/AOccl/Trans Reflections/DDGI = On

…RT + DLSS /FSR:

• Anti Aliasing – Nvidia DLSS/AMD FSR2 , Anti Aliasing Mode = Balanced

F1 23 – Settings

F1 23 – Settings

F1 23 – Settings

F1 23 – Settings

F1 23 – Settings

F1 23 – Settings

Avatar

Integrated benchmark

…without RT:

• Ultra settings to start
• Scaling Mode = Off
• Sun Contact Shadows = Off
• Specular Reflections = Very Low (no off)
• Diffuse Reflections = Low (no off)

…with ray tracing:

• Scaling Mode = Off
• Sun Contact Shadows = High
• Specular Reflections = Very High
• Diffuse Reflections = High

…RT + DLSS /FSR:

Temporal Upscaler = Nvidia DLSS or AMD FSR 3

• Scaling Mode = On
• Scaling quality = Balanced

Avatar Settings

Avatar Settings

Avatar Settings

Avatar Settings

Avatar Settings

Avatar Settings

Avatar Settings

Avatar Settings

Avatar Settings

Cyberpunk 2077

Integrated benchmark

…without ray tracing/DLSS:

• Quick Preset = Ultra (turns to custom)
• Resolution Scaling = Off
• Ray Tracing Reflections/Sunshadows/Local Shadows = Off
• Path Tracing/Tracing in photo mode = Off

…with ray tracing:

• Quick Preset = RT Ultra (turns to custom with DLSS/FSR)
• RT Lighting = Ultra
• Ray Tracing Reflections/Sunshadows/Local Shadows = On
• Path Tracing/Tracing in photo mode = Off

…RT + DLSS/FSR:

• Resolution Scaling = DLSS or FSR
• DLSS/FSR Quality = Balanced

Cyberpunk 2077 – Settings

Cyberpunk 2077 – Settings

Cyberpunk 2077 – Settings

Cyberpunk 2077 – Settings

Cyberpunk 2077 – Settings

Cyberpunk 2077 – Settings

Cyberpunk 2077 – Settings

Cyberpunk 2077 – Settings

Far Cry 6

Integrated benchmark settings are as follows:

…without ray tracing:

• Graphics Quality = Ultra
• HD Textures = On
• Anti Alaising = TAA
• DXR Reflections/Shadows = Off

…with ray tracing:

• DXR Reflections/Shadows = On

Far Cry: 6 – Settings

Far Cry: 6 – Settings

Far Cry: 6 – Settings

Far Cry: 6 – Settings

Far Cry: 6 – Settings

Shadow of the Tomb Raider

Run the integrated benchmark using the following settings:

…without ray tracing:

• DX 12
• Graphics Preset = Highest (turns to custom)
• Ray Traced Shadow Quality = Off

…with ray tracing:

• Ray Traced Shadow Quality = Ultra
• Nvidia RTX DLSS = Off

… RT + DLSS:

• Ray Traced Shadow Quality = Ultra
• Nvidia RTX DLSS = Balanced

Shadow of the Tomb Raider – Settings

Shadow of the Tomb Raider – Settings

Shadow of the Tomb Raider – Settings

Shadow of the Tomb Raider – Settings

Additional Testing

(Updated: 3/2024)

The following are a few things that will provide additional value and detail to the testing.

Temperature

We are measuring typical temperatures to expect and testing cooler performance.

• Record ambient temperature in degrees Celsius (preferably close to the intake fan of the GPU)
• GPU-Z will be used for monitoring temperatures during the test
• Record the GPU’s idle core (not hot spot) temperature in degrees Celsius after the card has been on and in the idle state for a few minutes
• Run F1 2023 (5 laps) benchmark using 4K UHD 3840×2160 resolution (w/RT + DLSS/FSR) and record the peak temperature in degrees Celsius with the default fan profile (other manual fan speeds can be tested as well).
• Run Cyberpunk 2077 benchmark using 4K UHD 3840×2160 resolution (w/RT + DLSS/FSR) and record the peak temperature in degrees Celsius with the default fan profile (other manual fan speeds can be tested as well).
• When comparing temperature results, they must be normalized to the same ambient temperature (23C).

System Power Consumption

Have GPUz running while testing. Use the Total Board Power value (max) to find the peak power for the graphics card only.

• Record the system power consumption at idle
• Record peak power consumption during the following two benchmarks: F1 2023 and Cyberpunk 2077 benchmarks (same settings as recording temperature) at 3840×2160 resolution (w/RT and DLSS/FSR).

Folding @ Home

For all of the folders out there, we test this running for a full 24 hours to get the PPD across that time. We run at stock settings and record average power (the GPU) and list clockspeeds.

Conclusion

Nope, this definitely isn’t the “conclusion” of this article; it will be slowly evolving, just like our GPU testing procedures. We’ll post comments with changes as they happen, so please keep an eye on this, the comments, and the forum thread for updates. We are always open to feedback and suggestions as well. So, if you think something worth testing is not included, feel free to speak up in the comments below.

– Joe Shields (Earthdog)

Please leave any comments or suggestions in the thread!