Overclockers.com GPU Testing Procedures

This article will serve as a reference for those who would like to compare results with our reviews, whether you are writing an article for the site or just testing the performance of your new GPU. The plan is to keep this article updated as changes in our testing procedures occur. On to the gritty details…

Test Platform

(Updated:  6/25/2013)

Our test systems consist of mainstream parts since that is what the majority of people will be using, hence “mainstream.” This allows our results to be relevant to a wider audience, and also allows for that audience to repeat our tests to compare with their own systems. The required parts and clocks are as follows:

  • i7 4770K @ 4 GHz
  • Dual Channel DDR3-1866 9-9-9-24
  • GPU @ stock
  • Monitor capable of 1920×1080
Test Setup

Test Setup

Synthetic Benchmarks

(Updated:  6/25/2013)

Synthetic benchmarks are very consistent and easily repeatable, making them some of the best tests for direct card-to-card comparisons.

3DMark Vantage

DirectX 10 benchmark running at 1280×1024

  • Performance preset
  • Feature tests can be disabled since they don’t affect the score
3DMark Vantage

3DMark Vantage

3DMark11

DirectX 11 benchmark running at 1280×720

  • Performance preset
  • “Benchmark tests only” can be selected since the Demo tests don’t affect the score
3DMark11

3DMark11

3DMark – Fire Strike

DirectX 11 benchmark running 1920×1080

  • Standard Fire Strike test (not Extreme)
  • “Include Demo” can be unchecked since it doesn’t affect the score
3DMark - Fire Strike

3DMark – Fire Strike

Unigine Heaven (HWBot version)

Constant GPU load throughout the benchmark. The most game-like synthetic benchmark in our suite.

  • Heaven Xtreme DX11 preset
HWBot Heaven "Extreme" DX11

HWBot Heaven “Xtreme” DX11

Game Benchmarks

(Updated:  11/12/2013)

All but two of the following benchmarks are “canned” tests, making them as consistent as possible and more synthetic-like than typical gameplay. In general, we do our testing with the commonplace resolution of 1920×1080, all of the graphics settings maxed, and any manufacturer specific features disabled.

Batman: Arkham Origins

1. Adjust the graphical settings in-game to match the following:

Video Options

Video Options

2. Create a shortcut to the game

3. Right click the shortcut and go to Properties

4. Add “benchmark” to the end of the Target line so it looks like this:

“C:\Program Files (x86)\Steam\SteamApps\common\Batman Arkham Origins\SinglePlayer\Binaries\Win32\BatmanOrigins.exe” benchmark

5. Now clicking the shortcut will run the benchmark

6. Record the FPS shown when the test is finished

Battlefield 4

Adjust the in-game graphical options to match the following:

BF4 Settings

BF4 Settings

Record FPS during the jeep ride at the beginning of the Tashgar single player mission

  1. Open FRAPS, and be sure Min/Max/Avg is selected
  2. Start the Tashgar single player mission
  3. When the jeep ride starts, begin the FRAPS recording by pressing F11
  4. Stop the recording by pressing F11 when the jeep stops and Hanna says “Russians”
  5. The results will be placed in a .csv file located in the “C:\FRAPS\Benchmarks” directory
  6. Record the average FPS

Bioshock Infinite

1. Create a shortcut to the Benchmark.bat file in the install directory

2. Right click on the shortcut and go to Properties

3. Add “UltraDX11_DDOF -resx=1920 -resy=1080 -unattended” to the end of the Target line so that it looks like this:

“C:\Program Files (x86)\Steam\SteamApps\common\BioShock Infinite\Binaries\Win32\Benchmark.bat” UltraDX11_DDOF -resx=1920 -resy=1080 -unattended

4. Now when clicking the shortcut it will run the benchmark with the desired settings

5. When the test is done, you should get a popup asking if you want to open the results file, which is located in the “C:\Users\<username>\Documents\My Games\BioShock Infinite\Benchmarks” directory. When opened, it should look like below and I highlighted the desired FPS number in green

Bioshock Results File

Bioshock Results File

Crysis 3

Adjust the in-game graphical options to match the following:

Graphics Options

Graphics Options

Advanced Graphics Options

Advanced Graphics Options

Record FPS during part of the first single player mission

  1. Open FRAPS, and be sure Min/Max/Avg is selected
  2. Start the first single player mission
  3. After you receive the bow and Psycho throws a fit about something on a monitor then opens a door leading outside, begin the FRAPS recording by pressing F11
  4. Stop the recording by pressing F11 after you have shot down the helicopter and made your way to the other side of the area before opening a door to go back inside.
  5. The results will be placed in a .csv file located in the “C:\FRAPS\Benchmarks” directory
  6. Record the average FPS

Final Fantasy XIV: A Realm Reborn

1. Adjust the settings to match the following:

Graphics Options

Graphic Setings

Display Options

Display Settings

2. Save the changes and close the window

3. Click Start to run the benchmark

GRID 2

1. Go to GRID Online

2. Go to Options & Extras > Options > Graphics Options > Video Mode to adjust some basic settings, then go to Quality – Advanced to adjust the rest of the settings. They should match the settings below:

Video Mode

Video Mode

Graphics

Graphics

Graphics

Graphics

Graphics

Graphics

At the bottom of the Options & Extras > Options > Graphics Options screen, there should be an option called Graphics Benchmark to start the test

Metro Last Light

Run the Benchmark tool located in the install directory with the following settings:

Metro Last Light Settigns

Metro Last Light Settings

Optional Testing

(Updated:  6/25/2013)

The following are a few things that will provide additional value and detail to the testing, but are not required.

Temperature

Measuring typical temperatures to expect and/or testing cooler performance.

  • Record ambient temperature in degrees Celsius (preferably close to the intake fan of the GPU)
  • GPU-Z, PrecisionX, Afterburner, etc. will be used for monitoring temperatures during the test
  • Record the GPU’s idle core temperature in degrees Celsius
  • Run Heaven “Extreme” DX11 and record the peak temperature in degrees Celsius with the default fan profile (other manual fan speeds can be tested as well)
  • When comparing temperature results, they all need to be normalized to the same ambient temperature
Ambient Temp

Ambient Temp

System Power Consumption

Kill-a-Watt meters (or similar) are used for measuring at-the-wall system power consumption.

  • Record the system power consumption at idle
  • Record peak power consumption during the following two tests:  Heaven “Extreme” DX11 and 3DMark11 Combined test
Kill-a-Watt

Kill-a-Watt

Noise

Sound level meters are used to measure dBA as a means to quantify noise and perceived loudness.

  • Tested on an open bench, which gives worst case scenarios
  • Minimize all external sources of noise:  Turn off A/C, turn off ceiling fans, turn off TVs, turn CPU fan off or as low as possible, etc.
  • Manually set fan speed in varying increments via software such as PrecisionX or Afterburner
  • Record sound level at each increment. Be sure to get readings within your meter’s most accurate dBA range, regardless of distance. The most affordable meters are accurate at high dBA ranges, meaning the meter will have to be placed close to a quiet source for an accurate reading.
  • Readings can be estimated at dBA ranges outside of your meter’s capability to get estimated sound level at longer distances.
  • Typical linear percentages cannot be used when comparing dBA measurements because dBA uses a logarithmic scale. So, the rule-of-thumb is every 1 dBA difference is roughly equal to a 10% difference in perceived loudness.

The following equation is used for estimation of sound level at different distances.

L2 = L1 – 20 * log10(r2/r1)

  • L1 = Sound level at reference distance
  • L2 = Sound level at desired distance
  • r1 = Reference distance
  • r2 = Desired distance
Sound Level Meter

Sound Level Meter

Multi-Monitor

Nothing changes here except the use of multiple monitors in AMD’s Eyefinity or NVIDIA’s Surround. All benchmark settings remain the same.

  • Three 1920×1080 monitors are preferred for a total resolution of 5760×1080
Multi-Monitor

Multi-Monitor

Picture from complx‘s project log:  RGD – Custom Built Surround Monitor Stand.

Conclusion???

Nope, this definitely isn’t the “conclusion” of this article, it will be slowly evolving, just like our GPU testing procedures. We’ll post comments with changes as they take place, so be sure to keep an eye on this, the comments, and the forum thread for updates. We are always open to feedback and suggestions as well. So, if you think there is something worth testing that’s not included, then feel free to speak up in the comments below.

- Matt T. Green (MattNo5ss)

Please leave any comments or suggestions in this thread.

Tags: , , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>