Overclockers’ Updated Video Card Testing Procedure

(UPDATED ARTICLE:  https://www.overclockers.com/overclockers-gpu-testing-prodcedures) – MattNo5ss (6/27/2013)

With the launch of Ivy Bridge, we’ve decided to migrate our video card testbed to the new platform, primarily because of its PCIe 3.0 specification. Along with the move, we’ve updated our test suite as well to make it more relevant to our readers that always like the latest and greatest titles. Older games like HAWX 2 and STALKER just won’t cut it any longer.

We had to make some compromises for cost-effectiveness so they aren’t ALL the latest $60 titles. However, they are all still modern games that require a solid graphics setup to look good. We’ve also expanded a little bit to include not only several graphics-intensive first person shooters, but a driving game and (completely new to us) a strategy game.

As you probably know, we have reviewers across the country, literally from the east coast (NY, MD, NC) to the west (CA, WA), so this procedure is designed to maximize cross-reviewer capability. Whoever writes a review, if they use these settings, their results can be appropriately compared with all other reviewers’ results.

Testing Platform and Method

• PCIe 3.0 motherboard (Z68 or Z77)
• Intel i7 3770K
• Tests are run with the CPU at 4.0GHz:
  • This is mostly due to ASUS’ complete inability to run a CPU completely at “stock”.
  • 4.0 GHz is also easily obtained by simply raising the multiplier with no other adjustments. Except for the very rare (so rare I’ve yet to see one) example, every 3770K should be able to run that at the stock Vcore, even if just using the stock cooler.
• RAM should be set at DDR3-1866 / 9-9-9-24.
  • Here we’re aiming for a middle ground between overclockers with fast memory (DDR3-2133+) and your average gamer (who likely ends up running around DDR3-1600/1866).
• The GPU should be run through the battery of tests at stock and overclocked to a 24/7 stable level, able to complete all game tests & benchmarks.
• Benchmarks (where scores are in the multiple thousands or tens of thousands) should be run three times at stock and the result averaged.
• Game tests at stock and overclocked can be run once, benchmarks when overclocked can be run once as well.
• Lastly, crank down on both the CPU and GPU to show what sort of benchmark scores the combo can achieve. Take your pick of benches to run, preferably no less than two.

Game Tests and Settings

Game tests will all be run at 1080P resolution (1920×1080). For ease of cross-reviewer benchmarking, all games are to be run at their highest quality settings.

Aliens vs. Predator DX11 Benchmark

• This can be run with AvP Tool or manually with the config file set properly.
  • Resolution: 1920×1080
  • Texture Quality: Very High
  • Shadow Quality: High
  • DX11 Features – Hardware Tessellation & Advanced Shadow Sampling both on.
  • Full Screen AA Samples: 4x
  • Antistropic Filtering: 16x
  • SSAO: On

Batman: Arkham City

• Run the in-game benchmark using the following settings:
  • Resolution: 1920×1080
  • VSync: Off
  • AA: 8xMSAA
  • MVSS and HBAO
  • Tessellation: High
  • Detail Level: Extreme
  • Other Settings: Yes
  • PhysX: Off

Battlefield 3

• This must be a manual run, using the following settings:
  • Resolution: 1920×1080
  • Graphics preset: Ultra (using all defaults included in that preset)
• The method of testing is manual, but is remarkably consistent (we’re talking less than 1 FPS difference between runs in our tests).
• Instructions:

1. Turn on Fraps
2. Start the first mission: Operation Semper Fidelis
3. Begin recording FPS (F11 by default) after you pick up the pistol and are looking forward.
4. Work through the level as normal.
5. Stop recording FPS when you’re at the end of the level and the game takes over motion, hit F11 again when the guy puts his hands up.

Civilization V

• This one is a bit more complicated, but here is the procedure nonetheless.
• Go in-game (DX11) to set graphics settings as follows:
  • 1920×1080
  • 8x MSAA
  • VSync Off
  • High Detail Strategic View: Enabled
  • Other Settings: High
• Find the config file in the “Sid Meier’s Civilization 5” directory in your Documents folder and set “LoggingEnabled = 1”.
• Create a shortcut on the desktop pointing to “CivilizationV_DX11.exe”.
• Right click on the shortcut and go to Properties.
• On the Shortcut tab, add “-benchmark LateGameView” to the end of the Target field.
• Run Bench. It takes 3 minutes and 50 seconds to run. There’s no need to set the duration; I tested none set, 60s, and 180s. The time was 3:50 either way.
• Get the LateGameViewBench.txt log file from the Logs directory within the “Sid Meier’s Civilization 5” directory.
• There are three different scores in the log file: full render, no shadow render, and no render. The scores are the number of frames rendered in 60 seconds. The full render score is what we want to use to get FPS, and to get FPS divide the full render score by 60.
• Record the result.

Dirt 3

• Run the in-game benchmark using the following settings:
  • Resolution: 1920×1080
  • Anti-Aliasing: 8x MSAA
  • Settings: Everything Enabled and Maxed (High or Ultra depending on setting)

Metro 2033

• Run with the included benchmark tool using the following settings:
  • Resolution: 1920×1080
  • DirectX Version: DirectX 11
  • Quality: Very High
  • Anti-Aliasing: MSAA x4
  • Texture Filtering: AF x16
  • PhysX Off
  • DOF (Depth of Field) On
  • Scene: Frontline

Synthetic Benchmarks

As mentioned, at stock these are run three times, then run once overclocked. Typically, these are the benches (plus Heaven DX11) that you’d run maxed out on the CPU and GPU.

3DMark03

• Older bench, but still scales really well both with GPU power and the addition of more GPUs.

3DMark Vantage

• A DX10 benchmark, this still scales decently with GPU power.

3DMark 11

• Brand spanking new and DX11, this one is mostly GPU dependent still.

HWBot Heaven Benchmark Xtreme (DirectX 11)

• Solid DX11 bench, this is more GPU dependent than pretty much all other benchmarks.

Added Bonuses

Both of these are performed with the GPU at stock. You can add overclocked numbers as well if you want; I don’t usually.

Temperature testing

• Test the card both at idle and loaded.
• To make the data worthwhile, we will need to monitor ambient temperature to normalize results.
• Results will be normalized to 25 °C. If your ambient is 22 °C, add 3 °C to the recorded temperature of the GPU.
• We will use HWBot Heaven DirectX 11 test as it doesn’t give the GPU a break. Use Afterburner or any constant monitoring program and record the peak temperature with the default fan profile.
• As you can see, we’re getting away from FurMark. It’s just too unrealistic for us to use – either the temps will be too high relative to a “normal” load, or the card will throttle and throw readings off.

Power Consumption

• Measure the system completely at idle.
• We will record peak consumption in two situations.
• The first is while running HWBot Heaven DX11.
• The second is while running the 3DMark 11 Combined Physics test.
• Again, we’re moving away from FurMark because of the potential for throttling, which will throw everything out of whack.

Multi-Monitor Setups

• Most of our reviewers won’t be able to do this, but if you can run three monitors, run all of the game tests above preferably at 3 x 1080p (5760 x 1080) using the same quality settings as above. Hokie is the only reviewer set up to do triple-monitor setups currently, but if we get others with the capability, even better

  

The first review to use this method (I think) will be the ASUS GTX670 DirectCU II TOP that I’m working on now. MattNo5ss has kindly agreed to benchmark the EVGA GTX670 Superclocked using the method so we have a comparison card.

We will be re-benchmarking all of the cards we have, but it may take some time. You should see the comparison chart grow with each review until it is as built up as the one we had previously. Priority will be given to the closest competition to the card being reviewed. Thank you for your patience as we increase our comparison chart.

We’ll be using the same methodology we always have with our Approved / meh / FAIL award system. We only have three levels so they can be clearly defined, don’t rely on arbitrary numbering and -most importantly- to encourage you to read the reviews that go along with them.

While it’s pretty well set in stone at this point, with the hardware and games already purchased, feel free to chime in with questions or comments. We’re here to review this hardware for you, so let us know what you think!

– Jeremy Vaughan (hokiealumnus)