Why Most Cooler Tests Are Flawed: CPU Cooler Testing Methodology

[Kenrou]

TLDR: "This video defines our benchmarking methodology for the next several years of CPU cooler testing, highlighting common errors made that we've solved for."

"CPU cooler testing is always flawed, in every execution, almost no matter what you do. You can solve for the many flaws created by the test bench by using dummy heaters, but then new issues of realism are introduced, and resolution of real-world impact can be lost. We've worked for 6 months to narrow-down the list of issues on test benches and, over that time, we briefly crippled ourselves with the knowledge of how inherently flawed most cooler tests on real platforms are. Our new testing approach resolves much of this, but also introduces custom-built dummy heaters that we've collaborated with an engineering firm to create. Between the refined 'real' computer test bench and the dummy heaters, we now have enough to create what we believe to be among the most accurate cooler reviews we can do. There will always be flaws in CPU cooler testing, it's just a matter of mitigating them to a point where testing becomes accurate."

 

[Woomack]

I won't argue with how bad are most cooler reviews on the web. However, there is one issue that can't be tested using a heater. As long as heaters will give you exact heat in W then new CPUs don't have one, the same hot spot on the IHS. Because of that, coolers designed for 250W can barely handle some CPUs that suppose to have under 150W or even 95W TDP. There are also many variables so as long as all coolers are tested on the same CPU with the same TIM, in the same ambient temp and airflow then results should be close to accurate. We can argue about some variables, but end-users don't really care about +/- 1-2°C. They care if the cooler handles their CPU and if the cooler is quiet under full CPU load.

 

[trents]

I won't argue with how bad are most cooler reviews on the web. However, there is one issue that can't be tested using a heater. As long as heaters will give you exact heat in W then new CPUs don't have one, the same hot spot on the IHS. Because of that, coolers designed for 250W can barely handle some CPUs that suppose to have under 150W or even 95W TDP. There are also many variables so as long as all coolers are tested on the same CPU with the same TIM, in the same ambient temp and airflow then results should be close to accurate. We can argue about some variables, but end-users don't really care about +/- 1-2°C. They care if the cooler handles their CPU and if the cooler is quiet under full CPU load.

Sorry, this didn't communicate well for me and I'm not sure what you are trying to say. Could you elaborate or rephrase that?

 

[ehume]

The video explained one thing I observed as I was doing my tests: the hotter (cooled less well) heatsinks used a few more watts than their better competitors. The "leakage" they found explains that. As for using the same CPU, not the same model; giving the system time to heat up to a steady state; measuring ambient air temp close to the heatsink; I would hope that every serious reviewer follows those elementary precautions; I know I did.

 

[EarthDog]

Sorry, this didn't communicate well for me and I'm not sure what you are trying to say. Could you elaborate or rephrase that?

I believe he is saying a heat plate gives a constant load/geat/temperature across the entire face of the contact surface. Whereas with CPUs, the heat is more concentrated where the dies are situated under the IHS. Because of this, results can vary.

 

[trents]

I have long taken cooler performance comparison reviews with a grain of salt because of all the variables, except to the point of consistently seeing some models show up near the top of the heap. Having said that, the reviews from Frostytech I have taken the most seriously since they have always used a controlled environment similar to the one in the video. Although artificial, are at least comparing apples to apples.

 

[Zerileous]

A couple of points. First of all the heat plates are not being used for review data, rather they are being used to validate the hierarchy produced by tests done with actual PC hardware. Secondly these heat plates do actually mimic the chiplet design of Zen2. There are three resistors (two high power and one low power) set up to match both the heat production and layout of the compute and IO dies of a Zen2 AM4 CPU. Each can be powered and adjusted independently. Here is a video that goes into more detail about this