Heatsink Testing

SUMMARY: Testing CPU heatsinks using motherboard temp sensors may be possible, but the probability of unintentional errors is high. A more rigorous testing regime using thermocouples and digital thermometers is the “gold standard”.

I have been working with Andy Lemont at Millennium Thermal Solutions on heatsink testing. Andy believes that many of the reviews you see by various hardware sites are well meaning but unintentionally mis-represent the true performance of many heatsinks. This can be traced to the imprecise measuring tools we use, most notably motherboard temp sensors.

Coincidentally, Stephen Hoar at Burning Issues has written a very interesting article (What’s a review worth? ) that examined this in some detail. Andy Lemont summarized Stephen’s “review of reviews” in the following graph:

Stephen used Global Orb tests in this analysis. According to Stephen, some Global Orb tests miss the mark by over 100%; this means that a review temp of say 30C is actually more like 60C! One key point is that

“For some reason, maybe due to calculator-allergies, you don’t see °C/Ws-a figures in the innumerable fansink tests posted up on hardware sites. It’s a pity, because this is the only measure which might be called a fact, & is of real use to every reader worldwide.”

I think the dilemma reviewers face is to test CPU cooling in a way that users can affirm on their own systems. For example, Tillman Steinbrecher over at Anandtech just did a heatsink roundup (Socket-A & Socket-370 Cooler Roundup – November 2000) using

“…an Abit KT7-RAID motherboard, a Thunderbird-750 CPU (overclocked to 950MHz using 1.8V voltage), an Asus V6800 graphics card (GeForce1), and 256 MB PC133 RAM. We used an Asus/Elan Vital midtower case and an Antec power supply…The temperature measurements were made using the KT7’s onboard thermistor…”

Using the KT7’s socket thermistor raised issues that are covered in the following thread in Anandtech’s Forum. Basically I think it boils down to the following (arguable) points:

• Motherboard sensors are inherently inaccurate and not suitable for testing heatsinks;
• Boring small holes in heatsinks and inserting thermocouples is one acceptable test method (this is an Intel spec);
• Measuring CPU temps using a thermocouple on the back of the CPU is possible and can give accurate results if properly set up.

To the last point, Stephen Hoar at Burning Issues came up with an approach (Hot Air) that, I think, answers the call for an accurate and reproducible methodology. I have adopted this approach and can summarize it as follows:

• Attach a “T” type thermocouple to the backside of the CPU, directly under the core;
• Insulate the socket to avoid any “trapped air” interference’
• Use a calibrated digital thermometer (in my case, Omega HH23);
• Use another thermocouple to measure ambient air temps;
• Report absolute temps, motherboard sensor temps and calculate C/Ws.

For all future reviews, I will take the approach outlined above. As more background to why I am doing this, read on as to how I linked up with Andy Lemont, how we tested heatsinks in the lab and what we found.

Andy is developing a heatsink (AAGK-1 ) and, of necessity, must test it against others (BTW: typical engineer – calls it the “AAGK-1”; I call it the “ice cube”). We hooked up and started talking about the testing issue. It turns out that Andy’s office is less than an hour from me, so I visited a couple of times; we proceeded to do a test comparing a bunch of heatsinks using lab methods and on an ASUS A7V and ABIT KT7.

Now let me be the first to say Andy is out to prove a point here – that his heatsink is among the best around. In the tests that followed, I participated with Andy and have no reason to doubt what we found. However, this is about is heatsink testing – any future heatsink reviews that I perform will be on my premises with the equipment I outlined above.

Go to PAGE 2 for the Lab Setup.

Email Joe

What follows is how we tested heatsinks. As pictured below

Andy’s lab is probably typical – he uses calibrated power supplies and digital thermometers to control and measure variables. The testbed is as shown below:

Each heatsink is placed on a precise thermal source so that the heat does not vary; it looks like what you might see on the rear window of your car. Note that this is a “large contact area” heat source – it is much larger than Intel Coppermine or AMD Socket A CPU footprints.

The heatsink’s temp is then measured by a thermal probe placed in a hole drilled into the heatsink:

The thermistor is placed in the hole, epoxied in place and then lapped smooth. This is the methodology specified by Intel for heatsink testing with its CPUs. AMD specifies a similar method. Overall, heatsink testing in this manner is used by all manufacturers seeking certification from Intel or AMD.

Using this setup, we ran the following heatsinks through it with these results:

C/Ws were calculated as follows:

where Delta Temp = (Heatsink Temp – Ambient Air Temp). Note that C/Ws are for “joint to air”; this means that they include the C/W for the thermal grease used.

The next step was then to mount the heatsinks on the ASUS and ABIT boards and compare results to the more precise thermocouple and digital thermometer readings taken at the same time. Please go to PAGE 3 for these results.

Email Joe

The pic below show how this testing was performed; we mounted each heatsink on my ASUS A7V and ABIT KT7 running an AMD T-Bird at 1000 MHz, 1.8 volts, ran Prime95 until temps stabilized, and then measured temps using the Omega HH23 and each board’s temp sensor. The ASUS uses a temp probe placed next to the CPU core and the ABIT uses a socket thermistor which is off center to the CPU’s core.

Andy very kindly graphed out the results:

*Heatsink Contact uneven – result adversely affected.

Hmmm…Looking at this, I am hard pressed to see a great correlation between what we saw on the lab “large contact area” testing. Further, temp readings between what we saw using the heatsink thermistor and motherboard sensor readings are disturbingly off. There does not appear to be a consistent error.

This is an eye-opener. I thought going into this that there is a “consistent error” using motherboard temp sensors, so that even if they are off somewhat in absolute terms, at least relative testing, using exactly the same components at the same time, would give good results.

Well, looking at these results, I’m not so sure about that. And there is enough doubt in my mind that I ponied up the $219 to buy the Omega HH23 and associated gear to move our heatsink testing to a higher plane.

Let me editorialize for a moment:

Manufacturers and vendors of heatsinks and thermal greases spend a lot of time and money developing products. I have tried to test and present data as objectively as I can – people’s lives are impacted by what we review and post as results. Andy Lemont and Stephen Hoar opened my eyes to a better way and I am thankful they did.

As always, any comments etc, drop me a line.

Email Joe

NOTE:

I am including the raw data as well; I apologize for the busy tables, but I did want to lay out all results. Also note that I did not mount the Agilent on the motherboard – way too difficult and the danger of breaking off a socket lug prevailed. Motherboard Monitor 4.17 was used without any correction factors.

From this raw data, we then computed C/Ws:

*Heatsink Contact uneven – result adversely affected.

SUMMARY: After a day of CPU temp measurements, things get murky – the “best” CPU temp measure is….?? Please read this as “work in progress” rather than a heatsink review.

I spent a day with Andy Lemont precisely measuring CPU temps four different ways. What we found was four different measures, one of which is the “industry standard” measure (drill hole in heatsink and insert thermocouple to measure CPU’s top). We found that the relationship of heatsink C/Ws to measuring temps by other means to be at best tenuous and at worst, potentially misleading.

• ABIT KT7, no raid
• CPU Stress: Prime95 – avg 15 minutes
• Fans: YS Tech 26 cfm; Super Orb 2 fans; Vantec 60x15mm
• Heatsinks drilled with thermocouple by Andy Lemont
• Temps measured by Omega HH23 Digital Thermometer
• Arctic Silver thermal grease

Temps were monitored for stability; each time temps were recorded, no substantial fluctuations were observed. In addition, at boot up, thermocouple temps were about at ambient, so there was no “residual” heat observed from earlier testing. The socket interior was insulated with closed cell foam which served to hold the thermocouple against the back of the CPU.

The following table lays out our test data:

All temps measured in Centigrade by Thermocouples except MBM417; CPU Back = probe on center back of CPU; Heatsink = probe drilled into heatsink; CPU Side = probe next to CPU core; CPU was AMD T-Bird 1000 MHz at 1.82 volts – 57.4 watts.

Unfortunately, we noticed after we packed up that the Millenium had wrapped several turns of the thermistor wire around the fan, so the results are off. However, note the Super Orb stats – decent heatsink C/W, but alternative CPU core temps paint a very different picture – as does temp measured by ABIT’s socket thermistor. Also look at the FOP32 and Vantec numbers: Similar temps all along except at the Heatsink, leading to a very different picture. Finally, the Alpha PAL6035 and copper Hedgehog almost line up one for one except at heatsink temps.

There is an underlying assumption which needs testing:

Heatsink C/Ws are a reliable indicator of heatsink performance in a user’s system.

I don’t have an answer to this – the testing continues.

Many many thanks to Andy Lemont at Millenium, Stephen Hoar at Burning Issues, Jim at Benchtest.com, Tillman Steinbrecher at Heatsink Guide and John Carcich for unstinting efforts and time on this issue.

Email Joe

SUMMARY: Evaluating Heatsinks based solely on CPU Core temps can be misleading – CPU Back temps indicate total cooling performance accounting for the motherboard’s secondary heat path effects.

Andy Lemont at Millenium and I have been getting together about once a week to run heatsink tests. The more we test, the more questions come up about what we see during these tests.

One hypothesis we tested: Heatsink back temps are accurate indicators of heatsink performance. The reason this is of interest is that all the current Socket A motherboards use a thermistor in the socket to measure CPU temps. In turn, many review sites use these boards for heatsink tests.

We ran a bunch of heatsinks using the same 60 mm fan (except for the Super ORB), a YS TECH, to see what the correlation is between CPU Core Temps (measured by a thermocouple drilled into the top of the heatsink) and CPU Back Temps (measured by a thermocouple epoxied onto the CPU’s back). These were run on an ABIT KT7 with a Duron 800 @ 1000 MHz, 1.93 volts, in open air.

What we found was that, in fact, the relationship between CPU Core Temps and Back Temps was not consistent. In fact, what we found was that a very high difference between Core Temps and Back Temps might indicate CPU instability. The results for the Super ORB were consistent – it failed Prime 95 every time and the Super ORB exhibited one of the highest differences between front and back temps.

Andy plotted C/Ws against Case Temps and Back Temps, as shown above, which vividly illustrates the inconsistency between the two. It also looks like the more inefficient the heatsink (higher C/W), the closer it gets to CPU Back Temps. The best performers seem to have low CPU Core Temps and small differences between CPU Core Temps and Back Temps. As Stephen Hoar at Burning Issues termed it, this “holistic” view of CPU cooling adds another dimension to understanding CPU performance variables.

These are intriguing clues but only clues at this time – we need to do a lot more work before we can state unequivocally that heatsinks exhibiting smaller front to back temp differences are better performers overall (although intuitively it seems valid).

Now having found something we could not explain, Andy went into his “I have to know why” mode and found something very interesting. What follows are his findings:

“I have been testing a 1 GHz Athlon running Prime 95 on an ABIT and Gigabyte board, measuring CPU Top (through sink thermocouple) and Back temps. My observation of variations of heat sink performance between motherboards showed little change in C/W between boards.

However, CPU Back temps would flip flop between boards on some heat sinks and not others, with all things being equal.

Utilizing some air flow visualization techniques, it became apparent that there was a fair amount of recirculation with the boards operating in open air outside the case. An examination of directional air flow as a factor with board component interaction indicated that my Millennium heatsink should be rotated 90 degrees – similar to the Tasiol and FOP 32.

As their clipping orientation was counter to the extrusion profile [ED: english translation – the heatsink clip runs across the fins] , nonsensical in a manufacturing design but in hindsight, quite reasonable considering a board’s thermal feedback to the CPU. So, I modified and rotated my heatsink 90 degrees. CPU Top temp was 35 C and Back temp 50 C – a 12 degree drop over my previous ABIT temp.

With some recirculation still evident, I placed a 92mm fan about 60mm above the sink in suction mode and CPU Back temps dropped to 46 C while CPU Core temps remained @ 35 C. Testing a larger sample with the overhead fan showed drops in backside temps between 1 and 5 C for the other sinks with the exception of the ALPHA PAL and Hedgehog, which showed little or no change.”

Note that both the Alpha and Hedgehog exhaust air off the board rather than blow air down to it.

Now to some degree this is an affirmation of what many of you have found out by doing: That air circulation through the case is a critical factor for overclocking performance. What Andy highlights is that air flow around the CPU and motherboard is an important consideration.

Now back to the ORB:

I think the reason that it results in high CPU back temps is that the ORB’s airflow off the heatsink is 360 degrees and positioned just right so that it heats all those nice capacitors around the socket, leading to higher back temps (and, IMHO, instability). In addition, perhaps the two-tiered fan sets up a convection current such that waste warm air off the board is sucked back into the heatsink by the top fan. A flawed design?

I think what we are finding is that some heatsinks, like the Super ORB, will perform well in certain boards and not so well in others (seems like for every satisfied ORB user, there’s an unsatisfied one). Some heatsinks, like the Alphas, perform consistently well in all motherboards due to a superior design which accounts for secondary heatpath effects. Whether this is by accident or intentional design, I don’t know.

However, users should pay very careful attention to airflow around the CPU for maximum CPU cooling and heatsink efficiency.

Email Joe