Why posted temps are nearly useless for comparison

After reading several more 'temperature' posts lately, I decided to repost this in hopes of bringing some sanity to the situation.

I know that for this post I'm probably going to need a state of the art, heavy-duty asbestos flame suit, but I'm willing to take that risk.

I've had a lot of time to ponder the meaning of life and the evolution of PC cooling. This issue has been sitting in the back of my head for a long time and has bothered me to the point that I have to post about it, so here goes...

The temperature, flow rate, etc, numbers posted on this and other forums, in articles and across the web in general are nearly useless in a real sense. Sure, they are nice to see but in the reality of the way that the majority of numbers are obtained, they are virtually useless as any means of performance comparison.

Why? Where to begin...first off, there are numerous posts of numbers like "my cpu temp. is xx degrees using xxxxxx hsf" and someone else will come along and say "oh yeh, well my cpu temp is xx degrees using xxxxxx hsf and xx number of case fans, therefore my system kicks butt over your system" or some such dialog.

First off, no one, I don't care how careful, meticulous and anal your are, will ever get reliable, repeatable or scientifically acceptable numbers off of using a CPU as the heat source to compare cooling systems. The issues with using a CPU to compare cooling performance are nearly inumerable. First off, every CPU is different and I don't just mean a Pentium is different than an Athlon. Every single piece of silicon will perform in its own unique way and perform thermally in a unique manner. Then add in effects caused by different motherboards, cases, power supplies, blah blah blah and you start to see the problem. This isn't even considering environmental effects such as altitude (air density), barometric pressure, humidity, ambient temperature, etc. All of these things vary from location to location and from moment to moment.

Now take into account the methods of measuring the numbers that get tossed around: in-socket thermistors, on-die diodes, external thermistors and thermal couples, infra-red sensors, etc. All of which are calibrated (or in most cases, not calibrated) differently and have hugely varying degrees of accuracy.

I take every single number posted in product reviews with a grain of salt and every number posted on message boards with a couple good shakes of salt. I wouldn't even consider comparing results across different reviews with each other.

What it comes down to is this; while the numbers floating around the web are fun to look at and are interesting from a novelty point of view, they really are not overly useful. The testing that is most useful that I have read about across my web wanderings have been those done here by overclockers.com using their die simulator and BillA's die simulator and test equipment setup. The *only* way to get comparitive results is to use *exactly* the same test equipment under *exactly* the same conditions. Even the most advanced testing schemes floating in webspace still have several shortfalls that are very difficult to overcome.

By no means am I trying to discourage people from posting their numbers and how they got them. What I am trying to do is bring some reality to the situation. Too many people read that someone got some number and come to the conclusion that the number is perfectly accurate and can be related to their own personal system; and that simply isn't true.

yes but it does give u something to go for.. and u think to much about this stuff.. relax.. but i can see where u are geting at.. i know someone that had a athlon runing at 70c REALLY STABLE i could not understand it my guess was it was the probe.. but it does not really madder the way i see it is as long as the system is stable at the X temp its ok (of course once u get the itching for more cooling then this logic is tossed out the window )

I'll jump in on this one...

Extra variables... thermal grease, lap job...

and of course, human embelishment. "I WATER COOLED TO -75C!!!"

Ummm... the greatest of antifreezes would be frozen...

Numbers like the ones you're talking about make a terrible god (not that there any numbers worth worshipping). I see that SpamFrag69 at the [H] got 3 degrees below ambient with an orb (given that I buy it) and I think "good for him." I'm not into aircooling (I'll smash my mini copper orb when my waterblock fnally arrives), but if I were, I'd have to look for a reasonably strong correlation before I bought an HSF.
The numbers are goofy, but they're not entirely meaningless. You just have to know what to look for.

BTW, if SpamFrag69 really exists, it's news to me.

Well said, as far as temps go, there's onlt one thing you need to know: "If it don't crash it's cool enough"

I'll agree that the temp readings are hard to compare. What we really should be using more is benchmarks. That is really what cooling is all about. It's about getting that computer to run faster.

Sad, but it is always going to be like this. What is even worse
is when other sites (You know the ones.) have sloppy,
inconsistant testing procedures, but pass off their reviews
like they are the Word of God.

But maybe it's like the Lincoln Park song: "In the end
it doesn't even matter."

While I agree there are far too many factors involved to assume that one can get the same temperatures as someone else using the same cooling, the numbers still tend to give us a rough idea of how heat is generated and dissipated throughout the users system. I have to stress though..I agree..this is a very rough idea.

I am a watercooling fan myself, and I am appalled at the lack of substantial information regarding heatsinks, waterblocks, radiators, and fans, not to mention the heat output of various components involved in the overclocking process.(even though I agree these will also vary slightly)

I'm in the middle of building a system right now, and I have a lot of questions that it seems no one has answers to. How should the thermal output on the northbridge vary with FSB? What is the thermal conductivity of this or that heatsink?

For me, the problem is that there's just not enough solid info out there. As soon as my system's done, my next project will be a dedicated test rig. That way I can measure flow rates and temps myself, and MAYBE have some concrete information to pass on. Until then, I agree we should take these temps with a grain of salt, but they are still at least a little useful.

As I said, I don't want to discourage people from posting their temps. What I would like though, is for people to better understand what a temperature reading means. Every single day on this forum you can read more new posts that indicate that people see a temperature, believe it is 100% accurate, and try to compare it to someone else's temperature.

One of the very first and most important things anyone learns when learning about mechanical measurements is to understand the limits of such measurements. The numbers posted here are incredibly limited. As just a rough guess of accuracy, I wouldn't give any posted temps any better than +-5 degrees of dependablity.

Yes, numbers do give a ROUGH idea of comparison, but nothing more. I'd bet the average all of the posted water-cooling temps would be lower than the average of all the posted air-cooled numbers. What I wouldn't bet canadian penny on is any kind of quantitative comparison between the two.

So go ahead and post numbers to your heart's content, just please realize how limited in usefulness those numbers really are.

I just wanted to wade in here and thank Aesik for an excellent thread. Core temperatures are the result of so many variables that it is virtually impossible to compare them. You can compare trends but not absolute values.

ButcherUK, be careful with the truism that if it does not crash, the temperature is okay. I am retesting some water blocks. With the core of my XP1600 at 35C I can run it at 12.5x145 perfectly stable at 1.90V. If I allow it to rise to 45C, I have to jack up the voltage to 2.05V to get it stable. That puts more strain on the CPU, Core voltage regulator circuitry, ATX connector and the motherboard itself. Point being, there are justifications for seeking lower temps, even when the system is stable.

Hoot

To continue from Hoot, that's really NOT the only reason to find better cooling tehniques. For example, one of the major benefits of water-cooling my machine is that it's now REALLY quiet, despite being SERIOUSLY overclocked. I've got a 750Mhz Slot-A Athlon to 963Mhz and it never EVER crashes. But that's not all that's cooled in my system. My GFX card, hard disks and PSU also figure into the loop. Water-cooling keeps 'em nice and cool but near silent at the same time.

To get back on topic, I developed what I call my GLSensor a while back. Basically a device, not unlike the DigiDoc, which plugs into the parallel port allowing you to measure up to 8 temperatures (the software interface can be seen below).



Now before Aesik starts shouting at me about dodgy temperature measurements, lemme explain a few things...

1) None of these sensors are calibrated and when I collected them all together a while back and insulated them alltogether, there was about 1.5 degrees between the maximum and minimum sensor.

2) Unfortunately, my device is quite sensitive to noise so the sensors who's wires pass my cold-cathode are pretty noisy (most of them).

3) The sensors themselves are packaged in silicon (they're surface mount devices) and I used some aryldite to make them a little more rigid.

4) Because the sensors are so small, they tend to be affected by heat transferrred down the wires themselves which almost certainly throws the readings off a little.

As Hoot pointed out, there are a lot of variables, but having stuff like this can give you a reasonable idea of what works and what doesn't. For example, re-seating my CPU block with a thinner layer of AS dropped the measured core temp by 5 degrees (ambient temperature was taken into account) a while back. Whilst this figure might not be spot on, it certainly indicates that reported temperatures can be useful.

So what am I saying? Basically, that there are a lot of inaccurate readings out there, but I think you're over-stating the case. This is by no means intended to be a flame, I just think that if you've got 10 sites which all agree that one heatsink is better than another because it gives lower temps, then maybe those temps aren't that unreliable after all?

Good thread Aesik, but it will probably fall on deaf ears.

Case in point - Ozzman (no offense meant, Ozz). In his sig, he has his 1.33 Athlon at 34C under load using air cooling (Pal 8045).

Now, I mean no disrespect, but I don't believe it (not that his MB isn't reporting that temp, just the temp itself). Why you might ask?

Well, his CPU is putting out approx 65W or more. Even if the case temp is 22C, that's a C/W of 0.18. That's just too good for my tastes.

Why else, you might ask? Well, at the moment I am running my TB 1200/266 in an MSI K7T Turbo at stock (clocked down for the moment). I have a Lian Li PC60 tower, round cables, ASII, a 120MM side hole, 120MM top blowhole, 3 80MM case fans, and a Swifty MC462A with a Mechatronics 80MM 50CFM fan (swapped it in last night, must update sig). My temps? 22.8C board, 26.3C power/aux, and 40.4C CPU. Oh, my voltage is a liitle higher than stock - 1.82V. My C/W - around 0.29.

So, does the Pal perform better than the Swifty? Is my HS not seated properly? (it is). Is my ASII too thick? Maybe Ozzman's thermistor isn't touching the backside of the core? Or MSI miscalibrated their thermistor reading firmware. Maybe all of the above.

Maybe we should all leave our idle and load temps out of our sigs, to prevent C/W envy. Or maybe we should just have fun with it, play with our comps to get the temperature down, and brag about it over a few beers.

My AX-7 arrives via overnight Fed-Ex tomorrow

While I agree that using a CPU for test purposes is pretty much useless. I don't think that comparing simmilar setups should be dismissed as being completely inaccurate or useless. Comparing your system to someone/people who run(s) a nearly identical setup can give you a good idea of where your temperatures should be in general. If 15 people have the same setup, and person X builds one idetical to those, he/her temperatures should be close to the temperatures of the others. I'm not talking about outside contributing factors like ambient temperature, barometric pressure, etc... Throw all that aside for now. If the first 15 people all run at temperatures between 35°C to 42°C, but person X is getting readings of 73°C. Wouldn't that suggest that something is possibly wrong? I think Aesik sumed it up pretty much in his last two paragraphs...

What it comes down to is this; while the numbers floating around the web are fun to look at and are interesting from a novelty point of view, they really are not overly useful. The testing that is most useful that I have read about across my web wanderings have been those done here by overclockers.com using their die simulator and BillA's die simulator and test equipment setup. The *only* way to get comparitive results is to use *exactly* the same test equipment under *exactly* the same conditions. Even the most advanced testing schemes floating in webspace still have several shortfalls that are very difficult to overcome.
By no means am I trying to discourage people from posting their numbers and how they got them. What I am trying to do is bring some reality to the situation. Too many people read that someone got some number and come to the conclusion that the number is perfectly accurate and can be related to their own personal system; and that simply isn't true.

Click to expand...

That's a pretty sweet setup G33k. What's most important though is that you realize the limitations of your system. How many people even compare their thermistor temperatures to each other? I'm impressed and I'd encourage others to follow the example; well, not building your own system like G33k's, but at least understanding what is going on.

I can agree that if the majority of websites reviewing a heatsink agree that a certain one is the best, than most likely it is. What I don't agree with is how people and websites will say stuff like "switching from Y heatsink to X heatsink will net 4 degrees better performance."

What I really disagree with is how most of the the testing is done. Ignoring the fact that most of the time the heat source being used is a CPU, almost every review only uses a sample size of one. This creates such a huge margin of error that it nearly nullifies the results.

People running similar setups should get in the same ballpark as each other, but there are so many factors to consider that trying to compare a few degrees of performance IS impossible. Now like Crystal said, if everyone is getting mid 30s and somone else gets something in the 70s, there is definitely some cause for concern.

My case is still based on two facts: nearly all testing published on the internet is not done on sound testing principles. And secondly, people reading reviews and websites take temperatures and readings WAY too seriously. It would do them a world of good to understand better how temperatures are measured and what the limits are of trying to compare temperatures.

Ta very much I've often wondered whether anyone's done a decent article on how reliable temperatures recorded by the DigiDoc are "in the field" as it were.

I see your point and I pretty much agree with it. Any site that says "If you have heatsink X and you go and buy heatsink Y, you're system will be Z degrees cooler" should be slapped

As you pointed out earlier, even if you used scientific conditions and were very careful, there's so many factors that saying X is better than Y in one situation means very little in a lot of others. What you've got to remember is that there's only so much you can do and things will ALWAYS vary from setup to setup. The thing you're forgetting is that the differences aren't actually that big in most cases. I agree that a 4 degree difference in temperatures is quite slim in the grand scale of things, but if the experiment is repeated time and time again, with the same results, surely there must be some truth in them!?

The problem is that a lot of sites don't go out of their way to make it clear that their results are not the be-all and end-all. There will ALWAYS be problems measuring CPU temperatures but that's just something we'll have to live with Not sure who said it, but it was a damn good saying : "a little knowledge is a very dangerous thing"...

G33k
very slick
any more info on your "GLSensor" ?
hardware ?
software ?
ameinable to higher temp resolution ?

on the temp issue, I've been involved in several ugly contentious wrangles lately,
some really insist on applying their personal C/Ws to the world

so I coined an Overclockers' Mantra:

all temps are crap but Watts are worse
ooooohhhhmmmmmmmmmm

an overstatement, but this C/W thing is becomming wearysome

be cool

Praise indeed! Hehe, yeah I've "witnessed" some of your rants

More info on my GLSensor here for you.

one of the problems with the 'net is that there is SO much out there,
that most of the good stuff gets missed

G33k, that is the slickest DIY project I've seen
(of course I'm quite interested in temps also)

accuracy limitations notwithstanding,
I can't help but believe that there is a market out there for such a 'finished' device
- there are more and more OCers digging into the temp thing

now I guess I'll have to build one to see what can be done for it's accuracy

don't sweat the max CPU load thing, the CPU's 'accuracy' of heat generation is below the sensors' capability
so what's the net effect ?, zip as I see it

be cool

Yeah, I wrote that article ages ago, but never really saw it as much of a problem since I never intended to use my software to benchmark accurately. I just wanted to know that having built a water-cooling system, my chip wasn't sitting at like 60 degrees+.

I'd agree that there's a market - I always thought the DigiDoc was a bit limited, because you couldn't record temps. That's why I built the original. I've got some plans for Mark 2 which will be even MORE marketable. It'll work slower, but have the ability to read from up to 16 sensors and control 16 fans giving them a range of 0-12v (unlike the DigiDoc's off or on).

If you need any help building one, let me know and I'll see what I can do for you

what's the timing on the Mark 2 ?
should I hold off, I've no actual need as my testing stuff is what I work with

heres a suggestion:
insert a means (cermet pots ?) of individual sensor 'adjustment' so they can be set for a uniform 'zero' (ref temp)

please add me to your list of interested people
( [email protected])

be cool