The Effects of Benchmarking Bias

[Archer0915]

Hey I usually go up to PA a couple of times a year and I hit Jersey from time to time so who knows could have a benching party some time.

@BH I really do understand what you are doing but my thing is two fold one being what i have already mentioned and if the SSE extensions were so great and the non intel processors were not using them then why were they even competitive against the Intel processors.

 

[Neuromancer]

Hey I usually go up to PA a couple of times a year and I hit Jersey from time to time so who knows could have a benching party some time.

@BH I really do understand what you are doing but my thing is two fold one being what i have already mentioned and if the SSE extensions were so great and the non intel processors were not using them then why were they even competitive against the Intel processors.

Let me know. You know where to contact me. Hmm... Ln2 is spensive here unless I get 230L . (then its under 50 cents) under that its 3-5 bucks a liter :S

But winter time? im working on a swap water setup. be easy to remove it from a case and throw it out the window. Im not worried about antifreeze at load temps

Heh heh

the drunker guy gets to apply voltage

 

[Frakk]

Just continue the testing! Honestly I am bias but I am on the side of business not Intel. Perhaps I have been hardened toward all the poor victims out there who want to cry and sue and not stand up and git'er done. This in no way means I support the rampant commercialism I just don't support people who want big brother to fix everything for them.

I suppose you would be quite happy for a few in an old boys club in London to cary on manipulating the LIBOR rate screwing you and the rest of the world out of billions.

Or for Banks in general to take your hard earned money and gamble it away, lose it and then make you pay them back your money they lost.

This country has the most liberal and under regulated financial and media system on the planet, far more liberal than the USA. Look at what such a liberal ideology got us.

Power corrupts, ultimate power corrupts ultimately. some one somewhere has to curb that because if you leave them entirely to there own devices they will take the biscuit and then blame you for it.

 

[Archer0915]

I suppose you would be quite happy for a few in an old boys club in London to cary on manipulating the LIBOR rate screwing you and the rest of the world out of billions.

Or for Banks in general to take your hard earned money and gamble it away, lose it and then make you pay them back your money they lost.

This country has the most liberal and under regulated financial and media system on the planet, far more liberal than the USA. Look at what such a liberal ideology got us.

Power corrupts, ultimate power corrupts ultimately. some one somewhere has to curb that because if you leave them entirely to there own devices they will take the biscuit and then blame you for it.

My mistake for opening the door to politics here. My simple reply is: No I am talking about the slip and fall crowd (businesses) who sue because they feel screwed. The people (businesses) I am talking about are just as much or more to blame because of their inaction as the actions of the entity they are suing. On the bigger scale, it is business. Mistakes were made market share lost and dominance in the CPU processing power arena lost! Defeat snatched from the jaws of victory.

I am relating business to business. Business to consumer is another story and we will not go there.

 

[Frakk]

Bringing politics like that into it was perhaps a mistake on my part, its just the best example i could think of where (just business) suddenly becomes my business and your business.

I think its just to easy to parn a (two scorpions in bowl analogy) off as just business, here i go again, forgive me... Vince Cable: opposition business secretary, 2006,- "something is not right here, someone needs to step in and have a look at this, whatever is going on here its going to blow up in our faces"
Gorden Brown: Serving chancellor of the exchequer,- "no, nothing is wrong, all the banks and banker are wonderful and know exactly what they are doing, no regulation or interference is necessary, let them get on with it, its fine"
2008, BANG it all blows up in our faces....

But wait, Banks can't fail, they are to big to fail, we have got to plow trillions into them to prop them all up, oh and BTW, your paying for it all.
And so they go back to being masters of the universe ripping us all off with complete and utter impunity. or so they thought, and they would be right.
The true extent on the LIBOR scandal will never get out, it can't. If the 10 Currencies rates set by London Interbank Offered Rate, which include the USD and the Euro knew what it means to the average Joe and the rate he pays on his mortgage and hired purchases, there'd be murders... follower by complete financial meltdown across the globe.

So now that i have had my final of topic (i hate bankers rant)
i guess my point here is one should never let a single entity get to big, to powerful. because they will use that to become so called masters of the universe.

I know i would, if i had the opportunity to muscle my way to be the only provider of my product you bet your life i would push it.

One might say "well that's capitalism" one product, one provider, one price. sounds more like communism to me, so yeah, in the interests of capitalism do whatever it takes to keep competition alive.

Of course Intel don't like it, AMD would not like it and neither would i. yet as a consumer, i love it. and that's all which matters. stuff Intel and stuff AMD as corporations, i want a choice and i want it cheap, that's it.

 

[xander89]

Bringing politics like that into it was perhaps a mistake on my part, its just the best example i could think of where (just business) suddenly becomes my business and your business.

I think its just to easy to parn a (two scorpions in bowl analogy) off as just business, here i go again, forgive me... Vince Cable: opposition business secretary, 2006,- "something is not right here, someone needs to step in and have a look at this, whatever is going on here its going to blow up in our faces"
Gorden Brown: Serving chancellor of the exchequer,- "no, nothing is wrong, all the banks and banker are wonderful and know exactly what they are doing, no regulation or interference is necessary, let them get on with it, its fine"
2008, BANG it all blows up in our faces....

But wait, Banks can't fail, they are to big to fail, we have got to plow trillions into them to prop them all up, oh and BTW, your paying for it all.
And so they go back to being masters of the universe ripping us all off with complete and utter impunity. or so they thought, and they would be right.
The true extent on the LIBOR scandal will never get out, it can't. If the 10 Currencies rates set by London Interbank Offered Rate, which include the USD and the Euro knew what it means to the average Joe and the rate he pays on his mortgage and hired purchases, there'd be murders... follower by complete financial meltdown across the globe.

So now that i have had my final of topic (i hate bankers rant)
i guess my point here is one should never let a single entity get to big, to powerful. because they will use that to become so called masters of the universe.

I know i would, if i had the opportunity to muscle my way to be the only provider of my product you bet your life i would push it.

One might say "well that's capitalism" one product, one provider, one price. sounds more like communism to me, so yeah, in the interests of capitalism do whatever it takes to keep competition alive.

Of course Intel don't like it, AMD would not like it and neither would i. yet as a consumer, i love it. and that's all which matters. stuff Intel and stuff AMD as corporations, i want a choice and i want it cheap, that's it.

All bow to maggie T this is her legacy....what a witch . this isnt something that is solely exclusive to the last 10 years. Ultimately this is capitalism, boom and bust baby, its the way the system works. If you give people free rein to do anything, they will eventually **** it up, it crashes, and we are back at square one. It will boom again and then bankers will be given just as much free rein because they are bringing wealth and opulence to the general population then it will go bust again. Ultimately all we can do is sit back and watch.

Its a pipe dream that britain will adopt a socially responsible form of capitalism (like a lot of the Scandinavian countries) so it will never stop, prepare to be angry for the rest of your life, i know i am! haha

anyhuuuuuuu

on topic.....wait what were we talking about?

 

[Automata]

I'm not going to say this again, get it back on topic.

I don't like breaking out the greenie-meanie text, but this is getting annoying.

 

[xander89]

until we get results this thread will take a few detours along the way. I think that talk on the actual topic has come to somewhat of a stalemate until we get some more hard numbers. I for one am quite enjoying the tangents, it gives my brain a brake for the stuff i dont understand! keeps a thread interesting/ alive in the absence of OP's results. But fair enough thid. *grumbles and curls up in the corner till bubba gets back with the numbers* hehe

 

[Automata]

until we get results this thread will take a few detours along the way.

Except for the discussion that is off-topic, this thread was going in a good direction and was moving at a decent rate.

 

[xander89]

Except for the discussion that is off-topic, this thread was going in a good direction and was moving at a decent rate.

i dont know, the back and forth between arch and bubba seem to be going in circles somewhat in the last few pages / a lot of repeating themselves, differences of opinion in essence but anyways this is going off topic again. *curls up in his corner*

 

[Frakk]

Sorry, back on topic then Bubba, what do you have for us?

 

[Bubba-Hotepp]

Except for the discussion that is off-topic, this thread was going in a good direction and was moving at a decent rate.

Thank you thid. To everyone else, this thread is not about politics or "capitalism". If you want to talk about the benchmarks, compilers and the effects they may or may not have etc. great. If you're going to talk about politics or which companies make better CPU's then please do it in another thread.

Sorry, back on topic then Bubba, what do you have for us?

I'm chugging right along with the testing but I'm not going to keep updating the first post over and over and over. I'll do it all at once.

Aida64 3 runs each = Completed
PCMark7 3 runs each = 80% complete
WPrime = Completed
Cinebench = Completed
Novabench = Completed
Passmark PT7 3 runs each = 90% complete
PCWizard 2012 = Completed
POV Ray 3.7 = Completed
Super PI = Completed

To do list:
3DMark 06 3 runs each
3DMark 11 3 runs each
PCMark05 3 runs each
MaxMem^2 3 runs each
Unigen Heaven 3 runs each
x264 HD
x264 FHD
list of games
WinRAR
WinZip
7Zip
etc. etc.

 

[Frakk]

I will keep my eye on page 1 to then, thanks.

PS: your last Spoiler code is broken.

 

[Neuromancer]

I'd really rather not have this thread degenerate into an Intel vs AMD "my CPU is bigger than your CPU" thread. This is all about the software not which CPU is better or "faster". And yes I'm going to include ALL results regardless of whether they show differences or not (cinebench being one of those that I've found to not show any difference).

Wasn't meant to be, I was messing with Kev who owns a ton of AMD chips and started reviewing z77 boards recently Was all in good fun.

Will stop cluttering up the thread though

 

[JigPu]

Not to discount all the work you've done Bubba-Hotep, but simply wading out into the unknown (and without knowledge of the underlying code or what compiled it, that's precisely what it is) will get you data, but of pretty limited use. Its eye-opening to see that e.g. Aida64 takes the vendor string into consideration, but not really telling of what's actually going on. It could be the libraries, or Aida itself could have hand-written assembly tuned to show each architecture in its best light (that would be especially probable given the warning it popped up...).

Echoing back to the calls for tests with other compilers, probably the easiest way to address this question would be to compile a program against Intel's libraries and those of a 3rd party and see if the former are more significantly affected by a change in vendor string than the latter. The Intel libraries are free for non-commercial use if anyone's interested. I'm too busy to do anything at the moment, so I'm hoping somebody else knows their way around a compiler. One possible idea would be to try to compile the SETI@Home source code which should be compatible with both FFTW and MKL.

JigPu

 

[Neuromancer]

Not to discount all the work you've done Bubba-Hotep, but simply wading out into the unknown (and without knowledge of the underlying code or what compiled it, that's precisely what it is) will get you data, but of pretty limited use. Its eye-opening to see that e.g. Aida64 takes the vendor string into consideration, but not really telling of what's actually going on. It could be the libraries, or Aida itself could have hand-written assembly tuned to show each architecture in its best light (that would be especially probable given the warning it popped up...).

Echoing back to the calls for tests with other compilers, probably the easiest way to address this question would be to compile a program against Intel's libraries and those of a 3rd party and see if the former are more significantly affected by a change in vendor string than the latter. The Intel libraries are free for non-commercial use if anyone's interested. I'm too busy to do anything at the moment, so I'm hoping somebody else knows their way around a compiler. One possible idea would be to try to compile the SETI@Home source code which should be compatible with both FFTW and MKL.

JigPu

I assume (way out of my league) that is what angers blog is all about. Someone that actually uses compilers and differnet Architectures. The OP was trying to see how it affected us as end-users.

I do like your idea though Might be better for folding.... maybe you should start that up in the folding community forum? Intel compiled for Intel and other compiled for AMD? Or do they offer that already?

 

[Bubba-Hotepp]

Not to discount all the work you've done Bubba-Hotep, but simply wading out into the unknown (and without knowledge of the underlying code or what compiled it, that's precisely what it is) will get you data, but of pretty limited use. Its eye-opening to see that e.g. Aida64 takes the vendor string into consideration, but not really telling of what's actually going on. It could be the libraries, or Aida itself could have hand-written assembly tuned to show each architecture in its best light (that would be especially probable given the warning it popped up...).

Echoing back to the calls for tests with other compilers, probably the easiest way to address this question would be to compile a program against Intel's libraries and those of a 3rd party and see if the former are more significantly affected by a change in vendor string than the latter. The Intel libraries are free for non-commercial use if anyone's interested. I'm too busy to do anything at the moment, so I'm hoping somebody else knows their way around a compiler. One possible idea would be to try to compile the SETI@Home source code which should be compatible with both FFTW and MKL.

JigPu

I understand what you're saying. BUT, I'm showing the "end result". If everyone wants to know "causality" then they'll have to push for the programmers to tell them why or decompile it themselves and examine the code. But nobody will know to even look if they never know there's a problem to begin with. And no matter WHAT the cause is, the end result is the same, programs that for whatever reason change their behavior depending on the "vendor ID" detected. The implications of that alone are clear and significant. The actual "cause" will only tell you why it's happening but you don't need to know the "cause" to know that it is happening. Here's what Agner said in his last email response after I sent him a screenshot of the Nano with "Bubba Hotepp" as the vendor ID and Aida64 giving the message "Please note that this Aida64 program is not fully optimized for the CPU of this computer"

Your screenshot proves that the AIDA has CPU-specific optimizations. This is also implied by the description at http://www.aida64.com/product/aida64-extreme-edition/benchmarks
For example, the description says that it uses the VIA specific "PadLock" instruction set or the "AES-NI" instruction set for AES benchmark. But it wouldn't have to check the vendor string for that. The "PadLock" instruction set is only supported by VIA, but the Intel AES instructions are also supported by AMD Bulldozer. Testing for "GenuineIntel" to see if the CPU has AES is wrong - it should only test the CPUID flag for AES.

 

[Neuromancer]

Does that mean I was right about aid64? Oops.

 

[Bubba-Hotepp]

Does that mean I was right about aid64? Oops.

You'll have to refresh my memory

 

[Neuromancer]

You'll have to refresh my memory

I said that the built in sensors of AIDA64 overrided the VIA vendor ID string change.