Even in today’s world, there’s some things you can always count on. The sun shows up every day. The earth revolves around the sun.
And Apple comes up with some nonsense that whatever it just came out with is faster than x86 processors.
Apple just announced its new systems. These are what I call the MacHammer systems, simply because they are at least superficially rather alike. They both use SOI, both have a 32/64-bit architecture, and both run at around the same speed.
Anyway, Apple presented some numbers to indicate that the G5 is faster than the PIV or Xeon in single and dual-CPU configurations. These measurement included Spec2000 measurements.
Spec2000 is a platform-independent benchmark often used to compare the CPU performance of different platforms. Manufacturers submit their scores to Spec for posting on their website. You can see the lastest posting of results here.
Anyone familiar with recent x86 spec scores would have smelled something rotten; the x86 scores presented by Apple were just too low. Chris Tom from AMDZone has already taken the official spec scores and written an article about it
To present the data a bit more directly:
Here are the four spec benchmarks presented by Apple:
The first set of benchmark use one processor; the second set two (except for the single-CPU Dimension 8300):
This would lead you to lead the G5 is very competitive in this benchmark, especially in dual-processor mode.
Wrong.
Now look at the numbers when you use official spec scores for x86 systems rather than Apple’s:
When scores for exact systems aren’t available, closest equivalents are used (i.e. a 3Ghz PIV with an 875p board for a Dell Dimension 8300). A number bracketed after the system
description for the CPU Base Rate benchmark indicates the number of CPUs used)
The G5 gets beaten by everything current, and walloped by the latest x86 technology.
Only the Athlon MPs lose decisively in this contest; the Xeons win big, the Opterons win huge.
Let’s compare a couple benchmarks for the Dell Precision 650. Same equipment, only difference is that Apple configured one machine and Dell configured the other:
These are huge differences.
Why?
Apple usually doesn’t make configuration data available, but if you burrow enough, you can find it here. Compare the configuration data to the configuration data found at spec, and you begin to see why.
It’s not hardware. For a CPU-intensive benchmark like this one, it’s the CPU and RAM that count, and all systems had 2Gb of RAM.
It’s the software that differs. For starters, the official spec scores usually use a version of Windows (2K, XP or Windows Server 2003), while these tests were run using Red Hat Linux 9.0.
The more likely major culprit, though, is the compiler used to get the spec source code ready for processing and what was done with it. The x86 manufacturers (outside of the Opteron tests, which often used a beta version of Suse Linux 64-bit) used the general standards: either the Intel and MS C++ and Fortran compilers, while Apple used a compiler you’ve never heard of (at least in the x86 world).
Finally, in the Apple testing, the code for the Apple machines were tweaked to the max with all kinds of libraries installed, while the code for x86 machines were largely default out-of-the-box experience.
It shows. If you look at the individual breakdown of subcomponents in the spec testing, only a couple ran roughly the same, while many showed a 25-30% difference, and a few showed up to a 60% difference.
And That’s The Best of their Tests!
Intel and AMD occasionally fib a little in their benchmark testing, but when they do, it’s usually a matter of a handful of percentage points, and if you look closely (or sometimes even not-so-closely) you can find the differences fairly easily.
They never do these kind of things.
What’s scary is that this is an improvement over the usual Apple benchmark. If you look at the Apple Performance White Paper and see the other tests being run.
They compare their hard drive Firewire performance not to PC Firewire performance, but USB 2.0, ignoring the general availability of Firewire in PC systems these days.
They run a number of apps (at least half of which you’ve never heard of), but provide no configuration data. Hell, sometimes they even use different programs. A couple of them run on Windows (in which case, why couldn’t they do that for spec?), and others required compiling (and we already know how well Apple did that with spec).
These folks even claim that their machine blows the doors off x86 machines in Quake. They don’t even identify the OS the x86 machine uses.
It could well be that the G5 does do some things better than an equivalent x86 machine, for instance, Photoshop has always liked the Mac a lot, but when you find dirt in every spot where you can look, it’s hard to expect things to be spanking clean in the places where they won’t let you look.
No Credibility
If Apple were part of the x86 market, they’d be ripped to shreds in a second presenting this kind of data. It would be a huge scandal. PC companies get torn a new one for attempting even a small fraction of what we have here.
But it’s a different world in Macdom. The average Macster is woefully ignorant of hardware, and tends to believe anything and everything Apple tells him. For many, it is guru and flock.
This is not an honest business, but nobody in the PC world has ever approached Apple in the level of sheer and consistent technically accurate but totally misleading information when it comes to performance.
It’s really a shame. I thought the G5 would be pretty good. From what I can figure out, it isn’t, and I don’t trust the company to tell the honest truth for the rest of it.
P.S. I’ve seen some pretty bizarre defenses of Apple so far. One said that the scores from Dell et. al. weren’t proper because those machines were tweaked. Uhhh, just what did Apple did, they even said so deep in the fine print. What’s fairer, measuring tweaked machine to tweaked machine, or tweaked machine to non-tweaked machine.
Another said that some version of Linux had to be used to compare apples to apples. Well, MacOS X isn’t Linux, and the desktop standard for x86 machines is Windows (not that using a properly optimized Linux bothered the Opterons very much). You want to know what machine is fastest, you test in their native environment.
Be the first to comment