Windows 8: The Times They Are a-Changin'

[EarthDog]

I think that the difference in score happens when you run a benchmark and are changing the clock speed of the CPU mid-bench.

downclocking in windows was the trigger, assuming I am reading things right...?

The problem builds on the problems we faced with Heaven. When downclocking the system under Windows8, the Windows RTC is affected as well. The biggest difference between Windows7 and Windows8 is that now all benchmarks (no exception) are affected.

Examples: Benchmarks and Windows Time.

Let us make this more practical. On our Haswell test system we downclocked the BCLK frequency by about 6% from 130 MHz to 122MHz. Using a CPU ratio of respectively 32x and 34x, the resulting CPU frequency remains 4160MHz. Then we ran comparison benchmarks. Here are a couple examples:

 

[Janus67]

I just reran every benchmark on windows 7 and got within a margin of error of my submissions under windows 8 with the exception of firestorm extreme which was lower on windows 7. This sounds like a witch hunt.

Edit: Also wouldn't it make more sense to simply add a subcategory for operating system and make it required for all submissions? Silly folk...
Second edit: I've written code to import info from hwbot api. Now I have to decide whether it's worth using it.

As for your second point, a subcategory for OS would just weaken the submissions of other users, seeing how OS used and their tweaks can make a large difference in scores. Also, there are far too many operating systems between desktop and server and their sub-versions.

 

[Bobnova]

Downclocking the FSB/QPI/HT/BCLK makes the windows RTC run more slowly.
Then raising the multiplier to get the speed back does not make it run more quickly, as the RTC is purely bclk based (or at least, a different multiplier).

 

[Janus67]

https://www.reddit.com/r/technology/comments/1knvit

Story made it to reddit and was posted on extremetech

Trying to fill in the questions and answer what benchmarking means there

 

[dejo]

I will give Pieter some credit here, as removing the scores via this is prolly the proper thing to do. And will prolly upset some of his circle of buddies

 

[EarthDog]

I will give Pieter some credit here, as removing the scores via this is prolly the proper thing to do. And will prolly upset some of his circle of buddies

This... no complaints here really.

 

[Overclockers.com]

Over the weekend HWBot.org uncovered a critical flaw with the time keeping method of Windows 8.

Due to a problem with the Windows 8 Real Time Clock (RTC) service, when changes are made to system frequency by software, the internal clock in Windows 8 begins to skew. The greater the change in frequency, the greater the clock skew changes. Frequency changes in BIOS do not have any impact, this only affects Windows 8 if using popular manufacturer tools like Asus TurboV...

... Return to article to continue reading.

 

[wagex]

figures.

 

[Bobnova]

Yeah I'm all for removing the scores. It's just that I'd prefer all of them were removed, as opposed to records and the ones Massman and Co. don't like.

 

[wagex]

This is so important because I use my computer as a $1200 clock.

you read the whole post right?

 

[Nimblor]

Yes. I even almost watched the video. Good thing he did not use an iPhone to verfiy the clock glitch. That would have been quite the conspiracy.

 

[wagex]

so you noticed the thread wasnt really about the time being off... but that the hwbot was pulling all windows 8 submissions because some people found out and abused this problem? just making sure.
its a bit bigger deal than the time just being off.

 

[awktane]

Ah I re-read the post and now I understand. The place that I read the story, as is far too common, had sensationalized it and removed most of the facts. That makes sense. I still think it would be interesting to split the operating system as a subcategory even if just for searching for results. It would be interesting to see which operating systems the benchmarks are happiest under and would also which operating system is the king in different categories.

In the end results from most benchmarks can be faked using virtualized hardware. I can tell a virtualized system that it has a specific cpu even if it doesn't. I can also affect the rtc with a rigged ntp server. Both of these are outside of the operating system. Conclusion? Cheaters going to cheat. Build a better benchmark.

 

[EarthDog]

I don't think the time has anything to do with it as doesn't your PC sync up with M$ servers for the time by default?

But yeah, an alarm clock shouldnt be a worry, surely not in this context. Just do not adjust clockspeeds in windows and all is well. Or after you adjust them, reboot, then its not a windows based app changing the time at that point (I dont think?).

 

[I.M.O.G.]

Ya, NTP typically syncs by default if you are internet connected. It would probably get broken by this kind of clock skew over time though, as what most people don't know about NTP, is that it just quits if the clock is too out of sync.

NTP is designed to fix very small skews, in the realm of seconds... If something makes you off by minutes or hours, there's a good chance it won't correct your time and just throw its hands up.

Of course, I know more about NTP on linux than I do about windows... So I could be wrong about how MS syncs time.

 

[Bobnova]

MS network time on win7 "fails" if the time is off too much, but also updates the time to whatever network said.

 

[awktane]

When it comes to world records even just removing a second from every minute would get you there. My point wasn't to point out ways to cheat my point was more that cheaters will cheat and that there will always be a way to skew results. Any way to skew a benchmark to me is laziness or a bug with the benchmark software not the operating system. Unfortunately some of the benchmarks that are still in use are getting quite old and likely have not been changed in years. It's along the same lines as anti-cheat software in games. A necessary evil.

Edit: Default on Ubuntu and Debian is to eat the time fed to it. I've seen clocks change by 20 minutes once they start synching. The same applies to windows. Yes both under normal operation work with drift or skew but in the end the system is designed to fix time and fix it does. It could be on a very small scale borderline undetectable and still skew benchmark results if that is what they are using.

 

[wagex]

with my machines it wont pull anything from the microsoft time servers if it isnt within the same month? as it supposed to be i believe.

 

[awktane]

As an aside I think it's silly that by default windows synchs to a single server. NTP becomes much more accurate if you are pulling time from multiple sources. A good ntp client given multiple servers can determine candidates and survivors thereby making your time much more accurate... Not that time being off by a few seconds matters in most cases.

I won't go into detail on the off chance that it does affect benchmarks but I just confirmed that with a batch file I can make my time have seizures. I can remove half a second off of every few seconds or even a tenth of a second off of every second. It took me about 5 minutes to write.

 

[I.M.O.G.]

As an aside I think it's silly that by default windows synchs to a single server. NTP becomes much more accurate if you are pulling time from multiple sources. A good ntp client given multiple servers can determine candidates and survivors thereby making your time much more accurate... Not that time being off by a few seconds matters in most cases.

I won't go into detail on the off chance that it does affect benchmarks but I just confirmed that with a batch file I can make my time have seizures. I can remove half a second off of every few seconds or even a tenth of a second off of every second. It took me about 5 minutes to write.

That wouldn't have any effect on the time observed from benchmarks. Changing the actual windows clock doesn't affect benchmark scores.

In order to skew benchmark scores, you need to make what windows thinks is a second, actually .9 or 1.1 seconds.