UL Adds New Storage Benchmark to 3DMark Suite

First a copy/paste where I put my thoughts on this elsewhere:

In concept it isn't bad. They took traces of accesses for several gaming scenarios to recreate on the test system. Apparently it needs a 30GB test file to work off, so it does kinda represent a porky game install. The headline output is a benchmark score, with detailed output of average transfer rate and access latency.

I'd be curious to see how it performs if it wasn't for one hurdle. This is a paid DLC to the paid 3DMark Advanced, which is probably the version enthusiasts most likely have as it is frequently on sale. While the cost is not huge, the main question is what value does it give? If you have the hardware already, you get a score and... then what? Unlike CPUs and GPUs, there's not really much you can do to overclock or otherwise improve performance of a SSD. Maybe if you're testing software caching strategies? That's a bit of a stretch. Or just checking system performance, but there are other ways to do that.

Also the headline score is an abstract value, and only useful if viewed in context of others. I also have to wonder, how the tests are weighted. For example, one is for recording gameplay via OBS. I do wonder if they would have been better to split scores into several sections, for example, loading time (probably most important generally), install time (mix of read/write and/or copy operations?), and game video capture as another output.

Benchmark wise, how much does it vary with the SSD itself, or does it get impacted by the rest of the system? If you use it to decide what to buy, then the reviewer will buy and use it.

I think this opens more questions than it answers. Think I'll stick to CrystalDiskMark as at least it is clear what the outputs are of that.

Some additional thoughts since then:

I have since looked at the technical description given for the bench, which partially answers the "what" but is still unconvincing in its relevancy.

I have seen some others have run it and report not a lot of disk activity going on, and getting low average transfer rates. Far lower than what the drives are capable of. This might be realistic in the sense that games may still be coded in a way that benefits HD owners, at the cost of SSD owners. I don't know how true that is today. For a slow storage device, CPU performance impact cost is lower than storage performance impact cost. For faster storage like a SSD, the CPU quickly becomes the limiting factor. The traces that were taken and used in the bench would probably have such limitations baked in. Then it is no longer that much of a SSD test. Well, a fast SSD might still perform better than a slower SSD or even HD, but we kinda already know that already. Does this bench rank SSDs in a gaming usage case better than other test tools?

I think that it isn't free to the masses will kill it off from much usage.

Reply Like

So far it's crashing for me after about 70% pass with no clear info why. Clean OS/Win11, CPU at auto, RAM at XMP, M.2 PCIe 4.0 SSD. This benchmark is really long and I have a limited time right now so will check it some other time.

Most of the tests are not meant to be used for masses and actually barely anyone is comparing that anymore. Overclockers community is getting smaller each year and I don't count all those who run 1 test per year and call themselves overclockers.

The idea of a storage benchmark based on games is actually good as there are no benchmarks like that and storage performance affects gaming much more than it used to be some years ago. I haven't seen how all test results look like so I can't say much about it. There are only two results that seem interesting and it's MB/s and access time. If there will be only some weird score then I won't use it as it's impossible to compare to daily usage.

Reply Like

At $2.99 I believe that is a pretty small hurdle. I do agree that most users won't care about this or will run it one time to get the results and then forget about it. As far as the single score at the end, I view this the same as Anvil Storage Utility. Where Anvil produces a single synthetic score, 3DM SB generates a real-world single score. By no means should a potential buyer base their purchase based on one benchmark. This just gives reviewers another tool in the toolbox.

I also agree that the results from this bench should be broken down to the individual benches before tallying them up for the final score. This would help determine what area a drive may be lacking or excelling in.

If memory serves me correctly there was an older 3DMark that measured complete system performance. I hated that one because the storage area always dragged my final score down.

Reply Like

Haven't touched it. Likely won't. There's other, better(?) tools out there.

I do like how it goes through several different scenarios though to test different things. Worth noting is that just because the drive isn't 'maxed out' doesn't mean the tests are bad or not doing enough. It's just the test and how the data is processed that determines what kind of load is on the device. For example, I know game loads don't slam the SSD to 100%. Perhaps other tests there are like that too? Not sure.

If you want to get a good idea of performance on your drive(s), I suggest looking at multiple tests. I use a combination of CDM and ATTO and get a mix of results in CDM and 'best case' (read: box specs) out of ATTO.

RE: a score, can't say I mind an all-ecompassing value at the end, but it would be nice to see the details, absolutely.

Reply Like

Real-life results are almost always far from results in synthetic benchmarks. PCMark storage tests usually end at about 500MB/s on modern SSD, but this is random performance in some specific scenarios. The SSD can't make more in this scenario so it still runs at its maximum speed, just can't push as high as its specified maximum sequential bandwidth.
This is why the fastest storage for daily work or gaming, doesn't mean the one that can make the highest bandwidth in a synthetic, sequential bandwidth benchmark. As long as ATTO or similar benchmarks are great for comparison, then are telling us barely anything about the storage performance in daily work or games. Here appears PCMark and now 3DMark with storage tests based on popular applications. The only problem is to present test results in the right way, so they will be clear and comparable for everyone.

@Blaylock, you mean PCMarks in general. There is a new PCMark 10 that has some more storage tests and shows results in MB/s. Tests are based on popular software so the same as the new 3DMark test which uses popular games as a test environment. So PCMark covers the office/home environments while 3DMark focuses almost only on gaming.

Reply Like

Some tests require MS Office or Adobe software to be installed. It's actually a problem as most of that soft is signed to the PC or user's account in some other way. Even business licenses have limited activations. I'm just not using these benchmarks. I think that's why I wasn't using Procyon too. I don't remember what benchmarks were there. I have a license but it required too many other things. It's just too problematic, especially that I reinstall OS quite often. I already download 250GB+ 3D benchmarks for comparison each time I reinstall OS, as Ubisoft or other games with benchmarks can't be just copied or installed offline.

I really hope that the 3DMark storage benchmark will be improved soon as I wish to use it in storage or even motherboard reviews. Most benchmarks give only synthetic bandwidth results which mean nothing to most users. I just wish to make storage reviews a bit more interesting.

Reply Like

That's why you set it up and image your OS. Then, simply restore the image. Once the base OS is setup and the image captured, it's literally a 10-15 minute process (for me and my test OS) to have a fresh OS up and running from scratch with all of my applications/benchmarks/etc ready to go. :)

Procyon requires MS Office, Photoshop/Lightroom Classic and Premiere Pro. I've been through 4 boards so far with the key I have for Procyon and the Adobe apps and haven't run into issues (outside of having to log in every time). Occasionally I get a warning that I'm signed in on another device, but I just log it out and move on. So yes, it's tied to your account. I just log in before I bench and go! :)

Reply Like