- Joined
- Nov 1, 1998
-
Welcome to Overclockers Forums! Join us to reply in threads, receive reduced ads, and to customize your site experience!
FRONTPAGE UL Adds New Storage Benchmark to 3DMark Suite
- Thread starter Overclockers.com
- Start date
- Joined
- Mar 7, 2008
First a copy/paste where I put my thoughts on this elsewhere:
In concept it isn't bad. They took traces of accesses for several gaming scenarios to recreate on the test system. Apparently it needs a 30GB test file to work off, so it does kinda represent a porky game install. The headline output is a benchmark score, with detailed output of average transfer rate and access latency.
I'd be curious to see how it performs if it wasn't for one hurdle. This is a paid DLC to the paid 3DMark Advanced, which is probably the version enthusiasts most likely have as it is frequently on sale. While the cost is not huge, the main question is what value does it give? If you have the hardware already, you get a score and... then what? Unlike CPUs and GPUs, there's not really much you can do to overclock or otherwise improve performance of a SSD. Maybe if you're testing software caching strategies? That's a bit of a stretch. Or just checking system performance, but there are other ways to do that.
Also the headline score is an abstract value, and only useful if viewed in context of others. I also have to wonder, how the tests are weighted. For example, one is for recording gameplay via OBS. I do wonder if they would have been better to split scores into several sections, for example, loading time (probably most important generally), install time (mix of read/write and/or copy operations?), and game video capture as another output.
Benchmark wise, how much does it vary with the SSD itself, or does it get impacted by the rest of the system? If you use it to decide what to buy, then the reviewer will buy and use it.
I think this opens more questions than it answers. Think I'll stick to CrystalDiskMark as at least it is clear what the outputs are of that.
Some additional thoughts since then:
I have since looked at the technical description given for the bench, which partially answers the "what" but is still unconvincing in its relevancy.
I have seen some others have run it and report not a lot of disk activity going on, and getting low average transfer rates. Far lower than what the drives are capable of. This might be realistic in the sense that games may still be coded in a way that benefits HD owners, at the cost of SSD owners. I don't know how true that is today. For a slow storage device, CPU performance impact cost is lower than storage performance impact cost. For faster storage like a SSD, the CPU quickly becomes the limiting factor. The traces that were taken and used in the bench would probably have such limitations baked in. Then it is no longer that much of a SSD test. Well, a fast SSD might still perform better than a slower SSD or even HD, but we kinda already know that already. Does this bench rank SSDs in a gaming usage case better than other test tools?
I think that it isn't free to the masses will kill it off from much usage.
In concept it isn't bad. They took traces of accesses for several gaming scenarios to recreate on the test system. Apparently it needs a 30GB test file to work off, so it does kinda represent a porky game install. The headline output is a benchmark score, with detailed output of average transfer rate and access latency.
I'd be curious to see how it performs if it wasn't for one hurdle. This is a paid DLC to the paid 3DMark Advanced, which is probably the version enthusiasts most likely have as it is frequently on sale. While the cost is not huge, the main question is what value does it give? If you have the hardware already, you get a score and... then what? Unlike CPUs and GPUs, there's not really much you can do to overclock or otherwise improve performance of a SSD. Maybe if you're testing software caching strategies? That's a bit of a stretch. Or just checking system performance, but there are other ways to do that.
Also the headline score is an abstract value, and only useful if viewed in context of others. I also have to wonder, how the tests are weighted. For example, one is for recording gameplay via OBS. I do wonder if they would have been better to split scores into several sections, for example, loading time (probably most important generally), install time (mix of read/write and/or copy operations?), and game video capture as another output.
Benchmark wise, how much does it vary with the SSD itself, or does it get impacted by the rest of the system? If you use it to decide what to buy, then the reviewer will buy and use it.
I think this opens more questions than it answers. Think I'll stick to CrystalDiskMark as at least it is clear what the outputs are of that.
Some additional thoughts since then:
I have since looked at the technical description given for the bench, which partially answers the "what" but is still unconvincing in its relevancy.
I have seen some others have run it and report not a lot of disk activity going on, and getting low average transfer rates. Far lower than what the drives are capable of. This might be realistic in the sense that games may still be coded in a way that benefits HD owners, at the cost of SSD owners. I don't know how true that is today. For a slow storage device, CPU performance impact cost is lower than storage performance impact cost. For faster storage like a SSD, the CPU quickly becomes the limiting factor. The traces that were taken and used in the bench would probably have such limitations baked in. Then it is no longer that much of a SSD test. Well, a fast SSD might still perform better than a slower SSD or even HD, but we kinda already know that already. Does this bench rank SSDs in a gaming usage case better than other test tools?
I think that it isn't free to the masses will kill it off from much usage.
So far it's crashing for me after about 70% pass with no clear info why. Clean OS/Win11, CPU at auto, RAM at XMP, M.2 PCIe 4.0 SSD. This benchmark is really long and I have a limited time right now so will check it some other time.
Most of the tests are not meant to be used for masses and actually barely anyone is comparing that anymore. Overclockers community is getting smaller each year and I don't count all those who run 1 test per year and call themselves overclockers.
The idea of a storage benchmark based on games is actually good as there are no benchmarks like that and storage performance affects gaming much more than it used to be some years ago. I haven't seen how all test results look like so I can't say much about it. There are only two results that seem interesting and it's MB/s and access time. If there will be only some weird score then I won't use it as it's impossible to compare to daily usage.
Most of the tests are not meant to be used for masses and actually barely anyone is comparing that anymore. Overclockers community is getting smaller each year and I don't count all those who run 1 test per year and call themselves overclockers.
The idea of a storage benchmark based on games is actually good as there are no benchmarks like that and storage performance affects gaming much more than it used to be some years ago. I haven't seen how all test results look like so I can't say much about it. There are only two results that seem interesting and it's MB/s and access time. If there will be only some weird score then I won't use it as it's impossible to compare to daily usage.
At $2.99 I believe that is a pretty small hurdle. I do agree that most users won't care about this or will run it one time to get the results and then forget about it. As far as the single score at the end, I view this the same as Anvil Storage Utility. Where Anvil produces a single synthetic score, 3DM SB generates a real-world single score. By no means should a potential buyer base their purchase based on one benchmark. This just gives reviewers another tool in the toolbox.
I also agree that the results from this bench should be broken down to the individual benches before tallying them up for the final score. This would help determine what area a drive may be lacking or excelling in.
If memory serves me correctly there was an older 3DMark that measured complete system performance. I hated that one because the storage area always dragged my final score down.
I also agree that the results from this bench should be broken down to the individual benches before tallying them up for the final score. This would help determine what area a drive may be lacking or excelling in.
If memory serves me correctly there was an older 3DMark that measured complete system performance. I hated that one because the storage area always dragged my final score down.
Haven't touched it. Likely won't. There's other, better(?) tools out there.
I do like how it goes through several different scenarios though to test different things. Worth noting is that just because the drive isn't 'maxed out' doesn't mean the tests are bad or not doing enough. It's just the test and how the data is processed that determines what kind of load is on the device. For example, I know game loads don't slam the SSD to 100%. Perhaps other tests there are like that too? Not sure.
If you want to get a good idea of performance on your drive(s), I suggest looking at multiple tests. I use a combination of CDM and ATTO and get a mix of results in CDM and 'best case' (read: box specs) out of ATTO.
RE: a score, can't say I mind an all-ecompassing value at the end, but it would be nice to see the details, absolutely.
I do like how it goes through several different scenarios though to test different things. Worth noting is that just because the drive isn't 'maxed out' doesn't mean the tests are bad or not doing enough. It's just the test and how the data is processed that determines what kind of load is on the device. For example, I know game loads don't slam the SSD to 100%. Perhaps other tests there are like that too? Not sure.
If you want to get a good idea of performance on your drive(s), I suggest looking at multiple tests. I use a combination of CDM and ATTO and get a mix of results in CDM and 'best case' (read: box specs) out of ATTO.
RE: a score, can't say I mind an all-ecompassing value at the end, but it would be nice to see the details, absolutely.
Real-life results are almost always far from results in synthetic benchmarks. PCMark storage tests usually end at about 500MB/s on modern SSD, but this is random performance in some specific scenarios. The SSD can't make more in this scenario so it still runs at its maximum speed, just can't push as high as its specified maximum sequential bandwidth.
This is why the fastest storage for daily work or gaming, doesn't mean the one that can make the highest bandwidth in a synthetic, sequential bandwidth benchmark. As long as ATTO or similar benchmarks are great for comparison, then are telling us barely anything about the storage performance in daily work or games. Here appears PCMark and now 3DMark with storage tests based on popular applications. The only problem is to present test results in the right way, so they will be clear and comparable for everyone.
@Blaylock, you mean PCMarks in general. There is a new PCMark 10 that has some more storage tests and shows results in MB/s. Tests are based on popular software so the same as the new 3DMark test which uses popular games as a test environment. So PCMark covers the office/home environments while 3DMark focuses almost only on gaming.
This is why the fastest storage for daily work or gaming, doesn't mean the one that can make the highest bandwidth in a synthetic, sequential bandwidth benchmark. As long as ATTO or similar benchmarks are great for comparison, then are telling us barely anything about the storage performance in daily work or games. Here appears PCMark and now 3DMark with storage tests based on popular applications. The only problem is to present test results in the right way, so they will be clear and comparable for everyone.
@Blaylock, you mean PCMarks in general. There is a new PCMark 10 that has some more storage tests and shows results in MB/s. Tests are based on popular software so the same as the new 3DMark test which uses popular games as a test environment. So PCMark covers the office/home environments while 3DMark focuses almost only on gaming.
Some tests require MS Office or Adobe software to be installed. It's actually a problem as most of that soft is signed to the PC or user's account in some other way. Even business licenses have limited activations. I'm just not using these benchmarks. I think that's why I wasn't using Procyon too. I don't remember what benchmarks were there. I have a license but it required too many other things. It's just too problematic, especially that I reinstall OS quite often. I already download 250GB+ 3D benchmarks for comparison each time I reinstall OS, as Ubisoft or other games with benchmarks can't be just copied or installed offline.
I really hope that the 3DMark storage benchmark will be improved soon as I wish to use it in storage or even motherboard reviews. Most benchmarks give only synthetic bandwidth results which mean nothing to most users. I just wish to make storage reviews a bit more interesting.
I really hope that the 3DMark storage benchmark will be improved soon as I wish to use it in storage or even motherboard reviews. Most benchmarks give only synthetic bandwidth results which mean nothing to most users. I just wish to make storage reviews a bit more interesting.
That's why you set it up and image your OS. Then, simply restore the image. Once the base OS is setup and the image captured, it's literally a 10-15 minute process (for me and my test OS) to have a fresh OS up and running from scratch with all of my applications/benchmarks/etc ready to go. 
Procyon requires MS Office, Photoshop/Lightroom Classic and Premiere Pro. I've been through 4 boards so far with the key I have for Procyon and the Adobe apps and haven't run into issues (outside of having to log in every time). Occasionally I get a warning that I'm signed in on another device, but I just log it out and move on. So yes, it's tied to your account. I just log in before I bench and go!
Procyon requires MS Office, Photoshop/Lightroom Classic and Premiere Pro. I've been through 4 boards so far with the key I have for Procyon and the Adobe apps and haven't run into issues (outside of having to log in every time). Occasionally I get a warning that I'm signed in on another device, but I just log it out and move on. So yes, it's tied to your account. I just log in before I bench and go!
- Joined
- Mar 7, 2008
I haven't tried it myself, but Ubisoft Connect launcher does have a "locate installed game" function. Implies you can copy that between installs.
That's why you set it up and image your OS. Then, simply restore the image. Once the base OS is setup and the image captured, it's literally a 10-15 minute process (for me and my test OS) to have a fresh OS up and running from scratch with all of my applications/benchmarks/etc ready to go.
With so many tests on various platforms, I would be crazy to reinstall everything each time I test something
I used to have the same OS which after a driver refresh was working on every motherboard. So I had 2 SSD with all benchmarks and everything was fine. I could clone the SSD and was like 10 mins max and I could bench. Right now when I move AMD OS to Intel mobo then it's crashing. On Win10, storage drivers are causing boot issues and I can't repair OS. On Win11 there are sometimes additional problems. I can move the same OS between different AMD setups or between Intels but not between AMD and Intel. It used to work as OS was loading standard AHCI driver, but not anymore. I'm not sure what exactly has changed in the last months.What I said is more a problem about additional licenses. I'm not going to buy other software licenses only to run some tests. Even in business, an Office license is now signed to the PC and you can't activate it more than 20 something times (after 3 or 5 activations you have to call support to unlock it). I don't remember how many exactly but I was talking with the MS rep and he said that their system is locking licenses which were too many times activated on different computers. In all cases that I had, it was ~20-25 activations. The same for retail and business licenses.
Now Adobe is more flexible but also causes problems. If you use the license on too many various computers then they suspect you are not using it within license terms. One of my clients had recently a visitation from BSA because Adobe reported them. They were using Photoshop and Illustrator on 5 computers, switching users from time to time. All legal licenses and now have problems.
I was trying it once but it couldn't find the game. I will try it in case I reinstall OS the next time. Right now all works fine on Z590/Z690 mobos and I'm not planning to test any AMD anytime soon.
- Joined
- Nov 1, 2009
- Location
- Austin, TX
So far I've used this on 2 drives, a SATA 6GB/s 860 EVO 2TB SSD, and a pcie 4.0 NVMe 1TB 980 Pro. So far, it seems to be good enough to say that newer tech is faster than older tech, so least recommendations can be made for PC's when it comes to value for money where the tech is concerned.
You'll notice how at least it gives storage bandwidth (MB/s) and storage average read times (in microseconds)
EDIT: I should also note that i do NOT optimize for benchmarks. My machine is running in "daily driver" mode, not "racetrack" mode. it's running steam, origin, battle.net, epic gamestore, precision X1, ICUE, virus software, and whatever else windows uses, plus more i don't know about, in the background. and i'm not sure if the web-browser was open during the benchmark.
You'll notice how at least it gives storage bandwidth (MB/s) and storage average read times (in microseconds)
EDIT: I should also note that i do NOT optimize for benchmarks. My machine is running in "daily driver" mode, not "racetrack" mode. it's running steam, origin, battle.net, epic gamestore, precision X1, ICUE, virus software, and whatever else windows uses, plus more i don't know about, in the background. and i'm not sure if the web-browser was open during the benchmark.
Similar threads
