Storage setup for programming purposes

[Dolk]

I'm trying to think of a good storage system that would be best suited for local programming development. My case is hardware programming. FPGA dev programs can generate thousands of small to a few hundred meg files rapidly. My bet is to go SATA for where I'll be doing the active development, but would M.2 be worth it?

I'd think the most optimized would be RAID-0 SATA to build up my storage as needed.

 

[freakdiablo]

Unless you think you'd need large - as in >4TB - drives m.2 is still a why not. If we're talking many multiple hundred Meg files, then the added bandwidth w would come in handy.

Going with the latest 5.0 x4 drive may be overkill, but a high end 3.0 or mid range 4.0 drive should have a boost over SATA, even SATA in raid.

 

[EarthDog]

Going with the latest 5.0 x4 drive may be overkill, but a high end 3.0 or mid range 4.0 drive should have a boost over SATA, even SATA in raid.

This.


Edit: Unless you need loads of capacity to be fast (storing them in SATA drives sounds like a better use case, and faster m.2 for scratch/working file, I wouldn't think sata is even a consideration versus m.2. Even the 3.0 x4 drives are orders faster than R0 SATA. But I'm with freakD, 4.0 m.2 or bust.

 

[freakdiablo]

This.
. But I'm with freakD, 4.0 m.2 or bust.

(For the record, that's Uncle Freak and the East Street Band from here on out, thanks.)

 

[mackerel]

FPGA dev programs can generate thousands of small to a few hundred meg files rapidly.

What's the balance of small to large files? I'm wondering if random times will be more important than sequential rates. To cover both bases, I'd aim for a higher end 4.0 with DRAM model. Ones I've personally used and are happy with include Samsung 980 Pro, 990 Pro, WD SN850, and Kingston Fury Renegade. To help preserve random performance I'd avoid raid.

 

[Woomack]

I guess that the IOPS is the number you are looking for when comparing SSDs for this task. Then the best price-to-performance ratio gives high, but not necessarily the most expensive PCIe 4.0 SSD, like Crucial T500. Kingston Renegade/KC3000 (the same stuff, even though specs suggest that the Renegade is faster) is also pretty good for these things, but overall, it is slower than the T500 and costs the same or more (depending on the store). Another advantage of the T500 is that it runs cooler than most other SSDs from the higher PCIe 4.0 shelf. I also wouldn't set RAID for these tasks, but I already said what I think about it in other threads.

 

[Dolk]

Yeah my thought was to have a large M.2 (2-4TB) as the storage of the working code. Thats where a lot of high traffic random read/write of <1k to <500Meg files will be generated.

Agree with you, Woomack, on the RAID.

 

[infinitevalence]

if you want some extra umph there are still some Optane M.2 drives out on the market, when it comes to IOPS they crush traditional multilayer SSDs and for things like compiling and syncing can endure far more writes than TCL NAND.

I would look at either a mobo with 4 M.2 or an add in card that lets you put more M.2 into your system. It may require PCIe Bifurcation support and for that Gigabyte AMD boards have been awesome, all 3 of mine have solid support for splitting the 16x into either 8x8 or 8x4x4 or 4x4x4x4x.

 

[Dolk]

https://www.dell.com/en-us/shop/rev...rationId=8b258ac9-8475-4853-b02e-07c9f4f77954

Simple but I think it will be enough for my local compiling.

 

[EarthDog]

Simple, LOL...you need a $7.5K workstation for this activity?

 

[Dolk]

Someone's paying

Each compile instance can take up to 20-30GB of RAM. I'll typically have a few running at a time. Cores will be fine, RAM and storage throughput become the issue.

 

[infinitevalence]

that will get it done, do you need an $8000 Dell, or could you get something more reasonable?

Part List - AMD Ryzen 9 9950X, Radeon PRO W7700, Fractal Design North ATX Mid Tower - PCPartPicker

Part List - AMD Ryzen 9 9950X, Radeon PRO W7700, Fractal Design North ATX Mid Tower

pcpartpicker.com

Granted this is DIY so maybe wont have the support and service your looking for, but IMO this offers much better value.

Another option is https://www.pugetsystems.com/workstations/ryzen/r121-l/ these are more custom than Dell but again, better value IMO.

 

[EarthDog]

Each compile instance can take up to 20-30GB of RAM. I'll typically have a few running at a time. Cores will be fine, RAM and storage throughput become the issue.

I still think you're fine on the latest and greatest. They support up to 256GB of DDR5 and it's practically quad channel anyway with how it's being addressed (4x32).

It's not your money, so, whatevs!

 

[infinitevalence]

Intel based Sapphire Rapids systems also may offer a really good value here as you can get decent thread counts, and great memory capacity.

 

[Dolk]

256GB option was like 2.5x of 128GB. I think I'll be fine with 128GB but we'll see.

infinitevalance, please read my sig

 

[infinitevalence]

lol, FINE cant really go wrong with TR.

 

[freakdiablo]

Simple, LOL...you need a $7.5K workstation for this activity?

*Checks Map*

*Flips it around*

I'm sorry, I thought this was OCF, not BasicB*tchville!

*Runs*

 

[EarthDog]

LOL! Fo sho!

But there's a difference between spending money and wasting it. But it is EOY and there's probably some budget that needs to be spent. Otherwise, I'd imagine he could easily get away with something half the cost and on X870E/Z890. That workstation doesn't offer much over what he needs.

A 9950X + 128GB of DDR5-6400 = Win at 1/2 the cost.