- Joined
- May 7, 2011
- Location
- Cook->Kent
Howdy all.
So I've decided to delve into the foray that is ZFS. I'm trying to learn as much as I can and I hope there's some low hanging fruit/easy wins here. I'm consolidating/upgrading/whatever you want to call it my storage setup. In particular moving from a DAS to a NAS. Current DAS is a RAID 5 on an Areca 1880i with 4x 2TB WD Blacks. 128k stripe on 4k block. Current performance is below.
NAS, specific to the replacement, is 6x 3TB WD Reds behind an Areca 1882xi in JBOD mode configured as raidz2, lz4 compression, 128k record size, with two datasets behind it on an SMB share. FreeNAS is a VM with 8 cores and 24GB memory at the moment. Host has 64GB but as one can imagine, I have a few other VMs, though I'm only dedicating another 8GB to one, so everything else is a bit dynamic. I'll eventually add another 64GB. As configured, my current performance difference from the DAS and a 10Gb link to the NAS is here...
I have a PLEX VM as well. Locally, that is to say an SMB share across VMs within the same host, my sequential numbers double at QD8 and about a 25% boost at QD1 for read. Not quite as dramatic on writes, but close. Random stays the same. So in this case, network latency is obviously to blame for a bit of it (about ~258us RTT)
I'm not concerned about the issues around these cards and smart data at the moment. I'm simply looking at it from a performance perspective and intend to get a new HBA at some point.
I've seen reference to cut VM dedicated memory and bump memory dedicated to ARC for a pretty good boost, which is to say just get a bigger cache in a manner of speaking, though I've not yet found details on how to do that. I'd assumed it used what it needs/can automatically. While the performance boost I already have with the current conf is substantial, I'm curious to know how to get more with the configuration alone. I don't know what the easy way is to provide the current configuration other than the above.
For the record, in this instance, NICs are Chelsio T4 LL versions. Server is in my sig. NIC in the server is in slot 2. Both connected to a Unifi XG-6-POE. Haven't looked at any NUMA considerations, if even warranted at all in this scenario.
So I've decided to delve into the foray that is ZFS. I'm trying to learn as much as I can and I hope there's some low hanging fruit/easy wins here. I'm consolidating/upgrading/whatever you want to call it my storage setup. In particular moving from a DAS to a NAS. Current DAS is a RAID 5 on an Areca 1880i with 4x 2TB WD Blacks. 128k stripe on 4k block. Current performance is below.
NAS, specific to the replacement, is 6x 3TB WD Reds behind an Areca 1882xi in JBOD mode configured as raidz2, lz4 compression, 128k record size, with two datasets behind it on an SMB share. FreeNAS is a VM with 8 cores and 24GB memory at the moment. Host has 64GB but as one can imagine, I have a few other VMs, though I'm only dedicating another 8GB to one, so everything else is a bit dynamic. I'll eventually add another 64GB. As configured, my current performance difference from the DAS and a 10Gb link to the NAS is here...
I have a PLEX VM as well. Locally, that is to say an SMB share across VMs within the same host, my sequential numbers double at QD8 and about a 25% boost at QD1 for read. Not quite as dramatic on writes, but close. Random stays the same. So in this case, network latency is obviously to blame for a bit of it (about ~258us RTT)
I'm not concerned about the issues around these cards and smart data at the moment. I'm simply looking at it from a performance perspective and intend to get a new HBA at some point.
I've seen reference to cut VM dedicated memory and bump memory dedicated to ARC for a pretty good boost, which is to say just get a bigger cache in a manner of speaking, though I've not yet found details on how to do that. I'd assumed it used what it needs/can automatically. While the performance boost I already have with the current conf is substantial, I'm curious to know how to get more with the configuration alone. I don't know what the easy way is to provide the current configuration other than the above.
For the record, in this instance, NICs are Chelsio T4 LL versions. Server is in my sig. NIC in the server is in slot 2. Both connected to a Unifi XG-6-POE. Haven't looked at any NUMA considerations, if even warranted at all in this scenario.
Last edited:
