RAID Array Stripe Alignment affect Performance

[klau1]

Just found out that RAID arrays can have alignment issues, separate from the well known 4K sector issue.

I suspect it to be the cause of slow transfer rates on my Adaptec RAID 3805 RAID6 (5x Hitachi 2TB 7200 RPM) array.

Nodesoft Disk Bench only gets 100MB/s writes and 80MB/s reads (these figures are more in line my Windows File Copy performance)
http://www.nodesoft.com/diskbench

But I don't know why the following benchmarks report so much higher

Atto/HDtach:



Crystal:




However,
I think I have confirmed that my array is not aligned, but am not sure.

State of Array:



Results:

Alignment Calculator:
http://www.techpowerup.com/articles/other/157




Alignment Checking Script:
http://www.ocztechnologyforum.com/f...ax-performance&p=518021&viewfull=1#post518021

Device ID: Disk #2, Partition #0
=======================================================
Total Volume Size: 5 TB
Starting Offset: 134235136 (131089 KB)
Partition Alignment: ** NOT ALIGNED!! **

VM Check Alignment Tool:
http://ctxadmtools.musumeci.com.ar/VMCheckAlign/VMCheckAlignment10.html






My Calculation:
Formula: ((Partition offset) * (Disk sector size)) / (Stripe unit size)
Source: http://support.microsoft.com/kb/929491

Partition off set = 134235136b
Disk sector size = 512b
stripe unit size = 262144b
block size = 512b

answer = 262178

This is an integer, thus indicates the array is aligned. But this conflicts with the conclusions of the other calculators, so WTF? I have no idea what I'm doing lol






Array Info:
Adaptec RAID 3805 RAID6
HD 5x (Hitachi 2TB) Sector Size: 512Bytes
Stripe Size: 256K
Disk Format: GPT
Windows Disc mode, Basic or Dynamic: Basic

 

[Automata]

Those numbers are certainly odd. The graph in the first picture makes it seem like something is limiting the transfer, because it should be a declining line. A misaligned stripe wouldn't cause that sort of number discrepancy, but it would hurt performance.

Do you have the BBU unit for that card and do you have write back cache enabled?

 

[klau1]

Yes, BBU is installed, Write and Read Cache enabled on the RAID controller.
I've tried enabled and disabled modes, and the transfer speeds are both pitiful.

 

[Solder]

When I recently reinstalled Windows, I used diskpart to create and align the partitions of both of my RAID 0 array's. I had been informed here in OC forums that the alignment should
match the stripe size. I set the stripe size to 128KB and the alignment to 128 as well. Holy cow! What a difference that made. I rebooted into the disk again and began the install.
Figuring I had some time to kill, I stepped out for a few minutes (about 5), when I came back in, it was sitting there waiting on me having completed the install already. I can't believe
it took less time to install Windows than it did to smoke a cig! What a treat that was. FYI, I set the same stripe and alignment on my HDD RAID as I have on my SSD RAID,
128 stripe and 128 alignment. It's a rockn' good time

 

[Automata]

Yes, BBU is installed, Write and Read Cache enabled on the RAID controller.
I've tried enabled and disabled modes, and the transfer speeds are both pitiful.

Is there a program you can install in Windows to check the status of the card or configure it? The CrystalDiskMark benchmark of 32 queue depth write should be much faster than the read because it can store it in memory and write it sequentially to the disk. From what you've posted, I don't think write back cache is enabled or there is another option limiting your transfer.

Right now, it is doing 179.2 IOPS, which seems low.

 

[Automata]

To show how bad it is, here is a virtual machine running CrystalDiskMark running on 9x 2tb Hitachi drives in RAID 6 under a LSI RAID card. Performance should be similar to yours, if not lower due to more disks in the array. This VM doesn't have the benefit of direct access to the driver on the host OS, so the performance is lower running directly on the host itself (host is Linux, so I can't run it directly).

 

[klau1]

These are the settings on Adaptec Storage Manager.

 

[klau1]

Here is the test with RAID0 using 2x Kingston HyperX 3K 128GB SSD








Network Copy (1GPs)










This test should show the upper limits of the Adaptec 3805.

Does it look normal for a Adaptec 3805?

The purpose of this test is to rule out a problem with limits on the Adaptec 3805. The stripe, size is still 256K . The write and read caches are on.

The SSD is aligned by Paragon Alignment Tool.

 

[shadowdr]

I am not sure how it relates to what you are looking for but the ASSSD bench of the HyprX drives is really poor for an SSD raid0. If you test one of those drives on a regular sata port it would be faster. I looks like you are more limited by the platform you are running than the raid controller. That is a socket A mobo with ATA 133 bandwidth limitations? I don't remember what the PCI lane bandwidth limitations are but I would think that is the problem you are experiencing and not a problem with the card itself.

 

[klau1]

the motherboard is actually a Socket 775 runnng Intel E2160.

Motherboard: P5E VM DO
Adaptec 3805: PCI-E 8X
Adaptec 3805 installed in the 16X PCI-E slot. (no 8x slot is available on board)

 

[shadowdr]

Ok. guess it has been a while since you updated your signature. This does suggest some problems with the card or the way it is setup. Here is a shot of a single SSD using the AS SSD bench, keep in mind that this is an M4, much slower than the HyperX drives that you have in a raid0 using the card. Perhaps the slot is shared with the onboard video or other IRQ's or perhaps the cards drivers are very old, I just don't know much about adaptec cards. In any case if that is the best that the card would provide the onboard sata III slots would seem to be much better.

 

[klau1]

This is a SATA 2 (300) card.

Would that change your suspicion?

 

[shadowdr]

This is a SATA 2 (300) card.

Would that change your suspicion?

Not at all, even a sata II sould only have the bandwidth limitation on each individual port. So with a raid0 it should realize double the sata II limitation, about the same as a single SSD or a little more. I would bet that if you attached just one of those drives to the onboard sata port it would beat the AS SSD score of the raido card.

 

[klau1]

In any case, poor alignment should affect only access performance of small files (ie: <1MB)

NOT

large gigabyte range files

Right?

 

[Automata]

It will affect everything as it has to request twice the data, but sequential data shouldn't be hurt as much as very small files. The problem is, alignment problems aren't going to give flat graphs with mechanical disks.

 

[shadowdr]

That^, but also there must be something else holding the card back. I wish that I knew more about the alignment that you suspect might be giving you poor performance but again, the AS SSD bench of raid0 is really bad. If it was installed in win7 or Vista the partition alignment should always be good and yet the score is super low. I was not even aware that the nand size would have any effect on performance but I don't think that it has anything to do with the mechanical drive array and the Alignment Calculator appears that it is for SSD's. The flat graphs on most of the screenies do look like it has a bandwidth limitation somewhere. Perhaps there is a newer bios or driver for the card?

 

[shadowdr]

Just found out that RAID arrays can have alignment issues, separate from the well known 4K sector issue.

I suspect it to be the cause of slow transfer rates on my Adaptec RAID 3805 RAID6 (5x Hitachi 2TB 7200 RPM) array.

Nodesoft Disk Bench only gets 100MB/s writes and 80MB/s reads (these figures are more in line my Windows File Copy performance)
http://www.nodesoft.com/diskbench

My Calculation:
Formula: ((Partition offset) * (Disk sector size)) / (Stripe unit size)
Source: http://support.microsoft.com/kb/929491

Partition off set = 134235136b
Disk sector size = 512b
stripe unit size = 262144b
block size = 512b

answer = 262178

This is an integer, thus indicates the array is aligned. But this conflicts with the conclusions of the other calculators, so WTF? I have no idea what I'm doing lol






Array Info:
Adaptec RAID 3805 RAID6
HD 5x (Hitachi 2TB) Sector Size: 512Bytes
Stripe Size: 256K
Disk Format: GPT
Windows Disc mode, Basic or Dynamic: Basic

I did some reading on the KB article that you linked to above and it looks like the answer you came up with is correct however the number 262478 is not evenly divisible by 256 or 512. It looks to be 34b off because the resulting 262144 would be the correct offset and is divisible evenly by 265 and 512.
The 134235136 offset divided by the block size 512= 262178
The correct off set should be 134217728 / 512= 262144

4.To determine how many disk sectors a given partition starts from the beginning of the disk, divide the value for StartingOffset by the value of BlockSize. In the example in step 2, the following calculation yields the partition starting offset in sectors:

32256 / 512 = 63

from the article so,
262178 /512 = 512.0664062 this should be 512
I don't know if you can align without hosing the data on the array but there is a link on that article specifically for multi partition disk alignment on raid arrays.

 

[Woomack]

Adaptec controllers are generally slow. Some ( if not all ) have additional delay on access times. Noone is using these controllers for SSD just because they are limiting transfers by ~20% comparing to the same based on LSI.
Running RAID5/6 will also limit transfers ( especially writes ). Good or bad alignment, it won't be much better in this config especially that's older SAS3G controller.
Card is based on older generation processor and in this case I bet that integrated controllers on newer boards will be faster especially for random transfers.

 

[shadowdr]

This card does have the ability to use an SSD for MaxCache which may with proper alignment boost read and write speed for files smaller than the SSD to a very reasonable rate. It looked like the misalignment was costing the writes about a third of it's normal speed while the reads suffered, less probably because many of the smaller files may not have crossed the boundries.