ICH9R RAID 5 SLOWqr

[brooklynbeany]

I built a brand new VISTA 32 bit PC (see spec in sig file) and have a single drive C drive and a D Drive (Raid 5 of five disks). The D drive is on ICH9R and the C drive is on the GSATA. The ICH9R has writebackcache enabled and the clip is removed for the 300.
I have only used Windows copy (10+ gig zip file) and get slow slow copy speeds. Here are some of the results:
Pre-Raid – disk to disk – 30meg transfer rate
Built Raid - C drive to D drive – 20 meg transfer rate
Built Raid – external USB drive to C drive – 30meg transfer rate
Built Raid – external USB drive to D drive – 20meg transfer rate

I believe that I should have at least 80meg transfer rate. I am really stuck and have spent at least 30 hours trying to work on this and have rebuilt the ICH9R drives many many times.

THANKS
Beany from Brooklyn

GIGABYTE GA-P35-DS3P LGA 775 Intel P35 (ICH9R)
Intel Q6600
AMS DS-3151SSBK Aluminum 3.5B SATA 5-drive hot-swap
6 Seagate 500 GB SATA II HDD (RAID-5 for 2 TB storage + C Drive)
Thermaltake Armor Series VA8000BWS
DDR2-800 RAM - 4 gig total
SeaSonic 550 watt silent PSU
Leadtek PX8800 GTS TDH 320MB

[Sam__]

I'm not sure what the problem is that but can we see some benchmarks please? so we can get more information

[brooklynbeany]

Thanks Sam for the quick reply.
I am not sure what benchmark to use. I thought the best was the real world Vista copy.
Please let me know what program to use. Should I use a separate program and if so, which one? I saw on another post ATTO but I went to their website and could not find anything.
Also, when I duplicate a file on the D drive (Raid 5 w/five drives) the windows copy speed is the same as a copy from the C drive.

[conway]

Try Hdtune

[Madwand]

With RAID 5, you should measure write performance in addition to read.

http://kb.ciprico.com/lore/article.php?id=268

Try different stripe sizes (e.g. 16k) and perhaps reducing the array size to 4 drives. It should perform better, but with RAID implementations, there are no guarantees that theory matches reality, and performance quirks arise from time to time.

[jmorgan]

Just to give you an idea of what you should be able to do. My raid 5 on 4 seagates on HDtune do

171mb/s avg and 13.2ms I have a raid 0 slice of 120gb on the faster portion of the drives so you should be higher then 170mb/s in HDtune

ATTO benchmark does read and write, you should do that bench also

But when transferring to or from a 500gb drive I have it does about 50mb/s I think thats more of a limit of the single drive

Oh yea you might try with one less drive also and hook up the 4 drives to the first 4 sata ports, alot if not all boards have 4 sata ports that use the southbride, I have heard sata ports 5-6 can use another onboard controller, they may not be playing nice together, mine are currently on 2-5 ports but im going to change mine to 1-4 next time I shut down.

[bing]

For ICHxR Raid 5, the "optimum" strip size is 64K, what is yours ?

[brooklynbeany]

I ran ATTO both with direct I/O enabled & disabled.
I also HD Tune 2.54.
This was all run on Intel software version 7.6.
I noticed that the 2 ATTO tests were vastly different. Also I can not coorelate the HD Tune & ATTO tests at all. Also I am wondering why the Windows Vista copy is so so slow compared to the high speed results I am getting.
My MB comes with 6 ports that are spec’ed to be on ICH9R and 2 other on GSATA. The drives are on ports 1-5 that way my drives are in order in the housing. Also, the strip size is the default size by ICH9R - 64.

I am willing to break the RAID array to test it but given it takes many hours to do it I was wondering what to do first, second, etc to get the maximum test information.

[Madwand]

I run ATTO with Direct I/O enabled and "Neither". Forget disabling Direct I/O as it obviously gives wacky results.

There is some correlation between ATTO and HDTune -- look at the 64k accesses. ATTO reported ~230 MB/s and HDTune ~284 MB/s max. These aren't that far off considering how different the tests are. Besides, the issue is not nit-picking ~20% performance, but the huge difference between read and write speeds here. Your copies are slow because the write speed is slow, not because of the difference between 230 MB/s and 280 MB/s at max. Apparently the write caching is either not enabled properly or not working as it should be.

If you have the OS on a separate drive, and not much data on the array to save elsewhere, you should be able to re-create the array without much trouble -- use a quick format if you haven't already.

[brooklynbeany]

The first test I will run will be to upgrade to v 2.8 of the Intel Raid controller software and then rebuild the Raid array as is.
When I shut off the writecacheing the Windows copy speeds are 50% slower (although I have not run ATTO or HD Tune that way yet).
Also, I have run the Seagate test on the driives as a Raid with no problems ID'ed with the disks.
PS: I used a full format which took all day!

[Sam__]

If those are your benchies then the read speeds are okay. The write speeds on the first two are a bit off.

[brooklynbeany]

I upgraded the Intel Raid Driver driver from v2.6 to v2.8. As you can see the write speeds have increased in the majority of the tests using ATTO. I have not rebuilt the raid yet.
Does this give me (us) any valuable info?

[bing]

WB cache enabled ?

[brooklynbeany]

The writebackcache is enabled according to the Intel® Matrix Storage Manager (v2.8).

[jmorgan]

run atto transfer size 1024kb, total length 32mb

I got more realistic results of 134mb/s read and 50mb/s write


Just transfered 11.5gb (mostly movies) and it was transferring from my raid 0 to raid 5 at 80mb/s

[brooklynbeany]

JMORGAN: DO you get these results if you copy from an external USB or firwaire hard drive to the Raid 5?
Those results are what I would have expectedfor my setup.

I ran the benchmarks again as requested but I also had an interesting result that I duplicated. When running ATTO with direct I/O off, I had some terrible speeds and some amazing speeds. Weird and not sure if others see that. The results are similar to the ones posted eariler by that title.
Also, i tested with writebackcache off and the numbers were horrible (reults not shown)

My C drive is a non-raid drive. When I copied between 2 non-raid HD's, I got 30mb/s. I now get 20 mb/s to my riad 5.

I think I have to break the Raid (yet again). Should I use the Intel Matrix software or the Intel Bio setting to do that? What tests or configuration (it was suggested that I bring it down to 4 drives)? Should I try 4 then 3 drives?
What tests besides running ATTO/HD Tune and windows copy tests should I try? Should I try it as a Raid 0 also? I am willing to do much more to try to resolve this I just want to try to be wise about it.

PS: I really appreciate all your help.

[Madwand]

Run ATTO with Direct I/O enabled and option "Neither".

Enable write-back caching on the array.

Back up any data and use quick format when re-creating.

Test different stripe sizes and drive counts.

E.g. 5 drives 32k, 16k
4 drives 64k, 32k, etc.

Stop when you get reasonably good figures. Note that ATTO is a quick & dirty tool -- don't worry about moderate run to run differences etc.; look at the overall pattern and ballpark instead.

E.g. the following are results for a 4-drive ICH8DO RAID 5 array with 64 KiB stripe size and write-back cache enabled, using drives which do around 70 MB/s max individually:

http://i89.photobucket.com/albums/k203/Madwand0/atto-ir5-4-64-4-Vista64.png

[brooklynbeany]

I am now getting somewhere.
When I broke the Raid and made a 4 disk array Raid 5 with standard setting (writebacke & 64bit). It finally started working at speeds like others have shown.
I then tested the stand alone drive and it was pretty fast also.
I then re-created the Raid5 w/5 disks and got the same junk performance.
Please note the ATTO tests.
I am going to now try smaller sizes 32 & 16 with 5 disks and see what the results are.
QUESTIONS:
1) After creating the new Raid, I then went into the Disk Manager (Vista) and it asked me to initialize the disks before logical disk manager can access and had 2 options: MBR (master boot record) and GPT (GUID partition table). Which one do I use?
2) After that I had to format the drives/volumes. I had the choice to do Simple or Spanned or Stripped. Which one do I use?
3) Also, (since I did this several times) sometimes it would make me initialize the volume which took 7+ hours. Any idea why? Either way is it a good idea to do it once I get this working right?

[Madwand]

I am now getting somewhere.
When I broke the Raid and made a 4 disk array Raid 5 with standard setting (writebacke & 64bit). It finally started working at speeds like others have shown.
I then tested the stand alone drive and it was pretty fast also.
I then re-created the Raid5 w/5 disks and got the same junk performance.


Great! -- though a bit disappointing because much of the attraction of ICH9R is the ability to have a single array with > 4 drives. Well, the other big attraction is the ability to support single arrays > 2 TiB, which I guess still applies if you use 4 750 GB or larger drives.


QUESTIONS:
1) After creating the new Raid, I then went into the Disk Manager (Vista) and it asked me to initialize the disks before logical disk manager can access and had 2 options: MBR (master boot record) and GPT (GUID partition table). Which one do I use?
2) After that I had to format the drives/volumes. I had the choice to do Simple or Spanned or Stripped. Which one do I use?
3) Also, (since I did this several times) sometimes it would make me initialize the volume which took 7+ hours. Any idea why? Either way is it a good idea to do it once I get this working right?

1. You need GPT if you have an array > 2 TiB. Otherwise, you can keep it simple and use MBR.
2. Simple.
3. I've never seen this feature / issue in ICH8x, so don't know.

[brooklynbeany]

Thanks for your review team. I have completed a number of tests:
- 64kb strip as shown in previous posts.
- 32kb strip – ATTO, windows copy & windows duplicate
- 16kb strip – ATTO, windows copy & windows duplicate
- For the windows copy & duplicate testes I used 2 folders. 1 folder had four ~10gigabyte files in it and the other folder had 22k jpeg’s (mainly ~800k & ~3megg in size)
- I also took the 16k strip drive and formatted it (windows format) with different allocation sizes. The upshot of this test was that anything except the default size was slow. Also I did not post the ATTO pics for this but can if people want to see them.

This was pretty time consuming because each time I redid the strip size the system initialized the volume (8 hours for 64, 11 hours for 32 and 14 hours for 16kb strip). I then (thanks to someone on another post struggling with this issue, carved out a 100meg volume and did the allocation size tests very quickly.

My Summary:
- 64kb strip works for a 4 disk array BUT not for a 5 disk array.
- I wonder if this is a bug or that the Intel driver should use a different default stripe size. Anyone know how to share this with Intel?
- Does this make sense to people?
- Since my computer will be used for pictures (with 5 kids this is growing fast) and video, I need to create the Raid5 volume with a stripe size of 16kb.

Have I missed anything? Thoughts or feedback before I finalize this configuration? Are there any crazy results that I should retest?

[Madwand]

I'm not exactly sure if the 32 KiB stripe size or the 16 KiB stripe size results look better from your measurements. Perhaps the 32 KiB because it does a bit better in the common 64 KiB access size.

However, any way you look at it, the configurations you've found give very good results for on-board RAID 5. Congratulations!

(P.S. don't forget to have some sort of external backup (e.g. an external drive) as well for your important personal pictures. RAID is good, but having an external backup on top of that is better -- RAID wouldn't do anything about user error, malware, software error, PSU failure damaging multiple drives, etc.)

[adv4nced53]

Sorry to hi-jack this thread but looking at this, I realized that I am going to need some help :-/

Here are my benchmarks:
http://www.speedrave.com/pics/computer/HDTune2.png
http://www.speedrave.com/pics/computer/ATTO1.jpg

What do you guys use to make changes? I realized that my HDTune graph is sooooo wavy due to the write speeds (correct?).
This is a 4 Seagate RAID5 and I need some guidance. Keep in mind I am pretty noob when it comes to this.

[Madwand]

Enable write-back caching, as shown here:

http://www.ocforums.com/showpost.php?p=5379140&postcount=13

Use the 256 MB ATTO test. The version which supports 256 MB tests can be found through here if needed:

www.hugesystems.com/supportspace/bench32.exe

Edit: Woops, didn't see that the above post was about nVIDIA RAID this time. Way off-topic, see the other thread. Intel tricks and issues do not apply here.

[adv4nced53]

Doesn't install for me. Different Mobo?

[Madwand]

Yes, as per my recent edit, your problem is with nVIDIA RAID, not Intel RAID. These are completely different. All the discussion, tweaks, measurements, experience and knowledge applicable to Intel RAID do not apply to you.

Your previous thread already has the appropriate information for nVIDIA. You could follow up on that further in that thread or in the linked StorageReview thread (or get a motherboard with an Intel RAID chipset).

[brooklynbeany]

madwand, et al,
The titles on the images are a bit off and also not as clear as I wanted. The biggest difference between the 64kb, 32kb & 16kb strip was that the speed of the Windows Vista copy kept getting (overall) much better.

[Madwand]

madwand, et al,
The titles on the images are a bit off and also not as clear as I wanted. The biggest difference between the 64kb, 32kb & 16kb strip was that the speed of the Windows Vista copy kept getting (overall) much better.

That's cool. You're right to look at actual application performance for doing the final bits of tuning.

[brooklynbeany]

I went to Intel (thru normal channels) and they believe there is nothing wrong and never heard of this issue before. They have told me to go the MB manufacturer (Gigabyte). Both companies are not very responsive.....

[fritzman]

This may be a bit off the wall, but had you considered grabbing a 150Gb Raptor for the O/S and running the 6 x 500's in Raid 10 on the ICH9R... reason I say that, is you would see huge performance benefits and still have 1.5Tb of storage (plus a mirror of the whole shebang).

[jhonyl]

brooklynbeany,

According to your post from '12-10-07, 11:02 PM' the problem that you have in copying files in windows is due to that you copy from the drive to itself. From drive D: to drive D: Hard disks work the fastest if they operate sequentially. By copying from the same drive to itself you ensure that the copying would not occur sequentially as the drive(s) head(s) go back and forth between the source file location and the destination file location. This movement of the drive(s) head(s) is a very time consuming action, and is called seek time, or access time. The HDTune image on one of your previous posts describes your access time as 11.4ms. More over even if copying from a drive to itself could have been done without seeking, since you both read and write to the same drive, this actions don't occur at the same time, so only half of the time you read and half of the time you write, and you measure what you "transfer", so your bandwidth will be half of what it really is. So when you get performance of only 20MB/s it really is 40MB/s because this 20MB where both read and written every second and That is 40MB/s. And when you get 40MB/s instead of 150MB/s, it just means that your drive was seeking during the difference: 150*pt+st=40*(pt+st),pt+st=1 => pt=39/149=0.26, st=0.74, and that means that in every second your raid array has spent 74% of the time on seeking and just about 26% of the time on actually reading or writing anything. Recalling that 11.4ms access time post: 0.74/0.0114=64, which means that your disks did 32 trips from the source to the destination every second, carying 0.625MB from source to dest' each time. If you want to test better, make a program that does not force seeks on the disk, and either read or write, like write a sequence of numbers from memory like 1234 1234 1234.... and then check that the data is correct. Timing the write would give you the write speed, and checking the data the read speed. Though I would guess that these disk benchmark programs already do just that.

*st=the amount of time in one second that is spent seeking
*pt=the amount of time in one second that is spent on actually reading or writing.