- Joined
- May 15, 2006
Perc 5/i (Dell, LSi) RAID card benchmark thread
About this card:
This is a PCI Express based RAID card made by LSI for Dell. It is normally found in their PowerEdge servers. The RAID card will fit in any "video card slot" (PCI Express) and is an 8x card physically and electrically. There are two SFF-8484 connectors at the top to give a total of eight SATA or SAS hard drives. There is also a DDR2 slot that is used for the "Write Back Cache".
Why this card and why benchmark it?:
RAID cards are normally many of hundreds, if not thousands of dollars for a good one. This Perc 5/i can be had for anywhere from $75 - $125, which may even include the BBU (Battery Backup Unit) and breakout cables. This makes the card an extremely good choice for anyone that doesn't want to spend a lot of money on a good RAID card. To get the most out of the card, we need to break down how settings affect performance. This requires quite a bit of time and thought to get it right; I've done both of those for you.
NOTE: In my tests, I enabled "Force write back" which is EXTREMELY DANGEROUS to data integrity without a BBU installed and working on your system. I did this only because I don't have the required hardware (BBU) to enable this feature normally and I have absolutely no data to lose on these hard drives. DO NOT use the "Force write back" option when using the system for storage.
Modification needed for desktop systems:
Since this is a card originally made for Dell servers, there are some obstacles that we must overcome. The first is the most obvious, cooling. The servers that these came from where specifically designed to cool this card. Most desktop systems will have no where near the airflow these cards need to stay cool. The stock heatsink is extremely small and has little surface area. Many people have modded this card and there really isn't a wrong way to do it as long as it stays cool. Here is how I modded it.
I ripped the stock retention bar off the stock heatsink (the black heatsink in the above pictures) and used it on the Zalman northbridge heatsink to secure it. I then used some 3M thermal tape to secure the stock heatsink to the other chip on the card. With this combination, I can run two Perc 5/i cards in a normal SLi/Crossfire based motherboard. Cooling wise, I just use the normal case fans with no modifications, the heatsink is large enough that it does not need a ton of airflow to stay cool. If you need the space, you can either use a smaller heatsink or "fan" out the pins of the heatsink. You will also want to add extra cooling if you do this.
The other important modification is a trickier one. Most motherboards will have a conflict with the power management of the card, either causing the motherboard not to POST or for the card to not be detected. There are two pins on the card that need to be covered up. There are multiple ways to do this. If you want a more permanent solution, you can use an adhesive (finger nail polish, etc). If you want a temporary solution, you can use electrical tape. The only issue with tape is it moves easily if you need to install or remove the card frequently.
There is a very good thread here that shows how to modify the pins. It is listed under the "SMBus Issue with Intel Chipsets". I had to do this mod on my motherboard and it is not an Intel based one. Be prepared to do this mod on any board.
Testing Methodology:
To properly test this card, we need all variables to stay the same except a single, controlled one. Here is a breakdown of the hardware that I used for these tests:
Motherboard: Asus M2N32-SLi
CPU: Phenom X4 (AM2) 2.6GHz
HDD:
----320gb Seagate (Operating system drive)
----7x 1tb Hitachi and 1x Western Digital Green (Perc 5/i)
Raid controller: Perc 5/i PCI-e
PSU: Corsair HX620
Case: Norco 4020
OS: Windows 2003
Pictures 1
Pictures 2
I will be using a multitude of programs to determine what settings are best as no one program will give definite results. The first program I will use is the "ATTO" benchmark to show how each option changes the performance of each RAID level and at what file size. The second I will use is HDTune Pro to test sequential throughput, random access and how fast it is with different file sizes.
Program: ATTO Disk Benchmark
Transfer Size: 0.5 to 8192kb
Total length: 256mb
Direct I/O: Enabled - Overlapped I/O
Queue depth: 4
Program: HDTune Professional
File length: 512mb
The hard drives were put in each specified RAID level with a size of 200gb, initialized in Windows, converted to a GPT disk and formatted with NTFS (quick). After that, the test was launched and started. Screenshots were taken after each benchmark. The RAID was then deleted from the RAID array and recreated with the new settings to prevent settings from not changing between tests. The server was NOT restarted between tests and I used the Dell OpenManage software suite to configure the Perc 5/i from a web browser.
The RAID levels used are 0, 5, 10 and 50. There will be a total of 6 tests per RAID level. Each "Read ahead" setting will be tested along with "Write back". To enable "Write back", I had to use the "Force Write back" option since I don't have the BBU (Battery Backup Unit); a built in safety mechanism that you can override. Stripe size was kept to a constant 64kb throughout all tests. Here are the settings used for each RAID level:
Read Ahead - Write Back
Adaptive Read Ahead - Write Back
No Read Ahead - Write Back
Definitions:
Adaptive Read Ahead - Adaptive read ahead is a read policy that specifies that the controller begins using read-ahead caching if the two most recent disk accesses occurred in sequential sectors. If all read requests are random, the algorithm reverts to Non read ahead; however, all requests are still evaluated for possible sequential operation.
Read-Ahead - A memory caching capability in some controllers that allows them to read sequentially ahead of requested data and store the additional data in cache memory, anticipating that the additional data will be needed soon. Read-ahead supplies sequential data faster, but is not as effective when accessing random data.
Write-Back - In write-back caching mode, the controller sends a data transfer completion signal to the host when the controller cache has received all the data in a disk write transaction. Data is written to the disk subsystem in accordance with policies set up by the controller. These policies include the amount of dirty/clean cache lines, the number of cache lines available, elapsed time from the last cache flush, and others.
Write-Through - In write-through caching mode, the controller sends a data transfer completion signal to the host when the disk subsystem has received all the data and has completed the write transaction to the disk.
Definitions pulled directly from Dell's own website here:
http://support.dell.com/support/edocs/storage/RAID/PERC5/en/UG/HTML/glossary.htm#wp1044435
About this card:
This is a PCI Express based RAID card made by LSI for Dell. It is normally found in their PowerEdge servers. The RAID card will fit in any "video card slot" (PCI Express) and is an 8x card physically and electrically. There are two SFF-8484 connectors at the top to give a total of eight SATA or SAS hard drives. There is also a DDR2 slot that is used for the "Write Back Cache".
Why this card and why benchmark it?:
RAID cards are normally many of hundreds, if not thousands of dollars for a good one. This Perc 5/i can be had for anywhere from $75 - $125, which may even include the BBU (Battery Backup Unit) and breakout cables. This makes the card an extremely good choice for anyone that doesn't want to spend a lot of money on a good RAID card. To get the most out of the card, we need to break down how settings affect performance. This requires quite a bit of time and thought to get it right; I've done both of those for you.
NOTE: In my tests, I enabled "Force write back" which is EXTREMELY DANGEROUS to data integrity without a BBU installed and working on your system. I did this only because I don't have the required hardware (BBU) to enable this feature normally and I have absolutely no data to lose on these hard drives. DO NOT use the "Force write back" option when using the system for storage.
Modification needed for desktop systems:
Since this is a card originally made for Dell servers, there are some obstacles that we must overcome. The first is the most obvious, cooling. The servers that these came from where specifically designed to cool this card. Most desktop systems will have no where near the airflow these cards need to stay cool. The stock heatsink is extremely small and has little surface area. Many people have modded this card and there really isn't a wrong way to do it as long as it stays cool. Here is how I modded it.
I ripped the stock retention bar off the stock heatsink (the black heatsink in the above pictures) and used it on the Zalman northbridge heatsink to secure it. I then used some 3M thermal tape to secure the stock heatsink to the other chip on the card. With this combination, I can run two Perc 5/i cards in a normal SLi/Crossfire based motherboard. Cooling wise, I just use the normal case fans with no modifications, the heatsink is large enough that it does not need a ton of airflow to stay cool. If you need the space, you can either use a smaller heatsink or "fan" out the pins of the heatsink. You will also want to add extra cooling if you do this.
The other important modification is a trickier one. Most motherboards will have a conflict with the power management of the card, either causing the motherboard not to POST or for the card to not be detected. There are two pins on the card that need to be covered up. There are multiple ways to do this. If you want a more permanent solution, you can use an adhesive (finger nail polish, etc). If you want a temporary solution, you can use electrical tape. The only issue with tape is it moves easily if you need to install or remove the card frequently.
There is a very good thread here that shows how to modify the pins. It is listed under the "SMBus Issue with Intel Chipsets". I had to do this mod on my motherboard and it is not an Intel based one. Be prepared to do this mod on any board.
Testing Methodology:
To properly test this card, we need all variables to stay the same except a single, controlled one. Here is a breakdown of the hardware that I used for these tests:
Motherboard: Asus M2N32-SLi
CPU: Phenom X4 (AM2) 2.6GHz
HDD:
----320gb Seagate (Operating system drive)
----7x 1tb Hitachi and 1x Western Digital Green (Perc 5/i)
Raid controller: Perc 5/i PCI-e
PSU: Corsair HX620
Case: Norco 4020
OS: Windows 2003
Pictures 1
Pictures 2
I will be using a multitude of programs to determine what settings are best as no one program will give definite results. The first program I will use is the "ATTO" benchmark to show how each option changes the performance of each RAID level and at what file size. The second I will use is HDTune Pro to test sequential throughput, random access and how fast it is with different file sizes.
Program: ATTO Disk Benchmark
Transfer Size: 0.5 to 8192kb
Total length: 256mb
Direct I/O: Enabled - Overlapped I/O
Queue depth: 4
Program: HDTune Professional
File length: 512mb
The hard drives were put in each specified RAID level with a size of 200gb, initialized in Windows, converted to a GPT disk and formatted with NTFS (quick). After that, the test was launched and started. Screenshots were taken after each benchmark. The RAID was then deleted from the RAID array and recreated with the new settings to prevent settings from not changing between tests. The server was NOT restarted between tests and I used the Dell OpenManage software suite to configure the Perc 5/i from a web browser.
The RAID levels used are 0, 5, 10 and 50. There will be a total of 6 tests per RAID level. Each "Read ahead" setting will be tested along with "Write back". To enable "Write back", I had to use the "Force Write back" option since I don't have the BBU (Battery Backup Unit); a built in safety mechanism that you can override. Stripe size was kept to a constant 64kb throughout all tests. Here are the settings used for each RAID level:
Read Ahead - Write Back
Adaptive Read Ahead - Write Back
No Read Ahead - Write Back
Definitions:
Adaptive Read Ahead - Adaptive read ahead is a read policy that specifies that the controller begins using read-ahead caching if the two most recent disk accesses occurred in sequential sectors. If all read requests are random, the algorithm reverts to Non read ahead; however, all requests are still evaluated for possible sequential operation.
Read-Ahead - A memory caching capability in some controllers that allows them to read sequentially ahead of requested data and store the additional data in cache memory, anticipating that the additional data will be needed soon. Read-ahead supplies sequential data faster, but is not as effective when accessing random data.
Write-Back - In write-back caching mode, the controller sends a data transfer completion signal to the host when the controller cache has received all the data in a disk write transaction. Data is written to the disk subsystem in accordance with policies set up by the controller. These policies include the amount of dirty/clean cache lines, the number of cache lines available, elapsed time from the last cache flush, and others.
Write-Through - In write-through caching mode, the controller sends a data transfer completion signal to the host when the disk subsystem has received all the data and has completed the write transaction to the disk.
Definitions pulled directly from Dell's own website here:
http://support.dell.com/support/edocs/storage/RAID/PERC5/en/UG/HTML/glossary.htm#wp1044435
Last edited: