Notices

Overclockers Forums > Hardware > Storage
Storage SSDs, HDDs, CD/DVD/Bluray
Forum Jump

Expandable RAID 5?

Post Reply New Thread Subscribe Search this Thread
 
 
Thread Tools
Old 02-06-09, 02:42 PM Thread Starter   #1
karkas
Member

 
karkas's Avatar 

Join Date: May 2008
Location: Phoenix, AZ

 
Expandable RAID 5?


I did a google search on this and got a few hits from a few years back with some contradicting info.

I'm planning setting up a HTPC/file server and would like to set up a RAID 5. Right now I only have a few TB worth of movies, but most of what I buy from here out will be HD, so I expect my needed storage to increase drastically over the next few years. Is there any way to easily set up a small volume RAID 5 now, then easily add in new drives in the future? I really don't want to spend $1.2k on 4 2TB drives just to get 6TB of storage that I won't even get half filled for a year or 2.

Is there a specific RAID controller or software I should be looking into to save myself headaches in the future? It would be great if I could just use the ICH10R on my mobo, but I doubt it?

Thanks in advance for suggestions.

__________________
Heatware
karkas is offline   QUOTE Thanks
Old 02-06-09, 03:53 PM   #2
Kayden
Member

 
Kayden's Avatar 

Join Date: Dec 2008

 
If you want long term RELIABLE storage, get a good midrange RAID card. Don't got with any onboard raid solution for something important like massive amounts of content. If you get a raid card you can migrate your data to a different system just by moving the card and drives.

Intel raids (as far as I've seen) are very limited. About the only options you have are create an array and delete an array. If you want raid level migration or online capacity expansion, you're going to have to pony up.

I bought an Areca 1220 last year for $300. It has 4 1TB drives on it in a raid 5. It has no problem keeping up with running torrents, converting media (to and from the same drive) and streaming HD content.

If you want to get really fancy you can look for a controller with a DIMM slot for adding your own memory and a battery back up. Models like this can be spendy though.
Kayden is offline   QUOTE Thanks
Old 02-06-09, 03:59 PM   #3
Dermen
Member



Join Date: Dec 2001
Location: Virginia

10 Year Badge
 
The Dell Perc5i are pretty popular since you can get them on ebay for around $100, they can handle up to 8 sata drives and have an onboard processor and 256mb cache (expandable to 512mb). I just grabbed one from the classifieds.

__________________
FX-6300 | Gb 970A-UD3P | EVGA GTX 750 Ti | Corsair 8GB 1866 | 120GB Hyper-X 3k | Seasonic G 550w
Heat
Dermen is offline Heatware Profile   QUOTE Thanks
Old 02-06-09, 04:16 PM Thread Starter   #4
karkas
Member

 
karkas's Avatar 

Join Date: May 2008
Location: Phoenix, AZ

 
So these controllers are capable of expanding the RAID's size without moving all data & rebuilding that array? Honestly, if the array falls apart I'm not too worried, because I'll be backing up with carbonite.... and while that kind of data won't be fast to DL back, it is at least safe.

__________________
Heatware
karkas is offline   QUOTE Thanks
Old 02-06-09, 04:30 PM   #5
noegruts
Member



Join Date: Jun 2007

 
Nobody has yet answered your question. The feature you are looking for is called "Online Capacity Expansion" or OLCE. If you find a RAID card with this feature, it will allow you to dynamically expand the array.

No motherboard RAID currently supports this feature, as far as I am aware.
noegruts is offline   QUOTE Thanks
Old 02-06-09, 04:50 PM   #6
Kayden
Member

 
Kayden's Avatar 

Join Date: Dec 2008

 
Didn't I say that
Kayden is offline   QUOTE Thanks
Old 02-06-09, 07:02 PM   #7
ou_phidelt
Member

 
ou_phidelt's Avatar 

Join Date: Mar 2004
Location: Rockingham, NC

 
Quote:
Originally Posted by Kayden View Post
Didn't I say that
I didn't read it that way either.

OP: Another option you could look at is Windows Home Server. The dynamic storage aspect is the whole reason I went with it. It is all software based. Basically you choose what you want backed up and it will be stored on at least two drives, however you get no control over what drives that is. In fact without addons you can't see whats on the drives, it just appears as one large storage pool. Basically everything first goes to the HD the OS is stored on and then it decides where it goes from there. When you add a drive it just adds to the pool. Conversely it is also very easy to swap drives for when you run out of room and want to swap a smaller drive for one of larger capacity.

It certainly isn't without its downsides but it has served me well. I use it for the same thing you are intending, video storage. Mine is built on a low power AMD LE-1150, 780B motherboard, with a 1.5TB Seagate with a second on the way.

__________________
HEAT
ou_phidelt is offline   QUOTE Thanks
Old 02-07-09, 01:18 AM   #8
Zerix01

 
Zerix01's Avatar 

Join Date: Mar 2007

 
You could go the Linux route and you would be able to expand that array with two commands. My RAID5 array is on my onboard controller but with the RAID options disabled, I'm simply using the ports, Linux takes care of everything else through software and it is fast (much faster than Windows software arrays).

You could also you use freeNAS which is based on BSD and has a GUI to help you through the setup.

I like to caution people that using a dedicated card can cause headaches. The array is paired with the card and if the card fails and you can't replace it with the same model then you will loose the data on the array. If you want to add more drives than what you have ports for on the card then you will need a new card and will need to re-build the array. In Linux you can fill all the on board ports plus add a cheap card (or two or three...) for more ports and they can all be added to the array. The drives in this setup can also be moved to a new computer on new controllers, as long as the drives are on the same ports, you can still re-sync the array and not loose data.

I would like to suggest RAID6 if you are looking to have a multi terabyte array. And keep a backup like you said you would. Please read this.

__________________
Deepthought 2.0:
AMD FX 8320 4.4GHz Water Cooled
ASUS Sabertooth 990FX
8GB Patriot DDR3 2000MHz (clocked@2133Mhz)
2x Gigabyte 550 Ti 1GB Water Cooled
2x64GB Crucial m4's in RAID 0 + 3x500GB Hitachi HDD's in RAID 5
Fedora 16 64-bit KDE Spin


Folding User Stats
Zerix01 is offline   QUOTE Thanks
Old 02-09-09, 05:41 PM Thread Starter   #9
karkas
Member

 
karkas's Avatar 

Join Date: May 2008
Location: Phoenix, AZ

 
Thanks for all the suggestions guys, I really appreciate it.

Up till now I was thinking of simply adding drives to my main PC to keep costs down. I could be persuaded to try to talk the wife into letting me build one, but to be honest I really don't understand the subtle differences between a NAS, WHS, and simply adding extra drives in a RAID to my existing desktop (aside from obvious increased resource demand).

@Zerix01 - The idea of the linux route is intriguing, but to be honest I'm a total linux noob, and outside of playing around with (and breaking) ubuntu for a few weeks I suspect I'd be fairly lost... any good/easy linux guides you know of?

__________________
Heatware
karkas is offline   QUOTE Thanks
Old 02-09-09, 06:56 PM   #10
ou_phidelt
Member

 
ou_phidelt's Avatar 

Join Date: Mar 2004
Location: Rockingham, NC

 
Quote:
Originally Posted by karkas View Post
Thanks for all the suggestions guys, I really appreciate it.

Up till now I was thinking of simply adding drives to my main PC to keep costs down. I could be persuaded to try to talk the wife into letting me build one, but to be honest I really don't understand the subtle differences between a NAS, WHS, and simply adding extra drives in a RAID to my existing desktop (aside from obvious increased resource demand).
WHS is a NAS. Basically you set it up and then you can disconnect everything but the power cable and network cable. You use a piece of software that comes with to control it from your PC. It really is very slick.
@Zerix01 - The idea of the linux route is intriguing, but to be honest I'm a total linux noob, and outside of playing around with (and breaking) ubuntu for a few weeks I suspect I'd be fairly lost... any good/easy linux guides you know of?
FreeNAS requires absolutely zero linux knowledge. Its set-up is very similiar in interface to a wireless router interface.
You could put together a system for dirt cheap. $50 motherboad, $25 processor, cheap case, and $30 powersupply. Then $100 for WHS or a half way decent RAID card depending on what route you take. It can be done on the cheap and being able to easily access files from any PC in the house is more convenient than you may think. Adding a media extender to you TV really opens up a new realm on easily attainable media.

Your HD cost will remain constant no matter what route you take.

__________________
HEAT
ou_phidelt is offline   QUOTE Thanks
Old 02-09-09, 07:38 PM Thread Starter   #11
karkas
Member

 
karkas's Avatar 

Join Date: May 2008
Location: Phoenix, AZ

 
Quote:
Originally Posted by ou_phidelt View Post
OP: Another option you could look at is Windows Home Server. The dynamic storage aspect is the whole reason I went with it. It is all software based. Basically you choose what you want backed up and it will be stored on at least two drives, however you get no control over what drives that is. In fact without addons you can't see whats on the drives, it just appears as one large storage pool. Basically everything first goes to the HD the OS is stored on and then it decides where it goes from there. When you add a drive it just adds to the pool. Conversely it is also very easy to swap drives for when you run out of room and want to swap a smaller drive for one of larger capacity.
So going by what you said here a NAS needs a RAID controller, but WHS doesn't? Are both of them easy to add in new HDDs? Do you only have half of the the total hardware storage availible bacause it is mirroring like raid 1?

Thanks for humoring the noob questions, I just want to make sure I set up my network well from the start rather than half-ass it and have to nuke & redo it in a year...

P.S. I was considering media extenders too... I need to research how well they work with WHS.

__________________
Heatware

Last edited by cw823; 02-10-09 at 07:11 AM.
karkas is offline   QUOTE Thanks
Old 02-10-09, 03:56 AM   #12
Zerix01

 
Zerix01's Avatar 

Join Date: Mar 2007

 
Quote:
So going by what you said here a NAS needs a RAID controller, but WHS doesn't? Are both of them easy to add in new HDDs? Do you only have half of the the total hardware storage availible bacause it is mirroring like raid 1?
NAS simply stands for Network Attached Storage. Whether or not it is using a RAID controller does not determine if it is a NAS. WHS is just software that turns a computer in to a NAS but tries to make it easy.


Quote:
@Zerix01 - The idea of the linux route is intriguing, but to be honest I'm a total linux noob, and outside of playing around with (and breaking) ubuntu for a few weeks I suspect I'd be fairly lost... any good/easy linux guides you know of?
This is the guide I used when I setup my RAID 5 array. http://bfish.xaedalus.net/?p=188

I can list more or less what you would need to do.

Build a cheap computer like what was listed above, low power sempron, 512MB RAM, I would suggest an Nvidia based chipset.

Install Ubuntu on to a USB flash drive, the installation should be simple enough for anyone who has installed Windows.

If you have three drives installed for your array then the drive on SATA port 1 would be /dev/sda1, on port 2 /dev/sdb1, and so on. Which is the same as C: and D: in Windows.

Use synaptic and install Gparted and mdadm.

Use the Gparted GUI to add partitions to the three installed drives.

Then from a console do this:
Code:
sudo mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1
/dev/md0 will be your new array drive. level=5 means RAID5, and you have your three drives being added to the array sda1, sdb1, and sdc1.

And that is it, your array will be doing it's initial sync which on my 3x 500GB RAID 5 array it took about 2.5 hours.

To watch the status run:
Code:
sudo watch cat /proc/mdstat
Then to format to ext3:
Code:
sudo mke2fs -j /dev/md0
Should only take a minute.

Now all you will need to do is use Ubuntu's GUI's to share /dev/md0 on your network.

In the furture when you want more space. Install the new drive, which should be sdd1, then:
Code:
sudo mdadm --add /dev/md0 /dev/sdd1
sudo mdadm --grow /dev/md0 --raid-devices=4
Watch the status again with:
Code:
sudo watch cat /proc/mdstat
Once the new drive is synced you need to expand the file system:
Code:
fsck.ext3 /dev/md0
Makes sure the filesystem is in order, think scandisk
Code:
resize2fs /dev/md0
will expand the file system to the new array size.

This looks like a lot but really you only have one command that makes the array and one that formats it. Then only two commands to expand it and two to expand the file system.

__________________
Deepthought 2.0:
AMD FX 8320 4.4GHz Water Cooled
ASUS Sabertooth 990FX
8GB Patriot DDR3 2000MHz (clocked@2133Mhz)
2x Gigabyte 550 Ti 1GB Water Cooled
2x64GB Crucial m4's in RAID 0 + 3x500GB Hitachi HDD's in RAID 5
Fedora 16 64-bit KDE Spin


Folding User Stats
Zerix01 is offline   QUOTE Thanks
Old 02-10-09, 05:02 AM   #13
ou_phidelt
Member

 
ou_phidelt's Avatar 

Join Date: Mar 2004
Location: Rockingham, NC

 
Quote:
Originally Posted by karkas View Post

So going by what you said here a NAS needs a RAID controller, but WHS doesn't? Are both of them easy to add in new HDDs? Do you only have half of the the total hardware storage availible bacause it is mirroring like raid 1?

Thanks for humoring the noob questions, I just want to make sure I set up my network well from the start rather than half-ass it and have to nuke & redo it in a year...

P.S. I was considering media extenders too... I need to research how well they work with WHS.
WHS does not need a RAID controller, actually you would be better off without one, it doesn't always play nice with them. The amount of storage is up to you. All the drives are added to a storage "pool". Essiantly what you decide to have duplicated would double the size need to store it. I have my photos, music, and video duplicated. My OS back-ups and some other random software I have stored are not. Right now I have two 1.5TB drives with it showing 2TB's free. But I know that is more like 1TB because 99% of what I add is video that is going to be backed-up. It did get to enjoy its flexibility yesterday when my second drive came in. Simply format it and add to the pool, the OS take care of the rest. No rebuilding of arrays or other mess to deal with, simple and quick. Including physical install the whole process to less the 15 minutes.

FreeNas either is easy to expand or impossible depending on how you look at it. You would need to create a new array when you want to add onto it. You could not add drives to a current array. You would either need to add drives as a second array or nuke the whole thing, add the new drives, and create the array again.

__________________
HEAT

Last edited by cw823; 03-02-09 at 10:55 AM.
ou_phidelt is offline   QUOTE Thanks
Old 02-10-09, 08:32 AM   #14
Zerix01

 
Zerix01's Avatar 

Join Date: Mar 2007

 
[QUOTE=ou_phidelt;5978120]
Quote:
Originally Posted by karkas View Post
The amount of storage is up to you. All the drives are added to a storage "pool". Essiantly what you decide to have duplicated would double the size need to store it.

I did get to enjoy its flexibility yesterday when my second drive came in. Simply format it and add to the pool, the OS take care of the rest. No rebuilding of arrays or other mess to deal with, simple and quick. Including physical install the whole process to less the 15 minutes.
That seems like it doesn't use RAID at all. It is just a backup solution. There is nothing wrong with that and I could see where that could be a good thing depending on the solution needed. But it doesn't sound like you get any performance gain like you would by using RAID. I also question the ability to carry the drives over to a new system and still be able to recover the data.

__________________
Deepthought 2.0:
AMD FX 8320 4.4GHz Water Cooled
ASUS Sabertooth 990FX
8GB Patriot DDR3 2000MHz (clocked@2133Mhz)
2x Gigabyte 550 Ti 1GB Water Cooled
2x64GB Crucial m4's in RAID 0 + 3x500GB Hitachi HDD's in RAID 5
Fedora 16 64-bit KDE Spin


Folding User Stats
Zerix01 is offline   QUOTE Thanks
Old 02-10-09, 01:52 PM   #15
ou_phidelt
Member

 
ou_phidelt's Avatar 

Join Date: Mar 2004
Location: Rockingham, NC

 
It's not RAID, never claimed it to be. This is not a system meant for performance, its a NAS limited by network speed. Its meant for easily accessible storage and backups. Even relatively slow drives will be faster than gigabit.

The data will easily carry over to a new system as long as the OS stays the same. It creates a 20GB partition on the main drive for the OS. I have reinstalled the OS a couple times without issues. Its not a system for everyone by anymeans but is good at what it was designed for.

__________________
HEAT
ou_phidelt is offline   QUOTE Thanks
Old 02-10-09, 04:08 PM   #16
noegruts
Member



Join Date: Jun 2007

 
Quote:
Originally Posted by ou_phidelt View Post
It's not RAID, never claimed it to be. This is not a system meant for performance, its a NAS limited by network speed. Its meant for easily accessible storage and backups.
This is the critical point that a lot of people here (probably since this is a "performance" oriented message board) seem to miss.

There is no point spending extra on a disk system that outperforms the network it is attached to in the home-user scenario.
noegruts is offline   QUOTE Thanks
Old 02-10-09, 04:33 PM   #17
Mr.Guvernment

 
Mr.Guvernment's Avatar 

Join Date: Feb 2003
Location: Costa Rica - and toronto sometimes

10 Year Badge
 
Quote:
The array is paired with the card and if the card fails and you can't replace it with the same model then you will loose the data on the array.
this is not always the case, often if yo ucan get a card with the same chipset /setup or one with in the same family, it can often work fine on getting the databack.

but this is also whay raid in ANY form is NOT a backup, it is for redundancy.

what happens if the drive your linux install is on dies? there goes your array no?
same thing with windows home server the OS takes a crap, so does your array...

__________________

,`, Poor Planning On Your Part Does Not Constitute An Emergency On My Part ,`,
,`, The trouble with life is theres no background music ,`,
,`, Life isnt short, you're just dead for too long,`,


Mr.Guvernment is offline Folding Profile   QUOTE Thanks
Old 02-10-09, 04:55 PM   #18
Zerix01

 
Zerix01's Avatar 

Join Date: Mar 2007

 
Quote:
Originally Posted by Mr.Guvernment View Post
but this is also whay raid in ANY form is NOT a backup, it is for redundancy.
Yes, that point needs to be stressed. There should always be another backup.

Quote:
what happens if the drive your linux install is on dies? there goes your array no?
same thing with windows home server the OS takes a crap, so does your array...
I could remove my drives and put them in my friends computer, then take a different version of Linux on a Live CD, boot from it and mount my array with all data intact.

__________________
Deepthought 2.0:
AMD FX 8320 4.4GHz Water Cooled
ASUS Sabertooth 990FX
8GB Patriot DDR3 2000MHz (clocked@2133Mhz)
2x Gigabyte 550 Ti 1GB Water Cooled
2x64GB Crucial m4's in RAID 0 + 3x500GB Hitachi HDD's in RAID 5
Fedora 16 64-bit KDE Spin


Folding User Stats
Zerix01 is offline   QUOTE Thanks
Old 02-10-09, 06:46 PM   #19
noegruts
Member



Join Date: Jun 2007

 
Quote:
Originally Posted by Mr.Guvernment View Post
same thing with windows home server the OS takes a crap, so does your array...
There are no arrays in WHS. If the machine dies you can take any of the storage drives and plug them into another machine and get the files off them. If you have duplication enabled the files should be stored on more than one of the physical drives in the machine.
noegruts is offline   QUOTE Thanks
Old 02-11-09, 02:15 PM   #20
mbentley
Gloriously Lead
Overclockix Chief Architect
In stealth mode

 
mbentley's Avatar 

Join Date: Sep 2002
Location: Indianapolis, IN

10 Year Badge
 
Quote:
Originally Posted by Zerix01 View Post
I could remove my drives and put them in my friends computer, then take a different version of Linux on a Live CD, boot from it and mount my array with all data intact.
...and that is the exact reason that i went with software raid in linux

i just went with raid 1 as it is just storage and nothing that i need high speed access to.

__________________
intel i5-750 -- msi p55-gd65 -- 4 x 4gb ddr3 1600 -- msi gtx550 ti cyclone oc 1gb -- antec p182 -- ocz fatal1ty 750w -- windows 8.1 x64
intel i5-3330 -- msi z77a-gd65 -- 4 x 8gb ddr3 1600 -- intel hd graphics 2500 -- fractal design define r4 black -- debian wheezy amd64

forum rules | overclockix | overclockers geforce survey | mbentley.net
mbentley is offline Author Profile Benching Profile Heatware Profile   QUOTE Thanks

Post Reply New Thread Subscribe


Overclockers Forums > Hardware > Storage
Storage SSDs, HDDs, CD/DVD/Bluray
Forum Jump

Thread Tools Search this Thread
Search this Thread:

Advanced Search


Mobile Skin
All times are GMT -5. The time now is 12:19 AM.
Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
You can add these icons by updating your profile information to include your Heatware ID, Benching Profile ID or your Folding/SETI profile ID. Edit your profile!
X

Welcome to Overclockers.com

Create your username to jump into the discussion!

New members like you have made this the best community on the Internet since 1998!


(4 digit year)

Why Join Us?

  • Share experience
  • Max out your hardware
  • Best forum members anywhere
  • Customized forum experience

Already a member?