[cw823]

Two 10tb+ arrays, with expansion to 20tb possible. Two separate machines, same hardware but lower rpm drives in the "backup". Both are running raid6, ext4. Would like to synchronize, and have failover capability if possible. Not necessary, but would be nice to have failover.

I've seen a few options online that are clearly geared toward the expert user, not a ubuntu novice like myself.

Anyone have experience or input? I'd like it to be automatic.

For this example:

server1 - 192.168.9.1
server2 - 192.168.9.2

both have a share mounted @ /mnt/md0
In that folder are about 8 other folders with all my data. Lots of it. Little to nothing exists on server2 right now, it's all on server1

[Adragontattoo]

Ubuntu novice, storage novice, linux novice or what?

You went right over to the deep end and dove to the bottom by the looks.

[thideras]

How often does the information need to be updated? Weeks? Days? Hours? Instant?
Does it actually need to have a complete failover (server1 goes offline and server2 picks up with no hesitation/user intervention)?

You could do something as simple as a Rsync that copies between the servers and as complicated as a storage cluster/SAN/GFS system that keeps everything live and has HA. Scripts are easy to setup once you get one working and you shouldn't have to touch them unless you need to update folders or add functionality. The script that I wrote for my server has not changed for a very long time and it has been rock solid.

Can you explain exactly what you want this to do?

[${freak}]

Something with crontab and rsync would probably do the job, unless you want to get more complex

[cw823]

Okay. let's go with the following folders that I'd like synced to both locations

TV
Movies
Files

TV, if I had a weekly "job" that synced files from the main server to the backup, that would be sufficient.
Movies, same as TV
Files, nearly instant backup as this would include family videos, pictures, documents, etc....

I've used ubuntu for f@h and for software raid, that's about it. My knowledge ends shortly thereafter.

It doesn't have to be HA setup, just figured if I could, then why wouldn't I?

Either way it's crucial that this setup work between two software raid arrays on separate ubuntu servers, that ARE attached to the same lan. Both servers have dual NICs, I was planning to have one nic per server to my local lan, then a crossover cable between the other nics to keep the traffic off my regular lan.

There's really no "constant" traffic from these servers, other than using media portal for movies/tv and listening to music.

[Stratus_ss]

It sound like Linux HA is probably more then you want to get into at this time, like freak said, cron job and rsync is the way you probably want to go

[mbentley]

+1 for using rsync. that is what i use to create a backup of my raid 5 array. i figure running a raid 5 array is pretty safe but keeping the data on another machine as well adds an extra level of protection and with rsync, it couldn't be easier:

Code:

rsync --delete -av /mnt/storage/ /mnt/backups/storage

i just have a cronjob setup on my backup box that pulls the data from a nfs share (/mnt/storage) and it creates a backup on a local drive mounted at /mnt/backups

my backup script also includes a brief email which gives a status report once it has completed the rsync:

Code:

#!/bin/bash

tmp=/tmp/weekly-storage-backup-`date +%F`
touch $tmp && chmod 600 $tmp
echo -e "subject: weekly backup of 'storage' has completed\n" > $tmp
echo -e "weekly backup of '/mnt/storage' has completed at `date`" >> $tmp
echo -e "" >> $tmp
rsync --delete -av /mnt/storage/ /mnt/backups/storage >> $tmp
/usr/sbin/sendmail -f mbentley@mbentley.net root < $tmp
rm $tmp

nothing really fancy but it does the trick.

[Shelnutt2]

You might want to look into inotifywait (/watch) if you want to rsync right after a file change

[cw823]

Swapping hardware from dual noconas to dual sossamans Monday or Tuesday, so I'll get everything back up and going then post here asking stupid noob questions.

[mbentley]

hopefully you should see a massive drop in power consumption. my noconas ate power plants for breakfast

and thanks shelnutt. i might have to look at inotifywait myself.