How do I copy from HDD A to B hundreds of GBs (1000s files)?

[gamefan]

The vast majority is several hundred video files (/several hundred GBs). Dozens of GBs (/thousands) of mp3 and other audio files. Then many hundreds of jpgs and other image files, and a tiny product of several txt name type type files. Maybe 600GB to 1TB all total.

I attempted this once before recently as a single load, but it was on a worse pc than the main pc I will be using and use regularly. See my sig. The HDDs are todays models (not listed in sig). USB interconnect(s) will need to be used unless otherwise instructed NOT too.i.e go inside young man. Because it either hung, crashed or said so long I gave up and did it in chunks of maybe 5% per.

Is there any reason I should not copy, say, all the video files in one chunk? If so then the rest should be easy - a total of just a few or half dozen copy selections. From just a few source HDDs Only one target HDD.

The copy and rename filneame to filename (2) problem - Do this for all n next files is NOT an issue. ie I can answer it immediately and let it get on with copying.

How do I copy from HDD A (&A.1 & A.2...) to HDD B hundreds of GBs (1000s files)?

[thideras]

Depending on what operating system, it changes. For example, I wouldn't copy that many files with XP because if one fails, it stops the copy completely. In Vista/7, you can choose to re-do the file or skip it.

If this was across a network, I would suggest DeltaCopy (Rsync for Windows). Beyond that, just normal copy if you aren't on XP.

[Adragontattoo]

I moved over a TB of data between two arrays a few months ago and then moved about half a TB over to a second rig as well.

My suggestion is to do the following:

Create target folders on destination drive(s)
Copy a group of files (all pics for instance) as a single item.
DO NOT USE USB unless you have to, it will take FOREVER.
After Group 1 is done, move to group 2, etc..

OR

Copy all at once BUT there can be issues.

Reason for this is twofold. Bulk copy will cancel the entire move at a single error OR it will stop the move until OK is clicked to continue. Moving per group (pics, movies, videos about cats, all your text docs) will copy over all of each item and not prevent the next group from continuing. Usually what I will do is start Pics, after a few minutes start Movies, after a few minutes start docs etc and then go off and do stuff OFF the computer (let it run, staring at it doesnt make it go faster) but I will check it to see if it has any alerts popping up (out of space, do you want to replace, why are you doing this to me, etc) and I will confirm/deny/ignore as it goes. This way will increase the time it takes in some ways but decrease the chance that a full fail happens. I have had some files that had screwed up permissions fudge a full move if I used a single bulk move before.

USB will take time, if you REALLY want to do that, expect it to take 2-3x as long if not more.

[Jmtyra]

First, I would *highly* suggest using TeraCopy (http://www.codesector.com/teracopy.php). It's free, it's fast, and it's awesome. With that many files, if one fails, your screwed. With TeraCopy, it'll resume. Score! Plus, it's much faster, just make sure you go into the options and uncheck "use system write cache" as it, sadly, actually makes things a bit slower.

Secondly, I'd make sure that both HDDs are in the same machine. Once that's the case, make sure you've enabled AHCI in the BIOS if you're using W7.

Other than that, you'll just need a *lot* of time. Oh and good cooling for those drives. :D

[dark_15]

I know this may be a little late, but what about RichCopy? It's a multithreaded app used to copy tons of files. http://technet.microsoft.com/en-us/magazine/2009.04.utilityspotlight.aspx

[visbits]

DD on linux is by far the easiest way.

dd if=/dev/sourcedrive of=/dev/destdrive bs=1M

Sit and wait.

[habbajabba]

DirSyncPro would work fine and so would teracopy which is what I use daily.
The best way to do it is to copy only. That way if you get a connection error you have not lost anything. DirSyncPro would work perfect for that too and works on all platforms.

[dark_15]

DD on linux is by far the easiest way.

dd if=/dev/sourcedrive of=/dev/destdrive bs=1M

Sit and wait.

dd for windows: http://www.chrysocome.net/dd

[Kowalski3500]

(let it run, staring at it doesnt make it go faster) but I will check it to see if it has any alerts popping up (out of space, do you want to replace, why are you doing this to me, etc)

I actually L-ed O L at that.

I don't know if I'm missing something here. But what's to stop the OP from putting both drives in one computer, and simply copying files over? I did this recently with about 100GB of stuff with no problems.

[Adragontattoo]

I actually L-ed O L at that.

I don't know if I'm missing something here. But what's to stop the OP from putting both drives in one computer, and simply copying files over? I did this recently with about 100GB of stuff with no problems.

I was wondering if anyone saw that.

I just went through literally 14 drives ranging from 40gb all the way through 1.5tb. I didnt have the room or inclination to shut a rig down, install the drive, check it, copy whatever data and then repeat on the next drive.

It took me 4 days to copy all the data that was useful and I found a few things that I thought were lost.

External made it go a ton faster in some ways but transfer rates took a hit but I am pretty sure that in the grand scheme, external was faster overall.

[Badbonji]

Do it overnight, and use Beyond Compare or some file comparison application so that you only copy the files you need to, not duplicates so it won't pop up with any renaming thing.

This is what I did with 1.8TB of files when transferring them.

[silkshadow]

I do huge array transfer more often that I like. Invest in gigabit equipment and quality cat5e (or cat6 cables for long runs) and have a look at Terracopy pro. I find it noticeably cuts down transfer time.