EXTREMELY SLOW disk to disk copy in Windows 10

[Ben333]

Running Win 10 Pro on a R5 3600 /w 16 GB RAM. Doing an archive of my firefox downloads, 80 GB from my 1TB WD black NVME to my 4 TB WD Blue. The WD blue always gets over 100 MB/sec, reading or writing.

For some reason though today, this transfer has been dipping below 1 Mb/sec... and even hovering around 14MB/sec. Just way too slow, and I closed out and tried a second time.

Yesterday I moved a 40+ GB archive from the HDD over to the NVME in order to extract it faster, and that moved very quickly. Today... 20 minutes for 80 GB???

It first said 2 or 3 minutes for the 80 GB. Now I'm 5 minutes in and it is saying 21 minutes remaining.

I've had this problem before. Why is this happening? Hoping someone here knows what is going on. There are many, many tiny files included... but would that slow this down by that much?

A reboot would likely fix it, but I normally leave this machine on 24/7 without rebooting for days at a time... reboots are annoying! Any info appreciated!



I haven't used windows since Ubuntu 16.04 came out... so only been back on M$ for two or three months. Had been using linux on everything since I last used Win 8.1, so I'm 4+ years outta date on anything windows.

 

[Automata]

Quick math shows the average file size is 25 KB over 277,432 files. If the disk is fragmented (and/or doing other operations), this seems fairly normal to me. Small files have a lot of overhead, and if they are scattered around the platter, are going to take a lot of seek time to read.

If you are copying the Firefox AppData folder over, clearing your cache should eliminate a lot of files.

 

[Ben333]

It was a LOT of tiny files. But would fragmentation matter when the NVME SSD was the source? It was my web downloads folder. Stuff I'd downloaded off the net. Moving it to the archive drive to free up SSD space. Here is how the rest of it went. (No small files) And the machine was only doing the copy & running maybe 12 firefox tabs. So nothing else disk-intensive.

 

[Automata]

I entirely misread the part about the drive. I saw "1tb WD black" and thought of the HDD. While small files have a lot of overhead, if NVME was the source, it shouldn't be that slow. Have you run benchmarks on the drive to see how it performs with small file operations?

 

[Ben333]

The drive is blazingly fast at anything I do. Boots take just seconds... hardly any reason to use Stand-By or Hibernate anymore, thought I'd never see the day lol

This was really the only instance of this slowness. I've had one other on this box, while trying to copy from an old 500 GB WD Blue onto the new 4 TB WD Blue.

Here are some benches I did /w the WD Black SSD (shown here /w a 500 GB main/boot partition) & the 4TB Blue:



I can try different settings if that helps. I'm wondering if it was just the OS being dumb... for lack of a better explanation...? But I opened task manager and everything was consistently under 6% to 9% CPU load. And the disk usage in task man mostly matched up /w the copy dialog window.

 

[Ben333]

Slightly slower running crystal disk @ 16MB option.

 

[EarthDog]

The source isn't relevant when the destination (4TB WD Blue) is the slowest part. It's still writing tiny files to a HDD so speeds are quite low as you see for the 4K size in CDM (2 MB/s). The write speeds on the larger set of files at 116 MB seems appropriate. But nickel and diming a platter with tiny files will run slower

 

[Robert17]

Write validation on a slower drive will impact it as well.

 

[Ben333]

Earthdog, I was more so clarifying to thid that fragmentation and random / seek shouldn't be an issue. I'd imagine it is much easier to write 1,000s of little files to a source fresh than to do the reads on them... But thanks because that explains a good bit of why it chugged so bad.

 

[Automata]

Earthdog, I was more so clarifying to thid that fragmentation and random / seek shouldn't be an issue. I'd imagine it is much easier to write 1,000s of little files to a source fresh than to do the reads on them... But thanks because that explains a good bit of why it chugged so bad.

Yes, even if you are writing a bunch of small files, unless the destination disk is heavily fragmented, it will write fairly quick because it can group and batch the writes. It won't be anywhere near sequential write speeds, but it shouldn't be that slow.

Do you have an application scanning the contents of the files? Security software, backup software, etc.

 

[Ben333]

Only Windows Defended. I've never used any AV software on my personal systems, they drive me nuts. And like I said though, I checked task manager and it reported that both disks were pretty much idle while that copy was crawling by...

So... not sure...

 

[Automata]

I'm not sure either, then. I'll try to think of something else.

Are other copies fast, or was the problem only this folder?

 

[dejo]

Looking at the 4k writes of the 4TB Blue drive (assuming the 2.52MB/s is a best case, and the files you are writing are of at least 4k size) I would not think the speeds are too far off. Small files decimate writes speeds as to what people are used to seeing. Many never pay attention to small file write capability of storage devices.
It is almost hilarious to see video reviews of drives and people freaking out and blasting a manufacturer for slow speeds when they never consider those small files (Brand X said this drive transfers AT UP TO this speed and it is only moving 4k files at a fraction of that speed- I want a refund!)

 

[Automata]

Looking at the 4k writes of the 4TB Blue drive (assuming the 2.52MB/s is a best case, and the files you are writing are of at least 4k size) I would not think the speeds are too far off. Small files decimate writes speeds as to what people are used to seeing. Many never pay attention to small file write capability of storage devices.

Those are random 4k writes, though. If the disk has space, the OS/disk will batch many small files into a single operation. Batching small files is more akin to sequential writing, but won't reach "real" sequential writing speeds.

 

[dejo]

I would still guess that even at sequential 4k capabilities, that 4k writes would still be nowhere near advertised sequential speeds of any drive.

 

[Janus67]

have you tried using xcopy to do a copy instead of windows explorer?

 

[Automata]

I would still guess that even at sequential 4k capabilities, that 4k writes would still be nowhere near advertised sequential speeds of any drive.

Right, that is what I'm saying. It won't be anywhere near as fast as sequential writing, but it should be a noticeable increase over random 4k writing.

 

[SPL Tech]

Looks like you need an SSD. problem solved. Hard drives are obsolete in the consumer sector at this point. SSDs are cheap enough.

 

[Ben333]

No thanks. The 4 TB is an archive drive. Much rather spend $100 for that than thousands. My file operations can take place while I sleep, so not really an issue. And the 3 gigaBYTES per second on my C drive... lovin it