Multi-rank memory: better latency and bandwidth?

[magellan]

Does multi-rank memory decrease latency and increase bandwidth relative to single-rank memory running at the same frequency?

Is there any difference in o'clocking potential between dual-rank and single-rank DDR4?

 

[mackerel]

To answer the first part, in synthetic benchmarks it seems the bandwidth is increased, but so is the latency. In some applications I use (Prime95, LLR) the performance difference going from dual to single rank is significant, with single rank memory resulting in ~20% slower app performance if ram speeds/timings are otherwise the same. I suspect the synthetic benchmarks are not exercising the ram in the way the application does so doesn't show as great a difference.

Not saying it is the same thing going on, but it may be a parallel to hard disks. You might have two with similar sustained transfer rates, but one might be significantly better at IOPS.

 

[magellan]

To answer the first part, in synthetic benchmarks it seems the bandwidth is increased, but so is the latency. In some applications I use (Prime95, LLR) the performance difference going from dual to single rank is significant, with single rank memory resulting in ~20% slower app performance if ram speeds/timings are otherwise the same. I suspect the synthetic benchmarks are not exercising the ram in the way the application does so doesn't show as great a difference.

Not saying it is the same thing going on, but it may be a parallel to hard disks. You might have two with similar sustained transfer rates, but one might be significantly better at IOPS.

In an old IBM whitepaper I read on x3550 servers (I think these are nehalem or westmere) they suggested "quad-rank devices (4R) are becoming more prevalent." I guess that prediction never came true did it?

Here's what they said about memory ranks in that article:
"For instance, a system populated with 6 x 2GB dual-rank DIMMs outperforms a
system populated with 6 x 2GB single-rank DIMMs by 7% for SPECjbb2005.
Dual-rank DIMMs are also better than quad-rank DIMMs because quad-rank
DIMMs will cause the memory speed to be down-clocked."

The whitepaper article was called Tuning IBM System X Servers for Performance.

I wonder if your Skylake memory ranking benchmark results would be at all applicable to other CPU architectures?

 

[mackerel]

It seems to work for Haswell too, but I haven't looked at that in as much in depth since I only have 4 DDR3 systems remaining, and 3 aren't overclockable. The remaining one is Broadwell so the L4 cache negates ram impact for my uses.

On a side note: today I was informed there exists dual rank DDR4 4GB modules! Crucial Ballistix Tactical. They only come in 2666, but if my earlier testing is correct, in theory it should perform as well as single rank 3200+. I've ordered a set to try it out.

 

[Woomack]

Depends from architecture higher rank may raise bandwidth or not. In most cases it raises latency. Bandwidth usually looks like 1 < 2 < 4 = 8.
Here you have comparison between laptop DDR4 single vs dual rank at about the same settings 2133 13-13-13 2N:
http://www.overclockers.com/forums/...ct-2x8GB-DDR4-2400-and-2x4GB-DDR4-2133-SODIMM
All results except Ralbench are repetable. Realbench gives +/- 15k score depends from run. You can also see that 3DMark is showing higher dual rank memory performance at higher benchmark settings but not at the lowest.

In bandwidth tests like AIDA64, read/write bandwidth is about the same but copy is worse on single rank kits. Most applications react good to memory read while barely to write. Copy sometimes affects performance, sometimes not. Windows internal memory performance test ( winsat ) uses mainly memory copy to show memory performance.

On the other hand single rank memory is overclocking better so if you can/want overclock it then single rank will be faster.

Generally difference between single and dual rank memory was much more visible in DDR3 than DDR4.
Here is one more thread with 2x4GB and 2x8GB in desktops:
http://www.overclockers.com/forums/showthread.php/763132-What-memory-is-better-for-Skylake-Z170
Read/write are not much different while difference in copy bandwidth is 2GB/s. It's still nothing significant while Skylake can make 40GB/s on average DDR4-2666 memory.

In DDR3 in my tests, single rank 2666 CL11 kit was performing about as good as dual rank 2133 CL9. At the end best was dual rank 2400 CL10.

 

[trents]

What does the term "rank" refer to? Is that the same as "channel" as in dual channel v. single channel? Never heard the term "rank" before in this context.

 

[EarthDog]

It is not the channels. You can have single/dual/triple/quad channel systems with single rank sticks.

A memory rank is a set of DRAM chips connected to the same chip select, which are therefore accessed simultaneously. In practice they also share all of the other command and control signals, and only the data pins for each DRAM are separate (but the data pins are shared across ranks).[citation needed]

The term “rank” was created and defined by JEDEC, the memory industry standards group. On a DDR, DDR2, or DDR3 memory module, each rank has a 64 bit wide data bus (72 bit wide on DIMMs that support ECC). The number of physical DRAMs depends on their individual widths. For example, a rank of x8 (8 bit wide) DRAMs would consist of eight physical chips (nine if ECC is supported), but a rank of x4 (4 bit wide) DRAMs would consist of 16 physical chips (18 if ECC is supported). Multiple ranks can coexist on a single DIMM, and modern DIMMs can consist of one rank (single rank), two ranks (dual rank), four ranks (quad rank), or eight ranks (octal rank).[citation needed]

There is little difference between a dual rank UDIMM and two single rank UDIMMs in the same memory channel, other than that the DRAMs reside on different PCBs. The electrical connections between the memory controller and the DRAMs are almost identical (with the possible exception of which chip selects go to which ranks). Increasing the number of ranks per DIMM is mainly intended to increase the memory density per channel. Too many ranks in the channel can cause excessive loading and decrease the speed of the channel. DRAM load on the CA (Command/Address) bus can be reduced by using registered memory.

https://en.wikipedia.org/wiki/Memory_rank

Hopefully that helps.

 

[magellan]

When did RAM banks become RAM ranks? Memory interleaving w/banked memory (done w/FPM RAM) seems to be the same thing as memory interleaving w/DDR ranked memory, but are they the same thing?

https://www.phy.ornl.gov/csep/ca/node19.html

 

[EarthDog]

I believe banks and ranks are different???

http://www.ccs.neu.edu/course/com3200/parent/NOTES/DDR.html

 

[mackerel]

Can't claim to understand this stuff myself, but rank seems to apply at a module level, and banks applies at a device (ram chip) level.

 

[trents]

Thanks, guys. That helps.

 

[bobhumplick]

When did RAM banks become RAM ranks? Memory interleaving w/banked memory (done w/FPM RAM) seems to be the same thing as memory interleaving w/DDR ranked memory, but are they the same thing?

https://www.phy.ornl.gov/csep/ca/node19.html

banks became ranks to clarify. banks normally meant a single stick on a channel. but then some sticks started having 2 sticks in one basically. so is this a single bank or 2 banks in one? what is it? well its a single dimm with 2 ranks. so it was mainly for better comunication.

to anyone else not understanding the difference between rank and channel, think of it this way. your cpu is dual channel. its only got the wiring to talk to 2 sets of memory at the same time. but your board could have 4 slots. how does that work? well theres a chip select wire that when the switch is flipped it will tell one set of ram to ignore what the cpu is saying. so the cpu sends the info to both (since they are both connected to the same wires essentially) and the other does the talking with the cpu.

now you could have 2 slots with a single set of ram (rank) or you could have 2 slots with 2 sets (ranks) of ram on a single stick.

by chopping up a bit of info accross 2 ranks you can increase performance. newer intel and amd cpus respond pretty well to it. well pretty good considering its still only 2 channels and not 2 ranks. ddr3 probalby responded better becuase ddr3 had such lower bandwidth. now that speeds are nearly doulbed theres less pressure on bandwidth. but with more and more cores coming to dual channel setups that could change soon.

the reason dual rank increases performance a bit is because ram takes a second to "warm up" once you tell it you need data. you can tell rank 1 to warm up and while youre waiting tell rank 2 to warm up another area that has the second half of the data. so time that would be wasted waiting on rank 1 to warm up can be regained by warming up rank 2 and have the data spread across both ranks.

but thats only if you can get the same timings and speed as single rank which you cant always do. newere intel ddr4 systems handle it really well. and you can see a gain. ryzen will see a gain too but if you lose clocks youll break even or go backwards.

if you can keep the exact same timings on an intel system then you can actually lower latency. but in reality it could go backwards if things dont time correclty