Another option might be the water cooling pump. Even though they are "all new" components, sometimes you get bad stuff right out of he box. Do you have the OEM cooler that was boxed with the CPU? I would swap that component out temporarily at least to check that possibility. The sound you hear could be failing pump bearings. And it's right down close on the motherboard so geographically that would fit the scenario. If you are overclocked then set the CPU speed and voltage back to stock first, however. The one thing that doesn't fit this is the fact that the noise doesn't start until Windows loads. That would indicate to me it is a component that requires a Windows driver or is somehow controlled by a Windows based program.
One easy thing you might try is to burn a Linux Live distro onto a DVD and boot from it. That would by pass the Windows drivers.