It would be good to get some big heatsink with slow fan. Zalman-style coolers, as well as Thermalrigt will perform nicely. If you don't OC hard your server, Arctic Cooling coolers are very silent and cheap. A fan controller would be useful too (not really with AC).
PSU - it often produces more noise than CPU cooler. I'd get silent one. Most ofsilent PSUs are very expensive, comparing to normal, but for example one of my friends has bought 400W Fortron with 120mm fans that is almost inaudiable, including night.
HDDs - they produce some noise, but they also produce lots of vibrations - with some case resonance, they are noisy. Think about dampening HDD vibrations - using some silencing set from shop, or suspend HDD in case in some way that it doesn't produce that much vibration - like make it hang on some strings or zip ties. In this case hdd needs more aiflo as it gives les heat to case.
case - it shocked me when I put my comp back tocase and it produced more noise than when it was laying on desk. Ther eare basically 3 kinds of sound damping materials. First is known wavy accoustic foam - relatively cheap, reduces sound signifiently. The second is rubber-like sound damping material usually used with car audio systems - it rather damps vibrations than noise itself, and can be put on case sides, top and bottom, and you can put some accoustic foam on it. That will give nice sound and vibration damping. If you use that foam, better keep your case relatively clean. The third option are some specialized noise dampening materials. They work pretty nice,but are expensive. Expect higher temps, but not much higher. Case fans mufflers also work.
And water cooling can be very silent, if only you use proper parts.
