You should read some of our other
threads. That's just one, but there are a couple more floating around there somewhere. Some of that is redundant, or maybe some of this thread is redundant, but either way...
HDD were never meant to be frozen, let alone immersed. If you are just going to cool it, not a problem, sub-0 though, they will need to be outside the enclosure. That's way out of HDD specs. PSU should be immersed. It by no means needs to be, but WTF, why not? I would just drop my whole machine in the vat if I thought it would work.
As for the vertical idea, I am still in between on how to do it. I had designed, but not built obviously, a plexi tray that you could slide out, identical to a regular mobo tray. You could probably just use a tray from an old case. It would be vertical and have the connectors not exposed to the coolant. I want it air tight though, so I have come away from that and gone to total immersion. It really depends on the coolant though. Alcohol would need to be sealed, and fluorinert too. Cheaper coolants, or less toxic ones really don't matter. With oil it's not a problem, but with other coolants, water will be absorbed through those holes and then you could be up you know what without a paddle.
Obviously, cdroms and floppies can't be immersed, but they could be frozen maybe. Depends on how nice they are and whether you are willing to try it. I thought about mounting them in my freezer, and again WTFN. Or you could create an external housing for the drives and run cables to it. Then you need to factor in distance between the board and the housing during your design.