if you have to balance space considerations against shroud standoff - the minimum standoff should be no less than 25mm
a good balance between size and standoff is 38mm of standoff which puts the typical 50mm thick core right at 5" thick after a 38mm x 120mm fan is mounted to it.
other than that, the usual formula for guestimating theoretically optimal standoff is: standoff = width of the fan hub [which is usually right at 50mm on a DC 120mm fan]
the difference in temp between 38mm standoff and 50mm standoff?
dunno - my thermistors can't tell a difference in coolant temps between the two but there is likely an ever-so-slight edge with the extra 12mm standoff.
For the record, the usual standoff on commercial rads or the flat plexi "shrouds", which is normally around 10mm-12mm, is basically pathetic.
edit: the performance difference between a core with no shroud and the core with a proper shroud is approx. 40% improvement in favor of the core with the shroud.