Are you talking about the idle for the CPU or the GPU? It's unclear...
Sounds like you need a couple more case fans... the bigger diameter the better for noise at least... by the way the temp drops so much with the side off.
Pictures would be really helpful!