IMO performance (actually temps, not performance) starts to degrade the moment you choose to go with low cfm fans and a thinner radiator. Considering that, there really isnt a 'too low' threshold. It would be nice if you could try slower fans before you buy them but obviously thats not realistic. Remember, silence and performance are a balancing act.
Personally I don't hold temps up that high, if the temps are in reason I consider the other aspects.
Just some thoughts.