Is the existing cooler the type that has a single blower fan or one with two or three conventional looking fans on it?
The blower type can be helped with case pressurization as I mentioned (just mount a variable speed Delta in the case blowing in), while the fan type benefits a huge amount from a fan in the side panel, as I have done on my PC. In many cases, it's possible to replace the existing fans on a fan type GPU cooler with some more powerful ones.
On the blower type cooler, if the rear grill is too restrictive, it might help to cut the grill, but beware that the fins on the radiator behind it are easily damaged. If you're willing to do some case modding, you could even cut the metal separating the expansion slot openings in order to get some more airflow.