ok, here's what i do to get thermal pads off. i use an amazing tool, the fingernail. scrap off as much as you can. fingernail is good because unlike a knife or something hard it wont scratch up the heatsink. i use rubbing alcohol to get any leftovers off.
The only way to get all of the pad off is to lap the heatsink. There are microscopic surface imperfections in your heatsink that can't be cleaned with solvents. These surface imperfections are where a heatsink compound is needed most. LAP IT.