Ok lesson learned, I shouldn't have bothered... no gain and no point.
BTW the stuff I removed wasn't super thin, not too thick ether. I'd say regular chipset sticky goo, just a bit thinner than on LGA775 generation boards.
I just double checked it, the shim is very soft sponge-like and level with the z77. It won't interfere with the chip-heatsink contact. But a thicker shim on other boards might make for too big of a gap for good contact.