if your motherboard had temperature sensors built in it would show the temp in the bios and you should be able to get a prog like motherboard monitor or hardware doctor, etc. to check the temp in windows.
I wouldn't even try to use a regular thermometer, though I suppose you could get the case temperature... sort of...
You could buy a temperature probe like compunurse or something and then just stick it b/w the heatsink and chip or on the back of the chip if possible.