For not OC'ing it is OK, but that is it. The stock HSF is marginal at best. I would recommend a good HSF, if you are even considering OC'ing. Thermalright AX-7, Millenium Glaciator 2, Alpha Pal 8045 are all good.
I found that most of my stability problems were solved when I went to a better hs/f.
You can do a lot of good with case fans also. The advantage to case fans is that they continuously change the air in your case. Air absorbs heat faster when the there's a big diffference between the heatsink temperature and the temperature of air in the case. In a (bad, closed) case with few fans or vents the air will eventually heat up to the temperature of the heatsink and do nothing no matter how fast the air moves.
If you mostly surf the web then the OEM heatsink alone is fine, but if you play games for hours on end, you'll run more dependably with extra fans and/or a better heatsink.
Your new fan will plug directly into the motherboard, yep.
I was running the retail AthlonXP 1700+ (Hs/F) for a while, until I needed a better sink for my FPS experiances ..
In a full server tower with no case fans (no mods) here were the specks:
CPU: 50 C (Full load), 45 C (Idle)
Mobo: 25 C
Ambient: 37 C
Amazing what a couple of fans can do. After modding (120mm in front, 2 80's in back) a heatsink upgrade and a new fan on the CPU, dropped 9 degrees off the CPU and 10 - 15 everywhere else. Thinking of adding an extra fan on the top...