If you can possibly get to it qith a Qtip, you can put some Isopropyl Alcohol on the Qtip and use that to clean it. It's best to use the 90-ish% stuff, not the 70%. As close to pure alcohol as you can get is best, that way it evaporates away and doesn't leave a bunch of byproducts.
If that isn't an option, i think you will have to go with ben333's advice and take it all the way apart and clean it. If you aren't real physically technical, you might consider taking a few pictures with a digicam as you dismantle so you know how to put it back together. But it should be fairly simple in there unless it's a fancy wireless mouse or something with 20 buttons.
