Latex gloves would have the same problem, because then you would be a statically charged capacitor, with the gloves as the dielectric.
Wether your case is unplugged or not (grounded or not), really doesn't matter. The point is that if you touch the case, you and the computer have equalized charge potential, therefore there's no shock to you or a component. Static shocks occur when two bodies with different potentials come into contact with each other. It doesn't matter what the actual voltages are, only that they are different.
For example, if you and the computer are both at 3,000 volts potential, there wouldn't be a static discharge. But if you are 3,000 volts higher or lower than the computer, there will most certainly be one when you touch it.
3,000 volts is not an unlikely number either, as a simple styrofoam coffee cup can transfer up to 3,500 volts static potential to you, carpeting can be even higher. Some components only need a 50 volt discharge to fry or scramble stored data (like bios chips).
Simply leaning your forearm on a case edge while working is plenty enough to remain safely equalized with your computer case.
If you do use a ground strap, hook it to the case, not a separate ground. A grounded strap only works if the object your touching is also grounded (zeroing the potential difference between you and it).
Have a great night, ya'll!