My best guess would be that it's a bad ground in the USB port/wire. When you insert a USB thumb drive (or other USB device) it completes an electrical circuit and the electrical feedback from this trips some kind of protective measure in the power supply (most likely Over-Voltage Protection or Over-Current Protection) causing the system to shut down in order to protect itself and prevent serious damage from the electrical short.
I've seen this happen on several computers, including some owned by family members, clients, and co-workers.
It doesn't seem to be limited to a specific motherboard manufacturer, case manufacturer, or manufacturer of the USB device(s) though. It's usually just a random occurrence where sometimes a USB device or USB port isn't properly made (or grounded) and results in an electrical short now and then (or in some cases a short that happens every time a device is plugged in to a specific port (or sometimes any port at all).
Strangely though, I did encounter one where every time the person plugged in the charging cord for their iPhone the computer would restart, but only with that one USB device. It never restarts when thumb drives, card readers, or any other devices are plugged in, just the iPhone charging cable. Got two new iPhone cables since then, and they both caused the restart too, weirdly.