Realistically, if you wanted to just "lock out" your USB ports, there is a way, but it's waaaaaaaaaaaay more difficult than just password proctecting your PC.
Here goes-
On bootup, go into BIOS, and go into Integrated Periphials-i think. Turn OFF your USB ports...not just downgrade them to USB1.1. Then, password-protect BIOS.
Now, whenever your brother uses your computer, the USB ports won't even exist, and when you use your computer, you can just reenable them, but make sure you constantly turn them off and on if you do things this way.
Hope this helps!