Many years ago I almost started a small repair shop and component reseller with a friend. he ended up opening a dollar store, which of course tanked within a year. The idea of needing a local hardware supplier is real. The only options are Best Buy and Computer Rennaisance. CR is horrible and overpriced. Best Buy limited. There's smaller mom/pop used stores but, well, they suck too for lack of modern components.
Summer 2010 I drove down to Microcenter in Yonkers (less than 2 hours drive) to get all the parts for my current build. It's better than 3+ hours to Boston. When I'm ready for my next build I'll likely be doing the exact same thing, or ordering the parts online. I get 99% of my random and business related parts online anyway, but sometimes it's nice to buy and build the same day.
I don't know if they still have them, but there used to be a regular computer fair at Hudson Valley Community College. You could get most components for a good build there back-in-the-day (2003/2004).