If you keep it on auto it will set a value that's way too high for that overclock.
You could keep all on default and see with cpu-z what the average vcore is when running prime95. My 2600k does 1.224 under full load with the turbo at 38x.
So I set it manual at that value and found 4.2Ghz only needs 1.2000 on my system
That's even lower compared to the stock voltage under load.
The 2500k has slightly lower vcores, but you can find them out with cpuz.
I find the stock cooler is a total waste of space in the cpu box. Even with stock settings it hits 80°C quite easy. It would really be wise to invest in a decent cooler, like the hyper212+ which does a good job, and isn't expensive. It will allow for higher overclocks should you desire to go that route eventually.
To be honest, going from 3700 to 4000 will not make a difference in gaming. Even going for the 4.4/4.6 Ghz gives little extra fps in games.
One thing to keep in mind, I started with a fixed vcore, this will be on 100% of the time, although the cpu clocks down, vcore doesn't when idling. Now I use offset, and control the overvolt with cpu offset voltage, takes a little experimenting before you get to the desired vcore when under full load, but will give the benefit of the vcore going low when doing simple tasks. Don't know if this really matters though, I ran my E8500 for 3 years at a fixed voltage without a problem.
Maybe you should go for a higher overclock, seeing your temps reaching 95°C and watching your system getting throttled,sure wont take you longer than an hour to decide for a better cooling device.