Hi Vukasin,
Before messing with the FSB, make sure to set the PCIe frequency to 100Mhz solid. If you leave it to
Auto it tends to go up along with the increased FSB.
About figuring out where the problem lies:
1. Did you try different straps / latches for the memory or just left the setting to auto? Sometimes a different latch could work better than the one AUTO selects...Also, try loosening the timings a bit to see if the memory is the issue.
2. Do you have a
DRAM Performance Level setting in the BIOS? If so, try setting it to
Standard
3. Do you have an option to mess with advanced timings for the MCH? Like MCH clock skew?
4. What's the BIOS version you have? Have you upgraded to the latest BIOS for your motherboard?
Another thing to do to see why the board can't handle the overclock is to try pulling out one of the memory DIMMs. I've had similar issues with 2 DIMMs set up in dual channel on my board. Single channel overclocked like a charm, but I had to go through a lot of trial and error to find a way to stabilize the system with a dual channel setup.
PS.:You can use CPU-Z to give us a shot of your memory/cpu/mobo specs