Super Micro X8DTH-iF WS build

Disturbingly interesting video, changed my firewall rules....

TBH I should firewall off my whole management VLAN and block internet access from that subnet.

EDIT:
And now that's done.

The IPMI has its own dedicated Ethernet port, but unfortunately it will piggyback on the other two ports if it gets blocked on its own port. It broadcasts it's there too. I'll need to figure out how to completely wall that sucker off. If I need that feature, I'll have to figure out a way to secure that port with high encryption of some kind. It's tough enough to secure the front door without worrying about a wide open or missing back door lol.

I have a layer 3 switch and keep my UPS, VMWARE management, ILO, IPMI/BMCs on a separate VLAN than that of my desktop LAN. In all I have 5 VLANS in use
My perimeter firewall now blocks outbound packets from that VLAN (a cidr /24 subnet)

All I have now is a couple of consumer routers wired and wireless and a cable modem. Got VoIP on the hardwired router. I wonder if this company can secure my fort (especially IPMI)?

https://www.ubnt.com/unifi-switching-routing/usg/

It would also be nice to have secure WiFi when I'm out in the back 40 and not have to worry about 1337 pedophile wardrivers trying to hack in.

Got that 200 mm fan mounted on the side panel pointing at the heart of the NB/PCIe slots. System temps went down in spite of two graphics cards running at 100% and closing the case. FAH is up and running with a vengeance.

I also fold on my main rig. You guys are complaining about losing programs because of a fresh install. To me, its a clean purge. I tend to load too many programs over time and then not able to completely remove them.

We use sysprep at work and its a great little tool when used with MS image server. It seems to me that we had sysprep with win 98 but we used a 3rd party program to copy drive images.

Just as well I did a fresh install. There were some outdated BIOS issues that needed to be addressed and an old win 7 installation would have added another layer of doubt / confusion to complicate the troubleshooting process.

Ubiquiti is quite good, cannot speak for the UTM/security though.

I have a 5KM wireless bridge accros a lake with some wounderfully stable ubiquiti kit.

Getting some unifi UAP-PRO for home.

Along with a load of cisco stuff I have an atom running pfSence, I rate this.

aftermath said:

Ubiquiti is quite good, cannot speak for the UTM/security though.

I have a 5KM wireless bridge accros a lake with some wounderfully stable ubiquiti kit.

Getting some unifi UAP-PRO for home.

Along with a load of cisco stuff I have an atom running pfSence, I rate this.

Click to expand...

unifi UAP-PRO for home?

I was having issues with BSoD while running GPU_FAH. Went into the BIOS and disabled some server specific stuff. So far so good. This board has worked hard for 4 years and it should last another 4 easy as a workstation/desktop system. I'll look up all the settings and shut off any unnecessary services as I find them. I want it mainly focusing on FAH and my apps.

Audioaficionado said:

unifi UAP-PRO for home?

Click to expand...

Those are the ones I want, I have some meshed some hard wired with the existing system.

Audioaficionado said:

I was having issues with BSoD while running GPU_FAH. Went into the BIOS and disabled some server specific stuff. So far so good. This board has worked hard for 4 years and it should last another 4 easy as a workstation/desktop system. I'll look up all the settings and shut off any unnecessary services as I find them. I want it mainly focusing on FAH and my apps.

Click to expand...

I had random crashes when I first built, really frustrating. What would happen is after a few successful boots it would crash the next time within a few minutes.
No BSOD.
No Errors logged.

I disabled HT and the Cache Optimizations which made the crashes less frequent.

In the end I think it was the PCIe Native mode power management settings

Theory

PCIe Goes Off --> GPU Insufficient Power; Hangs
PCIe Goes Off --> Fiber Chanel Goes off no disk; Hangs
PCIe Goes Off --> Network Cards go off; Hangs?

These boards have a boat load of settings that aren't well documented by SM. They assume that if you have to ask, you aren't a professional IT guy and thus shouldn't have an SM product. A ton of these server boards are hitting the used market as businesses retool to the newest tech and lower power consumption. I'm hoping to find a glossary of sorts that explains what is what and the basic function.

I might have allready found some.
Fire them over and I'll help where I can. I've been using Supermicro and Tyan for a while.

I'll get into the BIOS and get some screen shots up. An interesting thing is that you can force 1333 DDR3, but it still defaults to 1066. It might be limited to the CPU bus speed.

I think the Force speed X-Mhz is only for the situation where ram is un/miss-read but the CPU supports X-Mhz.
I don't know for sure but doubt that it works if you over populate on DIMMS. If its over populated its supposed to also run at lower clocks.

I only have 6 of 12 slots populated (blue) for 24GB. It's no biggie as the SPD timings are tighter to partially compensate.

Wow... Glad I found this *recent* thread on this board. Reading these problems kind of pisses me off and makes me reconsider.

First, before I get to a lot of the problems in this thread, I need to ask: Have you, or the guy with the X8DAH, measured idle wattage from the wall? I'm on the border between building a X8DTH vs an X10S. I know, the Haswell X10s are much lower idle wattage and I've seen those single and dual CPU wattage. But, I haven't seen much on the X8DTH and X8DAH idle watts.

My problem with the C612 chipset Supermicro boards is - COST. It will literately double my entire server budget for roughly a tad more CPU Passmark over what I have.

Sitting next to me:

2x L5640 Hex core 60W CPUs (CPU passmark 10500), 12 cores
6x 4GB ECC RDIMM 1333Mhz 1.35V
*Need motherboard...*

My goals are to have a low-powered server, hopefully around 110W (ehh), and still have enough muscle for 5 to 7 transcoding streams of Plex PMS + the 3 VMs I tend to run, one of which is my SABnzbd unraring.

I want the option to install a few GPUs at a later time and do some decrypting and Tor-hashing, and maybe get back into FAH as well. But reading these issues in this thread has me re-thinking the X8 platform all together... I may just sell my CPUs/memory now.

It's hard for me to spring for the X10 platform, at a staggering $900 to $1100 more. But that platform, with a single E5-2690 v3 would give me the same 12 cores at a slightly higher 13000 passmark, but the "idle" wattage would be way down around 40 to 60W. Extremely tempting, but the cost... Yowser.

X8DTH

Anyhoot, now onto a few things listed in this thread. I've been studying this board for a while as I, like yourself with the Commodore and Pentium Pro dual socket setups (been there!), I carefully build up my systems.

PCIe Slots 1-3 are connected to CPU 1.
PCIe Slots 4-7 are connected to CPU 2.
The BIOS does not have the NVIDIA SLI nor ATI Crossfire embedded instruction set.

If you dig deep into the manual, it states that only SR or DR (Single Rank or Dual Rank) memory, occupying only 1 slot per channel, will run at 1333. What that means is you can only run 3 memory DIMMs per CPU, or 6 for 2x CPUs (3 each CPU) to get 1333. And they must be either SR or DR. That's the secret to 1333 Mhz memory in these board. Actually, I found a few people running 1600 Mhz in these boards, Unbuffered ram.

QR (Quad Ranked) will be forced down to 1066 or 800 no matter what.

Using 2x memory dimms on a single channel will download to 1066 automatically. So, for each CPU, instead of 3 you install 6 and BAM you are at 1066.

^- this is all in the manual.


X8DAH

The guy with the X8DAH... Yes, this is the ONLY X8 LGA1366 board that supports Nvidia SLI. "Nvidia SLI" is an embedded BIOS setting; so, it's up to the vendor of the BIOS to specify it or not.

I'm sure you know, but the two farthest x16 slots are the full bandwidth x16 slots. The middle one is x8.

A note about PCIe 2.0... I know there are a lot of reviews that say PCIe 3.0 doesn't matter over PCIe 2.0. I am here to tell you my reviews show otherwise, gaming at 6000x1080 with 2x and 3x SLI systems. Just a note in case you ever go tri-monitor.


IPMI

I've been desperately trying to find both onboard 6Gbps SAS2 versions with "F" or IPMI destinations. But after that video you posted, wow... Frack that.

Did you try to OC already with SetFSB or RW-Everything?

One reason I'm still at 1066MHz memory speed is that my CPUs won't support 1333. I have to get faster CPUs to get there. I haven't found any setfsb files written for my SM chipset. Not familiar with RW-Everything.

I had a lot of hassles getting my SM system to work, but it's working great now. I have no regrets. I don't think I'm going to bother with OCing this system. I'll do it the Intel way and just get a faster CPU lol. They're dirt cheap now (compared to a V3 processor).

The SR-2 is crazy expensive and you don't know how much it was abused by the previous owners. I've heard they are somewhat fragile boards while SM boards are usually very well designed and built. I know my SM was used 4 years as a server and then retired a year before I got it. Once I got all the old settings cleared out and updated the BIOS/firmware, it's been running 24/7 serving as my desktop/workstation/FAH machine very well.

I read just about all of the X8D* manuals. You can only get 1333 Mhz if using 1 DIMM per channel. The X8 series is the LGA1366 socket, which has 3 channels. So, that means 3 DIMMs per CPU. If a dual CPU board, that's 6 DIMMS total (3 per CPU). If you double up and put two DIMMs per channel (6 DIMMS per CPU), it will drop down to 1066 or 800 Mhz, according to the manuals. They also need to be single or dual ranking.

And yes, you need a CPU to support them. The L5640s are a nice, at 6 core and only 60W TDP.

For the record, to update this thread on my last post...

I ended up with an X10SRA and an E5-2690v3 ES I snagged off of ebay. CPU passmark is a tad over 20,000 - much higher than the 2x L5640s I was looking at. It still added $700 to my overall costs though...

...but, my idle wattage in the SC846 chassis w/Platinum 660W ATX PSU, 64GB 2133 Mhz ECC ram, Linux w/C6 enabled and all HDDs spun down (except the SSD for OS), I am pulling 58W from the wall (via Kill-a-Watt readings). You just can't beat that. Just running 10 total drives as the board only supports 10 SATA drives. I don't need more than that at the moment. But if I do, my wattage will increase by 9W when installing the LSI 9211-8i I have (already measured).