The OpenPOWER Saga Continues: Can You Get POWER Inside 1U?
by Johan De Gelas on February 24, 2017 8:00 AM ESTTyan's GT75-BP012
Getting to the meat of today's article, we have the Tyan GT75-BP012. Anton has already described the Tyan GT75 servers in great detail here, so we will recap and add a few details.
The Tyan GT75 machines (just like the Tyan TN71-BP012 servers launched a year ago) are based on one IBM POWER8 Turismo Single Chip Module (SCM) processor, offering either eight or ten cores. This CPU finds itself paired with Tyan's Habanero motherboard, the same as in IBM's most affordable OpenPOWER server, the S812LC.
The board has 32 DIMM slots using four IBM Centaur memory buffer chips (MBCs). Since the operational voltage of the Centaur chip PHY maxes out at 1.43V, only low power DDR3 DIMMs are supported. The largest supported DIMMs are the quad ranked 32 GB DIMMs with 4 Gbit chips, allowing the server to have up to 1 TB of RAM. Unfortunately, the latest 8 Gbit based DIMMs are not supported. Tyan ships the server with eight 16 GB DIMMs – for a total of 128GB – if you take the standard configuration.
| Tyan GT75: IBM POWER8 Turismo CPU Options | |||||
| POWER8 8-Core | POWER8 10-Core | ||||
| Core Count | 8 | 10 | |||
| Threads | 64 | 80 | |||
| Nominal Freq. Turbo |
2.33 GHz 3.025 GHz |
2.095 GHz 2.926 GHz |
|||
| L2 Cache | 512 KB per core | 512 KB per core | |||
| L3 Cache | 8 MB eDRAM per core 64 MB per CPU |
8 MB eDRAM per core 80 MB per CPU |
|||
| DRAM Interface | DDR3L-1600 (Low Power Only) | ||||
| PCI Express | 3 × PCIe controllers, 32 lanes | ||||
| TDP | 130W 169W |
130W 169W |
|||
As the OpenPOWER POWER8 has to fit and operate within a 1U home, the clockspeed is limited to 2.328 GHz nominal. However, that is just a paper spec just like the clockspeed of the Xeon E5. In reality, the power governor defaults to "on demand". In that case, the CPU runs at 2.06 GHz at low load, and boost up to 3.025 GHz when the CPU is fully loaded. The speedsteps are very small, only +/- 30 MHz, so the second highest speedstep is 2.99 GHz. Below you find the configuration table of all Tyan GT75 servers.
| Comparison of Tyan GT75 Servers | ||||
| BSP012G75V4H-B4C | BSP012G75V4H-Q4T | BSP012G75V4H-Q4F | ||
| CPU | IBM POWER8 8-Core 2.328 GHz 130 W/169 W TDP |
IBM POWER8 10-Core 2.095 GHz 130 W/169 W TDP |
IBM POWER8 10-Core 2.095 GHz 130 W/169 W TDP |
|
| Installed RAM | 8 × 16 GB R-DDR3L | 16 × 16 GB R-DDR3L | 32 × 16 GB R-DDR3L | |
| RAM (subsystem) | Up to 1 TB of DDR3L-1333 DRAM, 32 RDIMM modules, four IBM Centaur MBCs | |||
| Storage | 2 × 512 GB SSDs | 2 × 1 TB SSDs | 4 × 1 TB SSDs | |
| Tyan Storage Mezzanine | MP012-9235-4I (4-port SATA 6Gb/s IOC w/o RAID stack) |
|||
| LAN | 4 × GbE ports | 4 × 10 GbE ports | 4 × 10 GbE ports | |
| Tyan LAN Mezzanine | MP012-5719-4C Broadcom 1GbE LAN Mezz Card |
MP012-B840-4T Qlogic+Broadcom 10GbE LAN Mezz Card- |
MP012-Q840-4F Qlogic 10GbE LAN Mezz Card | |
In today's article we're review the basic model, the BSP012G75V4H-B4C. Notice the twelve (!) fans.
The Tyan GT75-BP012 makes use of Tyan's mezzanine cards for networking and for the storage controller. As a result, it can equipped with up to four 3.5” hot-swappable SATA 6G HDD/SSDs and four network controllers (1 GbE or 10 GbE) without using the 8-lane PCIe riser.
Now if you've been counting the CPIe lanes required for all of this, it seems like we should be a bit short, and indeed that's the case. Digging a bit deeper, we'll find that the server is using a PLX PEX8748 PCIe switch to take a PCIe 3.0 x8 root port from the CPU and switch it among the LAN riser, SATA riser, and the two black PCIe x8 slots.
Facebook
Twitter
RSS
28 Comments
View All Comments
Amandtec - Friday, February 24, 2017 - link
Only an amateur here but I did read some of your previous articles on this. I thought the big advantage of POWER was it was encouraging 3rd parties to bring ASIC's into their server space, while Intel wan't to own the whole hardware setup? Surely ASIC's is where the whole performance per watt game ends - see Bitcoin mining.ddriver - Friday, February 24, 2017 - link
There is nothing preventing you from building your own accelerators and hooking them to a free PCIE slot.JohanAnandtech - Friday, February 24, 2017 - link
True. But you have only one PCIe 8x and AFAIK it is not an OpenCAPI one, nor an NVME capable.JohanAnandtech - Friday, February 24, 2017 - link
I meant NVLink, not NVME :-)SarahKerrigan - Friday, February 24, 2017 - link
The GT75 is a weird system and I've never been clear on why it exists. The Supermicro 1U (S821LC) at least gets you two CPU's for cheap in 1U... I'm not overly surprised by poor performance here. This is a 130W CPU, not a 170 as Anandtech says, running at low clocks. 170 is only the turbo power. I'm also guessing it's loud as hell with those fans.That being said - Anandtech also has a history of interestingly Intel-centric interpretations of results. Remember when Intel's uarch was "a lot more sophisticated" than P8 according to Anandtech based purely on ST 7zip results, and then when ThunderX did well on 7zip, Anandtech said 7z was a meaningless benchmark irrelevant to server workloads? I've also noticed that since gcc's Power output improved (~4000 MIPS single-thread compression with gcc 6.2 on 3.3GHz P8), 7zip has conveniently vanished from Anandtech's Power reviews. The idea of drawing conclusions about microarchitectures based on a balance of a range of tests seems alien to them. Looked at through that lens, Power8 (well, in systems not named GT75 :P) looks decent at some things, less decent at others, but overall pretty good. Database perf/W, per Anandtech's testing, is better than Haswell. This isn't a bad place for P8 to be, considering it's been shipping since early 2014 and is on the verge of replacement.
Tl;dr - GT75 is a turd but Anandtech sees what they want to see. I have to wonder if Ryzen is going to be reviewed the same way.
JohanAnandtech - Friday, February 24, 2017 - link
7zip is indeed a bad indicator for server performance, but at the time, we only had access to a virtual machine on top of a POWER8 with 2 GB of RAM and we thought it might give us a first glimpse of what the P8 was capable off.As time progressed, we understood that is mostly a TLB/latency sensitive benchmark. So it has no place in a server oriented article, it is mostly interesting to discuss micro arch details.
Ian Cutress - Friday, February 24, 2017 - link
Anandtech has different editors focusing on different areas - Johan on Server, I'm on CPU, Matt on mobile, etc. Feel free to reach out via email if you have suggestions for us.Kevin G - Friday, February 24, 2017 - link
This isn't the best showing of POWER8. Those chips need robust cooling to keep their clock speeds up to be competitive. The 1U form factor does place a lot of constraints on the design.As for the Tyan system itself, I'm kinda of surprised that they went with the Centaur buffer that uses DDR3 as IBM has reportedly been shipping buffers that support DDR4 for nearly half a year now. That'd lower power consumption greatly, though likely not enough to be competitive with Intel on a system level performance/watt metric. Moving to DDR4 would give the system a massive increase in memory capacity as 128 GB LR DDR4 DIMMs are shipping with 256 GB LR DIMMs on the horizon. Using 256 GB DIMMs, a system like this would support 8 TB which is a lot for a 1U server.
Considering the internals of the system I'm not surprised but a secondary PSU option would have been nice, even if it was external. PSU redundancy is remarkably common, even for 1U systems.
I'd also be worried about IO performance on this system with all the networking and SATA ports hanging off of a single PCIe uplink to the CPU.
I do agree with the conclusion that POWER9 looks to be very promising. The nice thing is that IBM is going to be offering both the SMT4 and SMT8 cores in both types of sockets (essentially four different dies!).
ddriver - Saturday, February 25, 2017 - link
Form factor wont change the facts - it is a power, it is too expensive, and it is too slow. Power sounds great on paper, but it doesn't seem to deliver in practice.Kevin G - Tuesday, February 28, 2017 - link
Form factor does matter when it changes the cooling system which in turn can invoke thermal throttling.Even on the x86 side of things, half wide 2U servers are popular as they can use larger fans despite having the same effective density as a full width 1U server. The increased air flow into a chassis is great for keepings cools and helps maintain high turbo levels for performance.