cy384

a few years of power9

2025/10/13

I like weird computers, and I've become fairly dedicated to running FOSS as much as possible; obviously, as soon as I learned about Raptor Computing Systems' (RCS) Power9-based machines, I knew I was target market. They were launched as the fastest 100% open computers on the market, and even now, I believe they still are. After a few years in service, I figure I should make some comments on my Blackbird-based server.

software

Power9 software support is actually pretty good. All the major Linux distros have support and install easily (I use Ubuntu at the moment), and 99.9% of open source projects work fine. A fair number of projects like ffmpeg and llama.cpp even have optimized assembly. IBM puts a solid amount of work (and probably salaries/donations) into ensuring Power remains viable (likewise for s390x). On the other hand, while basically everything can be compiled for ppc64le, nobody builds containers for it. You'll probably need to build everything you want and manage your own registry. For my uses, this isn't a problem, but caveat emptor.

The only other software pain point I'll mention is lack of GPU compute drivers:

• I'm not sure what the status of ROCm is, but I don't think anyone has managed to get it going well on ppc64le yet. Someone got it running on a raspberry pi, surely it's doable? I will say that AMD's open source drivers are fine for standard desktop usage, even if ROCm isn't.
• IBM paid up to get nVidia to port CUDA over, but the supported kernel versions are way out of date now.
• Intel seems to still be hashing out their driver situation. Maybe it's better than it was the last time I looked, especially on their newest hardware.

hardware

I only have a Blackbird, so that's what I'll be commenting on. It's a standard mATX form factor with a normal complement of ports. I have a pile of complaints:

• Only two RAM slots, seriously limiting memory bandwidth and capacity
• Only two PCIe slots (this one drives me insane) and no bifurcation support (actually bifurcation is used, but for stuff on the board, not to the slots)
• CPU is mounted such that you can blow hot air up (over the voltage regulators) or down (right at the top PCIe card) instead of front/back
• Pin header positioned to interfere with most cards in the second PCIe slot (had to desolder it)
• Four SATA ports is not quite enough for a file server that's starved on PCIe slots
• No heatsinks on the voltage regulators (which the Talos has)

I won't complain about the price, because everyone else already has. I paid $2000 for the motherboard and 3U cooler from RCS, and $200 for a CPU on ebay. I have a 02CY231, with:

• 16 cores, 64 threads
• 4MB L2 cache, 160MB L3 cache
• 2.50 GHz, boosting to 3.80 GHz
• 160W TDP

Idle CPU power usage is around 30 watts.

The rest of the setup:

• 128GB of DDR4 RAM
• 8x 3.84TB SATA SSDs
• Mellanox MCX416A ConnectX-4 100Gb NIC (Mellanox has very solid mainline Linux drivers)
• Broadcom 9500-16i (closed source config utility is available as ppc64le binaries)
• Sliger CX3150x 3U case (probably not what I would buy for this system if I did it again)
• Corsair SF600 PSU
• A few Noctua fans

hacking and firmware

Since it's 100% open, you can mess with absolutely anything on this system, from the embedded power management cores on the CPU to the BMC (running OpenBMC). I modified some parts of the firmware to adjust the power and CPU clock settings, since the chip (not provided by RCS) didn't have custom settings and ran at the (completely adequate) defaults. I posted the details over on the user forum. I did also stick some little heatsinks to the voltage regulators.

Unfortunately, I don't think RCS has the software resources to maintain this, and there's not enough of a community to keep things up to date. I would be shocked if RCS has sold more than 500 systems to the public (the serial number on my board is, uh, very low!). For example, while the software is stable, to me a large part of the appeal in using OpenBMC is in contrast to the conventional BMCs being wildly outdated and insecure always. It would probably not be too hard to modernize, but who has the time or the will? Sadly, not me.

competitors

RCS released their first Power9 system about six years ago. Let's look at the current processor landscape:

• amd64 remains fast, cheap, readily available, and supported by all software. Every single amd64 processor made in the last ten years has a pile of secret "security" bullshit.

• arm64 still has a huge gap in the market between high end systems and SBCs. There are a million Raspberry Pi competitors, which are generally slow, lack mainline kernel support, and have non-standard form factors. There's basically no secondary market of high end arm64 servers. An Ampere Altra dev board seems to be the only beefier option available to an individual buyer, and they're fairly open. Apple's M series machines seriously lack expansion options, and despite some very impressive efforts from the Asahi project, are still missing lots of drivers. Qualcomm seems weirdly dedicated to open source lately, but still not quite there. I'm not aware of any arm64 systems faster than a Raspberry Pi that have completely open firmware.

• riscv is not even slightly competitive in any quantitative regard. There are some options that have open firmware.

• Every other CPU architecture is dead.

long term viability

Pros:

• completely open, could maintain it myself forever
• dedicated community

Cons:

• fast and expandable enough for my purposes
• nobody else is maintaining it, builds are starting to link rot
• how long will distros support Power9? POWER8 got deprecated kinda quickly
• no followup hardware and it's been a while

Overall, kinda sad state of things, but I'm still happy with it, and expect to keep this as my main home server for the long term. I do also have an Intel mini PC for convenient experiments and Kubernetes stuff.