Building a 25 Gbit/s workstation for the SCION Association

Summary

SCION 25 Gbit/s Workstation This is an LGA4677 socket and it's about to be fitted with a 12-core Intel Xeon CPU to power the 64 PCIe Gen5 lanes for 3x Mellanox NVIDIA BlueField-2 Dual-25G smart NICs, which will ultimately power the SCION Association's new 25 Gbit/s testbench workstation! I built it to develop and test a new AF_XDP underlay for the SCION OSS has received significant data plane performance improvements over the past years, but still requires further work. In this article, I'll walk you through the entire planning, building, and configuration process in almost full detail. It’s hard to say how many hours went into it, but it was clearly a multi-week endeavor. In total it cost us CHF ~3,741.34 (around ≈$4,700 USD) in materials. See the complete list of components at the end. Disclaimer: I spent many hours writing this article by hand, but I must confess, LLMs did help me formulate and polish parts of it. Background SCION SCION (Scalability, Control, and Isolation On Next-Generation Networks), is, in a nutshell, an IETF draft-stage technology of a growing alternative to the Border Gateway Protocol (BGP). It's an innovative inter-AS routing architecture designed to address BGP's fundamental flaws and security vulnerabilities. Maybe at some point in the future the Internet will run SCION, although a more likely scenario is that SCION and BGP will run alongside each other. What is clear, though, is that critical infrastructure should run on SCION, where: path authenticity explicit path control (e.g. geofencing) more consistent latency characteristics deterministic failover are required and best-effort BGP routing is an unacceptable risk. The national bank of Switzerland realized this and since 2024 Switzerland's banking infrastructure now runs on the SCION-powered SSFN, which relies on the commercial implementation from Anapaya Systems AG that currently provides up to 100 Gbit/s border router solutions. The free open source implementation github.com/scionpr...