NFD26 – Scaling Discussion w/ Arista

The NFD26 series of posts are my commentary from specific presentations held for Networking Field Day 26, a virtual event presented by Tech Field Day.

Something that really peaked my interest during Arista Network‘s NFD26 presentation was the time they spent discussing scaling caveats and opportunities in modern data centers. There continues to be debates, both in the data center and campus around which is better, fixed or chassis-based switches? Now that is obviously a broad and open-ended question, so let’s focus it up a little bit. In terms of data center scale, what makes the most sense, chassis-based switches, or a distributed line-card model leveraging a fixed (or chassis based depending on port density), spine/leaf architecture. Arista definitely came prepared to discuss this and spent time on a couple slides contrasting the two options.

Arista described the two routing architecture scaling methods as either scaling up or scaling out. Scaling up describes a chassis based solution in which physical line-cards can be added to the base (backplane) chassis when additional capacity is required. Conversely, scaling out describes a spine/leaf topology that leverages physically diverse routers/switches that can be distributed throughout the data center. The spines are the physically disaggregated backplane that provide connectivity between the leaf nodes. The leaf nodes then, are the edge nodes where the edge connectivity takes place. Each leaf connects to each spine and leverages ECMP allowing for efficient throughput. Also, because each leaf is connected to each spine, any leaf-to-leaf communication just needs to traverse up to the spine and down to the destination leaf. When bandwidth scaling is needed, spines can be added to provide that. Here are the slides that Arista used to contrast the two scaling methodologies.

Bert’s Brief

I don’t necessarily think this is a “one size fits all” scenario by any means, but I come at it from two different standpoints, design and operational. First, from a design standpoint, when it comes time to scale in the scale up methodology, in relation to port count, you need a whole new chassis and linecard(s) to get started. On the bandwidth side, as stated in the above slide, the chassis fabric determines the bandwidth scale. With scale out, if more leafs are needed (*as long as you have spine ports available) you just need to add leafs. However, if the fixed spine is at capacity, you can physically upgrade one at a time in a fairly simple fashion. From a bandwidth scaling perspective, it might just be a fixed spine that you need to add, connect your leafs and away you go. From an operational standpoint, I think about support. Granted, chassis architectures can grow and improve over time, but it scares me that a failure in the actual chassis (backplane) could require you to need to replace the chassis and migrate linecards over to the new chassis. In a spine/leaf architecture leveraging fixed gear alone, you just replace the single device without needing to mess with the “backplane”. To me, the distributed modular method that the spine/leaf architecture provides makes a lot of sense from both design and operational standpoints. While not directly product driven, I appreciate Arista spending time going through this topic.