A Scalable, Commodity Data Center Network Architecture

The problem: Data center networks are expensive and achieve poor bisection bandwidth due to their use of a tree topology with non-commodity switches at the highest levels. This is definitely a real problem - applications like Hadoop have to try to keep communication within a rack and are bound by network performance.

Solution: Use commodity switches in a fat tree topology, with a centralized scheduler to achieve load balancing.

How it differs from previous work: These kinds of topologies have been used before in high-performance computing. The new element here is trying to use them for Ethernet/IP/TCP so that apps don't have to be changed and commodity switches can be used. In particular, the authors want to maintain the ordering property of conventional LAN topologies to have TCP perform well.

Tradeoffs / criticism: At first sight there seems to be a tradeoff between preserving packet ordering and achieving high bisection bandwidth using commodity elements. However, we know that using random paths we can do pretty well if we give up strict ordering guarantees. What exactly is the potential variation in latency that would be added by a reordering buffer in front of TCP? Might it not be fine to incur this? It seems much more attractive to modify host OSes a little bit than to require a centralized scheduler which is a central point of failure and is tailored to specific assumptions about traffic patterns (e.g. they say they assume each host has only 1 large flow coming out of it).

Long-term impact: This paper definitely puts data center topologies in the spotlight and provides an economic argument for using commodity switches. While the solution presented here may not be the best one, it's not inconceivable that the idea of using commodity switches will stick around for a while.