End-to-end Arguments in System Design

This paper presents one of the most fundamental design principles in distributed systems, that of implementing functionality at the endpoints of the network rather than providing complex features in the intermediate hardware and software. The most powerful reason for this is that implementing functions in the network may be not only inefficient, but inadequate: Functions such as error recovery, duplicate data detection, acknowledgment transmission, and security can only be done end-to-end, because even if single links can provide them, there is no way to guarantee that the concatenation of multiple links has worked correctly without checking at the ends. If a function will be provided by the ends, then duplicating it in the network may do nothing more than add overhead. However, there are also other reasons to favor the end-to-end approach: Keeping the network's requirements simple provides the flexibility to run either new functions or improved implementations of existing functions at the ends, over the same hardware. In this light, the network should strive for a similar role to the RAM or disk in a computer, providing a minimal interface to some resource in order to let applications use it as they wish, rather than trying to supply a complete system which anticipates all possible higher-level functions.

Putting all functionality at the end-points is of course impossible, and the paper does say that functionality in the network may be useful for performance. The main example of this is error detection and correction: performing these checks at the link layer leads to much greater efficiency than sending a packet through multiple links and only then checking it. However, in general, this applies to any functionality where the problem size scales with the size of the network and the issue is not fully under the control of the application: For example, in the Internet, routing is a function performed by the network rather than the end-nodes.

One very important argument for end-to-end design is not explicitly stated in the paper - that by keeping the network functionality minimal, we can support multiple types of hardware devices and software on the same network. This is a social and economic argument - it means that vendors will be more likely to adopt the design, leading to an ecosystem of complementary and competing products, which improve the utility of the system for everyone. In fact, this phenomenon is called the network effect. While the Internet may not have been designed with creation of a commercial ecosystem as a explicit goal, the network effect let it grow rapidly and ultimately benefited even its original military users.

The main shortcoming of the end-to-end argument is that it does not explain exactly which set of functions should be provided by the network and whether there are limits to what can be done with, for example, the IP model. This is the source of many networking research topics, including multicast, active networks, mobility, denial-of-service prevention etc. This issue is still mostly unsolved today.

In summary, this is a good paper to begin a networking course with because it introduces both the wide range of concerns faced by a network (reliable delivery, acknowledgments, security, etc) and an important argument against the ever-present temptation to add more functionality into a system.