Erlang

I'm grouping all three readings into one blog post because they're all perspectives on the same thing.

Problem: Writing reliable, scalable, industrial-grade distributed systems. Not an easy problem! And it's definitely real and becoming realer for more types of applications.

Solution/Nugget: Erlang has a lot of interesting features, but the main idea is to give you a more restrictive programming model provides good primitives for writing distributed apps but makes it hard to get into trouble. In other words, some tasks are easier if you tie one hand behind your back.

The actual model chosen by Erlang is actors communicating by asynchronous messages at the high level, and functional programming within an actor. The use of functional programming makes it easy to manage memory and prevents the programmer from getting into trouble with shared state (since variables are read-only). It's also nice for making program shorter, although Erlang syntax has problems too (see below). The use of actors makes it possible to treat different request handlers / etc as different entities in the program while sharing a small number of physical threads (kind of like the lightweight threads in Cappricio). It also real models networks directly, as messages may get lost, reordered, etc; applications or libraries can choose how to handle these cases. Most importantly, actors are independent units of failure. Along with Erlang's error reporting and process supervision mechanism, this lets multiple modules with potentially buggy code coexist within one runtime and survive crashes of some of the actors without taking down the entire system. The use of message passing and functional programming is also what makes it easy to do garbage collection and runtime code replacement, because the interfaces between executing processes are clean.

Why is the solution different from previous work? It actually works: people have built large systems with many nines of availability out of it. This derives from the fact that Erlang started out in a very reliability and scalability conscious environment -- a telecom -- and added a minimal set of features to make programming distributed systems easier. The result is a language that might not fit nicely into most traditional programming paradigms (it's not really purely functional for example) but gets the job done. Most importantly, Erlang takes into account "production" issues like code replacement and isolating failures that many academic systems don't because academics are not as aware of these issues.

Fundamental tradeoffs: In designing programming languages, there seem to be two related tradeoffs:

1. Performance versus conciseness: Languages with high-level programming features like functional programming, dynamic typing and pattern-matching generally can't perform as well as languages that give you direct access to memory and an easy way to map your program onto CPU instructions. However, they do give you conciseness and as a result more debuggability and testability. Erlang opts for conciseness.
   
2. Range of features versus reliability/predictability: Both high and low level languages can have broad or narrow feature ranges. For example, C++, a "low-level" language according to the definition above, nonetheless allows all kinds of programming paradigms (OOP, functional [sort of], procedural, generic, etc). Ruby, a more "high-level" language, also offers all of these. These languages seem great when you learn them because they let you use a wide range of programming styles, but in doing this, they give you more opportunity to shoot yourself in the foot and they also become very hard for compliers and runtimes to analyze. In contrast, Erlang is a high-level language with a restricted feature set. The restrictions make it easier to debug Erlang programs and to run them reliably, at the cost of removing some programmer flexiblity. SQL is another high-level language with restricted feature set, although it doesn't have quite the predictability benefits of Erlang because it is declarative. I actually don't know any low-level language with restricted feature set but the way Google uses C++ for example is to restrict usage of the crazier/harder-to-debug features.

The interesting conclusion from this taxonomy is that it isn't just a three-way tradeoff between performance, abstraction and predictability. Abstraction can be both helpful and harmful for predictability. In the Erlang case, it's helpful, but in SQL, which is also very abstract, it's hard to predict performance. This is why I tried to split up abstraction into conciseness and range of features, although these aren't great terms for the distinction I'm trying to make.

Is it influential? Yes, people at Amazon, Facebook and other companies are now using Erlang for web systems. New high-level languages like Scala also support Erlang's concurrency model.

Criticisms: While Erlang is a very neat system, it's also 20 years old, and some of the design decisions are really weird. In particular, Erlang syntax can be unnecessarily restrictive, and Erlang string processing is really inefficient because strings are represented as lists of 32-bit integers. This post by a CouchDB developer does a great job explaining Erlang's limitations. So does Chris Newcombe's presentation on Erlang.