Monday, April 20, 2009

Scaling Out

Problem: Facebook needed multiple datacenters for reliability and performance (latency).

Importance: Relevant to everyone who hosts an Internet service.

Solution: The most interesting thing about this article is the fairly pragmatic use of existing technology rather than reinventing data center storage from scratch, which allowed them to deploy a second data center relatively quickly. The biggest "trick" or "nugget" is the extension to MySQL to cause it to invalidate caches when replication pushes a database change over, maintaining the invariant that MySQL never contains older data than memcached. The other interesting idea is read-your-own-writes by redirecting users to the California data center after a write.

Is it influential? Sure, many web companies use a similar strategy. The article clearly points out some of the issues that need to be taken into account when you have replication and caching together.

Criticism: The main problem with this article's approach is the flip side to the nugget - by using some simple hacks on existing technology (MySQL, memcached, load balancing) we might be missing out a more general, less hacky solution. Already the solution in the article only supports writes at one data center, for example. The worrying about memcached and MySQL at the application level is a serious concern for anyone interested in ease of programming and bug prevention in Internet services as well. Projects like SCADS try to take a more general and clean-slate approach to these problems.

No comments: