The five principles the slides point out are familiar to anyone who's read about GFS, Dynamo, MapReduce, etc. It's nice to see consensus! The most surprising one to me was that eBay uses machine learning for automation, although this seemed to be more for the search side than the operations side.
The design patterns provide some good ideas as well. While I didn't fully understand the point of the scalable multicast and event queue patterns from the slides, I really liked the idea of a global message bus with logging to deal with failures, and of rollback. This reminds me of some of the elements of the Facebook architecture - for example, a common RPC format, Thrift, for service interactions, along with an RPC logging layer (Scribe). However, the Facebook architecture is somewhat different because you don't have a very clear concept of independent services yet, whereas eBay seems to be a more traditional SOA.
Their view on clouds is pretty promising - although they identify some "faux PaaS", they are actually considering using external cloud providers. This provides a strong case for cloud computing. One difference between eBay and other web applications though is that much of the data on eBay is public, so for example putting a search service in the cloud does not introduce a risk of losing consumer data. If a company like Facebook wanted to put services in the cloud on the other hand, they'd have to think very carefully because most of the site is private information.
Finally, the presentations also do a good job talking about tradeoffs. They mention CAP directly, and they say that they go for the A and P as much as possible. They also talk about the friction between performance, reliability etc and extensibility, though they don't try to say that this is an unavoidable tradeoff.
3 comments:
"However, the Facebook architecture is somewhat different because you don't have a very clear concept of independent services yet, whereas eBay seems to be a more traditional SOA."
Do you think that the more Facebook matures, the more it will move towards SOA? It seems like SOA would better facilitate code reuse within a large company (e.g. Amazon, Y!, Facebook, etc.) Why isn't Facebook doing this?
I think the difficulty has been adding new features at a rapid rate as opposed to spending a lot of time to refine an existing product. A site like ebay doesn't change its basic service very often, but Facebook has had a lot of big changes, such as the introduction of news feed, the introduction of Facebook apps, the new profile, etc. The company was probably just not at the scale of something like ebay where they could set aside a team of people and tell them "okay guys, you design version 2 of the site from scratch". SOA is undoubtedly the way to go though, and a lot of features (e.g. chat, photos) are now separate "services" with their dedicated pools of machines and such.
Maybe one should come up with an architecture that allows rapid deployment of new features... Such an architecture should probably provide a pretty high level (declarative?) API to enable fast prototyping and development of new features. An important question here is what are the services exposed by such API. Maybe something similar to Google Apps?
Post a Comment