Video and slides synchronized, mp3 and slide download available at http://bit.ly/YpQCCJ. Sean Cribbs compares ACID with BASE, explaining the virtues and tradeoffs of eventually consistent systems and what developers should know in order to feel comfortable working with EC systems. Filmed at qconsf.com. Sean Cribbs is a Software Engineer at Basho Technologies, where he works on Riak, the fault-tolerant, highly-scalable distributed database. Prior to Basho, Sean was a freelance developer and consultant who also managed the development of the open-source Radiant web publishing system. He briefly studied Music Theory after receiving degrees in Computer Science and Music from University of Tulsa.

1. Fear No More:
Embrace Eventual
Consistency
Sean Cribbs
@seancribbs

2. Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/Embrace-Eventual-Consistency
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month

3. Presented at QCon San Francisco
www.qconsf.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. Distributed Systems Experts

6. FEAR
Photo from Wikimedia

7. ACID vs. BASE

8. ACID vs. BASE
• Strong consistency • Weak consistency
• Isolation • Availability first
• Focus on “commit” • Best effort
• Nested transactions • Approximate answer
• Conservative • Aggressive
(pessimistic) (optimistic)
Fox, Gribble, Chawathe, Brewer, Gauthier -
Cluster-Based Scalable Network Services (SOSP97)

9. ACID vs. BASE
“Inconsistency is “Being unavailable
the worst thing that is the worst thing
could happen.” that could happen.”

10. Why BASE / EC?

11. Why BASE / EC?
• “Omniscience” is expensive and slow.

12. Why BASE / EC?
• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.

13. Why BASE / EC?
• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.

14. Why BASE / EC?
• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.
“Any sufficiently large system is in a
constant state of partial failure.”
Justin Sheehy, Basho CTO

15. Why BASE / EC?
• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.
• You’re probably doing it already.

16. Safety &
Liveness
Leslie Lamport
1977

17. Safety

18. Safety
• “Bad things
don’t happen”
• Point-in-time
identifiable

19. Safety
• “Bad things • mutual
don’t happen” exclusion
• Point-in-time • partial
identifiable correctness
• first-come,
first-serve

20. Liveness

21. Liveness
• “Good things
eventually
happen”
• Always in
future

22. Liveness
• “Good things • starvation
eventually freedom
happen”
• termination
• Always in
future • guaranteed
service

23. ACID vs. BASE
• Strong consistency • Weak consistency
• Isolation • Availability first
• Focus on “commit” • Best effort
• Nested transactions • Approximate answer
• Conservative • Aggressive
(pessimistic) (optimistic)
Fox, Gribble, Chawathe, Brewer, Gauthier -
Cluster-Based Scalable Network Services (SOSP97)

24. Peter Bailis
Eventual
consistency is
not safe
“...it’s easy to satisfy liveness
without being useful... If all
replicas return the value 42 in
response to every request, the
system is eventually
consistent.”
http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/

25. Liveness of BASE
• Convergence - “eventual delivery”
• Responsiveness - “eventual service”
• Resilience - “eventual recovery”
• Consensus-free - “eventual progress”

26. Safety of BASE
• Durability - “accepted writes are not lost”
• Integrity - “data is not corrupted”
• Authenticity - “data is not forged”

27. Real BASE Systems
Photo from Wikimedia

28. Domain Name Service
• Federated, hierarchical database
• How qconsf.com becomes 77.66.16.106
• Layered system with caching

29. Domain Name Service
• Federated, hierarchical database
• How qconsf.com becomes 77.66.16.106
• Layered system with caching
Diagrams from Wikimedia

30. Domain Name Service
• Federated, hierarchical database
• How qconsf.com becomes 77.66.16.106
• Layered system with caching
Diagrams from Wikimedia

31. DNS Liveness
• Convergence - caches eventually expire
• Consensus-free - local authority over subtree
updates
• Responsiveness - intermediaries can cache
results and reply quicker
• Resilience - authority servers can be replicated/
load-balanced

32. DNS Safety
• Authenticity - forgery prevented by DNSSEC

33. BitTorrent
• Peer-to-peer
cooperative large-file
transfer
• Dynamic membership
and block discovery
through the “tracker”
node
• Epidemic effect
http://computer.howstuffworks.com/bittorrent2.htm

34. BitTorrent Liveness
• Convergence - all peers that remain connected
eventually become seeds
• Resilience - loss of one peer doesn’t impede
progress
• Responsiveness - closer, faster peers tend to be
preferred

35. BitTorrent Safety
• Integrity - each block is checksummed to
prevent corruption

36. The Web
• Sparsely-connected graph of hypertext
documents identified by URIs
• Rich caching semantics: expiration, validation,
control
• Fluid evolution through uniform interface
• Layered system (federated)

37. Web: Liveness
• Consensus-free - local documents can be
changed, moved, removed without coordination
• Convergence - caching semantics prevent
unbounded staleness, redirection
• Responsiveness - many parties can proxy, cache
• Resilience - failure of one server doesn’t stop
the system

38. Web: Safety
• Privacy & Authenticity - HTTPS/SSL/TLS
• Integrity - POST responses don’t pollute caches

39. Dynamo
1
• Key-value store: distributed,
replicated, partitioned
2
• Client requests can go to any
node
• Low-latency at high percentiles 3
• Many clones: Riak, Cassandra,
Voldemort

40. Dynamo: Liveness
• Convergence - read-repair, hash-tree
exchanges, vector-clocks
• Resilience - hinted-handoff, sloppy quorums
• Responsiveness - replication
• Consensus-free - loose coordination, concurrent
updates

41. Dynamo: Safety
• Authenticity - won’t serve data you didn’t store
• Durability - confirmed writes are not lost

42. ACID vs. BASE

43. CONFLICT
Photo by Associated Press

44. SPECTRUM
Photo by Associated Press

45. hu g s!
Embrace
Eventual
Consistency