Netflix morphed from a private datacenter based monolithic application into a cloud based Microservices architecture. This talk highlights the pros and cons of building software applications as suites of independently deployable services, as well as practical approaches for overcoming challenges - especially in the context of an elastic but ephemeral cloud ecosystem. What were the lessons learned while building and managing these services? What are the best practices and anti-patterns?

3. • Netflix – background and evolution

7. • Netflix – background and evolution
• Monolithic Apps

10. Load Balancer
Monolithic App
Account
Component
Catalog
Component
Recommendation
Component
Customer Service
Component
Database

12. Time

16. User Accounts
Shopping Cart
Product Catalog
Customer Service

18. Product Catalog Customer Service

22. Tipping Point
& &
Organizational Growth Disverse Functionality Bottleneck in
Monolithic stack

23. MicroServices
Perhaps?

24. • Netflix – background and evolution
• Monolithic Apps
• Microservices

27. MicroService
MegaService

28. Attribution: Adrian Cockroft, Martin Fowler …

29. • Write programs that do one thing and do it well.
• Write programs to work together.
tr 'A-Z' 'a-z' < doc.txt | tr -cs 'a-z' 'n' | sort | uniq | comm -23 -
/usr/share/dict/words
Program to print misspelt words in doc.txt

30. Comparing Monolithic to MicroServices

32. Load Balancer
Monolithic App
Account
Component
Catalog
Component
Recommendation
Component
Customer Service
Component
Database

33. Load Balancer
Account
Service
Catalog
Service
Recommendation
Service
Customer Service
Service
Catalog
DB
API Gateway
Customer
DB

34. • Netflix – background and evolution
• Monolithic Apps
• Microservices
– Why?

37. Your
App/Service
Service X
Service Y
Service Z
Service L
Service M

38. • Netflix – background and evolution
• Monolithic Apps
• Microservices
– Challenges and Solutions

40. Vs

41. Heterogeneity
Can lead to chaos if not designed right …

44. non-JVM

46. Service Discovery
What’s your address?

47. Account
Service
Catalog
Service
Recommendation
Service
Customer Service
Service
X
Service Y
Service
Z
Service
Registry
Service
(e.g. Netflix Eureka)

49. 2 Billion
~20 Billion Fan
1 Request 1 Request
Monolithic
App
MicroServices

51. Your
App/Service
Service X
Service Y
Service Z
Cache
Cluster
Cache
Cluster
Tip: Config your TTL based on flexibility with data staleness!

52. Your
App/Service
Service X
Service Y
Service Z
Fn {A, B, C}
Cache
Cluster
Cache
Cluster

54. A/B Test
Service
App Service X Service Y Service Z
User Account
Service

55. A/B Test
Service
App Service X Service Y Service Z
User Account
Service
reduces dependency
load

57. Data Serialization Overhead
Data
transformation
getMovies() getMovie() getMovieMetadata(
)
C
l
i
e
n
t
D
C
l
i
e
n
t
C
C
l
i
e
n
t
B
Service A Service B Service C Service D
X X Avro
JSON Xml
Overall Response Time

60. A single “bad” service can still bring your service down

63. Your
App/Service
Service X
Service Y
Service Z
Service L
Service M

64. Your
App/Service
Service X
Service Y
Service Z
Service L
Service M
Your
App/SYeoruvirc e
App/Service

65. Your
App/Service
Service X
Service Y
Service Z
Service L
Service M
Your
App/SYeoruvirc e
App/Service
Service X
Service Y
Service Z
Service L
Service M

66. http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/WhatIsAutoScaling.html

67. http://techblog.netflix.com/2013/11/scryer-netflixs-
predictive-auto-scaling.html

68. 1.

69. Account
Service 1
Recommendation
Service 1
Customer Service
Service 1
API Gateway
Account
Service N
Recommendation
Service N
Customer Service
Service N
Customer Service
Load Balancer
Reco Service
Load Balancer
Account Service
Load Balancer
ELB

71. Account
Service 1
Recommendation
Service 1
Customer Service
Service 1
API Gateway
Account
Service N
Recommendation
Service N
Customer Service
Service N
Account Service
LB
Recommendation
Service LB
Customer Service
Service LB
ELB

72. http://github.com/netflix/ribbon

74. Service Dependency Visualization

75. Your Service
XY Farm

76. How many dependencies does my service have?
What is the Call Volume on my Service?
Are any Dependency Services running Hot?
What are the Top N Slowest “Business Transactions”?
What are the sample HTTP Requests/Responses that had a
500 Error Code in the last 30 minutes?

77. You
Your Service Dependency
Graph

84. • Netflix – background and evolution
• Monolithic Apps
• Microservices
– Challenges and Solutions
– Best Practices

86. http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html

89. three

92. Fault Injection Testing (FIT)
Device ELB Service B
Service C
Internet Zuul
Edge
Device or Account Override
Service A
Request-level simulations
Refer Re:Invent Talk “Embracing Failure: Fault Injection and
Service Reliability” – PFC305

93. https://github.com/Netflix/SimianArmy

96. • Netflix – background and evolution
• Monolithic Apps
• Microservices
– Challenges and Solutions
– Best Practices
• InterProcess Communication

98. Netflix IPC Stack (1.0)
A
p
a
c
h
e
H
T
T
P
C
l
i
e
n
t
Server (Karyon)
HTTP Admin Console
Eureka (Service
Registry)
Apache
Tomcat
Client
H
y
s
t
r
i
x
E
V
C
a
c
h
e
Ribbon
Load
Balancing
Metrics
(Servo)
Eureka
Integration
Bootstrapping (Governator)
Metrics (Servo)
Eureka Integration
Registration
Fetch Registry
A Blocking Architecture

99. Netflix IPC Stack (2.0)
Client (Ribbon 2.0)
Server (Karyon)
Admin Console
Eureka (Service
Registry)
Ribbon Transport
Load
Balancing
Metrics
(Servo)
Eureka
Integration
Bootstrapping (Governator)
Metrics (Servo)
HTTP
Eureka Integration
Registration
Fetch Registry
Ribbon
Hystrix
EVCache
R
x
N
e
t
t
y
RxNetty
UDP
TCP
WebSockets
SSE
A Completely Reactive Architecture

100. Details: http://www.meetup.com/Netflix-Open-Source-
Platform/events/184153592/

102. interface
@Http(method = HttpMethod.GET, uriTemplate="/users/{userId}/recommendations")
@Hystrix(fallbackHandler = RecommendationFallbackHandler.class)
@EvCache(name = "movie-rec", appName = "movieService", cacheKeyTemplate =
"{userId}")
recommendationsByUserId @Var "userId"
MovieService movieService = Ribbon.from(MovieService.class);
Observable<Recommendations> result =
movieService.recommendationsByUserId(“user1”)
.toObservable();

103. NetflixOSS

104. NetflixOSS
http://netflix.github.co

105. NetflixOSS
• Eureka
• Karyon
• Ribbon
• Hystrix
• Archaius
• Servo
• EVCache
• Curator/Exhibitor

106. Talk Time Title
PFC-305 Wednesday, 1:15pm Embracing Failure: Fault Injection and Service Reliability
BDT-403 Wednesday, 2:15pm Next Generation Big Data Platform at Netflix
PFC-306 Wednesday, 3:30pm Performance Tuning EC2
DEV-309 Wednesday, 3:30pm From Asgard to Zuul, How Netflix’s proven Open Source
Tools can accelerate and scale your services
ARC-317 Wednesday, 4:30pm Maintaining a Resilient Front-Door at Massive Scale
PFC-304 Wednesday, 4:30pm Effective Inter-process Communications in the Cloud: The
Pros and Cons of Micro Services Architectures
ENT-209 Wednesday, 4:30pm Cloud Migration, Dev-Ops and Distributed Systems
APP-310 Friday, 9:00am Scheduling using Apache Mesos in the Cloud

107. • Netflix – background and evolution
• Monolithic Apps
• Microservices
– Challenges and Solutions
– Best Practices
• InterProcess Communication
• Takeaways

110. http://techblog.netflix.com
http://netflix.github.io
http://martinfowler.com/articles/microservices.html
http://microservices.io/

111. Please give us your feedback on this
presentation
Netflix Booth