Help our Mythical Mysfits find their forever homes. Our Mythical stack now supports automated deployments. It's now time to take it to the next level. In this workshop, we give you the hands-on experience you need to run microservices in the real world. We focus on optimizations, management, and troubleshooting with AWS Fargate, Amazon CloudWatch, and other common tools. Proficiency in Docker and AWS is recommended. For a more foundational workshop, consider CON214 and CON321, our other workshops in this series.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Mythical Mysfits: Management and
Operations for AWS Fargate
C O N 3 2 2
Mitch Beaumont
Solutions Architect
AWS
Tom Schultz
Solutions Architect
AWS
Andy Mui
Solutions Architect
AWS

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
AWS Fargate
Observability
Monitoring
Logging
Distributed Tracing
Alerting
Hands-on Lab

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Fargate
Scheduling and Orchestration
Cluster Manager Placement Engine
Availability zone 1 Availability zone 2 Availability zone 3
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon ECS Constructs
Cluster
• Resource grouping and isolation
• IAM permissions boundary
Service
• Maintains desired # of
running tasks
• Replaces unhealthy tasks
• Elastic Load Balancing
integration
Task
• Running instance of a task
definition
• One or more containers
Task Definition
• Template used by Amazon ECS
to launch tasks
• Parallels to docker run
parameters
• Defines requirements – e.g.
• CPU/Memory
• Container image(s)
• Logging
• AWS Identity and Access
Management (IAM) role
Container 1
Container 1
Container 1
Container 1Container 1
{ ; }
JSON

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Observability
“…a measure of how well internal states of a system can be inferred from
knowledge of its external outputs.” – Wikipedia

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Monitoring
Amazon CloudWatch ECS metrics
Service CPU/memory utilization
Amazon ECS metadata endpoint
Query task metadata and access Docker
stats for running tasks
169.254.170.2/v2/metadata
169.254.170.2/v2/stats
Partners
"read": "2018-11-17T01:39:05.558680889Z",
"preread": "2018-11-17T01:39:04.569462567Z",
"num_procs": 0,
"pids_stats": {},
"network": {},
"memory_stats": {
"stats": {
"cache": 6127616,
"mapped_file": 2117632,
"total_inactive_file": 1626112,
"pgpgout": 3056,
"rss": 3981312,
"total_mapped_file": 2117632,
"pgpgin": 5524,
"pgmajfault": 51,
"total_rss": 3981312,
"hierarchical_memory_limit": 536870912,
"total_pgfault": 5865,
"total_active_file": 4501504,
"active_anon": 3981312,
"total_active_anon": 3981312,
"total_pgpgout": 3056,

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Logging
Amazon CloudWatch Logs
Fargate awslogs log driver
STDOUT and STDERR I/O streams
Stream logs to Amazon Kinesis Data
Streams, Amazon Kinesis Data Firehose,
AWS Lambda
Subscription filters
Container 1
logs logs

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Logging
Sidecar container
Fargate provides 4GB ephemeral storage
Amazon Kinesis agent sidecar pattern (below)
Custom logging solutions
logs

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Distributed Tracing
User
AWS X-Ray
Analyze and debug distributed applications
Useful for microservices environments
X-Ray SDK for app instrumentation
X-Ray daemon runs as sidecar
Service
from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.ext.flask.middleware import XRayMiddleware
app = Flask(__name__)
xray_recorder.configure(service=‘Microservice’)
XRayMiddleware(app, xray_recorder)

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Distributed Tracing

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Alerting
Amazon CloudWatch
Collects metrics, events, logs produced by
services/apps
Triggers actions based on metric thresholds and
event/log patterns
Service Integration
Amazon SNS
User notifications – email, SMS, mobile app
System-to-system messaging – AWS Lambda,
SQS, HTTP/S endpoint
AWS Lambda
Event-driven compute
Rule Alarm
Traditional
server
metrics
events
logs
AWS Partner /
Other Consumer

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hands-on Lab
www.mythicalmysfits.com

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Our mission: Ethical, mythical creature care.
Our priority: Find homes for the abandoned, and often misunderstood, mythical creatures in our
community.
Help us find their forever homes!
Your mission: Modernize and innovate on the Mythical stack.
Lab 1: Monitoring with Amazon CloudWatch metrics and Amazon ECS task metadata endpoint
Lab 2: Log analysis with Amazon CloudWatch Logs and Amazon Elasticsearch Service
Lab 3: Distributed Tracing and Debugging with AWS X-Ray
Lab 4: Operational improvements – Alerting, Auto Scaling, Amazon CloudWatch dashboards
Welcome to Mythical Mysfits

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Mythical Stack
VPC
AWS Cloud
Private subnet Private subnet
users
/ + /fulfill-like
/like
/fulfill-like
/fulfill-like

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Mythical Stack
VPC
AWS Cloud
Private subnet Private subnet
users
/ + /fulfill-like
/like
/fulfill-like
/fulfill-like
???

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lab 1: Monitoring
VPC
AWS Cloud
Private subnet Private subnet
users
/ + /fulfill-like
/like
/fulfill-like
/fulfill-like
ECS Service
metrics
ECS task metadata endpoint
- 169.254.170.2/v2/stats
???
App Metadata proxy
side car

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lab 2: Log Analysis
VPC
AWS Cloud
Private subnet Private subnet
users
/ + /fulfill-like
/like
/fulfill-like
/fulfill-like
???

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lab 3: Distributed Tracing and Debugging
VPC
AWS Cloud
Private subnet Private subnet
users
/ + /fulfill-like
/like
/fulfill-like
???
App
(instrumented
with X-Ray SDK)
X-Ray daemon
side car

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lab 4: Operational Improvements
VPC
AWS Cloud
Private subnet Private subnet
users
/ + /fulfill-like
/like
/fulfill-like
/fulfill-like
???
Auto Scaling
Email
notification
HTTP
notification

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Instructions:
http://www.mythicalmysfits.com/fargate-ops
Raise your hand if you have any questions. Feel free to work together with folks at
your table. Near the end, we’ll hand out AWS credit codes to cover costs for the
workshop.
High-five your neighbors and have fun! Please fill out feedback forms and follow
the clean-up instructions once you are done!
Email us with comments/questions/feedback:
aws-mythical-mysfits@amazon.com
Logistics

22. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.