Continuous Deployment Strategies eBook from Snap CI. Written by Ketan Padegaonkar. Learn techniques and terms from Snowflake servers to Blue/Green Deployments.

1. Continuous
Deployment
Strategies
Ketan
Padegaonkar
TECHNIQUES FOR IMPLEMENTATION
@snap_ci
snap-ci.com

2. snap-ci.com © 2015 2
CONTENTS
Page
Author Bio 3
Chapter 1: Continuous Deployment Strategies 4-5
1.1 Snowflake Servers 6
1.2 Phoenix Servers 7
1.3 How Does this Apply to Continuous Deployment? 8
1.4 Blue/Green Deployments 9
1.5 Canary Releases 10
Chapter 2: Blue / Green Deployments 11-12
2.1 Changing the load balancer/router 13
2.2 Changing public DNS records 14
2.3 Load Balancer + Internal DNS records 15
Chapter 3: Implementation Techniques for Canary Releases 16-17
3.1 Long running migration phase 18

3. snap-ci.com © 2015 3
AUTHOR BIO
Ketan P
Senior Developer
Quintessential shaver of yaks, supporter and developer
for open source, long time ThoughtWorker and builder
of continuous delivery tools @goforcd and @snap_ci.
@ketanpkr
github.com/ketan

4. snap-ci.com © 2015 4
Continuous
Delivery
Strategies

5. snap-ci.com © 2015 5
CHAPTER 1
Not very long ago, during the days of Perl/CGI and PHP (and even today);
deployments involved ftp-ing your files to the webserver directory and running a
bunch of migrations by shelling into the webserver.
Modern webapps and application servers have evolved quite a bit since then, but
many developers continue to use similar strategies to perform deployments. FTP has
been replaced by git pull followed by bundle install or npm install and then restarting
the appserver that you use.
Along with similar deployment strategies, many developers also continue to use the
same servers for months and years on end. But even with the right patches, updates,
etc, after a while it’s easy to end up with a “snowflake server”: a server with a delicate,
unique configuration that does not lend itself to change. A solution to this is the
“phoenix server.” Here’s a little more about both these concepts…

6. snap-ci.com © 2015 6
SNOWFLAKE SERVERS
While this approach may work great for small webapps, it quickly falls apart when
you’re managing anything but a couple of servers.
It is generally difficult to set up a new server if your existing server has problems. If
you’re looking to scale by adding more servers, it’s difficult to keep each server’s
software, configuration and services in sync. You may not be in a position to replicate a
production (or production-like) environment for testing.
Over time, one loses track of the magic configuration files, packages and services that
are installed on the production machines.
1.1

7. snap-ci.com © 2015 7
PHOENIX SERVERS
Some these issues can be addressed by using a bit of configuration management tools
like Chef, Puppet, or Ansible, among a few others.
These tools help avoid configuration drift, the sort of one-off changes that go
unnoticed and undocumented.
An important point to note is that these tools only apply the configuration that they are
asked to. If you applied any additional configuration outside of what these tools are
aware of, or if you forgot to un-apply or remove some configuration, you’d be out of
luck. This is what Ranjib Dey calls the Accumulator Anti-Pattern.
A much better alternative is to tear down servers periodically and apply configuration
changes every once in a while. This helps avoid and catch any configuration drift
outside of configuration management tools. Eventually one can move to deploying new
servers on every configuration change. This strategy is known as an Immutable Server.
1.2

8. snap-ci.com © 2015 8
HOW DOES THIS APPLY TO CD?
Continuous Deployment requires that at a very minimum, you have: * a solid
foundation of tests that gives you confidence in your software * a set of automation
tools and scripts that give you confidence that your deployment will succeed, or
rollback in case of issues
A few of the more popular continuous deployment strategies include blue/green
deployments and canary releases, which I’ve briefly outlined below…
1.3

9. snap-ci.com © 2015 9
BLUE / GREEN DEPLOYMENTS
Blue/green is a technique for deployments where the existing running deployment is
left in place. A new version of the application is installed in parallel with the existing
version. When the new version is ready, cut over to the new version by changing the
load balancer configuration.
This makes rollback really simple and gives time to make sure that the new version
works as expected before putting it live.
1.4
This is one of the simplest CD strategies to implement and get started with.
v1.0
WEB
SERVER
ROUTER
APP
SERVER
DB
SERVER
v1.1
v1.0
v1.1
v1.0
v1.1

10. snap-ci.com © 2015 10
CANARY RELEASES
This is named after the “canary in a coal mine” metaphor. The metaphor originates
from the times when miners used to carry caged canaries into the mines with them: if
there were any dangerous gases (methane or carbon monoxide) in the mine, the
canary would die before the gas levels reached levels hazardous to humans.
Canary releasing is similar to blue/green, although only a small amount of the servers
are upgraded. Then, using a cookie or similar, a fraction of users are directed to the
new version.
1.5
This allows for the load and functionality of the site to be tested with a small group of
users. If the application behaves as expected, migrate more and more servers to the
new version until all the users are on the new version.
This technique can also be used to do some interesting multi-variant testing and
performance testing.
v1.0
v1.1

11. snap-ci.com © 2015 11
Blue/Green
Deployments

12. snap-ci.com © 2015 12
CHAPTER 2
In the previous article on Continuous Deployment Strategies, we explored at a high
level some of the popular CD strategies.
This article describes some of the implementation techniques for performing blue/
green deployments.
To recap – Blue/green is a technique for deployments where the existing running
deployment is left in place. A new version of the application is installed in parallel with
the existing version. When the new version is ready, cut over to the new version by
changing the load balancer configuration.
The basic idea behind this technique is to be able to route users to either the green set
of servers, or the blue set of servers.
This can be achieved in multiple ways:
Changing the load balancer/router
Changing public DNS records
Load Balancer + Internal DNS records

13. snap-ci.com © 2015 13
CHANGING THE LOAD BALANCER / ROUTER2.1
Using this technique you would change the load balancer or request router to point to
either the blue server or the green server.
v1.0
WEB
SERVER
ROUTER
APP
SERVER
DB
SERVER
v1.1
v1.0
v1.1
v1.0
v1.1

14. snap-ci.com © 2015 14
CHANGING PUBLIC DNS RECORDS2.2
Using this technique involves load-balancing at the DNS level by changing either the
CNAME or the A records. When users request a DNS entry for your website, your DNS
server could return the IP address for either the blue server or the green server. This
particular approach would require that the time to live (TTL) is set to a very small value.
This particular approach is not recommended for changing public DNS records because
some DNS servers run some browsers don’t respect the low TTL values on the DNS entry,
they will cache the DNS records for longer than the TTL. This may leave a subset of your
users not being able to access your website.
DNS
SERVER
example.com
1.2.3.4
OR
5.6.7.8
green.example.com => 1.2.3.4
blue.example.com => 5.6.7.8

15. snap-ci.com © 2015 15
LOAD BALANCER + INTERNAL DNS RECORDS2.3
This approach is a combination of the two techniques we’ve seen so far. In this technique,
instead of changing public DNS records, we’ll change internal (private) DNS records. Since
the internal network and the DNS server can be managed and configured to respect TTLs,
this approach works a lot better than changing public DNS records.
DNS
SERVER
internal.example.com
192.168.1.1
OR
192.168.2.1
green.internal.example.com => 192.168.1.1
blue.internal.example.com => 192.168.2.1

16. snap-ci.com © 2015 16
Implementation
Techniques for
Canary Releases

17. snap-ci.com © 2015 17
CHAPTER 3
In the previous article on Continuous Deployment Strategies, we explored at a high
level some of the popular CD strategies.This article describes some of the
implementation techniques for performing Canary Releases.
Just like Blue/Green Deployments, one would start by deploying the application to a
small subset of your servers. Once the subset of servers is deployed to, you may then
route requests for a few users to the new set of servers.
This strategy lets you do a lot of interesting things:
Test the performance of the application.
Perform A/B tests based on demographics and user profiles, for example “users
between the ages of 20 – 25 living in Iowa.”
Load Balancer + Internal DNS records.
As your confidence in the deployment improves, you can deploy your application to
more and more servers, and route more users to the new servers.
v1.0
v1.1

18. snap-ci.com © 2015 18
LONG RUNNING MIGRATION PHASE
Because one or more versions of your application may be running in production for
some period of time, your application and its components (webservices, microservices,
database) needs to be backward-compatible so that it works with at least one or two
previous versions of your application. This Parallel-Change Pattern is a simple and
effective way to implement backward-incompatible changes between application
interfaces.
3.1

19. © 2015
snap-ci.com