2. Agenda
Dealing with changes
Perfect solution or Continuous DB migration
Carbon5 Introduction
Maven-driven mode
Embedded mode
Pros/Cos and best practices
3. Dealing with Changes
Applications evolve. God, bless bugs, refactoring and
business changes.
We love applications and configure CI, VCS in order to
track source code/application version.
But what’s about DB?
4. Dealing with Schema Change
FACT: Some modifications might require changes in DB
schema.
5. Dealing with Schema Change
FACT: Some modifications might require changes in DB
schema.
Changes are simple and risky.
6. Dealing with Schema Change
FACT: Some modifications might require changes in DB
schema.
Changes are simple and risky.
Simple changes:
– add one more table
– add index
7. Dealing with Schema Change
FACT: Some application modifications might require
changes in DB schema.
Changes are simple and risky.
Simple changes:
– add one more table
– add index
Risky changes:
– rename/add column
– change foreign key
– migrate data from one table/column to another
– denormalization on a fly
8. Dealing with Schema Change
What’s about old apps upgrade and multiple schema changes?
App
v1
App
v10
9. The Impact
Risks to lose data
Painful downtime
Risks to break application (see #1 and #2)
Resources, efforts and budget
11. How people solve it
Deny Friday’s deployments
Manual deployment
Hibernate-like schema update + SQLexec
Self-developed tools based on version number approach
12. Perfect solution
Dedicated reusable framework
Every Feature request leads to separated SQL migration script.
Framework tracks which changes were not applied and applies
them.
Configurable time to launch apply procedure
Prevent Double changes
Have Change Log
Simple integration to existent project
15. Why Carbon V
DB-migration framework
Open Source project
Lightweight framework
Simple usage
16. How it works
Dedicated project folder for SQL migration scripts. This folder
should be available when framework takes control.
Each SQL migration script is pure SQL/DDL file with migration
SQL code for specific feature
Each Delta file should have name
YYYYMMDDHHMMSS_<FEATURE>.sql
When framework take control it checks and applies only new
changes.
C5 uses own JDBC connection.
Way to re-usue existent DataSources with connection params
17. No magic, just…
C5 creates own table “schema_version” inside your database
C5 controls this table by itself (no needs to create or update).
sql_file_name date duration
201405170900_user_auth.sql 2014-05-17
13:00:00
8 sec
201404130400_award_4565.sql 2014-04-14
14:00:00
6 sec
18. When to initiate migration procedure?
Maven-driven approach
Embedded mode
21. Configure Maven project
Update POM definition
Configure Explicit DB connection
Can be configured depending on specific maven profiles
(staging, production)
22. Add SQL migration files
Use Project/src/main/db/migrations directory
23. Launch it
performs migration
$ mvn db-migration:migrate
resets migration
$ mvn db-migration:reset
check if DB is up to date
$ mvn db-migration:validate
create new feature
$ mvn db-migration:new -Dname=new_feature
24. Use cases
Manual runs from developer machine
Part of Continuous Deivery process
– Add just one more Maven action inside specific CI configuration before
deployment action
26. Embedded mode
Schema Check is constituent part of application.
SQL changes are built-in into the application
Any checks and possible DB migration is performed every time
when application launches.
Dedicated bean to carry out this responsibility
27. Create application bean
import com.carbonfive.db.migration.DataSourceMigrationManager;
import com.carbonfive.db.migration.DriverManagerMigrationManager;
import com.carbonfive.db.migration.MigrationManager;
import com.carbonfive.db.migration.ResourceMigrationResolver;
public class DBMigratorImpl {
DataSource dataSource;
...
public void init(){
DataSourceMigrationManager migrationManager = new
DataSourceMigrationManager(dataSource);
migrationManager.setMigrationResolver(new
ResourceMigrationResolver("classpath:/db/migrations"));
migrationManager.migrate();
}
}
28. Inject the bean into application context
Seamless integrated with Spring DI
It is possible to initialize ResourceMigrationResolver
DataSourceMigrationManager directly though Spring DI without any line of code
Two places where bean might be integrated inside Spring context
Before Persistent beans
After Persistent beans
29. Embedded vs maven driven
E: Consistent and grace application upgrade
E: No delays between deployment and schema upgrade
M: Simple integration with CI
M: External commands like validate & reset
M: Quick manual migrations (hotfixes for instance)
30. Carbon5 Benefits
Lightweight framework
Brilliant extensibility (thanks to reasonable design)
Support many DBMS in unofficial mode
Simple integration
Real continuous DB migration
33. Flyway framework
Rich support for different build tools
The same integration approach on Carbon V
Pluggable architecture
Java-based migrations (two options: through JDBC Connection &
Spring JDBCTemplate)
Extended service table
34. Best practices
Feature driven development (each SQL feature change in own
file)
Make sql files safe in order to prevent it from double execution
Keep database connection settings in single place (use shared
DataSource)
Do not make structure changes to the DB outside of your
framework
37. NoSQL migration approach
No dedicated explicit migration procedure (solved on business
level in case of needs)
No shutdown or downtime
Rolling update
– Every domain entity has version number
– your v2 application should handle the v1 db format
– Write code to convert entity to v2 on a fly (repository level)
– Write modified entities to v2 on demand
Some DBs like RavenDB are able to perform auto-migration
39. Presentation in a nutshell
Keep in mind this problem
Do not invent wheels, use external frameworks/techniques
And do not be afraid of Friday’s deployment then.