Snap CI now enables public GitHub repositories to do Continuous Integration(CI) and Continuous Delivery(CD) for free. When doing CD, the goal is to automate the process of deployment and build software in a way that can be deployed to production anytime. To achieve this, we need to figure out a way to deploy the application that doesn't interrupt the users using the system. Learning about this process will allow you to reduce the frictions and delays for deploying new features and bug fixes to production. Your application needs to change in a way that understands this process. You will learn about application and deployment pipelines and the different ways to deploy. In this talk we will touch upon some common deployment pipelines. We will cover in detail about how we changed Snap to do deployments that didn't affect our customers, the challenges we faced and some patterns that helped us overcome them. What made this challenging was the data migrations and long running builds of our customers that made it difficult to figure out when to cut over.

1. Continuously serving the OSS
community with
Continuous Integration and Delivery
Akshay Karle
!
Fernando Júnior

2. “How long would it take your
organization to deploy a change that
involves just one single line of code?”
Mary and Tom Poppendieck

3. “(…) build software in such a way that
the software can be released to
production at any time.”
Martin Fowler

4. CONTINUOUS INTEGRATION

5. CONTINUOUS INTEGRATION

6. CONTINUOUS INTEGRATION

7. CONTINUOUS DELIVERY

8. CONTINUOUS DELIVERY
!
!
▫︎ Continuous Integration
!
▫︎ Automated tasks
!
▫︎ Repeatable/reliable process

9. THERE’S MORE…

10. CONTINUOUS DELIVERY

11. CONTINUOUS DELIVERY

12. DEPLOYING YOUR APPLICATION
!
!
▫︎ Database migrations
!
▫︎ Infrastructure update
!
▫︎ Restarting services

13. ZERO DOWNTIME DEPLOYMENTS
Deployment process of your application
has got to be transparent for end users

14. WHO WE ARE?
Developer,
Snap CI
/nandopaf
/fernando-alves
/nand0paf
Developer,
Snap CI
/akshay_karle
/akshaykarle
/akshay_ka

15. WHAT WE DO?
!
!
!
!
▫︎Continuous Integration for repositories on GitHub
▫︎SaaS
▫︎Helps you do Continuous Delivery
▫︎Free for open source projects

16. SNAP INSIDE-OUT

17. LIFECYCLE OF A BUILD

18. LIFECYCLE OF A BUILD

19. LIFECYCLE OF A BUILD

20. LIFECYCLE OF A BUILD
head
repository
commit

21. LIFECYCLE OF A BUILD
head
repository
commit

22. LIFECYCLE OF A BUILD
head
repository
commit
DATABASE

23. LIFECYCLE OF A BUILD
head
repository
commit
DATABASE

24. LIFECYCLE OF A BUILD
head
repository
commit
DATABASE

25. Babysitters
LIFECYCLE OF A BUILD
head
repository
commit
DATABASE

26. Babysitters
LIFECYCLE OF A BUILD
head
repository
commit
DATABASE

27. Babysitters
LIFECYCLE OF A BUILD
head
repository
commit
DATABASE

28. LIFE CYCLE
!
▫︎ Prepare the container
▫︎ Starts database
▫︎ Sets PATH
▫︎ Download artifacts
▫︎Git clone
▫︎Runs each pipeline stage
▫︎Upload artifacts
DATABASE

29. Babysitters
LIFECYCLE OF A BUILD
head
repository
commit
DATABASE

30. Babysitters
LIFECYCLE OF A BUILD
head
repository
commit
DATABASE

31. ARCHITECTURE OVERVIEW
L
B

32. ARCHITECTURE OVERVIEW
L
B
Database
web server

33. ARCHITECTURE OVERVIEW
!
!
▫︎ Rails app fronted by
apache
!
▫︎ Receives the hooks from
GitHub
Database
web server

34. ARCHITECTURE OVERVIEW
L
B
web server
Database

35. ARCHITECTURE OVERVIEW
L
B
Build Server
web server
Database

36. ARCHITECTURE OVERVIEW
!
▫︎ Background jobs
▫︎ Babysitters
▫︎ Build Queue
▫︎ Artifacts
!
▫︎ OpenVZ Containers
▫︎ Virtual machines where the
pipeline runs
Build Server

37. ARCHITECTURE OVERVIEW
L
B
Build Server
web server
Database

38. OUR DEPLOYMENT
!
▫︎ Do have automated scripts
!
▫︎ Deployment pipeline
!
▫︎ 1-click deploy
▫︎ Sort of…
Babysitters
DATABASE
L
B
Build Server
Database
web server

39. OUR DEPLOYMENT
!
▫︎ Wait for all builds to finish
!
▫︎ Put app on maintenance
mode
▫︎ No new requests picked up
!
▫︎ Deploy and wait for
migrations
Babysitters
DATABASE
L
B
Database
VZHOSTBuild Server
web server

40. ZERO DOWNTIME
DEPLOYMENTS

41. BLUE-GREEN DEPLOYMENT
Web Server App Server Database

42. BLUE-GREEN DEPLOYMENT
Web Server App Server Database
App ServerApp ServerWeb Server Database

43. BLUE-GREEN DEPLOYMENT
Web Server App Server Database
App ServerApp ServerWeb Server Database

44. BLUE-GREEN DEPLOYMENT
App ServerApp ServerWeb Server Database

45. BLUE-GREEN DEPLOYMENTS FOR SNAP
VZHOSTBuild Server
web server
VZHOSTBuild Server
web server
L
B
DATABASE

46. BLUE-GREEN DEPLOYMENTS FOR SNAP
VZHOSTBuild Server
web server
VZHOSTBuild Server
web server
L
B
DATABASE

47. CHALLENGES IN SNAP
Long running builds
Database migrations

48. CHALLENGES IN SNAP
Long running builds

49. LONG RUNNING BUILDS
VZHOSTBuild Server
web server
VZHOSTBuild Server
web server
L
B
DATABASE

50. LONG RUNNING BUILDS
VZHOSTBuild Server
web server
VZHOSTBuild Server
web server
L
B
DATABASE

51. LONG RUNNING BUILDS
!
▫︎ Builds should continue
to run
!
▫︎ Artifacts should continue
to be served

52. COLOR AWARE BABYSITTERS
Build Server
Green Stack
Build Server
Blue Stack

53. COLOR AWARE BABYSITTERS
Build Server
Green Stack
Build Server
Blue Stack
web server

54. BUILD LIFECYCLE
head
repository
commit
DATABASEDATABASE
L
B

55. BUILD LIFECYCLE
head
repository
commit
DATABASEDATABASE
L
B

56. BUILD LIFECYCLE
head
repository
commit
DATABASEDATABASE
L
B

57. THE SWITCH
L
B

58. THE SWITCH
L
B

59. THE SWITCH
L
B

60. THE SWITCH
L
B

61. THE SWITCH
L
B

62. THE SWITCH
L
B

63. THE SWITCH
L
B

64. CHALLENGES IN SNAP
Long running builds
Database migrations

65. CHALLENGES IN SNAP
Database migrations

66. DATABASE MIGRATIONS
Pipelines
Counter
Duration
App v1.0
App v1.0
L
B

67. DATABASE MIGRATIONS
Pipelines
Counter
Duration
App v1.0
App v1.0
L
B
Upgrading to v2.0,
migrating

68. DATABASE MIGRATIONS
Pipelines
Counter
App v2.0
App v1.0
L
B

69. Cannot have destructive migrations

70. MIGRATIONS IN PHASES
Pipelines
Counter
Duration
App v1.0
App v1.0
L
B

71. MIGRATIONS IN PHASES
Pipelines
Counter
Duration
App v1.0
App v1.0
Upgrading to v2.0
L
B

72. MIGRATIONS IN PHASES
Pipelines
Counter
Duration
App v2.0
App v1.0
L
B

73. MIGRATIONS IN PHASES
Pipelines
Counter
Duration
App v2.0
App v1.0
L
B

74. MIGRATIONS IN PHASES
Pipelines
Counter
Duration
App v2.0
App v1.0
L
B

75. MIGRATIONS IN PHASES
Pipelines
Counter
Duration
App v2.0
App v1.0
Upgrading to v3.0
L
B

76. MIGRATIONS IN PHASES
App v2.0
App v3.0
Pipelines
Counter
L
B

77. MIGRATIONS IN PHASES
App v2.0
App v3.0
Pipelines
Counter
L
B

78. LESSONS LEARNT

79. Automation is beauty

80. Watch out your data

81. Things will go wrong

82. THANK YOU
Questions?
https://snap-ci.com