Benoit Hediard
AgoraPulse
Grails & DevOps
CI & CD in the Cloud
RUNNING A LEAN STARTUP
DEVOPS & GRAILS & AWS
Gr8Conf Europe 2014
agorapulse
2
Benoit Hediard
@benorama
LEAN STARTUP
Part 1
3
DEPLOYMENT PIPELINE
WITH GRAILS & AWS
Demo
DEVOPS CULTURE
Part 2
#lean
#custdev
#startup
#agile
#dev...
About Me
4
A Social Media Management
Platform for Brands
Benoit Hédiard
Co-founder & CTO
Grails	
  Plugins	
  Author:	
  A...
Client Framework
Server Framework
Cloud provider
SaaS services
Our leanstack
5
Elastic
Beanstalk
RDS &
DynamoDB
SES & SQS
...
LEAN STARTUP
Part 1
6
What is a Lean Startup?
7
A Startup is a search for a repeatable and scalable business model. 
The Customer Development pr...
Feedback Loop
8
Ideas	
  
BUILD	
  
Code	
  
MEASURE	
  
Data	
  
LEARN	
  
BUILD – MEASURE – LEARN
The fundamental activi...
Paul Graham’s Startup Curve
9
Minimum Viable Product Product Market Fit
TROUGH OF SORROW
J
L
From search mode
Move fast,...
Minimum Viable Product
10
Manual Pipeline
✔ Collaboration

✗ Manual Build
✗ Manual Testing
✗ Manual Delivery
✗ Manual Provisioning
✗ Manual Deployme...
Semi-automated Pipeline
✔ Collaboration
✔ Continuous Build
✔ Continuous Testing
✔ Continuous Delivery
✗ Manual Provisionin...
DEVOPS CULTURE
Part 2
13
What is DevOps?
14
DevOps is a software development method that stresses
communication, collaboration and integration betw...
Pre-2000’s Developer…
15
The Game of Thrones era
Developers
Requirements	
  
Design	
  
Development	
  
Test	
  
Release	
  
Maintenance	
  
Pre-2000’s Waterfall Development
16
Dev	
...
2000’s Developer…
17
The Rocky
Balboa era
OpsDev
2000’s Iterative Agile Development
18
Develop	
  
Test	
  Release	
  
Design	
  
Dev	
  
Ops	
  
Business	
  
Idea
Custome...
2010’s Developer…
19
The X-Men era
Business Dev Ops
One team approach
2010’s Lean & DevOps
20
Develop	
  
Test	
  Release	
  
Design	
  
Dev	
  
Ops	
  
Business	
  
Idea
Customer
Cycle time:
...
Automated Pipeline
Demo – Part 3
21
Code	
   Build	
   Test	
   Release	
   Deploy	
   Monitor	
  
✔ Collaboration
✔ Conti...
Git Branching Model
22
develop beta master (PROD)
Snapshot merge
deploy	
  
Elastic
Beanstalk
BETA
feature
Feature merge
t...
Cultural shift
23
Developers must own application-layer code wherever it lives, while ops
must own the infrastructure wher...
IT Automation + IaaS
Enterprise DevOps
24
3. Deploy your App
1. Describe your “Infrastructure as Code”
2. Build and manage...
Lean DevOps / NoOps
25
PaaS + SaaS
1. Deploy your App and use Managed Services
Cloud SQL
Amazon RDS
Relational DB
Cloud Da...
Automated Pipeline
Demo – Part 4
26
Code	
   Build	
   Test	
   Release	
   Deploy	
   Monitor	
  
✔ Collaboration
✔ Conti...
Demo – Part 5
27
Zero Downtime Deployment (blue/green)
40 minutes ago…
28
MVP running on locale dev machine
(manual build pipeline)
2 Travis files & few Grails Plugins later…
29
“Unlimited” Horizontal Auto-Scaling
(hundred of web servers)
Push-Button Scal...
CONCLUSION
30
Conclusion
31
Such an exciting time to be a developer
AND an entrepreneur today…
Embrace Lean
Embrace DevOps
Become an ent...
LEAN STARTUP
Part 1
32
DEPLOYMENT PIPELINE
WITH GRAILS & AWS
Demo
DEVOPS CULTURE
Part 2
#lean
#custdev
#startup
#agile
#de...
About Me
33
A Social Media Management
Platform for Brands
Benoit Hédiard
Co-founder & CTO
Grails	
  Plugins	
  Author:	
  ...
Client Framework
Server Framework
Cloud provider
SaaS services
Our leanstack
34
Elastic
Beanstalk
RDS &
DynamoDB
SES & SQS...
LEAN STARTUP
Part 1
35
What is a Lean Startup?
36
A Startup is a search for a repeatable and scalable business model. 
The Customer Development p...
Iridium woes (Fail fast)
•  Conceived in 1987 by Motorala
•  Bought a fleet of 15 rockets from Russia,
the U.S. and China
•...
Feedback Loop
38
Ideas	
  
BUILD	
  
Code	
  
MEASURE	
  
Data	
  
LEARN	
  
BUILD – MEASURE – LEARN
The fundamental activ...
Paul Graham’s Startup Curve
39
Minimum Viable Product Product Market Fit
TROUGH OF SORROW
J
L
From search mode
Move fast...
Minimum Viable Product
40
Manual Pipeline
✔ Collaboration

✗ Manual Build
✗ Manual Testing
✗ Manual Delivery
✗ Manual Provisioning
✗ Manual Deployme...
Semi-automated Pipeline
✔ Collaboration
✔ Continuous Build
✔ Continuous Testing
✔ Continuous Delivery
✗ Manual Provisionin...
DEVOPS CULTURE
Part 2
43
What is DevOps?
44
DevOps is a software development method that stresses
communication, collaboration and integration betw...
Pre-2000’s Developer…
45
The Game of Thrones era
Developers
Requirements	
  
Design	
  
Development	
  
Test	
  
Release	
  
Maintenance	
  
Pre-2000’s Waterfall Development
46
Dev	
...
2000’s Developer…
47
The Rocky
Balboa era
OpsDev
2000’s Iterative Agile Development
48
Develop	
  
Test	
  Release	
  
Design	
  
Dev	
  
Ops	
  
Business	
  
Idea
Custome...
2010’s Developer…
49
The X-Men era
Business Dev Ops
One team approach
2010’s Lean & DevOps
50
Develop	
  
Test	
  Release	
  
Design	
  
Dev	
  
Ops	
  
Business	
  
Idea
Customer
Cycle time:
...
Automated Pipeline
Demo – Part 3
51
Code	
   Build	
   Test	
   Release	
   Deploy	
   Monitor	
  
✔ Collaboration
✔ Conti...
Git Branching Model
52
develop beta master (PROD)
Snapshot merge
deploy	
  
Elastic
Beanstalk
BETA
feature
Feature merge
t...
Cultural shift
53
Developers must own application-layer code wherever it lives, while ops
must own the infrastructure wher...
IT Automation + IaaS
Enterprise DevOps
54
3. Deploy your App
1. Describe your “Infrastructure as Code”
2. Build and manage...
Lean DevOps / NoOps
55
PaaS + SaaS
1. Deploy your App and use Managed Services
Cloud SQL
Amazon RDS
Relational DB
Cloud Da...
Automated Pipeline
Demo – Part 4
56
Code	
   Build	
   Test	
   Release	
   Deploy	
   Monitor	
  
✔ Collaboration
✔ Conti...
Demo – Part 5
57
Zero Downtime Deployment (blue/green)
40 minutes ago…
58
MVP running on locale dev machine
(manual build pipeline)
2 Travis files & few Grails Plugins later…
59
“Unlimited” Horizontal Auto-Scaling
(hundred of web servers)
Push-Button Scal...
CONCLUSION
60
Conclusion
61
Such an exciting time to be a developer
AND an entrepreneur today…
Embrace Lean
Embrace DevOps
Become an ent...
Reading recommendations
62
Upcoming SlideShare
Loading in …5
×

Grails & DevOps: continuous integration and delivery in the cloud

2,351 views

Published on

Nowadays, companies require very short release cycles, especially in lean startup environments.

But to release often:

deployments should be routine, not terrifying.
configuration should require a few clicks, not a thousand-line shell script.
problems should be easy to spot, not buried in a log file.
You are a developer that need to release every week or every day with a single git commit and zero-downtime? Easily spot release performance or bugs issues? If required, roll back to previous version in few seconds and one click? And you don't want to manage any dedicated repository, monitoring, build, staging, production servers? So this talk is for you!

We will explore Lean startup and DevOps concepts and share our experience on how to create a simple and fully automated build pipeline for Grails apps with a live demo, based on SaaS/cloud services: GitHub, Travis CI, NewRelic, AWS (ElasticBeanstalk, CloudFront), etc.

Published in: Technology, Business

Grails & DevOps: continuous integration and delivery in the cloud

  1. 1. Benoit Hediard AgoraPulse Grails & DevOps CI & CD in the Cloud
  2. 2. RUNNING A LEAN STARTUP DEVOPS & GRAILS & AWS Gr8Conf Europe 2014 agorapulse 2 Benoit Hediard @benorama
  3. 3. LEAN STARTUP Part 1 3 DEPLOYMENT PIPELINE WITH GRAILS & AWS Demo DEVOPS CULTURE Part 2 #lean #custdev #startup #agile #devops #groovylang #grailsfw #aws BUZZWORD BINGO
  4. 4. About Me 4 A Social Media Management Platform for Brands Benoit Hédiard Co-founder & CTO Grails  Plugins  Author:  AWS  SDK,  Facebook  SDK,  CDN  Asset  Pipeline,  Coveralls,   Raven,  SegmentIO…   Passionate about •  Lean/DevOps •  Cloud/AWS •  Java/Grails •  UX/UI •  Facebook/Twitter Platforms @benorama
  5. 5. Client Framework Server Framework Cloud provider SaaS services Our leanstack 5 Elastic Beanstalk RDS & DynamoDB SES & SQS S3 & CloudFront Route 53 Check http://leanstack.io !
  6. 6. LEAN STARTUP Part 1 6
  7. 7. What is a Lean Startup? 7 A Startup is a search for a repeatable and scalable business model.  The Customer Development process is the way startups quickly iterate and test each element of their business model. Agile Development is the way startups quickly iterate their product as they learn. A Lean Startup is Eric Ries’s description of the intersection of Customer Development, Agile Development and if available, open platforms and open source. – Steve Blank, author of The Startup Owner's Handbook
  8. 8. Feedback Loop 8 Ideas   BUILD   Code   MEASURE   Data   LEARN   BUILD – MEASURE – LEARN The fundamental activity of a startup is to turn ideas into products, measure how customers respond, and then learn whether to pivot or persevere. All successful startup processes should be geared to accelerate that feedback loop. – Eric Ries, author of The Lean Startup
  9. 9. Paul Graham’s Startup Curve 9 Minimum Viable Product Product Market Fit TROUGH OF SORROW J L From search mode Move fast, break things To execution mode Grow fast, scale things ScaleInitial enthusiasm Reality sets in Experimenting & pivoting Hockey stick Starts working
  10. 10. Minimum Viable Product 10
  11. 11. Manual Pipeline ✔ Collaboration ✗ Manual Build ✗ Manual Testing ✗ Manual Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring Demo – Part 1 11 Code   Build   Test   Release   Deploy   Monitor   Code repo Team communication Agile Project Management EC2 Hosting Idea Customer
  12. 12. Semi-automated Pipeline ✔ Collaboration ✔ Continuous Build ✔ Continuous Testing ✔ Continuous Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring Demo – Part 2 12 Code   Build   Test   Release   Deploy   Monitor   Continuous integration and delivery Idea Customer Test coverage history & stats Coveralls Plugin
  13. 13. DEVOPS CULTURE Part 2 13
  14. 14. What is DevOps? 14 DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) operations professionals. ✔ Collaboration ✔ Automation & Infrastructure as Code ✔ Continuous Integration / Testing ✔ Continuous Delivery / Deployment ✔ Continuous Monitoring GOALS: reduce friction and increase velocity
  15. 15. Pre-2000’s Developer… 15 The Game of Thrones era Developers
  16. 16. Requirements   Design   Development   Test   Release   Maintenance   Pre-2000’s Waterfall Development 16 Dev   Test  /  QA   Ops   Business   Idea Customer Silo Silo Silo Friction Friction Friction Cycle time: Months or Years
  17. 17. 2000’s Developer… 17 The Rocky Balboa era OpsDev
  18. 18. 2000’s Iterative Agile Development 18 Develop   Test  Release   Design   Dev   Ops   Business   Idea Customer Cycle time: Weeks or Months Stability Change
  19. 19. 2010’s Developer… 19 The X-Men era Business Dev Ops One team approach
  20. 20. 2010’s Lean & DevOps 20 Develop   Test  Release   Design   Dev   Ops   Business   Idea Customer Cycle time: Hours or Days DevOps
  21. 21. Automated Pipeline Demo – Part 3 21 Code   Build   Test   Release   Deploy   Monitor   ✔ Collaboration ✔ Continuous Build ✔ Continuous Testing ✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✗ Manual Monitoring Elastic Beanstalk S3 & CloudFront RDS PaaS CDN DB Managed Services Idea Customer Elastic Beanstalk Plugin CDN Asset Pipeline Plugin
  22. 22. Git Branching Model 22 develop beta master (PROD) Snapshot merge deploy   Elastic Beanstalk BETA feature Feature merge test   test   test   Snapshot merge Release merge Version upgrade deploy   Elastic Beanstalk PROD test   test   test   test   test   test   test  
  23. 23. Cultural shift 23 Developers must own application-layer code wherever it lives, while ops must own the infrastructure wherever that is. – Donnie Berkholz, analyst at RedMonk Dev/Test Production Dev   Ops   ApplicaMons   ApplicaMons   Infrastructure   Infrastructure   Dev/Test Production   Dev   ApplicaMons   ApplicaMons   Ops   Infrastructure   Infrastructure   Responsability Responsability Confuse of Dev or Ops? Simple rule: if you are praise for Web site success, you are Dev; if you are blame when Web site down, you are Ops – DevOps Borat Cultural shift
  24. 24. IT Automation + IaaS Enterprise DevOps 24 3. Deploy your App 1. Describe your “Infrastructure as Code” 2. Build and manage your “Automated Infrastructure” PROs Portability Control Flexibility CONs Complexity Maintenance Amazon EC2 Google Compute Engine IaaS (Infrastructure as a Service) Configuration Management Tools Portable Containers
  25. 25. Lean DevOps / NoOps 25 PaaS + SaaS 1. Deploy your App and use Managed Services Cloud SQL Amazon RDS Relational DB Cloud Datastore DynamoDB NoSQL Cloud Storage Amazon S3 Storage + Cloud EndPoints + Elasticache, AppStream, SWF, SQS, SES, FPS, Elastic Transcoder… Amazon RedShift Big Data BigQuery Google App Engine PaaS (Platform as a Service) Cloud DNS Elastic Beanstalk PaaS (Platform as a Service) Amazon Route 53 PROs Focus on building applications instead of managing Infrastructure
  26. 26. Automated Pipeline Demo – Part 4 26 Code   Build   Test   Release   Deploy   Monitor   ✔ Collaboration ✔ Continuous Build ✔ Continuous Testing ✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✔ Continuous Monitoring Application and server monitoring Business monitoring Exception monitoring Idea Customer Sentry Plugin NewRelic PluginSegmentIO Plugin
  27. 27. Demo – Part 5 27 Zero Downtime Deployment (blue/green)
  28. 28. 40 minutes ago… 28 MVP running on locale dev machine (manual build pipeline)
  29. 29. 2 Travis files & few Grails Plugins later… 29 “Unlimited” Horizontal Auto-Scaling (hundred of web servers) Push-Button Scalable Database with Multi-AZ (up to 32vCPU and 244Go of RAM) 100% Resilient (no single point of failure) Deployment in < 10 minutes (deployment pipeline) 100% Cloud-based (fully managed/no SSH)
  30. 30. CONCLUSION 30
  31. 31. Conclusion 31 Such an exciting time to be a developer AND an entrepreneur today… Embrace Lean Embrace DevOps Become an entrepreneur! #lean #custdev #agile #devops #groovylang #grailsfw #aws #PaaS #SaaS
  32. 32. LEAN STARTUP Part 1 32 DEPLOYMENT PIPELINE WITH GRAILS & AWS Demo DEVOPS CULTURE Part 2 #lean #custdev #startup #agile #devops #groovylang #grailsfw #aws BUZZWORD BINGO
  33. 33. About Me 33 A Social Media Management Platform for Brands Benoit Hédiard Co-founder & CTO Grails  Plugins  Author:  AWS  SDK,  Facebook  SDK,  CDN  Asset  Pipeline,  Coveralls,   Sentry,  SegmentIO…   Passionate about •  Lean/DevOps •  Cloud/AWS •  Java/Grails •  UX/UI •  Facebook/Twitter Platforms @benorama
  34. 34. Client Framework Server Framework Cloud provider SaaS services Our leanstack 34 Elastic Beanstalk RDS & DynamoDB SES & SQS S3 & CloudFront Route 53 Check out http://leanstack.io !
  35. 35. LEAN STARTUP Part 1 35
  36. 36. What is a Lean Startup? 36 A Startup is a search for a repeatable and scalable business model.  The Customer Development process is the way startups quickly iterate and test each element of their business model. Agile Development is the way startups quickly iterate their product as they learn. A Lean Startup is Eric Ries’s description of the intersection of Customer Development, Agile Development and if available, open platforms and open source. – Steve Blank, author of The Startup Owner's Manual
  37. 37. Iridium woes (Fail fast) •  Conceived in 1987 by Motorala •  Bought a fleet of 15 rockets from Russia, the U.S. and China •  Built and launch 72 satellites •  Nine months after the first call was made in 1998, Iridium was in Chapter 11 bankruptcy. 37 5.2 billion dollar business plan mistake No Business Plan Survives First Contact With A Customer – Steve Blank, author of The Startup Owner's Handbook
  38. 38. Feedback Loop 38 Ideas   BUILD   Code   MEASURE   Data   LEARN   BUILD – MEASURE – LEARN The fundamental activity of a startup is to turn ideas into products, measure how customers respond, and then learn whether to pivot or persevere. All successful startup processes should be geared to accelerate that feedback loop. – Eric Ries, author of The Lean Startup
  39. 39. Paul Graham’s Startup Curve 39 Minimum Viable Product Product Market Fit TROUGH OF SORROW J L From search mode Move fast, break things To execution mode Grow fast, scale things ScaleInitial enthusiasm Reality sets in Experimenting & pivoting Hockey stick Starts working
  40. 40. Minimum Viable Product 40
  41. 41. Manual Pipeline ✔ Collaboration ✗ Manual Build ✗ Manual Testing ✗ Manual Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring Demo – Part 1 41 Code   Build   Test   Release   Deploy   Monitor   Code repo Team communication Agile Project Management EC2 Hosting Idea Customer
  42. 42. Semi-automated Pipeline ✔ Collaboration ✔ Continuous Build ✔ Continuous Testing ✔ Continuous Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring Demo – Part 2 42 Code   Build   Test   Release   Deploy   Monitor   Continuous integration and delivery Idea Customer Test coverage history & stats Coveralls Plugin
  43. 43. DEVOPS CULTURE Part 2 43
  44. 44. What is DevOps? 44 DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) operations professionals. ✔ Collaboration ✔ Automation & Infrastructure as Code ✔ Continuous Integration / Testing ✔ Continuous Delivery / Deployment ✔ Continuous Monitoring GOALS: reduce friction and increase velocity
  45. 45. Pre-2000’s Developer… 45 The Game of Thrones era Developers
  46. 46. Requirements   Design   Development   Test   Release   Maintenance   Pre-2000’s Waterfall Development 46 Dev   Test  /  QA   Ops   Business   Idea Customer Silo Silo Silo Friction Friction Friction Cycle time: Months or Years
  47. 47. 2000’s Developer… 47 The Rocky Balboa era OpsDev
  48. 48. 2000’s Iterative Agile Development 48 Develop   Test  Release   Design   Dev   Ops   Business   Idea Customer Cycle time: Weeks or Months Stability Change Agile
  49. 49. 2010’s Developer… 49 The X-Men era Business Dev Ops One team approach
  50. 50. 2010’s Lean & DevOps 50 Develop   Test  Release   Design   Dev   Ops   Business   Idea Customer Cycle time: Hours or Days DevOps Lean
  51. 51. Automated Pipeline Demo – Part 3 51 Code   Build   Test   Release   Deploy   Monitor   ✔ Collaboration ✔ Continuous Build ✔ Continuous Testing ✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✗ Manual Monitoring Elastic Beanstalk S3 & CloudFront RDS PaaS CDN DB Managed Services Idea Customer Elastic Beanstalk Plugin CDN Asset Pipeline Plugin
  52. 52. Git Branching Model 52 develop beta master (PROD) Snapshot merge deploy   Elastic Beanstalk BETA feature Feature merge test   test   test   Snapshot merge Release merge Version upgrade deploy   Elastic Beanstalk PROD test   test   test   test   test   test   test  
  53. 53. Cultural shift 53 Developers must own application-layer code wherever it lives, while ops must own the infrastructure wherever that is. – Donnie Berkholz, analyst at RedMonk Dev/Test Production Dev   Ops   ApplicaMons   ApplicaMons   Infrastructure   Infrastructure   Dev/Test Production   Dev   ApplicaMons   ApplicaMons   Ops   Infrastructure   Infrastructure   Responsability Responsability Confuse of Dev or Ops? Simple rule: if you are praise for Web site success, you are Dev; if you are blame when Web site down, you are Ops – DevOps Borat Cultural shift
  54. 54. IT Automation + IaaS Enterprise DevOps 54 3. Deploy your App 1. Describe your “Infrastructure as Code” 2. Build and manage your “Automated Infrastructure” PROs Portability Control Flexibility CONs Complexity Maintenance Amazon EC2 Google Compute Engine IaaS (Infrastructure as a Service) Configuration Management Tools Portable Containers
  55. 55. Lean DevOps / NoOps 55 PaaS + SaaS 1. Deploy your App and use Managed Services Cloud SQL Amazon RDS Relational DB Cloud Datastore DynamoDB NoSQL Cloud Storage Amazon S3 Storage + Cloud EndPoints + Elasticache, AppStream, SWF, SQS, SES, FPS, Elastic Transcoder… Amazon RedShift Big Data BigQuery Google App Engine PaaS (Platform as a Service) Cloud DNS Elastic Beanstalk PaaS (Platform as a Service) Amazon Route 53 PROs Focus on building applications instead of managing Infrastructure
  56. 56. Automated Pipeline Demo – Part 4 56 Code   Build   Test   Release   Deploy   Monitor   ✔ Collaboration ✔ Continuous Build ✔ Continuous Testing ✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✔ Continuous Monitoring Application and server monitoring Business monitoring Exception monitoring Idea Customer Sentry Plugin NewRelic PluginSegmentIO Plugin
  57. 57. Demo – Part 5 57 Zero Downtime Deployment (blue/green)
  58. 58. 40 minutes ago… 58 MVP running on locale dev machine (manual build pipeline)
  59. 59. 2 Travis files & few Grails Plugins later… 59 “Unlimited” Horizontal Auto-Scaling (hundred of web servers) Push-Button Scalable Database with Multi-AZ (up to 32vCPU and 244Go of RAM) 100% Resilient (no single point of failure) Deployment in < 10 minutes (deployment pipeline) 100% Cloud-based (fully managed/no SSH)
  60. 60. CONCLUSION 60
  61. 61. Conclusion 61 Such an exciting time to be a developer AND an entrepreneur today… Embrace Lean Embrace DevOps Become an entrepreneur! #lean #custdev #agile #devops #groovylang #grailsfw #aws #PaaS #SaaS
  62. 62. Reading recommendations 62

×