Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Continuous Integration
at CartoDB
JUAN IGNACIO SÁNCHEZ
CAS 2015 Basic level talk
We’re hiring!
www.cartodb.com/jobs
First things first…
♡ YOU
LET’S GET TO KNOW
EACH OTHER!
Who has done
a software
deployment,
or release,
this year?
Who has done
a software
deployment,
or release,
during the last
6 months?
Who has done
a software
deployment,
or release,
during the last
month?
Who has done
a software
deployment,
or release,
during the last
week?
Who has done
a software
deployment,
or release,
during the last
day?
15 / week
peak: 7
Tools?
Team!
Team!
Process!
(just a little)
Continuous
deployment is
more of a deal
with the devil than
magic ;-)
Continuous deployment is
more of a deal with the devil
than magic ;-)
15 / week
peak: 7
Breaking CartoDB up
Components
1. Editor
2. Platform
1. SQL API
2. Maps API
3. CartoDB.js
Products
1. SaaS
2. On-premise
3. Open Source
Product Team
1. Editor: 8
2. Data services: 5
3. Web: 6
4. APIs: 2
5. Systems: 5
The GOALS in
the deal
Reduce risk
SLA-owned
Reduce risk
impact x chance
Add value sooner
Roadmap
7 big* features /
milestone (2 weeks)
Deliver
features
22 PRs
each
week
15 / week
peak: 7
users
♡
support
♡
users
Summary,
agile basics
(quick) feedback
(less) cost
HOW
Integration Delivery Deployment
Continuous…
< <
Continuous
Integration
[CI.2] “Continuous Integration is a software
development practice where members of a
team integrate...
Continuous
Delivery
[CD.1] “software engineering approach in
which teams keep producing valuable
software in short cycles ...
Continuous
Deployment
[CD.1*] “software engineering approach in
which teams keep producing valuable
software in short cycl...
Continuous Deployment
at CartoDB:
a deal with the devil?
JUAN IGNACIO SÁNCHEZ
CAS 2015 Basic level talk
[CI.1]
1. Maintain a code repository
2. Automate the build
3. Make the build self-testing
4. Everyone commits to the
basel...
The CartoDB Way™
[CI.1]
1. Maintain a code repository
2. Automate the build
3. Make the build self-testing
4. Everyone commits to the basel...
Step 0:
hiring
1. Initiative
2. Courage
3. Teamwork
4. Problem solving
Step 1:
onboarding
1. Big picture
2. Empowerment:
deploy from the very
beginning, senior
or junior
Step 2:
plan
1. Incremental
2. Iterative
3. Accept tradeoffs
Step 3:
code
1. Take issue
2. Branch master
3. Code + tests
4. Staging testing
Step 4:
pull request
1. Integration tests
2. Code style
3. Code review
Step 5:
merge
&
deploy
Step 6:
evil devil
• #devops
• Rollbar
• Kibana
The devil is in the
detail
Feature flags
[FF.1] “The basic idea is to have a
configuration file that defines a bunch of
toggles for various features you ...
Feature flags
Feature flags
Canary
Releases
[CR.1] “Canary release is a technique to
reduce the risk of introducing a new
software version in producti...
Quick rollback
Integration issues
Scaling issues
Corner cases
1
2
3
Quick rollback Drawback
• Migrations in its own
PR
• Previous PR for
“preparing” ORM
(ActiveRecord)
Post-mortem
• Affected components
• Timing
• What happened
• 5W
• Actions
Smoke tests
• Manual testing
• On-premise releases
• Tag
Wrapping it Up
Costs
• Riskier
• Have plan B
• backups
• rollbacks
• soft deletions
• Heavy
instrumentalized
• servers
• fast tests
• rea...
Costs
• Unexpected DB locks
• Integration issues
Benefits
• Cheaper
• Early detection &
diagnosis
• No “release day”
panic
• Small and easy
reverts.
Benefits
• Accelerated TTM
• Building the Right
Product
• Test in actual
environment
• Visible progress
Is Continuous Delivery
suitable for you?
Is Continuous Delivery
suitable for you?
¯_(ツ)_/¯
Questions?
Thank you!
Juan Ignacio Sanchez
Backend Engineer at CartoDB
juanignaciosl@cartodb.com
@juanignaciosl
References
Continuous Integration
[CI.1] Wikipedia: https://en.wikipedia.org/wiki/Continuous_integration
[CI.2] Fowler: ht...
Continuous integration at CartoDB
Continuous integration at CartoDB
Upcoming SlideShare
Loading in …5
×

Continuous integration at CartoDB

1,331 views

Published on

Te dicen "Move fast and break things" pero realmente quieren decir "Hay paladas de trabajo por hacer, hagámoslas rompiendo lo menos posible". En esta charla básica haremos una introducción a integración continua y hablaremos un poco sobre testing, Git, feature toggles, cultura de empresa, versionado (o no) y otros mecanismos que usamos en CartoDB para avanzar lo más deprisa posible.

Published in: Software
  • Be the first to comment

Continuous integration at CartoDB

  1. 1. Continuous Integration at CartoDB JUAN IGNACIO SÁNCHEZ CAS 2015 Basic level talk
  2. 2. We’re hiring! www.cartodb.com/jobs First things first… ♡ YOU
  3. 3. LET’S GET TO KNOW EACH OTHER!
  4. 4. Who has done a software deployment, or release, this year?
  5. 5. Who has done a software deployment, or release, during the last 6 months?
  6. 6. Who has done a software deployment, or release, during the last month?
  7. 7. Who has done a software deployment, or release, during the last week?
  8. 8. Who has done a software deployment, or release, during the last day?
  9. 9. 15 / week peak: 7
  10. 10. Tools?
  11. 11. Team!
  12. 12. Team!
  13. 13. Process! (just a little)
  14. 14. Continuous deployment is more of a deal with the devil than magic ;-)
  15. 15. Continuous deployment is more of a deal with the devil than magic ;-)
  16. 16. 15 / week peak: 7
  17. 17. Breaking CartoDB up
  18. 18. Components 1. Editor 2. Platform 1. SQL API 2. Maps API 3. CartoDB.js
  19. 19. Products 1. SaaS 2. On-premise 3. Open Source
  20. 20. Product Team 1. Editor: 8 2. Data services: 5 3. Web: 6 4. APIs: 2 5. Systems: 5
  21. 21. The GOALS in the deal
  22. 22. Reduce risk
  23. 23. SLA-owned
  24. 24. Reduce risk impact x chance
  25. 25. Add value sooner
  26. 26. Roadmap 7 big* features / milestone (2 weeks) Deliver features
  27. 27. 22 PRs each week
  28. 28. 15 / week peak: 7
  29. 29. users ♡ support ♡ users
  30. 30. Summary, agile basics
  31. 31. (quick) feedback
  32. 32. (less) cost
  33. 33. HOW
  34. 34. Integration Delivery Deployment Continuous… < <
  35. 35. Continuous Integration [CI.2] “Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.” XP (1999), Booch method (1991)
  36. 36. Continuous Delivery [CD.1] “software engineering approach in which teams keep producing valuable software in short cycles and ensure that the software can be reliably released at any time.” [CD.2] “You build software in such a way that the software can be released to production at any time”
  37. 37. Continuous Deployment [CD.1*] “software engineering approach in which teams keep producing valuable software in short cycles and ensure that the software can be is reliably released at any time.” [CD.2*] “You build software in such a way that the software can be is released to production at any time”
  38. 38. Continuous Deployment at CartoDB: a deal with the devil? JUAN IGNACIO SÁNCHEZ CAS 2015 Basic level talk
  39. 39. [CI.1] 1. Maintain a code repository 2. Automate the build 3. Make the build self-testing 4. Everyone commits to the baseline every day 5. Every commit (to baseline) should be built 6. Keep the build fast 7. Test in a clone of the production environment 8. Make it easy to get the latest deliverables 9. Everyone can see the results of the latest build 10. Automate deployment Best practices
  40. 40. The CartoDB Way™
  41. 41. [CI.1] 1. Maintain a code repository 2. Automate the build 3. Make the build self-testing 4. Everyone commits to the baseline every day 5. Every commit (to baseline) should be built 6. Keep the build fast 7. Test in a clone of the production environment 8. Make it easy to get the latest deliverables 9. Everyone can see the results of the latest build 10. Automate deployment Best practices
  42. 42. Step 0: hiring 1. Initiative 2. Courage 3. Teamwork 4. Problem solving
  43. 43. Step 1: onboarding 1. Big picture 2. Empowerment: deploy from the very beginning, senior or junior
  44. 44. Step 2: plan 1. Incremental 2. Iterative 3. Accept tradeoffs
  45. 45. Step 3: code 1. Take issue 2. Branch master 3. Code + tests 4. Staging testing
  46. 46. Step 4: pull request 1. Integration tests 2. Code style 3. Code review
  47. 47. Step 5: merge & deploy
  48. 48. Step 6: evil devil • #devops • Rollbar • Kibana
  49. 49. The devil is in the detail
  50. 50. Feature flags [FF.1] “The basic idea is to have a configuration file that defines a bunch of toggles for various features you have pending. The running application then uses these toggles in order to decide whether or not to show the new feature.” - Fowler
  51. 51. Feature flags
  52. 52. Feature flags
  53. 53. Canary Releases [CR.1] “Canary release is a technique to reduce the risk of introducing a new software version in production by slowly rolling out the change to a small subset of users before rolling it out to the entire infrastructure and making it available to everybody.”
  54. 54. Quick rollback Integration issues Scaling issues Corner cases 1 2 3
  55. 55. Quick rollback Drawback • Migrations in its own PR • Previous PR for “preparing” ORM (ActiveRecord)
  56. 56. Post-mortem • Affected components • Timing • What happened • 5W • Actions
  57. 57. Smoke tests • Manual testing • On-premise releases • Tag
  58. 58. Wrapping it Up
  59. 59. Costs • Riskier • Have plan B • backups • rollbacks • soft deletions • Heavy instrumentalized • servers • fast tests • real time alerting
  60. 60. Costs • Unexpected DB locks • Integration issues
  61. 61. Benefits • Cheaper • Early detection & diagnosis • No “release day” panic • Small and easy reverts.
  62. 62. Benefits • Accelerated TTM • Building the Right Product • Test in actual environment • Visible progress
  63. 63. Is Continuous Delivery suitable for you?
  64. 64. Is Continuous Delivery suitable for you? ¯_(ツ)_/¯
  65. 65. Questions?
  66. 66. Thank you! Juan Ignacio Sanchez Backend Engineer at CartoDB juanignaciosl@cartodb.com @juanignaciosl
  67. 67. References Continuous Integration [CI.1] Wikipedia: https://en.wikipedia.org/wiki/Continuous_integration [CI.2] Fowler: http://www.martinfowler.com/articles/continuousIntegration.html [CI.3] Duvall: “Continuous Integration. Improving Software Quality and Reducing Risk” http://martinfowler.com/books/duvall.html Continuous Delivery [CD.1] Wikipedia: https://en.wikipedia.org/wiki/Continuous_delivery [CD.2] Fowler: http://martinfowler.com/bliki/ContinuousDelivery.html [CD.3] Agile Alliance: http://guide.agilealliance.org/guide/cd.html XP [XP.1] eXtreme Programming: https://en.wikipedia.org/wiki/Extreme_programming Feature Flags [FF.1] FeatureToggle - Martin Fowler: http://martinfowler.com/bliki/FeatureToggle.html Canary Releases [CR.1] CanaryRelease - Danilo Sato: http://martinfowler.com/bliki/CanaryRelease.html Misc [M.1] Booch method: https://en.wikipedia.org/wiki/Booch_method [M.2] One Hacker Way - Erik Meijer: https://vimeo.com/110554082 [M.3] https://medium.com/gutefrage-net-engineering/how-to-escape-from-the- deployment-and-integration-hell-5c55e1628137#.o7d0n3a5h

×