Service Configuration Management
For Rapid Growth
Friday, September 6 2013
Echelon Ignite 2013 Thailand
Takashi SOMEDA
Echelon Ignite 2013 Thailand
https://twitter.com/tksmd
Tech Evangelist of Nulab,Inc.
About me
https://faceb...
Echelon Ignite 2013 Thailand
Backlog is a project management tool that 1,800 clients use.
In addition to issue management ...
950,000 users around the world draw wireframes, network diagrams,
UML, business plans and so on.
Cacoo provides basic func...
A new collaborative chat app just opened as preview beta.
• Revolutionize your team discussion
• Integration with nulab’s ...
User Growth of Cacoo
Echelon Ignite 2013 Thailand
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
900,00...
Tipping points
Echelon Ignite 2013 Thailand
Why is
configuration management
important ?
Extreme
Uncertainty
http://www.flickr.com/photos/bilal-kamoon/6835060992/
http://www.slideshare.net/mattangriffel/growth-hacking/82
http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version/19
http://www.slideshare.net/startuplessonslearned/eric-ries-the-lean-startup-google-tech-talk/25
1K+ deploys/hour
https://reinvent.awsevents.com/recap.html
Need to balance
Speed & Quality
GOAL
Business, Customer, Market
What
Growth Hack
Lean Startup
How
Team
Proces
s
Todays topic
Agenda
Echelon Ignite 2013 Thailand
 Configuration Management in essence
 Version Control System (VCS)
 Continuous Inte...
VCS
CI
Service
source
code
Server
Configuration
Server
Orchestration
app
servers
Test
Test
configure
deploy
Configuration management refers to the process
by which all artifacts relevant to your project,
and the relationships betw...
Keep everything in
Version Control System
http://www.flickr.com/photos/munaz/2498380666/
Without VCS
Echelon Ignite 2013 Thailand
VCS in a nutshell
Echelon Ignite 2013 Thailand
VCS Architecture
Echelon Ignite 2013 Thailand
 Centralized
 Repository maintained on server
 CVS, Subversion
 Distribu...
What to keep in VCS
Echelon Ignite 2013 Thailand
 Source
 Build Script
 Ant, Maven, Gradle, Rake, Makefile
 Database S...
Run “beta” in a real world
Echelon Ignite 2013 Thailand
 To know whether you’re going toward right
place as fast as possi...
flag or branch
Echelon Ignite 2013 Thailand
if(flagA){
}
if(flagA){
}
if(flagB){
}
flag
branch
production
feature A
featur...
Pros and cons of “flag”
Echelon Ignite 2013 Thailand
 Pros.
 Same code running on all servers
 Simple and quick release...
Pros and cons of “branch”
Echelon Ignite 2013 Thailand
 Pros.
 Keep source code clean
 Cons.
 Integration (merge) cost...
Beta strategy
Echelon Ignite 2013 Thailand
 Decide basic policy in your team
 If your code changes many things
 Use “br...
http://nvie.com/posts/a-successful-git-branching-model/
http://www.flickr.com/photos/alexmartin81/5883645329/
Every commit should be built
immediately
What’s CI
Echelon Ignite 2013 Thailand
 One of agile practice originally
 To detect integration problem earlier, do
inte...
Benefits of CI
Echelon Ignite 2013 Thailand
 Quick feedback
 Early detection of build issues
 Ensuring traceability
Sta...
What to build
Echelon Ignite 2013 Thailand
 Compile
 Run tests
 Assemble
 Inspection
 Generate Docs
 Etc…
Package Repository
Echelon Ignite 2013 Thailand
 Maintain your own repository
 Maven repo, cpan, pypi, gem
 Shared fold...
Failure feedback from CI
Echelon Ignite 2013 Thailand
 Notification via
 E-mail
 Group
 those who broke code
 IRC / C...
CI build time
Echelon Ignite 2013 Thailand
 Should be done within 30mins
 If build time exceeds it
 Split into smaller ...
Testing in CI
Echelon Ignite 2013 Thailand
 Unit Test
 Integration Test
 Server Configuration Test
 Smoke Test
Build Pipeline
Echelon Ignite 2013 Thailand
http://www.flickr.com/photos/naturegeak/5688437941/
Cloud-native server management
Why “Cloud” for startups
Echelon Ignite 2013 Thailand
 Extremely flexible
 Scalable
 Global infrastructure
NOT the chea...
Paradigm shift of “Cloud”
Echelon Ignite 2013 Thailand
 All resources are
 completely virtualized
 programmable
 alloc...
NOT
prevent failure
BUT
restore from failure
quickly
Design principle for failure
Echelon Ignite 2013 Thailand
 Keep each server small and simple
 Building a brand-new serve...
Problem of ad-hoc management
http://www.flickr.com/photos/getbutterfly/6317955134/
What servers are patched
and
What are n...
Provisioning Layer
Echelon Ignite 2013 Thailand
http://devops-toolchain.googlecode.com/svn/trunk/src/site/resources/presen...
Orchestration Tasks
Echelon Ignite 2013 Thailand
 Deploy
 System administration
 stop / start middleware
 Build tempor...
Migration from “bare” scripts
Echelon Ignite 2013 Thailand
 Try simple tool
 No server installation required
 Needs SSH...
Server Configuration Principles
Echelon Ignite 2013 Thailand
 Declarative
 Abstraction
 Idempotent
 Convergence
DO NOT...
Difference between them
Echelon Ignite 2013 Thailand
 Server Orchestration manages
“execution”
 Changing server state to...
Popular tools
Echelon Ignite 2013 Thailand
 Orchestration
 Capistrano
 Fabric
 Configuration
 Chef
 Puppet
 Ansible
Configuration Test
Echelon Ignite 2013 Thailand
httpd package installed
httpd service enabled
httpd service started
80 por...
VCS in server management
Echelon Ignite 2013 Thailand
 Use VCS same as application source
 Keep into VCS
 Configuration...
Summary
http://www.flickr.com/photos/nauright/2662160957/
VCS
CI
Service
source
code
Server
Configuration
Server
Orchestration
app
servers
Test
Test
configure
deploy
to balance
Speed & Quality
Speed = Automation
Quality = Testing
Service Configuration Management for Rapid Growth
Upcoming SlideShare
Loading in …5
×

Service Configuration Management for Rapid Growth

2,004 views

Published on

Configuration Management Workshop held at Echelon Ignite Thailand 2013

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,004
On SlideShare
0
From Embeds
0
Number of Embeds
333
Actions
Shares
0
Downloads
1
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Service Configuration Management for Rapid Growth

  1. 1. Service Configuration Management For Rapid Growth Friday, September 6 2013 Echelon Ignite 2013 Thailand
  2. 2. Takashi SOMEDA Echelon Ignite 2013 Thailand https://twitter.com/tksmd Tech Evangelist of Nulab,Inc. About me https://facebook.com/takashi.someda Write tech articles about AWS, SCM https://github.com/tksmd
  3. 3. Echelon Ignite 2013 Thailand Backlog is a project management tool that 1,800 clients use. In addition to issue management feature, Backlog provides • File sharing by WebDAV • Git and Subversion repository hosting. Visit http://backlogtool.com/
  4. 4. 950,000 users around the world draw wireframes, network diagrams, UML, business plans and so on. Cacoo provides basic functionality as a draw tool and powerful collaborative features like • Simultaneous Editing on same diagram by multiple users • Integration with Google Services like Google Apps, Google Drive and Google+ Hangouts
  5. 5. A new collaborative chat app just opened as preview beta. • Revolutionize your team discussion • Integration with nulab’s services • Provide easy-to-use API for developers Be the first to get notified when the app is launched. http://typetalk.in
  6. 6. User Growth of Cacoo Echelon Ignite 2013 Thailand 0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 800,000 900,000 1,000,000 Oct,2010 Oct,2011 Oct,2012 Sep,2013 62,000 200,000 600,000 950,000 All Japan Overseas
  7. 7. Tipping points Echelon Ignite 2013 Thailand
  8. 8. Why is configuration management important ?
  9. 9. Extreme Uncertainty http://www.flickr.com/photos/bilal-kamoon/6835060992/
  10. 10. http://www.slideshare.net/mattangriffel/growth-hacking/82
  11. 11. http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version/19
  12. 12. http://www.slideshare.net/startuplessonslearned/eric-ries-the-lean-startup-google-tech-talk/25
  13. 13. 1K+ deploys/hour https://reinvent.awsevents.com/recap.html
  14. 14. Need to balance Speed & Quality
  15. 15. GOAL Business, Customer, Market What Growth Hack Lean Startup How Team Proces s Todays topic
  16. 16. Agenda Echelon Ignite 2013 Thailand  Configuration Management in essence  Version Control System (VCS)  Continuous Integration (CI)  Server Orchestration & Configuration  Live Demo
  17. 17. VCS CI Service source code Server Configuration Server Orchestration app servers Test Test configure deploy
  18. 18. Configuration management refers to the process by which all artifacts relevant to your project, and the relationships between them, are sorted, retrieved, uniquely identified, and modified. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation Jez Humble, David Farley
  19. 19. Keep everything in Version Control System http://www.flickr.com/photos/munaz/2498380666/
  20. 20. Without VCS Echelon Ignite 2013 Thailand
  21. 21. VCS in a nutshell Echelon Ignite 2013 Thailand
  22. 22. VCS Architecture Echelon Ignite 2013 Thailand  Centralized  Repository maintained on server  CVS, Subversion  Distributed  Each client has its repository  Git, Mercurial, (Bazaar)
  23. 23. What to keep in VCS Echelon Ignite 2013 Thailand  Source  Build Script  Ant, Maven, Gradle, Rake, Makefile  Database Script  Schema, Migration  Server related files DO NOT put binary artifacts
  24. 24. Run “beta” in a real world Echelon Ignite 2013 Thailand  To know whether you’re going toward right place as fast as possible  Rule  Run on “real” data, not dummy  Speed >>> Quality  Limited user ( at least yourself )
  25. 25. flag or branch Echelon Ignite 2013 Thailand if(flagA){ } if(flagA){ } if(flagB){ } flag branch production feature A feature B
  26. 26. Pros and cons of “flag” Echelon Ignite 2013 Thailand  Pros.  Same code running on all servers  Simple and quick release  Cons.  Scattered “if” statements on your code
  27. 27. Pros and cons of “branch” Echelon Ignite 2013 Thailand  Pros.  Keep source code clean  Cons.  Integration (merge) cost  conflict risk  Need additional server to run “branch” code
  28. 28. Beta strategy Echelon Ignite 2013 Thailand  Decide basic policy in your team  If your code changes many things  Use “branch” and refine the code in the beta  Switch “flag” after the quality becomes satisfying and expose it to the production  Just turn the flag on at release
  29. 29. http://nvie.com/posts/a-successful-git-branching-model/
  30. 30. http://www.flickr.com/photos/alexmartin81/5883645329/ Every commit should be built immediately
  31. 31. What’s CI Echelon Ignite 2013 Thailand  One of agile practice originally  To detect integration problem earlier, do integrate repeatedly and frequently  Core of “Delivery Pipeline”  Lots of things can run on CI server
  32. 32. Benefits of CI Echelon Ignite 2013 Thailand  Quick feedback  Early detection of build issues  Ensuring traceability Start small, scale gradually
  33. 33. What to build Echelon Ignite 2013 Thailand  Compile  Run tests  Assemble  Inspection  Generate Docs  Etc…
  34. 34. Package Repository Echelon Ignite 2013 Thailand  Maintain your own repository  Maven repo, cpan, pypi, gem  Shared folder  Amazon S3  Used by both  Development  Testing  Server configuration
  35. 35. Failure feedback from CI Echelon Ignite 2013 Thailand  Notification via  E-mail  Group  those who broke code  IRC / Chat  ITS / BTS Take action as soon as possible
  36. 36. CI build time Echelon Ignite 2013 Thailand  Should be done within 30mins  If build time exceeds it  Split into smaller pieces  Build parallel  Get a higher spec machine
  37. 37. Testing in CI Echelon Ignite 2013 Thailand  Unit Test  Integration Test  Server Configuration Test  Smoke Test
  38. 38. Build Pipeline Echelon Ignite 2013 Thailand
  39. 39. http://www.flickr.com/photos/naturegeak/5688437941/ Cloud-native server management
  40. 40. Why “Cloud” for startups Echelon Ignite 2013 Thailand  Extremely flexible  Scalable  Global infrastructure NOT the cheapest solution
  41. 41. Paradigm shift of “Cloud” Echelon Ignite 2013 Thailand  All resources are  completely virtualized  programmable  allocated and available soon BUT everything will be possibly broken in a meanwhile
  42. 42. NOT prevent failure BUT restore from failure quickly
  43. 43. Design principle for failure Echelon Ignite 2013 Thailand  Keep each server small and simple  Building a brand-new server faster than fixing broken one  Distribute servers to different physical locations Need to manage much more servers than on-premises
  44. 44. Problem of ad-hoc management http://www.flickr.com/photos/getbutterfly/6317955134/ What servers are patched and What are not ?!
  45. 45. Provisioning Layer Echelon Ignite 2013 Thailand http://devops-toolchain.googlecode.com/svn/trunk/src/site/resources/presentations/Velocity2010/ Application Service Orchestration System Configuration Cloud or VM Image Launch OS installBootstrapping Configuration Command and Control
  46. 46. Orchestration Tasks Echelon Ignite 2013 Thailand  Deploy  System administration  stop / start middleware  Build temporary environment  batch / testing DO NOT include configuration
  47. 47. Migration from “bare” scripts Echelon Ignite 2013 Thailand  Try simple tool  No server installation required  Needs SSH connection only  Select tool which is written in your familiar language
  48. 48. Server Configuration Principles Echelon Ignite 2013 Thailand  Declarative  Abstraction  Idempotent  Convergence DO NOT touch environment directly
  49. 49. Difference between them Echelon Ignite 2013 Thailand  Server Orchestration manages “execution”  Changing server state to single direction  Focus on temporally control  Server Configuration manages “state”  Converging server into a “state”  can be tested repeatedly
  50. 50. Popular tools Echelon Ignite 2013 Thailand  Orchestration  Capistrano  Fabric  Configuration  Chef  Puppet  Ansible
  51. 51. Configuration Test Echelon Ignite 2013 Thailand httpd package installed httpd service enabled httpd service started 80 port listened : Is httpd package installed? Is httpd service enabled ? Is httpd running ? Is 80 port listened? : Configuration Test
  52. 52. VCS in server management Echelon Ignite 2013 Thailand  Use VCS same as application source  Keep into VCS  Configuration files  Administration scripts  Orchestration scripts  Configuration scripts
  53. 53. Summary http://www.flickr.com/photos/nauright/2662160957/
  54. 54. VCS CI Service source code Server Configuration Server Orchestration app servers Test Test configure deploy
  55. 55. to balance Speed & Quality
  56. 56. Speed = Automation Quality = Testing

×