• Like
Cloud so much more than a tool-fest - Qcon London 2012
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Cloud so much more than a tool-fest - Qcon London 2012

  • 3,380 views
Published

 

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,380
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
39
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Cloud... so much more than a tools fest London 2012 Q conhttp://localizationlocalisation.files.wordpress.com/2010/08/rolling-out-old-tools.jpg Patrick Debois
  • 2. Vagrant & Veewee http://jedi.be/blog
  • 3. @patrickdebois
  • 4. Freelancer
  • 5. Context:a traditional enterprise
  • 6. With a developer group on a mission to innovatehttp://1.bp.blogspot.com/-dJydLIEZCQA/TtO26OpjywI/AAAAAAAADBg/wo3Vpl-eesc/s800/Cat%2BArmy.jpg
  • 7. Realtime TV - Web interaction
  • 8. Second Screen Applications
  • 9. http://www.twylah.com/billycoover/topics/xamarin
  • 10. TV-show +/- 1 millionconcurrent viewers http://icanhascheezburger.files.wordpress.com/2011/09/2afc3f14-2ab1-49dc-819e-87490c58b16f.jpg
  • 11. http://www.twylah.com/billycoover/topics/xamarin
  • 12. Google App Engine
  • 13. Abstraction is AWESOME
  • 14. If it failshard to debug/ understand
  • 15. Too generic Java Rails Esper Redis Hadoop ETL toolsPython Nodejs Mongodb Mysql
  • 16. Need more control
  • 17. IAAShttp://3.bp.blogspot.com/-TgmC6DkVNiU/TzCTP_BxWqI/AAAAAAAAAb8/797iiSLQ3FY/s1600/funny-pictures-cat-fixes-your-computer.jpg
  • 18. I never did any cloud before.Only enterprise stuff http://meowcheese.com/files/lolpics/2010/05/dog-not-cat.jpg
  • 19. What I learned from managing Production Servers http://www.collthings.co.uk/2008/06/10-very-rare-clouds.html
  • 20. A Single Server WEB APP DB
  • 21. Starts GrowingWEB WEB WEBAPP APP APP DB
  • 22. Generic Servers become Specific Servers WEB WEB WEB APP APP APP DB
  • 23. Physical becomes VirtualWEB WEB WEBAPP APP APP DB xen/vsphere/kvm/...
  • 24. We learned cloning isn’t working WEBvI WEBv2 WEBv1 APP APP APP DB
  • 25. We introduce config managementWEB APP DB = = =WEB APP DBConfig Config ConfigJEOS JEOS JEOS
  • 26. Infrastructure as codeWEB APP DBConfig Config ConfigJEOS JEOS JEOS Config Management Code Repository Cfengine/Puppet/Chef
  • 27. What I learned from managing Test Servers http://www.collthings.co.uk/2008/06/10-very-rare-clouds.html
  • 28. Reuse Across EnvironmentsDEV TEST PROD WEB WEB APP APP DB DB Infrastructure Code Repository
  • 29. Explosion of VM creationDEV TEST PROD WEB APP DB WEB APP DB WEB APP DB WEB APP DB WEB APP DB WEB APP DB WEB APP DB WEB APP DB Infrastructure Application Code Repository Code Repository
  • 30. UI Interface toAutomated Provisioning of VMS APP WEB APP DB INFRA WEB APP DB VM VM VM Cobbler,Spacewalk, ...
  • 31. Metadata Registry for Systems APP WEB APP DB Config Puppet/ChefINFRA WEB APP DB Management Server VM VM VM
  • 32. I’VE NEVER INSTALLED A GUI FOR THAT.
  • 33. $ knife ec2 server create -r role[webserver] -I ami-7000f019 -f m1.small -A Your AWS Access Key ID -K Your AWS Secret Access Key
  • 34. $  puppet  node  create  -­‐-­‐image  ami-­‐XxXXxXXX  -­‐-­‐keypair  puppetlabs.admin  -­‐-­‐type  m1.small aka “cloudpack”
  • 35. Security Elastic Load groups balancersServers Keys S3 Storage E-Mail service http://fog.io @geemus Custom Scripts
  • 36. AWS Network Constraints Security Only 1 ELBGroups only on network not on non- creation interface standard ports http://iruntheinternet.com/lulzdump/images/cat-in-jar-cok-stuck-1298144751i.jpg
  • 37. Dynamic Monitoringnodes  =  search(:node,  "hostname:[*  TO  *]  AND  chef_environment:#{node.chef_environment}")
  • 38. VM creation failure, network hickups, disk erratic behavior http://icanhascheezburger.files.wordpress.com/2009/09/funny-pictures-cat-is-squished.jpg
  • 39. Embrace
  • 40. Re-architect
  • 41. “Quis custodietipsos Custodes” who watches the watchers
  • 42. Noops
  • 43. What I learned working in the (Amazon)cloud
  • 44. what’s the difference ? Internal Cloud Web,App,DB Web,App,DB Config Mgmt Config MgmtMetadata Registry Metadata Registry JEOS AMI VM Xen UI Provision AWS Console
  • 45. From console to API Web UI AWS Console“Internals” API AWS API Fog Abstracted Jclouds API Boto
  • 46. Beyond Servers Components Server(s) + Loadbalancers DNS Service IP Address Email Service EBS Volume Firewall Keys
  • 47. From server to stackCloudformation Json file specifyingorder of component creation and dependencies
  • 48. What I learned from managing development Servers http://www.collthings.co.uk/2008/06/10-very-rare-clouds.html
  • 49. Development movesfrom host into virtual machines VMWEB WEBAPP APP DB DBHOST HOST
  • 50. Development starts using config mgt VM VMWEB WEBAPP APP DB DBHOST Config JEOS HOST
  • 51. Reuse “code” across EnvironmentsDEV TEST PRODWEB WEB WEBAPP APP APP DB DB DB Infrastructure Code Repository
  • 52. Vagrant UP PROVISION VM HALT DESTROY VM Management WEB Simple CLI APP Vagrantfile DB Puppet/Chef Config Basebox JEOS HOSThttp://vagrantup.com/
  • 53. Integrate with Continuous Integration DEV TEST PRODAPP WEB APP DB WEB APP DB WEB APP DBINFRA WEB APP DB WEB APP DB WEB APP DB Infrastructure Application Code Repository Code Repository
  • 54. Development/VM workflowWEB APP DBConfig PROVISIONJEOS VM UP HALT DESTROY
  • 55. TEST WEB APP DB WEB APP DB WEB APP DB DEV WEB APP DB PRODAPP WEB APP DB WEB APP DB WEB APP DBINFRA WEB APP DB WEB APP DB WEB APP DB Setup Outgrew Setup Outgrew Peak Capacity VM on Laptop Test Lab needed in Prod CLOUD
  • 56. Reuse “workflow” across Environments “If it’s hard to it more often” DEV TEST PRODWeb,App,DB Web,App,DB Web,App,DBConfig Mgmt Config Mgmt Config Mgmt JEOS JEOS JEOS AWS AWS AWS Fog Library (API) Mccloud (simple CLI)
  • 57. +Reuse workflow across hypervisors customersWeb,App,DB Web,App,DB Web,App,DBConfig Mgmt Config Mgmt Config Mgmt JEOS JEOS JEOSVirtualbox KVM AWS Fog Library
  • 58. Cloud Libs (Jclouds/Fog/Boto) embracing old and personal“personal” “old” “hybrid” “new” VsphereVirtualbox Openstack AWS KvmVM Fusion Eucalyptus Rackspace Libvirt
  • 59. Abstracting OS Installation Debian Ubuntu Solaris Win Redhat Archlinux Suse Centos Kickstart Preseed Unattended.xml Virtualbox KVM Fusion Parallelshttp://github.com/jedi4ever/veewee
  • 60. Cloud Libs (Jclouds/Fog/Boto) beyond serversDNS Keys IPs Storage Securitygroups Loadbalancer
  • 61. Workflow beyond servers up provision VM halt destroy ipOther balance sorry
  • 62. Self Servicing
  • 63. Continuous Integration to Continuous Delivery Faster/Delivery DEV TEST PRODAPP WEB APP DB WEB APP DB WEB APP DBINFRA WEB APP DB WEB APP DB WEB APP DB Faster/Feedback Infrastructure Application Code Repository Code Repository
  • 64. Confidence
  • 65. Faster/DeliveryDEV OPS Faster/Feedback
  • 66. Infrastructure ~ Code
  • 67. Testing ~ Monitoring Cucumber-nagios
  • 68. Debugging ~ Metrics
  • 69. Metrics Reuse Traditional Ops TEST PROD WEB APP DB WEB APP DB WEB APP DB WEB APP DB Collectd,Ganglia, Graphite, Opentsdb
  • 70. Extend “metrics” to Development DEV TEST PRODWEB APP DB WEB APP DB WEB APP DBWEB APP DB WEB APP DB WEB APP DB Collectd,Ganglia, Graphite, Opentsdb
  • 71. Extend “logs” to Development DEV TEST PRODWEB APP DB WEB APP DB WEB APP DBWEB APP DB WEB APP DB WEB APP DB Logstash, Graylog
  • 72. Selfservicing “metrics injection” DEV TEST PRODWEB APP DB WEB APP DB WEB APP DBWEB APP DB WEB APP DB WEB APP DB StatsD
  • 73. Selfservicing “alerts” DEV TEST PRODWEB APP DB WEB APP DB WEB APP DBWEB APP DB WEB APP DB WEB APP DB Tatle
  • 74. Selfservicing “graphs” DEV TEST PRODWEB APP DB WEB APP DB WEB APP DBWEB APP DB WEB APP DB WEB APP DB Graphite
  • 75. Social ITradiate information to where it’s needed
  • 76. App App Continuous Load App Monitoring Alerting Metrics LoggingDeployment Integration Testing MetricsHeroku Cloudbees Blitz.io New Relic Pingdom Pager Duty DatadogHQ Loggly Repeating Service Pattern Simple API/CLI , Self-Servicing
  • 77. Abstracting It’s all events Timestamp - Key - ValueLogs Metrics Monitoring Meta Ops
  • 78. +Reuse workflow across monitoring toolsCollectd Ganglia GraphiteNagios Zenoss Sensu Dream Library
  • 79. Workflow reuseMonitoring UP
  • 80. So maybe you don’t haveunlimited resources http://www.collthings.co.uk/2008/06/10-very-rare-clouds.html
  • 81. But you can shape your internal IT as a cloud
  • 82. Questions?
  • 83. Thank you! http://www.collthings.co.uk/2008/06/10-very-rare-clouds.html