Cloud so much more than a tool-fest - Qcon London 2012

6,574 views
5,894 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,574
On SlideShare
0
From Embeds
0
Number of Embeds
529
Actions
Shares
0
Downloads
41
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Cloud so much more than a tool-fest - Qcon London 2012

  1. Cloud... so much more than a tools fest London 2012 Q conhttp://localizationlocalisation.files.wordpress.com/2010/08/rolling-out-old-tools.jpg Patrick Debois
  2. Vagrant & Veewee http://jedi.be/blog
  3. @patrickdebois
  4. Freelancer
  5. Context:a traditional enterprise
  6. With a developer group on a mission to innovatehttp://1.bp.blogspot.com/-dJydLIEZCQA/TtO26OpjywI/AAAAAAAADBg/wo3Vpl-eesc/s800/Cat%2BArmy.jpg
  7. Realtime TV - Web interaction
  8. Second Screen Applications
  9. http://www.twylah.com/billycoover/topics/xamarin
  10. TV-show +/- 1 millionconcurrent viewers http://icanhascheezburger.files.wordpress.com/2011/09/2afc3f14-2ab1-49dc-819e-87490c58b16f.jpg
  11. http://www.twylah.com/billycoover/topics/xamarin
  12. Google App Engine
  13. Abstraction is AWESOME
  14. If it failshard to debug/ understand
  15. Too generic Java Rails Esper Redis Hadoop ETL toolsPython Nodejs Mongodb Mysql
  16. Need more control
  17. IAAShttp://3.bp.blogspot.com/-TgmC6DkVNiU/TzCTP_BxWqI/AAAAAAAAAb8/797iiSLQ3FY/s1600/funny-pictures-cat-fixes-your-computer.jpg
  18. I never did any cloud before.Only enterprise stuff http://meowcheese.com/files/lolpics/2010/05/dog-not-cat.jpg
  19. What I learned from managing Production Servers http://www.collthings.co.uk/2008/06/10-very-rare-clouds.html
  20. A Single Server WEB APP DB
  21. Starts GrowingWEB WEB WEBAPP APP APP DB
  22. Generic Servers become Specific Servers WEB WEB WEB APP APP APP DB
  23. Physical becomes VirtualWEB WEB WEBAPP APP APP DB xen/vsphere/kvm/...
  24. We learned cloning isn’t working WEBvI WEBv2 WEBv1 APP APP APP DB
  25. We introduce config managementWEB APP DB = = =WEB APP DBConfig Config ConfigJEOS JEOS JEOS
  26. Infrastructure as codeWEB APP DBConfig Config ConfigJEOS JEOS JEOS Config Management Code Repository Cfengine/Puppet/Chef
  27. What I learned from managing Test Servers http://www.collthings.co.uk/2008/06/10-very-rare-clouds.html
  28. Reuse Across EnvironmentsDEV TEST PROD WEB WEB APP APP DB DB Infrastructure Code Repository
  29. Explosion of VM creationDEV TEST PROD WEB APP DB WEB APP DB WEB APP DB WEB APP DB WEB APP DB WEB APP DB WEB APP DB WEB APP DB Infrastructure Application Code Repository Code Repository
  30. UI Interface toAutomated Provisioning of VMS APP WEB APP DB INFRA WEB APP DB VM VM VM Cobbler,Spacewalk, ...
  31. Metadata Registry for Systems APP WEB APP DB Config Puppet/ChefINFRA WEB APP DB Management Server VM VM VM
  32. I’VE NEVER INSTALLED A GUI FOR THAT.
  33. $ knife ec2 server create -r role[webserver] -I ami-7000f019 -f m1.small -A Your AWS Access Key ID -K Your AWS Secret Access Key
  34. $  puppet  node  create  -­‐-­‐image  ami-­‐XxXXxXXX  -­‐-­‐keypair  puppetlabs.admin  -­‐-­‐type  m1.small aka “cloudpack”
  35. Security Elastic Load groups balancersServers Keys S3 Storage E-Mail service http://fog.io @geemus Custom Scripts
  36. AWS Network Constraints Security Only 1 ELBGroups only on network not on non- creation interface standard ports http://iruntheinternet.com/lulzdump/images/cat-in-jar-cok-stuck-1298144751i.jpg
  37. Dynamic Monitoringnodes  =  search(:node,  "hostname:[*  TO  *]  AND  chef_environment:#{node.chef_environment}")
  38. VM creation failure, network hickups, disk erratic behavior http://icanhascheezburger.files.wordpress.com/2009/09/funny-pictures-cat-is-squished.jpg
  39. Embrace
  40. Re-architect
  41. “Quis custodietipsos Custodes” who watches the watchers
  42. Noops
  43. What I learned working in the (Amazon)cloud
  44. what’s the difference ? Internal Cloud Web,App,DB Web,App,DB Config Mgmt Config MgmtMetadata Registry Metadata Registry JEOS AMI VM Xen UI Provision AWS Console
  45. From console to API Web UI AWS Console“Internals” API AWS API Fog Abstracted Jclouds API Boto
  46. Beyond Servers Components Server(s) + Loadbalancers DNS Service IP Address Email Service EBS Volume Firewall Keys
  47. From server to stackCloudformation Json file specifyingorder of component creation and dependencies
  48. What I learned from managing development Servers http://www.collthings.co.uk/2008/06/10-very-rare-clouds.html
  49. Development movesfrom host into virtual machines VMWEB WEBAPP APP DB DBHOST HOST
  50. Development starts using config mgt VM VMWEB WEBAPP APP DB DBHOST Config JEOS HOST
  51. Reuse “code” across EnvironmentsDEV TEST PRODWEB WEB WEBAPP APP APP DB DB DB Infrastructure Code Repository
  52. Vagrant UP PROVISION VM HALT DESTROY VM Management WEB Simple CLI APP Vagrantfile DB Puppet/Chef Config Basebox JEOS HOSThttp://vagrantup.com/
  53. Integrate with Continuous Integration DEV TEST PRODAPP WEB APP DB WEB APP DB WEB APP DBINFRA WEB APP DB WEB APP DB WEB APP DB Infrastructure Application Code Repository Code Repository
  54. Development/VM workflowWEB APP DBConfig PROVISIONJEOS VM UP HALT DESTROY
  55. TEST WEB APP DB WEB APP DB WEB APP DB DEV WEB APP DB PRODAPP WEB APP DB WEB APP DB WEB APP DBINFRA WEB APP DB WEB APP DB WEB APP DB Setup Outgrew Setup Outgrew Peak Capacity VM on Laptop Test Lab needed in Prod CLOUD
  56. Reuse “workflow” across Environments “If it’s hard to it more often” DEV TEST PRODWeb,App,DB Web,App,DB Web,App,DBConfig Mgmt Config Mgmt Config Mgmt JEOS JEOS JEOS AWS AWS AWS Fog Library (API) Mccloud (simple CLI)
  57. +Reuse workflow across hypervisors customersWeb,App,DB Web,App,DB Web,App,DBConfig Mgmt Config Mgmt Config Mgmt JEOS JEOS JEOSVirtualbox KVM AWS Fog Library
  58. Cloud Libs (Jclouds/Fog/Boto) embracing old and personal“personal” “old” “hybrid” “new” VsphereVirtualbox Openstack AWS KvmVM Fusion Eucalyptus Rackspace Libvirt
  59. Abstracting OS Installation Debian Ubuntu Solaris Win Redhat Archlinux Suse Centos Kickstart Preseed Unattended.xml Virtualbox KVM Fusion Parallelshttp://github.com/jedi4ever/veewee
  60. Cloud Libs (Jclouds/Fog/Boto) beyond serversDNS Keys IPs Storage Securitygroups Loadbalancer
  61. Workflow beyond servers up provision VM halt destroy ipOther balance sorry
  62. Self Servicing
  63. Continuous Integration to Continuous Delivery Faster/Delivery DEV TEST PRODAPP WEB APP DB WEB APP DB WEB APP DBINFRA WEB APP DB WEB APP DB WEB APP DB Faster/Feedback Infrastructure Application Code Repository Code Repository
  64. Confidence
  65. Faster/DeliveryDEV OPS Faster/Feedback
  66. Infrastructure ~ Code
  67. Testing ~ Monitoring Cucumber-nagios
  68. Debugging ~ Metrics
  69. Metrics Reuse Traditional Ops TEST PROD WEB APP DB WEB APP DB WEB APP DB WEB APP DB Collectd,Ganglia, Graphite, Opentsdb
  70. Extend “metrics” to Development DEV TEST PRODWEB APP DB WEB APP DB WEB APP DBWEB APP DB WEB APP DB WEB APP DB Collectd,Ganglia, Graphite, Opentsdb
  71. Extend “logs” to Development DEV TEST PRODWEB APP DB WEB APP DB WEB APP DBWEB APP DB WEB APP DB WEB APP DB Logstash, Graylog
  72. Selfservicing “metrics injection” DEV TEST PRODWEB APP DB WEB APP DB WEB APP DBWEB APP DB WEB APP DB WEB APP DB StatsD
  73. Selfservicing “alerts” DEV TEST PRODWEB APP DB WEB APP DB WEB APP DBWEB APP DB WEB APP DB WEB APP DB Tatle
  74. Selfservicing “graphs” DEV TEST PRODWEB APP DB WEB APP DB WEB APP DBWEB APP DB WEB APP DB WEB APP DB Graphite
  75. Social ITradiate information to where it’s needed
  76. App App Continuous Load App Monitoring Alerting Metrics LoggingDeployment Integration Testing MetricsHeroku Cloudbees Blitz.io New Relic Pingdom Pager Duty DatadogHQ Loggly Repeating Service Pattern Simple API/CLI , Self-Servicing
  77. Abstracting It’s all events Timestamp - Key - ValueLogs Metrics Monitoring Meta Ops
  78. +Reuse workflow across monitoring toolsCollectd Ganglia GraphiteNagios Zenoss Sensu Dream Library
  79. Workflow reuseMonitoring UP
  80. So maybe you don’t haveunlimited resources http://www.collthings.co.uk/2008/06/10-very-rare-clouds.html
  81. But you can shape your internal IT as a cloud
  82. Questions?
  83. Thank you! http://www.collthings.co.uk/2008/06/10-very-rare-clouds.html

×