The influence of "Distributed platforms" on #devops

628 views

Published on

Mesoscon Amsterdam 2016 Keynote ,

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
628
On SlideShare
0
From Embeds
0
Number of Embeds
77
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The influence of "Distributed platforms" on #devops

  1. 1. The influence of “distributed platforms” on devops Kris Buytaert @krisbuytaert
  2. 2. @KrisBuytaert@KrisBuytaert ● I used to be a Dev,I used to be a Dev, ● Then Became an OpThen Became an Op ● CTO and Open Source Consultant @CTO and Open Source Consultant @inuits.euinuits.eu ● Everything is a DNS ProblemEverything is a DNS Problem ● Evangelizing devopsEvangelizing devops ● Organiser of #devopsdays, #cfgmgmtcamp, #loadays,Organiser of #devopsdays, #cfgmgmtcamp, #loadays, ….….
  3. 3. What's thisWhat's this devops thingdevops thing anyhow ?anyhow ?
  4. 4. World , 200X-2009World , 200X-2009 Patrick Debois, Gildas Le Nadan, Andrew Clay Shafer, Kris Buytaert, JezzPatrick Debois, Gildas Le Nadan, Andrew Clay Shafer, Kris Buytaert, Jezz Humble, Lindsay Holmwood, John Willis, Chris Read, Julian Simpson, and lotsHumble, Lindsay Holmwood, John Willis, Chris Read, Julian Simpson, and lots of others ..of others .. Gent , October 2009Gent , October 2009 Mountain View , June 2010Mountain View , June 2010 Hamburg, Goteborg, Rome, ...Hamburg, Goteborg, Rome, ... 5th aniversary in Gent 20145th aniversary in Gent 2014 25+ events in 201625+ events in 2016
  5. 5. C(L)AMSC(L)AMS ● CultureCulture ● (Lean)(Lean) ● AutomationAutomation ● MeasurementMeasurement ● SharingSharing Damon Edwards and John WillisDamon Edwards and John Willis Gene KimGene Kim
  6. 6. The Old DaysThe Old Days ● ““Put this Code Live, here's a tarball” NOW!Put this Code Live, here's a tarball” NOW! ● What dependencies ?What dependencies ? ● No machines available ?No machines available ? ● What database ?What database ? ● Security ?Security ? ● High Availability ?High Availability ? ● Scalability ?Scalability ? ● My computer can't install this ?My computer can't install this ?
  7. 7. 10 days into operations10 days into operations
  8. 8. 11 days into operations11 days into operations
  9. 9. Some HistorySome History LessonsLessons
  10. 10. openMosixopenMosix (+/- 2001-2005)(+/- 2001-2005) ● 2.4 Kernel patch,2.4 Kernel patch, ● Loadbalance multiple similar processesLoadbalance multiple similar processes ● Both short (compilations) and longlived (calculations)Both short (compilations) and longlived (calculations) processesprocesses ● Parts of processes migrated to other nodesParts of processes migrated to other nodes ● OMFS allowed identical access to data from all nodesOMFS allowed identical access to data from all nodes
  11. 11. openMosix LimitationsopenMosix Limitations ● shared memoryshared memory • Limited type of applications could migrateLimited type of applications could migrate • Patches available but not successfulPatches available but not successful ● Filesystem accessFilesystem access oMFS : unstable => removedoMFS : unstable => removed
  12. 12. Lessons from openMosixLessons from openMosix ● Applications need to be adaptedApplications need to be adapted ● Files need to be accessible (oMFS)Files need to be accessible (oMFS) ● Huge gap between developers and consumersHuge gap between developers and consumers => Limited working use cases=> Limited working use cases
  13. 13. Linux-HALinux-HA (2005- now)(2005- now) ● Hearbeat, Heartbeat v2, Pacemaker , CorosyncHearbeat, Heartbeat v2, Pacemaker , Corosync ● Define resource (OCF)Define resource (OCF) ● Define constraints, clones, colocation, …Define constraints, clones, colocation, … ● Long running services that should not stopLong running services that should not stop
  14. 14. Pacemaker ArchitecturePacemaker Architecture ● Stonithd : The Heartbeat fencing subsystem. ● Lrmd : Local Resource Management Daemon. Interacts directly with resource agents (scripts). ● pengine Policy Engine. Computes the next state of the cluster based on the current state and the configuration. ● cib Cluster Information Base. Contains definitions of all cluster options, nodes, resources, their relationships to one another and current status. Synchronizes updates to all cluster nodes. ● crmd Cluster Resource Management Daemon. Largely a message broker for the PEngine and LRM, it also elects a leader to co-ordinate the activities of the cluster. ● openais messaging and membership layer. ● heartbeat messaging layer, an alternative to OpenAIS. ● ccm Short for Consensus Cluster Membership. The Heartbeat membership layer.
  15. 15. Working with developersWorking with developers ● State vs StatelessState vs Stateless • Most applications have state somewhereMost applications have state somewhere • Discuss how to share/access stateDiscuss how to share/access state ● Data accessData access • Local filesystem is kingLocal filesystem is king • We needed distributed / shared filesystemsWe needed distributed / shared filesystems ● HealthHealth ● MetricsMetrics
  16. 16. Isolating ServicesIsolating Services ● 1 application per VM1 application per VM • Easy configurationEasy configuration • No accidental changesNo accidental changes Anno 2005 – 2010 people called us crazyAnno 2005 – 2010 people called us crazy ● Multiple VM's with the same functions (A,B,001,002,256)Multiple VM's with the same functions (A,B,001,002,256) ● Isolate code from configIsolate code from config
  17. 17. Lessons from Linux-HALessons from Linux-HA ● Applications need to be adaptedApplications need to be adapted ● Files need to be accessible (shared/distributed FS)Files need to be accessible (shared/distributed FS) ● Monitoring strategy needs changeMonitoring strategy needs change ● Good collaboration between developers and ops folksGood collaboration between developers and ops folks => Many general purpose use cases=> Many general purpose use cases ● Works better with an Open Source MindsetWorks better with an Open Source Mindset
  18. 18. Impact on Monitoring/MetricsImpact on Monitoring/Metrics ● HA , monitoring a serviceHA , monitoring a service • Monitoring the individual serviceMonitoring the individual service • Monitoring/Ensuring the global serviceMonitoring/Ensuring the global service ● Health status of an applicationHealth status of an application ● Metric collectionMetric collection ● Alerting based on metrics rather than only status checksAlerting based on metrics rather than only status checks
  19. 19. Back to 2016Back to 2016
  20. 20. Cloud Adoption is (s)lowCloud Adoption is (s)low ● Enterprises are afraid cloudEnterprises are afraid cloud • Security, Cost, Control, Stability, (insert more fud here)Security, Cost, Control, Stability, (insert more fud here) ● ““Private cloud” will solve thisPrivate cloud” will solve this ● Please fill in 4 word documents for for each VM you wantPlease fill in 4 word documents for for each VM you want
  21. 21. Failed (Private) Cloud ProjectsFailed (Private) Cloud Projects ● Identical copies of Bare Metal or vm's moved to the cloudIdentical copies of Bare Metal or vm's moved to the cloud ● No config managementNo config management ● No monitoringNo monitoring ● No resilienceNo resilience ● Legacy procedures (still 5 week wait for a VM)Legacy procedures (still 5 week wait for a VM) ● No API usageNo API usage
  22. 22. These DaysThese Days ● ““Put this Code Live, here's Docker Container” NOW!Put this Code Live, here's Docker Container” NOW! ● No machines available ?No machines available ? ● What database ?What database ? ● Security ? What distro is this even ?Security ? What distro is this even ? ● High Availability ?High Availability ? ● Scalability ?Scalability ? ● How do we monitor his ?How do we monitor his ? ● How did you build this ?How did you build this ?
  23. 23. A typical Enterprise ContainerA typical Enterprise Container ● No different from a full vmNo different from a full vm ● Multiple services running inMultiple services running in on containeron container ● Ssh is the default connectionSsh is the default connection
  24. 24. Why ?Why ?
  25. 25. IT Departments have not adapted,IT Departments have not adapted, 'Shadow'-IT is winning'Shadow'-IT is winning
  26. 26. Elephant , MonolithElephant , Monolith vsvs MicroservicesMicroservices
  27. 27. Can we Slice the Elephants ?Can we Slice the Elephants ? {Devops/Cloud/Container}{Devops/Cloud/Container} adoption requires changeadoption requires change • In software architectureIn software architecture • In codeIn code • In infrastructureIn infrastructure • In mindsetIn mindset • In organisationIn organisation • In managementIn management
  28. 28. ContactContact Kris Buytaert Kris.Buytaert@inuits.euKris Buytaert Kris.Buytaert@inuits.eu Further ReadingFurther Reading @krisbuytaert@krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/ http://www.inuits.eu/http://www.inuits.eu/ Inuits.euInuits.eu Essensteenweg 31Essensteenweg 31 BrasschaatBrasschaat BelgiumBelgium 891.514.231891.514.231 +32 475 961221+32 475 961221

×