Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Melbourne Microservices Meetup: Agenda for a new Architecture


Published on

This presentation steps back to look at the current IT climate and context for microservices. I argue that we are experiencing a paradigm shift in how we build applications and that microservices may represent a new paradigm alternative.

I then look back at previous experience with application architectures, the driving forces acting today in terms of "crisis" and opportunities and what aspects of microservices we want to examine in more detail in future meetup events.

Published in: Technology
  • Be the first to comment

Melbourne Microservices Meetup: Agenda for a new Architecture

  1. 1. microservices an agenda for a new architecture
  2. 2. who am I?  Twenty years experience working in distributed systems  Slowly getting better at it  Background: integration, SOA, Event Processing, BPM, CAD/GIS, Astrophysics  CTO at Sixtree:  Editor at InfoQ:  Blogger since 2007:  Twitter: @scaganoff
  3. 3. overview To set out an agenda and discussion programme for microservices melbourne  a little philosophy  a little reflection  what’s happening?
  4. 4. a little philosophy  Epistemology: how do we gain knowledge?
  5. 5. scientific method  Karl Popper (1935): Theories can never be proven, only falsified.  Thomas Kuhn (1962): Scientists work within a “paradigm,” a set of rules & regulations that  defines boundaries and;  describes how to behave inside those boundaries.  Paul Feyerabend (1975): There is no method. It’s all just “marketing”
  6. 6. the copernican revolution 1543 1687
  7. 7. how is this relevant to IT?  Truth? But you can’t measure productivity ...or quality  Software is a social process cannot separate method from culture ...organizational change = “you're not doing it right”  It takes a long time for knowledge to converge …in that time tools/techniques have moved on  Is IT merely fashion?
  8. 8. crisis & change  Crisis  Discrepancy between theory and fact  Change in social/cultural climate  Change  Paradigms are resistant/resilient to change  Failures take a long time to build momentum  Shift  New candidate emerges and a battle ensues  Often the old paradigm doesn't die out completely
  9. 9. application architecture shifts  Mainframe to Client/Server Crisis: scarcity of compute resources Enabler: Emergence of PCs and midrange computers Impact: $$$ shifted from hardware to software  World Wide Web Crisis: Content distribution, e-commerce Enabler: HTTP, HTML Impact: $$$ shifted online  The current shift… Crisis of agility: change takes too long and costs too much Enabler: Commodity hardware, Virtualization, SOA, DevOps Impact: ….
  10. 10. many interacting shifts On-Premise → Cloud Hosted Owned → Pay per use Physical → Digital Local → Global Scale-up → Scale-out Sequential → Concurrent Static → Mobile Centralized → Distributed Waterfall → Agile Assembly Line → TQM
  11. 11. the current crisis  Business Agility, speed of change  The big ball of mud  Scalability  a global user-base  flavours: differential, elastic  Commodity vs Differentiation  Does IT matter?
  12. 12. the monolith  Have we been working with the wrong level of abstraction?  The bigger an application becomes, the harder it is to change.  Services feel like the right level of abstraction but there is a mismatch with the way applications are currently packaged and delivered.  Very coarse differentiation between “commodity capabilities” and “defining capabilities” (e.g. pace layering).
  13. 13. the leap of faith  The gap between business requirements and what an application vendor offers. the leap of faith Snip here to outsource, offshore, cloudify
  14. 14.  Cloud Scale, Cost, Agility, Abundance, Ephemeral  Services as a Service Buy services not software Compose “best of breed”  NoSQL DBs are cheap & malleable  Continuous Delivery Change can be automated  Agile Incremental change Small, productive teams  DevOps Remove the barriers from conception to production the opportunities
  15. 15. the composable enterprise  The pioneers are “providers”, Netflix, Warner Music Group  The mainstream will be “consumers”  …or a mixture Consume commodity SaaS Build their own “engines of differentiation” So far all the talk has been about providing… consuming, not so much
  16. 16. what is an application anyway?  First instances of the new paradigm resemble the old paradigm  What is other than Siebel in the cloud?  Is Netflix an enterprise or an application? It’s hard to tell…traditional system boundaries are dissolving.
  17. 17. a little reflection we’ve been here before
  18. 18. distributed systems frameworks  The way we conceptualize distributed systems reflects our programming languages  Sun RPC/DCE – distributed procedures  CORBA/DCOM – distributed objects  SOAP – distributed objects + XML serialization  REST – www for bots  Events, Asynchrony, Concurrency  Reactive Extensions  Promises  CSP (Go, Erlang) – a return to 1970’s message passing paradigms?
  19. 19. problems with distributed systems  Classic:  Serialization  Interface contracts  Service discovery  Waldo’s fallacies  State Management & Consistency  New:  Management  Visibility & Responsiveness  Resilience  Change  Coordination
  20. 20. waldo et al  Fallacies of distributed computing  There is a single natural object-oriented design for an application regardless of the context in which it will be deployed.  Failure and performance issues are an implementation detail to be added after the initial design.  The interface of an object is independent of the context in which that object is to be used …fundamentally: latency, performance & failure must be accounted for from the design up… Services address Waldo because they make network boundaries explicit.
  21. 21. serialization  A recurring problem in many systems  Sun RPC/DEC – wedded to C and Unix  DCOM – wedded to C and Windows  CORBA – IDL binding to multiple languages was a big problem  SOAP – XML binding to multiple languages was a big problem …mismatch between serialized objects and an XML document  JSON – better match to common language structures such as object attributes, arrays, maps
  22. 22. interface contracts  Waldo fallacy: an object interface is independent of its context  XSD fallacy: a document’s validation rules are independent of the process context  Postel’s Law: “be conservative in what you send and permissive in what you receive…”  Permissive Consumer  c.f. various object serialization libraries  worry that JSON is just reinventing XML
  23. 23. where to from here? what to consider for microservices
  24. 24. coupling vs cohesion  microservices favour extreme de-coupling ….what do we trade in terms of coherence?  Database  synchronization across systems of record  Code  How to handle shared libraries & code change  Ctrl+C/Ctrl+V = decoherence  Services  Vocabularies vs shared object models  Consumer/Provider coupling – avoiding rpc
  25. 25. service design  Granularity - how big is a “micro”?  LOC seems somewhat arbitrary  ….is it more that I can throw it away without grief?  Hypermedia & HATEOAS  Reiteration of object references – values vs urls  Seems like a great idea, but we are lacking good examples  Many choices: HAL, JSON.API, JSON Siren, Collection+JSON, JSON-LD  Automating consumers  Design-first specification  API Blueprint, RAML, Swagger 2.0  API Evolution - versioning
  26. 26. development concerns  Continuous Delivery  Testing  Client-driven testing: PACT  Integration testing  Runtime environment  Naked processes  Traditional containers – e.g. JVM  New containers – LXC, Docker, PaaS  Autoscaling
  27. 27. languages & frameworks  What are good languages & frameworks for microservices? ….why?  Popular  Go  node.js  Scala + Play  …?  Frameworks  Netflix OSS  Spring Boot, Mule APIKit  Seneca (node.js)
  28. 28. visibility & responsiveness  Self-healing Services  Logging  Trace, debug  Metrics – response times  Coordination – end-to-end monitoring, choreography  Tools  ELK: Elasticsearch, Logstash, Kibana  Splunk  Riemann  Alerting & Reaction  Circonus
  29. 29. coordination  Orchestration vs  Choreography  Mastering asynchronous coordination  Message passing  Events: sourcing, processing  State: application and system-wide  Layered architectures - “layers are bad”  Composition is a form of re-use, so where do you define your level of composition?
  30. 30. antifragile patterns  Adrian Cockroft (Netflix) – Dystopia as a Service  Cloud Native Architecture:  Embrace “broken & inefficient” to deliver “sooner” and “dynamic” "the new engineering challenge is not to construct perfection but to construct highly agile and highly available services from ephemeral and often broken components."  Microservices  Reactive APIs  Bulkheads  Circuit breakers  Chaos Monkey (and other Simians)
  31. 31. use-cases  Greenfields applications  Brownfields applications – building new microservices apps amongst traditional apps  Remediation – re-architecting legacy apps into microservices  Distributed Teams  Legacy Systems  The “build” vs “buy” equation
  32. 32. future agenda  Aim for a monthly meetup with two speakers each month  Looking for speakers to discuss:  DDD  API Design  Languages & frameworks  Development concerns  Testing  Continuous Delivery  Runtime Environment  Monitoring & Visibility  Coordination  Antifragile Patterns  Use-Cases/Case-Studies and….