Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
What to Upload to SlideShare
Loading in …3
×
1 of 29

Mars missions & Data meshes - a crash course to data meshes

1

Share

Download to read offline

Data meshes are the latest data architecture trend. Really a paradigm shift. But what actually happens is just the natural evolution of technological decentralization. This is a very short crash course, focusing in on the basics of data meshes.

Mars missions & Data meshes - a crash course to data meshes

  1. 1. MarsMissions&DataMeshes by Sven Balnojan; A short introduction to the topic
  2. 2. MarsClimateOrbiter,launchedDec.1998. Mission-Studyclimate,atmosphere&surfacechanges.
  3. 3. Targetorbit(150kmalt.) Possiblecorrections bymultiplethrusters
  4. 4. Lotsofsensors,sending data"0.42","1.002",... Thrusterscanbe controlledfromground (timelapse:~30mins)
  5. 5. Sensors:Builtbysome otherentity,inthiscasea companyformilitarytech called"Lockhead". Thrusters:Controlledby the"maincontrolling entity"whichevalutesthe data &thenprovidesitto decisionmakers - missioncontrol-touseit todecideonproper actions.
  6. 6. Decisionmakers Soundsfamiliar?Thetypicaldataorg Providers&analyzersof data Datasources Datasources 0.42 (1,2,20) inagreedupon specification
  7. 7. BOOOMMMMM!!!!!!!!!!
  8. 8. https://www.simscale.com/blog/2017/12/nasa-mars-climate-orbiter-metric/
  9. 9. Domainboundaries....That'swhathappenstocentraldatateamsalldaylong.
  10. 10. Theonlydifference?NASAhasasimplefix:interfacebeforehand=>be happy. Datateamsontheotherhand=>notachance,becausethebasechanges, andshouldchangeoften&frequently.
  11. 11. MarsMissions&DataMeshes by Sven Balnojan; A shortintroduction to the topic
  12. 12. Agenda • Theorganizationalproblem(Whoisresponsibleforwhat?) • Thesolution(Youareresponsibleforthat.) • Whenshouldweconsiderchanging? • Threebuildingblocksofdatameshes • Thisiscompletelynormal(Evolutionofdecentralizationintech.) • The"light"variant(X-as-a-Service) • Questions...
  13. 13. Theproblem?
  14. 14. Thesolution?
  15. 15. If/Whenshouldwechange?
  16. 16. Twoforcesdeterminewhetheryouneedtochange: 1)SourceCount+SourceComplexity 2)UseCases+(Quality)Demands =>willleadtoa"pullofdecentralization"whichyou shouldfeel. The"PullofDecentralization" Determiningforces
  17. 17. Whatisadatamesh?
  18. 18. "OldWineinNewBottles"(toquoteArifWider) ProductThinking • Data,notaby-productanymore,but wrappedina"data-service"subjecttoa productmanager. • Applyusualproductmanagement practicestoadataset(whoisthe customer?...). • Data-servicemaintainedinacross- functionalteam(withitspipelines,docs, interfaces,...). Domain-DrivenDistributedArchitecture • Thedataproducthasexactlyone domain. • Theteamcanbecomeadomainexpert. • Data-servicesbecomenodesinamesh =datamesh. • Mightbereasonabletodistinguish betweensource-orienteddataservices &higher-orderdataservices(e.g.aCLV) • Data-serviceonlyatrueone,iff....apply Platforms(X-as-a-Service) • Toboth,developdataservicesfastAND • connectthemtogetherwewant platformstoreduceduplication(justlike indeployment,...) • Animportantpointtofocuson:Domain- agnosticplatform!
  19. 19. Whataboutgovernance?
  20. 20. Decisionmakers Providers&analyzersof data Datasources Datasources 0.42 (1,2,20) inagreedupon specification Twosources,e.g.twosensorsmadebytwodifferentsuppliers, yetthedatacanbecombinedbecauseatleasttheformat (vectororscalar)isfixed.
  21. 21. EvolutionofDecentralizationinTech
  22. 22. Turnsout,we'vedonethisbefore!Thefourdecentralizationmovesintech. Monoliths=>SOA/Micro- Services • 2014formalizedbyMartin Fowleret.al.butgoingback tosomewhat2011 • WHAT: The"code",the "micro-service" Ops+Dev=>DevOps • "Phoenixproject"important book(2018)butroots sometime2007/2008. • Verymuchparalleltothe micro-servicesmove. • WHAT:Operationofa service. Frontends=>Microfrontends • FirstontheThoughtWorks TechRadarin2016 • Somegreatexamplesdone veryearlybySpotify • CamJacksonsarticlein2019 withlotsofexamples+tech implementationideas • WHAT:Thefrontendpartof aspecific service/component. Dataasby-products=>Data Mesh • Firstonstageby ThoughtWorks2019. • WHAT:Theraw&possibly transformed"data".
  23. 23. The"light"Variant
  24. 24. X-as-a-Serviceisacollaborationmodelbetween teams.Itaimstohelpteamsbuilddomainexpertise andremovingfrictions. HowdoesX-as-a-Servicework TeamAprovidingX-as-a-ServicetoTeamB means... • TeamAandTeamBalmostnever"talk" • TeamBcanusetheserviceasis,withoutasking anyone(becauseofgreatdocs&properAPI versioning) • TeamAunderstandtheneedsoftheirend-users andhelpsthemdeliverfasterbyprovidingX-as- a-Service X-as-a-Service X-as-a-ServiceDefinition
  25. 25. ExamplesofX-as-a-Service Models-as-a-Service GTM-as-a-Service Clusters-as-a-Service PushNServe-as-a-Service
  26. 26. Questions?
  27. 27. Ressources! • StuffbytheoriginatorsatThoughtWorks • 2019,https://martinfowler.com/articles/data-monolith-to-mesh.html • 2020,https://martinfowler.com/articles/data-mesh-principles.html • https://www.thoughtworks.com/de/webinar/data-mesh • Stuffbyothers • BarrMoses,https://towardsdatascience.com/data-mesh-101-everything-you-need-to-know-to-get-started-72087f5a7d91 • Stuffbyme • 2019,https://towardsdatascience.com/data-mesh-applied-21bed87876f2 • 2020,https://towardsdatascience.com/theres-more-than-one-kind-of-data-mesh-three-types-of-data-meshes-7cb346dc2819 • 2020,https://medium.com/swlh/microservices-data-meshes-micro-frontends-and-the-timeless-principles-of-decentralization-2ac2516b2951

×