Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big data == lean data

51 views

Published on

Big data is primarily associated with AI and new technology. It is as much a revolution in cooperation patterns, however. Big data entails the democratisation of data within an organisation, enabling agile, data-driven innovation in a manner that was previously unavailable. Knowing this, how can you work as an organisation to harvest the fruits and what can go wrong?

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Big data == lean data

  1. 1. www.mimeria.com Big Data == Lean Data Agila Sverige, 2018-05-30 Lars Albertsson www.mapflat.com, www.mimeria.com 1
  2. 2. www.mimeria.com Service-oriented architectures ● Services own data ● Heterogeneous coupling 2 Service Service Service App App App Poll Aggregate logs NFS Hourly dump Data warehouse ETL Queue Queue NFS scp DB HTTP DB DBDB
  3. 3. www.mimeria.com ● Teams own services Service-oriented organisations 3
  4. 4. www.mimeria.com ● Need data from teams ○ willing? ○ backlog? ○ collected? ○ useful? ○ extraction? ○ data governance? ○ history? Data-centric innovation 4
  5. 5. www.mimeria.com ● Need data from teams ○ willing? ○ backlog? ○ collected? ○ useful? ○ extraction? ○ data governance? ○ history? Innovation value stream mapping 5
  6. 6. www.mimeria.com Enter Big Data ● What is Big Data? 6 AI magic Clusters Weird technology ? Spoiled developers
  7. 7. www.mimeria.com A collaboration paradigm 7 Stream storage Data lake Data democratised
  8. 8. www.mimeria.com Onboard driven by use case 8 Data lake
  9. 9. www.mimeria.com Data platform == collaboration platform 9 Data lake
  10. 10. www.mimeria.com Balance of success 10 Data lake Balance planning & architecture ● Homogeneity ● Governance ● Coordination with business value driven activities
  11. 11. www.mimeria.com Coupling by design 11 Data lake ● Coordination >> autonomy ● Homogeneity >> heterogeneity
  12. 12. www.mimeria.com Data agility 12 Data lake ● Siloed: 6+ months ● Autonomous: 1 month ● Coordinated: days ∆ ∆ Latency?
  13. 13. www.mimeria.com A journey of learning 13 Data lake ● End-to-end == feedback, value ● Scale == cost ● All data now == waterfall
  14. 14. www.mimeria.com End-to-end >> scale 14 AI magic Clusters Weird technology ? Workflow orchestration Proof of value !
  15. 15. www.mimeria.com Can I have AI now? 15 ● Crawl, walk, run AI Deep learning A/B testing Machine learning Analytics Segments Curation Anomaly detection Data infrastructure Pipelines Instrumentation Data collection Credits: “The data science hierarchy of needs”, Monica Rogati
  16. 16. www.mimeria.com A journey of many years 16 ● Simple == max value ○ Reporting ○ Forecasting, risk ○ User notification ● AI first == waterfall AI Deep learning A/B testing Machine learning Analytics Segments Curation Anomaly detection Data infrastructure Pipelines Instrumentation Data collection Value Effort Credits: “The data science hierarchy of needs”, Monica Rogati
  17. 17. www.mimeria.com Who's talking? 17 Lars Albertsson Mapflat - independent consultant Mimeria - data-value-as-a-service AI Deep learning A/B testing Machine learning Analytics Segments Curation Anomaly detection Data infrastructure Pipelines Instrumentation Data collection Credits: “The data science hierarchy of needs”, Monica Rogati

×