Download to read offline
Uploaded byLars Albertsson
Big data == lean data
The document discusses the importance of data in service-oriented architectures, emphasizing on collaboration and data governance within teams. It explores concepts related to big data, agile methodologies, and the balance needed for successful data management and innovation. Additionally, it touches on AI applications and the progression of data capabilities from basic reporting to advanced machine learning techniques.
More Related Content
Similar to Big data == lean data
![[DSC Europe 25] Slobodan Dolinic - Smart and Intelligent Green Region.pptx](https://cdn.slidesharecdn.com/ss_thumbnails/0bribinjsp6ghwtvsvor-2-sigre-slobodan-dolinic-260115093812-c9c10e90-thumbnail.jpg?width=640&height=640&fit=bounds)
Big data == lean data
- 1. www.mimeria.com Big Data ==Lean Data Agila Sverige, 2018-05-30 Lars Albertsson www.mapflat.com, www.mimeria.com 1
- 2. www.mimeria.com Service-oriented architectures ● Servicesown data ● Heterogeneous coupling 2 Service Service Service App App App Poll Aggregate logs NFS Hourly dump Data warehouse ETL Queue Queue NFS scp DB HTTP DB DBDB
- 3. www.mimeria.com ● Teams ownservices Service-oriented organisations 3
- 4. www.mimeria.com ● Need datafrom teams ○ willing? ○ backlog? ○ collected? ○ useful? ○ extraction? ○ data governance? ○ history? Data-centric innovation 4
- 5. www.mimeria.com ● Need datafrom teams ○ willing? ○ backlog? ○ collected? ○ useful? ○ extraction? ○ data governance? ○ history? Innovation value stream mapping 5
- 6. www.mimeria.com Enter Big Data ●What is Big Data? 6 AI magic Clusters Weird technology ? Spoiled developers
- 7. www.mimeria.com A collaboration paradigm 7 Streamstorage Data lake Data democratised
- 8. www.mimeria.com Onboard driven byuse case 8 Data lake
- 9. www.mimeria.com Data platform ==collaboration platform 9 Data lake
- 10. www.mimeria.com Balance of success 10 Datalake Balance planning & architecture ● Homogeneity ● Governance ● Coordination with business value driven activities
- 11. www.mimeria.com Coupling by design 11 Datalake ● Coordination >> autonomy ● Homogeneity >> heterogeneity
- 12. www.mimeria.com Data agility 12 Data lake ●Siloed: 6+ months ● Autonomous: 1 month ● Coordinated: days ∆ ∆ Latency?
- 13. www.mimeria.com A journey oflearning 13 Data lake ● End-to-end == feedback, value ● Scale == cost ● All data now == waterfall
- 14. www.mimeria.com End-to-end >> scale 14 AImagic Clusters Weird technology ? Workflow orchestration Proof of value !
- 15. www.mimeria.com Can I haveAI now? 15 ● Crawl, walk, run AI Deep learning A/B testing Machine learning Analytics Segments Curation Anomaly detection Data infrastructure Pipelines Instrumentation Data collection Credits: “The data science hierarchy of needs”, Monica Rogati
- 16. www.mimeria.com A journey ofmany years 16 ● Simple == max value ○ Reporting ○ Forecasting, risk ○ User notification ● AI first == waterfall AI Deep learning A/B testing Machine learning Analytics Segments Curation Anomaly detection Data infrastructure Pipelines Instrumentation Data collection Value Effort Credits: “The data science hierarchy of needs”, Monica Rogati
- 17. www.mimeria.com Who's talking? 17 Lars Albertsson Mapflat- independent consultant Mimeria - data-value-as-a-service AI Deep learning A/B testing Machine learning Analytics Segments Curation Anomaly detection Data infrastructure Pipelines Instrumentation Data collection Credits: “The data science hierarchy of needs”, Monica Rogati