Download as PDF, PPTX
Uploaded byArne Roßmann
Modern Data Platforms
The document outlines the characteristics and reference architecture of a modern enterprise data and AI platform, emphasizing its discoverability, trustworthiness, interoperability, and security. It highlights the significance of an event-driven architecture using Apache Kafka as a key component and provides a real-world example related to car distribution. The platform aims to provide scalable, governed data services while utilizing Microsoft Azure services for data management and analytics.
In this document
Powered by AI
Overview of modern data platforms for analytics in companies presented by Arne Roßmann.
Arne Roßmann's educational qualifications in Computer Science and International Management, along with his key competencies in Enterprise Architecture and roles in the industry.
Outline of the presentation covering characteristics, reference architecture, key components like Kafka, and a real-world example.
Key features like discoverability, addressability, security, interoperability, and self-service capabilities.
Visual representation of an Enterprise Data & AI platform architecture showing various components.
Detailing a data-centric architecture that includes real-time processing, batch sources, and various Azure services.
Introduction to Kafka as a critical component in the architecture for data streaming and processing.
Discussion on event-driven architecture addressing challenges in data management and integration.
Explains how event-driven architecture supports machine learning and clean solutions for data management.
Presentation of a practical example showcasing the implementation of data strategies in car distribution.
A specific example link provided detailing metrics exporter related to car distribution.
Final remarks summarizing key points and inviting questions from the audience.
Detailed architecture of a modern enterprise platform highlighting components like data governance and analytics.
Information on a Capgemini virtual evening event and its agenda.
Invitation to attendees to visit Capgemini's virtual booth and engage.
Contact details for Arne Roßmann, Head of AI & Data Engineering at Capgemini.
Overview of Capgemini's services, company profile, and global presence in the consultancy landscape.
More Related Content
Similar to Modern Data Platforms
![[DSC Europe 24] Josip Saban - Buidling cloud data platforms in enterprises](https://cdn.slidesharecdn.com/ss_thumbnails/josipsaban-buidlingclouddataplatformsinenterprises-250217194546-5568421d-thumbnail.jpg?width=640&height=640&fit=bounds)
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Modern Data Platforms
- 1. What does amodern data platform for analytics in the company look like? München, 2021-02-10, Arne Roßmann
- 2. Arne Roßmann Education: - ComputerScience (BSc) with Focus on Business Intelligence - International Management (MBA) Key Competences: - Enterprise Architecture - DevOps / DataOps - Big Data / Cloud Projects / Roles: - Chief Architect - DevOps Coach - Solution Architect - (before: Developer, Scrum Master, Project Manager, …)
- 3. § What arethe characteristics of an Enterprise Data & AI Platform § How does a Reference Architecture for an Enterprise Data & AI Platform look like? § Why Kafka is a perfect key component in the modern platform? § A real world example § Q & A Agenda
- 4. 4 © Capgemini 2021.All rights reserved | Modern Data Platform| Arne Roßmann | 2021-02-10 Enterprise Data & AI Platform Characteristics Discoverable Addressable Trustworthy (defined & monitored SLO) Self Service Inter Operable (governed by open standards) Secure
- 5. Reference Architecture for anEnterprise Data & AI Platform 5
- 6. Concept_Templates_April_2018.pptx System of InsightsApproach 6 NEXT-GEN ENTERPRISE DATA & AI PLATFORM DATA CENTRIC MODELS BUSINESS DATA HUB DATA PROCESSING DATA CURATION STREAMING SOURCES BATCH SOURCES AI & ANALYTICS FOUNDATION SEARCH, AI & ANALYTICS OLAP PLATFORM MANAGEMENT & OPERATIONS PLATFORM SECURITY ACTIVE DIRECTORY ORCHESTRATION KEY VAULT MFA SECURITY CENTER OMS MONITOR STREAM PROCESSING STREAM SERVING STORE UNIVERSAL DATA LAKE DATA TRANSFORMATION DATA TRUST AZURE DEVOPS DATA INGESTION LANDING RAW DATA INGESTION TRANSFORMED DATA INGESTION STREAM INGESTION AI & ANALYTICS EXECUTION INTELLIGENT APPs SELF SERVICE & ENTERPRISE BI REAL-TIME APPs DATA SERVICES APP DATA SERVICES CUSTOM AI APPs AI & ANALYTICS SERVICES Multiple sources converge to a common Data & AI platform using Microsoft Azure centric services Goals: Consistent, repeatable methods and processes that can scale Govern locally where it matters, manage centrally AZURE DATA FACTORY AZURE DATA FACTORY AZURE DATABRICKS AZURE DATABRICKS AZURE DATABRICKS EVENT HUB KAFKA AZURE DATA LAKE AZURE DATA LAKE AZURE DATA LAKE COSMOS DB DATA LAKE SEARCH AZURE ML AAS ADLS POWER BI BOT FRAMEWORK COGNITIVE SERVICES APP SERVICE LOGIC APP FUNCTION SYNAPSE ANALYTICS MGMT & SECURITY AZURE INFORMATION PROTECTION AXON SECURE@SOURCE ENTERPRISE DATA CATALOG POLYBASE AZURE SYNAPSE ANALYTICS API APP
- 7. Kafka as akey component in the architecture 7
- 8. 8 © Capgemini 2021.All rights reserved | Modern Data Platform| Arne Roßmann | 2021-02-10 Event Driven Architecture – an approach to solve the Big Ball of Mud problem https://www.confluent.io/blog/using-apache-kafka-drive-cutting-edge-machine-learning/ https://www.confluent.io/blog/changing-face-etl/
- 9. 9 © Capgemini 2021.All rights reserved | Modern Data Platform| Arne Roßmann | 2021-02-10 Event Driven Architecture – “clean” architecture and AI integration https://www.confluent.io/blog/using-apache-kafka-drive-cutting-edge-machine-learning/ https://www.confluent.io/blog/changing-face-etl/ https://www.confluent.io/blog/importance-of-distributed-tracing-for-apache-kafka-based-applications/
- 10. A real worldexample 10
- 11. 11 © Capgemini 2021.All rights reserved | Modern Data Platform| Arne Roßmann | 2021-02-10 A real world example on Car Distribution https://github.com/arossmann/metrics_exporter
- 12. 12 © Capgemini 2021.All rights reserved | Modern Data Platform| Arne Roßmann | 2021-02-10 Summary & Q&A
- 13. 13 © Capgemini 2021.All rights reserved | Modern Data Platform| Arne Roßmann | 2021-02-10 A Modern Enterprise Data & AI Platform The AI Engineering Data Centricity Platform Reference Architecture Discoverable Addressable Trustworthy (defined & monitored SLO) Self Service Inter Operable (governed by open standards) Secure AI & ANALYTICS EXECUTION DATA CENTRICITY FOUNDATION AI & ANALYTICS FOUNDATION SOURCE DATA PLATFORM FOUNDATION DATA TRUST Architecture & Advisory Modern Enterprise Data & AI Platform AI & ANALYTICS EXECUTION AI & Analytics Execution AI & ANALYTICS FOUNDATIONS DATA CENTRICITY FOUNDATION DATA TRUST Data Engine Virtual Semantic Layer & Caching Data Ingestion Data Landing Zone Raw Data Files Batch Ingestion Streaming Ingestion Data Governance Strategy and Process Reference & Master Data Data Lifecycle Management Data Privacy Data Quality & Profiling Data Discovery & Lineage Data Catalogue Entity Relationship Management Business Glossary Data Lake, Data Hub, EDW Business Data Hub Modeled data store Universal Data Lake Curated Data Store Data Transformation Data Enrichment Data Curation Custom AI Solutions Intelligent Apps AI & Analytics - Data Services Internal Data External Data CRM ERP ANALYTICAL OTHER THIRD PARTY SOCIAL DATA OTHER SENSOR IOT AI Monitoring Data Science Workbench Model Catalogue Analytics Orchestration KPI Catalogue Application - Data Services Traditional BI Reporting Data Centricity - Data Services PLATFORM FOUNDATION Cloud Foundations I&D Platform Foundations & Run Platform Security & Governance Network Connectivity Cyber Security Cloud Infrastructure Platform Management & Operations DevOps/DataOps Automation Hybrid Cloud Services Edge Integration Data Modeling Aggregated Fact APIs Visualization Data Exploration & Search Analytics & Dashboards Self-Serve BI Reporting, BI & Visualization
- 14. 14 © Capgemini 2021.All rights reserved | Modern Data Platform| Arne Roßmann | 2021-02-10 SEC1 © 2020 Capgemini. All rights reserved. 14 Abendveranstaltung mit Capgemini Save the Date und sei bei unserem virtuellen Abendprogramm am 11.02.2021 von 18:00 Uhr bis 20:00 Uhr dabei. Es erwartet dich unter anderem eine Keynote „Coding at Capgemini – Revolution or Evolution for Clients” von unserem Head of Innovation Thilo Hermann. Erhalte spannende Einblicke und Beispiele, wie Capgemini seine Kunden revolutionär aber auch evolutionär mit Software unterstützt. Schnapp dir außerdem einen Drink für ein ausgelassenes Get-Together: lerne dabei unsere Bereiche kennen und vernetze dich mit Capgemini-Mitarbeitern. Du hast Interesse? Dann melde dich bis zum 11.02.2021 unter https://capgemini- events.de/abendveranstaltung_mit_capgemini/ an. Wir freuen uns auf dich!
- 15. 15 © Capgemini 2021.All rights reserved | Modern Data Platform| Arne Roßmann | 2021-02-10 SEC1 © 2020 Capgemini. All rights reserved. 15 Let‘s get in contact and visit us now at our virtual booth/258 here on the OOP! Architecting Our Future
- 16. 16 © Capgemini 2021.All rights reserved | Modern Data Platform| Arne Roßmann | 2021-02-10 Contact Roßmann, Arne Head of AI & Data Engineering Germany, Insights & Data Chief Architect Capgemini Nuremberg Bahnhofstr. 30 90402 Nuremberg, Germany arne.rossmann@capgemini.com https://www.linkedin.com/in/arnerossmann/
- 17. Capgemini is aglobal leader in consulting, digital transformation, technology, and engineering services. The Group is at the forefront of innovation to address the entire breadth of clients’ opportunities in the evolving world of cloud, digital and platforms. Building on its strong 50-year heritage and deep industry-specific expertise, Capgemini enables organizations to realize their business ambitions through an array of services from strategy to operations. A responsible and multicultural company of 265,000 people in nearly 50 countries, Capgemini’s purpose is to unleash human energy through technology for an inclusive and sustainable future. With Altran, the Group reported 2019 combined global revenues of €17 billion. About Capgemini Learn more about us at www.capgemini.com This presentation contains information that may be privileged or confidential and is the property of the Capgemini Group. Copyright © 2020 Capgemini. All rights reserved.