Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

What’s your Data Fabric Strategy for AI in a Multi-Cloud-World?


Published on

Modern workloads no longer run within the confines of the Data Centre – they operate across Data Fabrics. Use cases such as AI not only require fast and dynamic performance and scaling, but also agility and rapid deployment options. Data supporting these workflows is the most important asset and is becoming increasingly diverse, dynamic and distributed across the Edge, Core and Multi-Cloud - implementing a Data Fabric strategy is imperative. In this workshop we highlight how AI is underlining the need for a Data Fabric, and will demonstrate how NetApp can help address performance and agility requirements through technologies integrating containers and multi-cloud environments.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

What’s your Data Fabric Strategy for AI in a Multi-Cloud-World?

  1. 1. What’s your Data Fabric Strategy for AI in a Multi-Cloud-World? Gerhard Wolf AI Solution Architect, NetApp
  2. 2. Edge Core Cloud – Multi Cloud 4 © 2019 NetApp, Inc. All rights reserved
  3. 3. Data and Data Services Simplifying and integrating the orchestration of data services across a choice of hybrid, multi-cloud environments 5 © 2019 NetApp, Inc. All rights reserved Distributed Dynamic Diverse Secure Efficient Future-proof Freedom of choice
  4. 4. Edge Core Cloud – Multi Cloud 6 © 2019 NetApp, Inc. All rights reserved CloudCoreEdge NETAPP Data Fabric
  5. 5. Workloads Data pipeline spanning Edge, Core, and Cloud 7 AI Analytics 1 2 3 Cloud Services © 2019 NetApp, Inc. All rights reserved CloudCoreEdge Enterprise Apps General Apps
  6. 6. Operate across Data Fabrics Modern Workloads 8 © 2019 NetApp, Inc. All rights reserved
  7. 7. Engineered Storage Systems Converged Networked Storage Traditional Systems Business Enterprise Business Apps File Services The effect of transformation is very broad A Data Fabric Strategy is essential 9 Time VolumeofData(Exabyte) Webscale Cloud-native Containers Core + Edge Software Defined Storage Hyperconverged Flash / SCM Technologies Infrastructure Emerging technologies will redefine applications … ...posing unique requirements on IT infrastructure Emerging Software Business Business Insights IoT AI AR / VR Surveillance Autonomous Vehicles Genomics © 2019 NetApp, Inc. All rights reserved
  8. 8. Transforming IT Along Multiple Dimensions Cloud is a design pattern for IT; not just a data center destination 10 1990-2005 2005-2015 2015-Future Fast Microservices Cont. Integration/ Delivery Containers Private, Public, Multi-Cloud Diverse including raw Distributed, Data Fabric Medium 3/N-Tier Agile Virtual Machines Private Cloud Structured, semi-struct. Centralized Data Lake Slow Monolithic Waterfall Bare Metal Data Center Structured, processed Centralized Data Warehouse Time to new capabilities Application Architecture SW Dev Methodology Deployment & Packaging Application Infrastructure Data Types Data Infrastructure “Traditional” “Virtualized” “Cloud”Design Pattern © 2019 NetApp, Inc. All rights reserved
  9. 9. Emerging models are building from the traditional data center Everything is becoming defined by Data! 11 Emerging areas where IT can run apps DEDICATEDSHARED ON-PREM OFF-PREM 1 Outsource on similar infrastructure 2 Rent the app as a service 3 Shift between private and public cloud 4 Build internal private cloud 5 Micro-services based infrastructure Internal Private PaaS Hyperscale PaaS IaaS SaaS 1 2 3 4 5 6 6 Visibility and Insights across the Hybrid environment © 2019 NetApp, Inc. All rights reserved
  10. 10. Core Capabilities That IT Must Now Deliver 12 © 2019 NetApp, Inc. All rights reserved Configuration Management Consistent environments CI/CD A secure streamlined development experience Cloud and PaaS Data access without infrastructure management Containers Application portability AI + Analytics Actionable insights into environments Code and Binary Management Fast, real-time data access
  11. 11. AI and Data 13 © 2019 NetApp, Inc. All rights reserved The ”Unreasonable Effectiveness of Data” – Peter Norvig (Google)
  12. 12. Edge to Core to Cloud Data pipeline spanning Edge, Core, and Cloud 14 Ingest Data prep. Unified data lake Training cluster Training sets Test Deployment IM3 IM2 IM1 Repo Analysis / Tiering • Aggregation • Normalization • Transformation • Data collection • Edge-level AI / Inferencing • (Training) • Exploration • Training • Inferencing • Deployment • Model serving • Data tiering • Cloud AI (GPU instances) • Data tiering • Data sync & burst • Aggregation • Inferencing • Training CloudCoreEdge Cloud AI GPU-aaS / Tiering© 2019 NetApp, Inc. All rights reserved 1 2 3
  13. 13. ONTAP driven AI 15 CloudCoreEdge © 2019 NetApp, Inc. All rights reserved
  14. 14. 16 Proven Architecture High utilization by low latency ONTAP as the multi protocol high efficiency storage platform Simple to Deploy Ansible Automation DevOps DataOps AIOps NFS easiness Performance and Scalability From small till 300 GB/s Throughput From small to several multiple AI compute cluster Integrated Data Pipeline Edge to core to cloud End-to-end data management VM or Kubernetes driven ONTAP driven AI Smart, Powerful, Trusted © 2019 NetApp, Inc. All rights reserved
  15. 15. A Platform for Innovation 17 ▪ Flexible ▪ Intelligently manage data and compute across edge, core & cloud ▪ Deploy AI Frameworks with confidence ▪ Powerful ▪ DLU optimized compute with massively scalable flash ▪ Start small and grow non-disruptively - Scale without limits © 2019 NetApp, Inc. All rights reserved NetApp Storage and ZDLS - a perfect combination
  16. 16. AFF A800: The World’s Fastest Flash for AI/DL Ultra low latency and massive throughput for your deep learning applications 18 500µs latency 25GB/s throughput 11.4M IOPS 300GB/s throughput; 4xhigher than competitor 1M IOPS 24-node Cluster raw capacity 79PB raw capacity * With high-performance NFS© 2019 NetApp, Inc. All rights reserved
  17. 17. Start Small, Scale Big Flexibility and future proof 19 * 1 AFF = 1 HA pair = 2 Nodes ** Based on ONTAP 9.4, NAS workloads Throughput ** Raw Capacity (Typical) Raw Capacity (w/ Expansion) Connectivity AFF A800 1 HA pair * 25GB/s 364.8TB 6.6PB 100GbE 40GbE 4 HA pairs 100GB/s 1.5PB 26.4PB 12 HA pairs 300GB/s 4.4PB 79.2PB AFF A700s 1 HA pair 18GB/s 367.2TB 6.6PB 40GbE4 HA pairs 72GB/s 1.5PB 26.4PB 12 HA pairs 216GB/s 4.4PB 79.2PB AFF A320 1 HA pair 10.7GB/s 367TB 734TB 100GbE 40GbE 4 HA pairs 42.8GB/s 1.5TB 3PB 12 HA pairs 128GB/s 4.4PB 8.8PB AFF A220 1 HA pair 5.3GB/s 91.2TB 4.4PB 10GbE4 HA pairs 21.2GB/s 364.8TB 17.6PB 12 HA pairs 63.6GB/s 1.1PB 52.8PB © 2019 NetApp, Inc. All rights reserved
  18. 18. ONTAP FlexGroup: A Single Namespace for the Unified Data Lake 20 Up to 400 billion files, 20 PB Handles 1 billion file additions per month Eliminated need to frequently provision storage 20% reduction in running time for Spark jobs © 2019 NetApp, Inc. All rights reserved
  19. 19. Cloud Bursting On-premises origin volume data cached to Cloud Volumes ONTAP Cloud Acceleration Cloud Volumes ONTAP origin volume data cached on-premises Cloud-to-Cloud Caching CVO origin data cached in the same cloud or a different cloud Hybrid Cloud Caching with FlexCache ONTAP 9.6 21 FlexCache with ONTAP 9.6 enables caching to and from Cloud Volumes ONTAP volumes Data Center O CCVO O CCVO CVO OCVO C © 2019 NetApp, Inc. All rights reserved
  20. 20. The History of NetApp Contributions to Containers 22 © 2019 NetApp, Inc. All rights reserved 2013 2016 2019 DOCKER Software container management platform that automates application deployment TRIDENT IS BORN The first external provisioner for Kubernetes CVS & APP MGMT. NetApp launches NFS services for top-3 clouds; Application mgmt. for Kubernetes KUBERNETES Automating deployment, scaling and management of containers and applications NKS AND CVS NetApp launches managed Kubernetes services 2014 2018 2015 NDVP NetApp Docker volume plugin
  21. 21. From DevOps to DataOps Artifact/ Binary Package Manager DATA MANAGEMENTINFRASTRUCTURE AUTOMATION REST APIs Define Code Build Test QA Deploy UAT Release 23 © 2019 NetApp, Inc. All rights reserved Your Data Fabric
  22. 22. Model Versioning and Auto Tiering Reproducibility, Shareability and Data Provenance 24 model.pkl data.csv Old Models and Data Shared Data lake for Training models logs 10101 11001 data.jpg 10101 11001 model.h5 10101 11001 data Multiple Data-scientists checkout large data files simultaneously Model and Data versions Snapshot © 2019 NetApp, Inc. All rights reserved
  23. 23. AI driven NetApp 25 CloudCoreEdge © 2019 NetApp, Inc. All rights reserved
  24. 24. NetApp Active IQ Actionable intelligence for optimal data management 26 Continuous Value from NetApp Support contract ▪ 300,000 assets create over 200 billion data points each day ▪ Logs / Performance Counters / Significant Events / Configurations ▪ Multi-petabyte data lake processes nearly 10 trillion data points each month ▪ Advancing towards AI-driven best next action recommendations and full stack analytics / ServiceNow © 2019 NetApp, Inc. All rights reserved
  25. 25. Hybrid, Multi-Cloud Architecture for Big Data and AI 27 © 2019 NetApp, Inc. All rights reserved. NetApp In Place Analytics Module
  26. 26. Active IQ Workflows Powered by AI 28 © 2019 NetApp, Inc. All rights reserved. Active IQ Face Active IQ Brain On the road Automatically inferring risks (rules) NLP applied to docs Predictive models for part failures Deep Learning Prescriptive Ops Deep Learning Predicting the consequences of risks Associative Rule Mining K-means clustering Accurate sizing of storage Machine Learning Ideal configuration NLP + Machine Learning Forecasts capacity and headroom Regression Dexter: root cause analysis of customer cases BOD + inferencing old telemetry Advanced workload detection Tagging + Heuristics
  27. 27. Monitor, Optimize and Secure Cloud Insights 29 © 2019 NetApp, Inc. All rights reserved.
  28. 28. Always-on Privacy and Compliance Controls Cloud Compliance 30 © 2019 NetApp, Inc. All rights reserved.
  29. 29. NetApp program in India/Bangalore StartUps 31 © 2019 NetApp, Inc. All rights reserved.
  30. 30. NetApp’s AI Award Wins 32 © 2019 NetApp, Inc. All rights reserved. Best “Corporate Innovation in AI” ▪ AIconics said NetApp’s AI solutions supported innovation throughout the corporation by mastering data ▪ The world’s only independently judged awards celebrating the drive, innovation and hard work in the international AI community “The brightest minds who are eager to find their next move in the industry.” Best “AI Solution for Big Data” ▪ With over 1,500 nominations, Tech Breakthrough said ONTAP AI deserved the top spot among products in the AI industry today ▪ Honors excellence and recognizes the creativity, hard work and success of companies, technologies and products in the field of AI “Recognizing the best companies, technologies, products and services in the field of Artificial Intelligence.”
  31. 31. The Data Fabric 33 © 2019 NetApp, Inc. All rights reserved
  32. 32. 34 © 2019 NetApp, Inc. All rights reserved Define your rules so the fabric can take care of itself Discover Activate Optimize Automate Understand the state of your data, applications, systems & services Stitch everything together to gain control Continually understand the state of your fabric and react Data Fabric
  33. 33. 35 © 2019 NetApp, Inc. All rights reserved Data Fabric Cloud Services on NetApp HCI ONTAP AI StorageGRID Cloud Analytics Cloud Controls Data Protection Public Cloud System Analytics Application Controls Data Placement Private Cloud Cloud Insights Cloud AI Cloud Backup SaaS Backup Active IQ NetApp Kubernetes Service Cloud Tiering Cloud Sync Cloud Manager Cloud Orchestrator Azure NetApp Files Cloud Volumes Service Cloud ONTAP Simplify, Unify, and Integrate Your Data Fabric
  34. 34. Key Takeaways ▪ NetApp is a data and AI driven company, delivers AI based insights and services to you ▪ NetApp’s Data Fabric unifies Data Management across distributed resources ▪ It gives you easy access to protected live data for shared planning and analysis around the globe ▪ NetApp is a strong and innovative partner well-founded by his ecosystems of partnership and cooperation ▪ NetApp delivers you a seamless data pipeline from Edge-to-Core-to-Cloud 36 © 2019 NetApp, Inc. All rights reserved
  35. 35. Q&A