Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

To The Cloud and Back: A Look At Hybrid Analytics

424 views

Published on

To The Cloud and Back: A Look At Hybrid Analytics

Published in: Technology
  • Be the first to comment

  • Be the first to like this

To The Cloud and Back: A Look At Hybrid Analytics

  1. 1. To The Cloud and Back: A Look at Hybrid Analytics Keith Manthey
  2. 2. © Dell EMC 2016 - All rights reserved2 ITIL-based IT processes Client-server scale-up apps Infrastructure resiliency Traditional CIO challenge: next-generation infrastructures are needed
  3. 3. © Dell EMC 2016 - All rights reserved3 ITIL-based IT processes Client-server scale-up apps Infrastructure resiliency Cloud-native Coexisting IT paradigms Traditional DevOps based IT processes Distributed scale-out apps Application resiliency CIO challenge: next-generation infrastructures are needed
  4. 4. © Dell EMC 2016 - All rights reserved4 Traditional Cloud-native Modern IT Off-Premises On-Premises Available both on and off premises
  5. 5. CLOUD ENABLED STORAGE DATALAKE DATACENTER
  6. 6. CLOUD ENABLED STORAGE DATALAKE DATACENTER Through 2020, IDC predicts 4.5x faster spending on cloud based big data analytics then on-premise solutions By 2020, IDC predicts that usage of big data analytics for actionable intelligence will double over today On premise and Cloud use of Hybrid Analytics will become a dominant use of the technology More specifically – Moving data from On-Premise to the Cloud and Back
  7. 7. The journey to here
  8. 8. © Dell EMC 2016 - All rights reserved8 Data Management DATA LAKE SOLUTION FOR EDW MODERNISATION Clickstream Web & Social Geolocation Sensor & Machine Server Logs EXISTINGSOURCES ERP CRM Commodity Compute DATA SERVICES OPERATIONAL SERVICES HORTONWORKS DATA PLATFORM HADOOP CORE Business Analytics Visualization & Dashboards IT Applications NEWSOURCES 2 3 1 ETL/ELT OFFLOAD ACTIVE ARCHIVE ENRICH WITH NEW DATA TYPES MULTI-PROTOCOL ACCESS ENTERPRISE-GRADE DATA MANAGEMENT 5 NFS, SMB, HTTP, Swift 1 2 3 4 5 4 New Data Flow Current Data Flow Legend OFFLOAD Isilon
  9. 9. © Dell EMC 2016 - All rights reserved9 1. Active Archive – Optimise Enterprise Data Warehouse storage by archiving cold data and still analyse it as needed 2. ETL Offload – Improve EDW performance by offloading ETL processing to Hadoop 3. Semi/Unstructured Data Analytics – Increase confidence in business decisions with new data sources 4. Multi-protocol Access – Enable applications to access/update Hadoop data using NFS, SMB, HTTP, Swift and other file/object based access methods 5. Data Management – Enterprise-grade data management at Hadoop economics DATA LAKE BENEFITS Unique to Isilon
  10. 10. © Dell EMC 2016 - All rights reserved10 ISILON MOMENTUM 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 • >12,000 Clusters in the Field • Approaching 1PB Average Cluster Capacity • 1.3EB of Hadoop Usage • 7,000+ Customers World Wide • 1,300 New Customers in 2015 • #1 in Data Lakes >1,200 HDFS Customers Scale-Out NAS Leader
  11. 11. A look forward
  12. 12. Ability to: • Enact Policy Based Data Movement • Migrate data from On-Prem to Cloud and vice versa • Leverage Hybrid Analytics to greatest Effect To The Cloud and Back
  13. 13. Limits: 100 Gb/Sec 1 Petabyte (1 million gigabytes) ~1 Day to transfer one way To The Cloud and Back
  14. 14. Data Silos OnPremOnlyfiles CloudOnlyfiles Strong Bandwidth Typically Hand Scripted Expensive Extraction Limit outbound data movement
  15. 15. Our Research
  16. 16. Problem Domain 1. File movement automation in place today is labor intensive and fraught with peril 2. Non-prescriptive file movement out of the cloud will be extremely expensive with limited value in return. 3. Most file movements might not return to exactly the same target location. For example: Location 1, File 1 Location 2, File 1 – begets File 2 Location 3, File 2
  17. 17. What if…. 1. There was a way to move files by policy (i.e., rules based) to various locations, including the cloud and back 1. For Example – only target net new files created in the cloud in certain directories for movement. 2. The rules based file movement could allow for multiple targets and / or destinations for files. 1. For Example – move net new files in one directory to a single argetand net new files in a second directory to three targets.
  18. 18. OneFS in the Cloud
  19. 19. Our Journey to Here 1. OneFS already is a software defined asset. 1. SDEdge is a software defined storage offering 2. CloudPools is a software feature in SDEdge that uses the Public Cloud (Azure, AWS, Google) as a back up target 2. OneFS has been loaded and run in the public cloud as noted in the previous slide. 3. Dell EMC has a long history of data mobility across our product suite (replication, backups, etc) 4. Dell EMC has a long history of policy based file movement features.
  20. 20. Futures 1. We are exploring what OneFS with all its features and abilities to move data up to the cloud and back would look like. 2. What about a cloud environment that contained a OneFS daemon that allowed policy based file movement.? 3. The cloud environment file movement (up and back) could be controlled by a Isilon cluster
  21. 21. On-Prem Files Movement policies Future State? Cloud Files OneFS Daemon

×