Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SAP HANA SPS09 - Dynamic Tiering

6,316 views

Published on

SAP HANA SPS09- What's new? SAP HANA Modeling

Published in: Technology
  • For SAP online training register at http://www.todaycourses.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

SAP HANA SPS09 - Dynamic Tiering

  1. 1. 1 ©2014 SAP SE or an SAP affiliate company. All rights reserved. SAP HANA SPS 09 - What’s New? HANA Dynamic Tiering SAP HANA Product Management November 2014 (Delta from SPS 08 to SPS 09)
  2. 2. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 2 Public Disclaimer This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP’s strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.
  3. 3. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 3 Public Agenda Positioning What is “SAP HANA Dynamic Tiering”, and what is its value to the customer? Technical Details Implementation choices Use Cases SAP BW and native HANA applications Future Direction Where are we headed?
  4. 4. Positioning What is “SAP HANA Dynamic Tiering”, and what is its value to the customer?
  5. 5. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 5 Public IDC predictions for 2014 Data explosion Data volumes will continue to explode to 6 billion petabytes Social networking Social networking will become embedded in cloud platforms and most enterprise apps and processes Cloud Cloud spending will surge by 25%, reaching over $100 billion. There will be a doubling of cloud data centers. Internet of Things 30 billion devices, sensors in 2020 – driving $8.9 Trillion in revenue Mobile CRM Data Planning Opportunities Transactions Customer Sales Order Things Instant Messages Demand Inventory Big Data Sales Order Things Mobile Demand Big Data CRM Data Customer Planning Transactions
  6. 6. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 6 Public SAP End to End Data Management for Real Time Business Business & Consumer Applications Big Data SAP DATA MANAGEMENT STORE TRANSACT PREDICT ANALYZE Custom Development ISVs & OEMs ERP Internet of Things Workforce of the Future Cloud Industries
  7. 7. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 7 Public e SAP HANA platform Processing Engine Application Function Lib. & Data Models Integration Services SAP HANA PLATFORM Real-time transactions + end-to-end analytics Operational Analytics Big Data Warehousing Predictive, Spatial & Text Analytics REAL-TIME ANALYTICS Sense & Respond Planning & Optimization Consumer Engagement REAL-TIME APPLICATIONS SAP ESP SAP ASE Replication Server SAP SQL Anywhere SAP IQ SAP Data Services Extended Application Services SAP Data Management Portfolio End-to End Data Management & App Platform for Real-Time Business Database Services SAP HANA dynamic tiering
  8. 8. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 8 Public Time Value of Data Time Value Last time accessed Value of immediate data access declines When you need it again Archive Access Event •Regulatory audit •Business critical reference data •Source data
  9. 9. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 9 Public Multi-Temperature Storage Options with SAP HANA Data Temperature Storage Option SAP BW on HANA SAP Business Suite on HANA SAP HANA Native hot SAP HANA In-Memory    cold SAP HANA dynamic tiering (1)  (2) Data Aging (Next Gen ILM)  3  Near-line Storage (NLS)    frozen Data Archiving (ADK)     Generally available  Combination not available 1 Early shipment available for SAP BW 7.4; General availability planned Q4/2014 2 General availability with limited scope planned Q4/2014 3 For selected business objects
  10. 10. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 10 Public SAP HANA Dynamic Tiering Key aspects at a glance Add-on Product to SAP HANA Manage data of different temperatures Hot data (always in memory) – classical HANA Cold data (disk based data store) Introducing a new type of table: Extended table – disk-based columnar table SPS 9 release focus Operational integration Common installer Unified monitoring and administration Integrated backup/recovery Initial functional scope Transparent query processing Cross-store optimizer Use extended table in calculation views Applications manage data temperatures (no active support for aging) SAP HANA Database Data for daily reporting, other high-priority data Other data required to operate the application Hot Warm
  11. 11. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 11 Public Introducing SAP HANA Dynamic Tiering Requirements from our customers Manage data cost effectively, yet with desired performance based on SLAs Handle very large data sets – terabytes to petabytes Update and query all data seamlessly via HANA tables Application defines which data is “hot”, and which data is “warm” Native Big Data solution to handle a large percentage of enterprise data needs without Hadoop SAP HANA hot store (in-memory) SAP HANA warm store (dynamic tiering) Extended table (definition) Extended table (data) Fast data movement and optimized push down query processing All data of extended table resides in warm store SAP HANA Database System Hot table (definition/data)
  12. 12. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 12 Public Hot/Warm Data Management Questions about SAP HANA Dynamic Tiering Size and cost constraints may prohibit all in-memory solution Not all data has the same value Warm data has lower latency requirements than hot data Why is warm data management important for SAP HANA? SAP HANA dynamic tiering utilizes disk backed, smart column store technology based on Intellectual Property from SAP Sybase SAP HANA dynamic tiering excels at ad hoc queries on structured data from terabyte to petabyte scale SAP HANA dynamic tiering is a deeply integrated, high performance solution in a single system Why is SAP HANA dynamic tiering the best solution for warm data management? Hadoop has unlimited capacity for raw data processing Hadoop is best suited for batch processing of raw, unstructured data Hadoop is an external data store with technical integration into HANA – with higher TCO in order to manage the additional system What about Hadoop for warm data storage and processing?
  13. 13. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 13 Public SAP HANA Dynamic Tiering Key aspects at a glance Data in the database Different data temperatures Maximum access performance Hot data - always in memory Reduced access performance: Warm data - not (always) in memory All part of the database’s data image Data moved out of the database Different data qualities Available for read access BW Near-line storage Not accessible without IT process Traditional archive Data is stored and managed outside of the application database SAP HANA Database Data for daily reporting, other high-priority data Other data required to operate the application Hot Warm NLS Data that is (normally) not updated, infrequently accessed Traditional Archive Data that‘s kept for legal reasons or similar Externalize
  14. 14. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 14 Public Problems with temperatures There are too many options – across system boundaries In DB In memory No restrictions, all features available External to DB Near-line Storage Read access, no updates In DB On disk No restrictions, all features available hot warm cold ??? External to DB Archive storage No read access or updates Performance and Price Priority and Data Volume HANA Archive HANA column and row store Warm store of dynamic tiering / Non-Active Data Concept BW Near-line Storage Traditional Archive
  15. 15. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 15 Public Problems with temperatures There are too many options – across system boundaries In DB In memory No restrictions, all features available External to DB Near-line Storage Read access, no updates In DB On disk No restrictions, all features available hot warm cold ??? External to DB Archive storage No read access or updates Performance and Price Priority and Data Volume HANA column and row store Warm store of dynamic tiering / Non-Active Data Concept BW Near-line Storage Traditional Archive hot warm BW NLS Archive
  16. 16. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 16 Public SAP HANA Dynamic Tiering Map data priorities to data management Hot Store Classic HANA tables Primary data image in memory DB algorithms optimized for in-memory data Persistence on disk to guarantee durability Warm Store Extended Tables Primary data image on disk Data processing using algorithms optimized for disk-based data Main memory used for caching and processing. SAP HANA Database Primary image in memory Durability Cache / Processing Primary Image on disk Dynamic Tiering Hot data Warm data All in one database Hot Store Warm Store RAM
  17. 17. Technical Details Implementation choices
  18. 18. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 18 Public SAP HANA Dynamic Tiering – one database / one experience for HANA application developers and admins SAP HANA Dynamic Tiering Reduced TCO Optimized for performance Single database experience Centralized operational control Centralized monitoring / admin High speed data ingest Common installer and licensing model Unified backup and restore Integrated security Optimized query processing SAP HANA Dynamic Tiering
  19. 19. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 19 Public SAP HANA Dynamic Tiering The overall system layout SAP HANA with Dynamic Tiering consists of two types of hosts: Regular worker hosts (running the classical HANA processes: indexserver, nameserver, daemon, xsserver,…) –HANA hosts can be single-node or scale-out; appliance or TDI “ES host” (running nameserver, daemon, and esserver) –esserver is the database process of the warm store One single SAP HANA database: one SID, one instance number All client communication happens through index server / XS server Hot Store Fast data movement and optimized push down query processing SAP HANA System with dynamic tiering service Worker host(*) Worker host Worker host Client Application Connect ES host Column Table Row Table Extended Table Warm Store Common Storage System (*) Standby hosts not shown
  20. 20. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 20 Public Database Catalog HANA Extended Tables HANA Database Warm Store Data HANA extended table schema is part of HANA database catalog HANA extended table data resides in warm store HANA extended table is a first class database object with full ACID compliance Hot Store Table Definition Data Table Definition Classical HANA column/row table Extended table (warm table)
  21. 21. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 21 Public High Speed Data Ingest Import from CSV files: IMPORT FROM CSV FILE ‘bigfile.csv’ INTO t1 Bulk array insert: INSERT INTO t1 (col1, col2, col3...) VALUES (val1, val2, val3...) High-speed data movement between HANA tables and HANA extended tables: INSERT INTO t_extended select c1 FROM t_hana Concurrent inserts from multiple connections: A HANA extended table may be a DELTA enabled table, which allows multiple concurrent writes Warm Extended Table IMPORT FROM CSV FILE ‘data.csv’ INTO t_extended CSV DATA Hot HANA column Table INSERT...SELECT Materialization Data movement between hot and warm store HANA Database
  22. 22. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 22 Public Optimized Query Processing Parallel query processing •Data is pulled from HANA hot store into HANA warm store query processing engine using multiple streams, and processed in parallel Push/Pull query optimization and transformation •Query operations ship to hot or warm store as appropriate for native performance Extended tables may be used in HANA CALC views •HANA Calc engine and HANA SQL engine share extended table query performance optimizations Joining Grouping Ordering T3 T4 T1 T2
  23. 23. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 23 Public Example Query Plan select "account_num", count(*) as account_count from VXM_FOODMART.CUSTOMER C where "lname" >= 'Ga' and "lname" < 'Gb' and exists ( select * from VXM_IQSTORE.PRODUCT P where "product_id" = "customer_id" ) group by "account_num" order by "account_num"; Customer is a native HANA table in HANA memory Product is a HANA extended table in the warm store
  24. 24. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 24 Public HANA Monitoring and Administration HANA Cockpit: New, web based monitoring and administration console for HANA Extended Storage HANA Studio will be used for design and modeling of HANA extended tables HANA Cockpit displays status, CPU/memory/storage resource utilization, table usage statistics Provides access to and search of server logs and custom traces Shows alerts triggered by extended storage Enables administration of extended storage: add and drop storage, or increase size of file
  25. 25. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 25 Public Unified Backup and Restore HANA backup manages backup of both hot and warm store Point in Time Recovery (PITR) is supported Extended Storage HANA Data backups (manual or scheduled) Log backups (automatic, or none) Data backup Log backup System crash Restore Time t1 t2 t3 Data backups with log backups allow restore to Point in Time or most recent state: t1- > t3 Data backups alone allow restore to specific backup only: t1 or t2 Log area Backup History
  26. 26. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 26 Public High Availability and Disaster Recovery High availability Compute node failure will result in failover to standby node (manual for warm store nodes) Storage failure will depend on inherent storage vendor disk mirroring and fault tolerance capabilities Hot and warm store should use the same storage to facilitate auto-failover in the future Disaster recovery HANA without Dynamic Tiering supports continuous replication to maintain a disaster recovery site HANA with Dynamic Tiering will maintain a disaster recovery site through backup and restore capabilities only –Disaster recovery through system replication is planned for a future release –Disaster recovery through storage replication may be added independently from software releases Classical HANA services Compute node Hot Store Warm Store Service Compute node Standby node Manual Failover Standby node Warm Store Auto- Failover mirror mirror
  27. 27. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 27 Public Support in SAP HANA multiple database containers (MDC) MDC: One SAP HANA system can have multiple tenant databases Each tenant database can be associated with zero or one extended stores Each extended store is dedicated to exactly one tenant database SAP HANA system with MDC and dynamic tiering Compute node System Database Compute node Compute node Tenant Database <B> Extended Store Tenant Database <A> Tenant Database <C> Extended Store Classical HANA (single-node or scale-out) ES Host <B> ES Host <C>
  28. 28. Use Cases SAP BW and native HANA applications
  29. 29. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 29 Public SAP NetWeaver BW powered by SAP HANA Data Classification by Object Type Frequent reporting and/or HANA-native operations BW – Operational Data Data Categories in a BW System Staging Layer Analytic Mart Business Transformation EDW Propagation EDW Transformation Corporate Memory Archive/NLS “Old”, “out-of-use” data – Archive, read-only, different SLAs Limited reporting, limited HANA-native operations Archived
  30. 30. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 30 Public SAP HANA database Database Catalog Extended Tables in HANA BW Use Case: Staging and Corporate Memory Object Classification in BW Data Sources and write-optimized DSOs can have the property “Extended Table” Generated Tables are of type “Extended” All BW standard operations supported – no changes Only minor temporary RAM required in HANA InfoCubes and Regular or Advanced DSOs Generate standard column table Hot Store Warm store BW System Corporate Memory Write-optimized DSO Staging Area Data Source Table Schema Data PSA Table Table Schema Data Active Table Data Mart InfoCube Table Schema Data Fact Table
  31. 31. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 31 Public SAP HANA Dynamic Tiering for Big Data Cutting edge, in-memory platform Transact/analyze in real-time Native predictive, text, and spatial algorithms Petascale extension to HANA with disk backed, columnar database technology Expand HANA capacity with warm/cool structured data in HANA warm store Tight integration between HANA hot store and HANA warm store for optimal performance SAP HANA with Dynamic Tiering provides native Big Data solution Hot data SAP HANA Petascale, warm structured data HANA extended tables
  32. 32. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 32 Public SAP HANA with Dynamic Tiering Native Big Data solution for a multitude of use cases SAP HANA Dynamic Tiering for Big Data Use Cases across Industries Airline route profitability analysis: SAP HANA analyzes revenue, variable operating costs (fuel, landing fees...), and fixed operating costs in real time to make decisions on network, pricing, and marketing to determine where to fly, when, and how often. All data must be analyzed in real time. Financial services: Stock tick data streamed into SAP HANA for immediate price fluctuation analysis and trading actions, with historical stock price data stored in HANA extended tables for trend analysis and portfolio management. Telecommunications: Network service data in HANA extended tables analyzed and correlated with customer loyalty data in SAP HANA, to anticipate customer churn and initiate customer retention response activities. Public utilities: enterprise data stored in SAP HANA and large amounts of smart meter data stored in HANA extended tables, to identify operational problems, and establish incentive pricing for more efficient energy use.
  33. 33. Future Direction Where are we headed?
  34. 34. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 34 Public SAP HANA Dynamic Tiering roadmap SAP HANA dynamic tiering available to be used by any HANA application (if the application supports the feature) Common installer Unified administration and monitoring using HANA Cockpit Extended Storage (ES) engine is part of HANA topology Single authentication model Single licensing model Combined error log / trace handling Integrated File-based backup/recovery, including point-in time recovery HANA ES host scale-out and auto-failover (HA) Disaster Recovery (SAP HANA system replication) Further integration with respect to backup/recovery Hybrid extended tables with rule based automatic data movement / aging Optimization of communication between hot and warm store Further unification of DDL and DML for HANA extended tables Further optimizer enhancements Further extension of unique HANA capabilities to warm store FUTURE PLANNED
  35. 35. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 35 Public Hybrid extended tables Automatic, rules-based, asynchronous data movement between hot and warm stores Hot partitions in HANA memory; remaining partitions in warm store Single HANA table that spans hot and warm stores Hot data in HANA tier Warm data In warm tier 2012 2012 Hybrid Extended Table aging regulatory audit
  36. 36. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 36 Public How to find SAP HANA documentation on this topic? •In addition to this learning material, you can find SAP HANA platform documentation on SAP Help Portal knowledge center at http://help.sap.com/hana_platform. •The knowledge centers are structured according to the product lifecycle: installation, security, administration, development: SAP HANA Options SAP HANA Advanced Data Processing SAP HANA Dynamic Tiering SAP HANA Enterprise Information Management SAP HANA Predictive SAP HANA Real-Time Replication SAP HANA Smart Data Streaming SAP HANA Spatial •Documentation sets for SAP HANA options can be found at http://help.sap.com/hana_options: SAP HANA Platform SPS What’s New – Release Notes Installation Administration Development References •
  37. 37. ©2014 SAP SE or an SAP affiliate company. All rights reserved. Thank you Contact information Richard Bremer, Courtney Claussen, Balaji Krishna, and Robert Waywell SAP HANA Product Management AskSAPHANA@sap.com
  38. 38. ©2014 SAP SE or an SAP affiliate company. All rights reserved. 38 Public © 2014 SAP SE or an SAP affiliate company. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices. Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward- looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

×