Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Bring your SAP and Enterprise Data to Hadoop, Apache Kafka and the Cloud

1,111 views

Published on

The world’s largest enterprises run their infrastructure on Oracle, DB2 and SQL and their critical business operations on SAP applications. Organisations need this data to be available in real-time to conduct necessary analytics. However, delivering this heterogeneous data at the speed it’s required can be a huge challenge because of the complex underlying data models and structures and legacy manual processes which are prone to errors and delays.

Unlock these silos of data and enable the new advanced analytics platforms by attending this session.

Find out how to:
• To overcome common challenges faced by enterprises trying to access their SAP data
• You can integrate SAP data in real-time with change data capture (CDC) technology
• Organisations are using Attunity Replicate for SAP to stream SAP data in to Kafka

Published in: Technology
  • Be the first to comment

Bring your SAP and Enterprise Data to Hadoop, Apache Kafka and the Cloud

  1. 1. Bring Your SAP and Enterprise Data to Hadoop, Apache Kafka and the Cloud Lessons from Fortune 100 Companies about Data Ingestion Ted Orme – VP Technology EMEA Ted.Orme@Attunity.com
  2. 2. CONFIDENTIAL INFORMATION © 2017 Attunity Technology Investment Areas 2016 2015 2014 BI/Analytics 1 1 1 Cloud 2 3 5 Mobile 3 5 3 Digitization/Digital Marketing 4 6 7 Infrastructure and Data Center 5 2 2 ERP 6 4 4 Security 7 7 7 Industry Specific Applications 8 10 9 CRM 9 9 10 Networking/Voice-Data Comm’s 10 8 6 Top CIO Priorities: Sources: Gartner CIO Surveys 2013-2015 Big Data adoption Migration & Analytics
  3. 3. Data Integration for Analytics and IoT
  4. 4. CONFIDENTIAL INFORMATION © 2017 Attunity • Corporate Data Lake Initiative for Real-time Compliance • Reduce fraudulent activity, reputational damage Global Bank • IoT Data Lake for New Analytics • Predictive Maintenance Global Auto Maker • IoT Data Lake integrated with SAP data • Improve food production efficiencies and fleet maintenance Global Food Processing • Data Lake for real-time access to online and legacy data • Increase revenues with dynamic policy changes Global Insurance • Data Lake for Design Analytics with Real Time Data • Expects 3x improvement in manufacturingAirline Manufacturer Examples – Fortune 100 Customers Building Data Lakes for Analytics and IoT
  5. 5. CONFIDENTIAL INFORMATION © 2017 Attunity • 100s to 1000s of Data Sources • Business and Machine data Analyze Everything • On-premise or in the Cloud • In DB, DW, Hadoop, In-Memory, etc. Analyze Anywhere • Capture new/changing data • Process/stream in motion Analyze in Real-time • Capture new/changing data • Process/stream in motion Analyze in Real-time ‘Be Prepared’ – build architecture so you can:
  6. 6. CONFIDENTIAL INFORMATION © 2017 Attunity Transfer TransformFilterBatch CDC Incremental In-Memory File Channel Batch Attunity Replicate Hadoop Files RDBMS Data Warehouse Mainframe Cloud On-prem Cloud On-prem Hadoop Files RDBMS Data Warehouse Kafka Management Automation
  7. 7. CONFIDENTIAL INFORMATION © 2017 Attunity Needs & Challenges in SAP Analytic Environments Real-time access to SAP data for reporting and analytics Flexibility to access and analyse SAP data across different BI tools and environments like Cloud, other DW & Hadoop Remove lock-in and dependency on SAP BW Reduce complex and costly projects to integrate SAP data Decode Complex SAP data for business users for common data model integration
  8. 8. CONFIDENTIAL INFORMATION © 2017 Attunity Attunity Replicate for SAP Extending Attunity Replication Leadership and SAP Integration RDBMS | EDW | Hadoop | Kafka On Premises | Cloud Core and Industry-Specific SAP Modules Attunity Replicate RDBMS | EDW | Hadoop On Premises or Cloud Bulk Load CDC 2017 Attunity
  9. 9. CONFIDENTIAL INFORMATION © 2017 Attunity Use Cases 1. Data Lakes: Live SAP Data Ingest for Hadoop Data Lakes / Kafka 2. Cloud Analytics: Live SAP Data Ingest/Migration for Cloud Analytics 3. ODS: Create an SAP ODS (operational data store) for Real-time BI 4. Real-time data warehousing: with SAP application data
  10. 10. CONFIDENTIAL INFORMATION © 2017 Attunity Configurable, Pre-defined Automation • Intuitive and easy to use web- interface • Simple to configure and manage replication tasks • Single interface for any source to target • Easy management with alerts notifications Intuitive User Experience
  11. 11. CONFIDENTIAL INFORMATION © 2017 Attunity Replicate for SAP – App-Level Replication • Business object level metadata • Real-time CDC aligned with business object • Unpacks pooled and clustered tables
  12. 12. CONFIDENTIAL INFORMATION © 2017 Attunity In Memory and File Optimised Data Transport Enterprise-class CDC for SAP Flexible and optimized CDC options • Transactions applied in real- time and in order • Changes applied in optimised batches • Integration with data warehouse native loaders to ingest and merge • Message encoded streaming of changes (for Kafka message broker) R1 R1 R2 R1 R2 R1 R2Batch CDC Data Warehouse Ingest-Merge SQL n 2 1 SQL SQL Transactional CDC Message Encoded CDC 2017 Attunity
  13. 13. CONFIDENTIAL INFORMATION © 2017 Attunity SAP certified agent to decode complex SAP data structures with metadata for replication Automated SAP Data selection via UI from pool, clustered or indexed tables Enable data transformation during replication for all SAP data sets Support for SAP ERP (ECC), CRM, and custom SAP modules Maintaining SAP data integrity during replication and during CDC Minimal performance impact on SAP system by adding support for RFC calls Key Enablers for SAP Environments
  14. 14. CONFIDENTIAL INFORMATION © 2017 Attunity Replicate for SAP TransformFilter Batch CDC Incremental In-Memory File Channel Batch Architecture Persistent Store Extract relationships for Pool and Cluster Tables Navigate and select SAP business objects Automated ABAP Mapping and Change- Data-Capture for Pool and Cluster tables 1. Replicate for SAP UI 2. Replicate for SAP RFC Calls RDBMS (Oracle, DB2, etc.) Redo/ Archive logs or Journal File ---------------- Transparent Tables On Premises Hadoop RDBMS Data WarehouseKafka Cloud
  15. 15. CONFIDENTIAL INFORMATION © 2017 Attunity SAP Platforms Supported by Attunity Replicate *Replicate 5.1.2 supports SAP as of November Supported SAP Versions Supported DBs for SAP • Primarily SAP ECC 6.0 + all EhP levels • Can also support ECC 5.0, 4.7 Enterprise and 4.6C
  16. 16. CONFIDENTIAL INFORMATION © 2017 Attunity SAP Applications Supported by Attunity Replicate ERP / ECC (Enterprise Resource Planning / ERP Core Components)* CRM (Customer Relationship Management) SRM (Supplier Relationship Management) GTS (Global Trade System) MDG (Master Data Governance) * All modules supported but HR
  17. 17. CONFIDENTIAL INFORMATION © 2017 Attunity • Ability to realise new analytics value in native target schema aka Hive, Kafka, Cloud • SAP data access via BI reports running directly against the replicated data with logical naming • End-to-end automation from SAP data selection to loading data with transformations via intuitive “Click to Load” interface • Faster performance with less overhead on SAP system by limiting RFC calls • Real-time change data capture (CDC) applied to SAP data sets Key Value of Attunity Replicate for SAP Rapid Data Access for More Users and Systems
  18. 18. CONFIDENTIAL INFORMATION © 2017 Attunity SOURCES OLTP, ERP, CRM Systems Documents, Emails Web Logs, Click Streams Social Networks Machine Generated Sensor Data Geolocation Data Data Integration & Ingest Attunity Replicate for HDP and HDF Accelerate time-to-insights by delivering solutions faster, with fresher data, from many sources - Automated data ingest - Incremental data ingest (CDC) - Broad support for many sources Integrating SAP Data into the Hortonworks Connected Data Platform
  19. 19. CONFIDENTIAL INFORMATION © 2017 Attunity SOURCES OLTP, ERP, CRM Systems Documents, Emails Web Logs, Click Streams Social Networks Machine Generated Sensor Data Geolocation Data Data Integration & Ingest Attunity Replicate for HDP and HDF Accelerate time-to-insights by delivering solutions faster, with fresher data, from many sources - Automated data ingest - Incremental data ingest (CDC) - Broad support for many sources Integrating SAP Data into the Hortonworks Connected Data Platform
  20. 20. CONFIDENTIAL INFORMATION © 2017 Attunity • Accelerate data delivery across enterprise and cloud • Empower rapid utilization of data by the business • Continually optimize with intelligent insight Attunity Corporate Overview Over 2000 Customers in 65 Countries Financial Services Manufacturing / Industrials GovernmentHealth Care Technology / Telecommunications Other Industries Enterprise Data Management On Premise | Cloud | Across Platforms Global Organization USA EMEA APAC
  21. 21. CONFIDENTIAL INFORMATION © 2017 Attunity Attunity – Trusted Technology and Partner  Trusted by Microsoft with 3 OEMs, bundled inside SQL Server  Trusted by Amazon (AWS) with technology licensing for cloud migration service  Trusted by IBM and Oracle with respective OEMs of Attunity technology  Trusted by Teradata as a reseller for Data warehouse and Hadoop market  Trusted by HP as a reseller for Data warehouse and analytics market  Trusted by Accenture, Capgemini and Cognizant as SI partners  Trusted by Hortonworks, Cloudera and MapR for Hadoop solutions  Trusted by over 2000 customers in over 65 countries
  22. 22. CONFIDENTIAL INFORMATION © 2017 Attunity Data Ingest for Hadoop Data Lakes Attunity Replicate Batch CDC Incremental Batch Cloud On- prem Cloud On- prem Persistent Store Hadoop Files RDBMS Data Warehouse Mainframe
  23. 23. CONFIDENTIAL INFORMATION © 2017 Attunity Data Ingest for Hadoop Data Lakes Attunity Replicate Batch CDC Incremental Batch Cloud On- prem Cloud On- prem Persistent Store Hadoop Files RDBMS Data Warehouse Mainframe
  24. 24. Thanks! attunity.com
  25. 25. CONFIDENTIAL INFORMATION © 2017 Attunity Technical Time - Cluster Table example in SAP RFBLG is a table cluster for Accounting Documents. It has a BLOB column that holds the BSEG, BSEC, BSED, BSES and BSET cluster tables within it. A cluster table is a special hash-keyed area of the database that allows for more efficient storage of redundant data. You cannot have a cluster table (like BSEG) within an OpenSQL join statement. There used to be an Oracle limitation. Oracle could only handle a certain number of columns in a single table, but BSEG was wider, so SAP came up with the Cluster design
  26. 26. CONFIDENTIAL INFORMATION © 2017 Attunity RFBLG copied Oracle to SQL Blob Column
  27. 27. CONFIDENTIAL INFORMATION © 2017 Attunity BSEG copied by R4SAP into SQL SAP Table Meta Data – Field Names BSEG is part of the Blob from RFBLG BSEG does not exist on its own in the DB

×