Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

From BI Developer to Data Engineer with Oracle Analytics Cloud, Data Lake

87 views

Published on

In this session look at the role of a data engineer in designing, provisioning, and enabling an Oracle Cloud data lake using Oracle Analytics Cloud, Data Lake. Attendees learn how to use data flow and data pipeline authoring tools and how machine learning and AI can be applied to this task, as well as how to connect to database and SaaS sources along with sources of external data via Oracle Data as a Service. Discover how traditional Oracle Analytics developers can transition their skills into this role and start working as a data engineer on Oracle Public Cloud data lake projects.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

From BI Developer to Data Engineer with Oracle Analytics Cloud, Data Lake

  1. 1. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com From BI Developer to Data Engineer with Oracle Analytics Cloud, Data Lake Mark Rittman, CEO and Founder, MJR Analytics Oracle Open World 2018, San Francisco
  2. 2. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Introducing MJR Analytics ● Specialists in Oracle Cloud Analytics ● Founded by Mark Rittman in 2018 ● 100% Cloud focus + project delivery ○ Oracle Analytics Cloud ○ Oracle Autonomous DW Cloud ○ Oracle Data Integration Cloud ○ Oracle Big Data Cloud ● Speak to us now during OOW 2018 info@mjr-analytics.com +44 7866 568246 https://www.mjr-analytics.com
  3. 3. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Oracle Analytics Cloud ● Oracle’s Cloud Analytics platform based on OBIEE and Oracle DV technology ● Customer-managed or Oracle-managed (Autonomous Analytics Cloud) ● Available in three editions ○ OAC Standard ○ OAC Data Lake ○ OAC Enterprise
  4. 4. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Three Key Components of OAC Data Lake Oracle Data Visualization (OAC Standard Edition) Oracle Essbase Cloud Data Flows & Data Lake Analysis
  5. 5. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com ● Explore, catalog and discover data in Oracle Big Data Cloud, Oracle Database ● Enrich and transform raw data into valuable information and insights ● Analyze at-scale data using Data Visualization ● Combine data from SaaS, social and real-time ● Create predictive and classification models ● Analyze the sentiment in social media feeds Data Flows Oracle Analytics Cloud, Data Lake
  6. 6. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com But what’s a Data Lake?
  7. 7. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com What Is a Data Lake? ● Complements a data warehouse ● Landing area for unstructured and semi-structured data for analysis ● Flexible data storage platform with cheap storage, flexible schema support + compute ● Use-cases include ○ Storing data intended for multiple query engines ○ Landing data for initial discovery ○ Storing high-volume granular event data from Event Hub
  8. 8. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com What Is a Data Lake?
  9. 9. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Creates new insights + models using tools such as R and sampled data Helps people understand insights from data that they’ve unearthed Data Engineers Makes at-scale data consumable in some form, either directly or by data scientists and data analysts Data Scientists Data Analysts Data Lake User Personas
  10. 10. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Data Engineers ● Can code, run clusters ● Create data pipelines & prepare data ● and build predefined ML models ● Knowledge of the math of ML limited ● They may be DBAs, BI developers ● Experience with DevOps, cloud
  11. 11. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com
  12. 12. OAC Data Lake Features for Data Engineers 12 ● Explore, catalog and discover data in Oracle Big Data Cloud, Oracle Database ● Enrich and transform raw data into valuable information and insights ● Analyze at-scale data using Data Visualization ● Combine data from SaaS, social and real-time ● Create predictive and classification models ● Analyze the sentiment in social media feeds ● Data engineering without the hand-coding
  13. 13. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Example OAC Data Lake Scenario
  14. 14. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com OAC Data Lake Cloud Components
  15. 15. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com15 Scenario : Ingest and Analyze Real-Time Feeds
  16. 16. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Scenario : Ingest and Analyze Real-Time Feeds 16
  17. 17. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com17 Scenario : Ingest and Analyze Real-Time Feeds
  18. 18. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com18 Scenario : Ingest and Analyze Real-Time Feeds
  19. 19. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com19 Scenario : Ingest and Analyze Real-Time Feeds
  20. 20. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com20 Scenario : Ingest and Analyze Real-Time Feeds
  21. 21. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com21 Scenario : Ingest and Analyze Real-Time Feeds
  22. 22. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com22 Scenario : Ingest and Analyze Real-Time Feeds
  23. 23. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com23 Scenario : Ingest and Analyze Real-Time Feeds
  24. 24. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com24 Scenario : Ingest and Analyze Real-Time Feeds
  25. 25. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com25 Scenario : Ingest and Analyze Real-Time Feeds
  26. 26. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com26 Scenario : Ingest and Analyze Real-Time Feeds
  27. 27. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Oracle Cloud Platform-as-a-Service Stack 27
  28. 28. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Oracle Big Data Cloud, Ambari and Hive ThriftServer 28
  29. 29. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Oracle Event Hub Cloud Service - Dedicated 29
  30. 30. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Managing and Cataloging the Cloud Data Lake 30 ● Catalog of all data assets in projects ● Connection to Hive Thrift Server ● IoT and Social Media Data Sets ● Data Flows and Sequences ● Managed data lake store ● Control the lifecycle of your data lake assets ● Security ● Scheduling
  31. 31. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Data Preparation Features from OAC Standard Edition 31 1. Split timestamp field that’s not in valid format 2. Choose “space” character as delimiter 3. Convert the first split column into a date datatype 4. Choose the correct date format for this field’s values 5. Repeat for the TIME split column, concatenate with ’T’ in-between and finally convert resulting field into TIMESTAMP
  32. 32. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com32 Data Flows are sequences of data transformations executed on the BI Server - Spark execution on roadmap for OAC DL Create Essbase Cube Time Series Forecast Sentiment Analysis Predictive / ML Model Train and Build Run custom R and other python scripts Extended Data Flow Capability for Data Lake Edition Data Flows are based on the technology previously announce as “Dataflow ML”, now delivered as part of Oracle Analytics Cloud
  33. 33. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Example : Enrich With Sentiment, Then Visualize 33 1. Add Sentiment Analyse step to data flow, persist final enriched dataset back to Hive table 2. Add a calculation to convert sentiment description values to positive/negative cumulative score 3. Analyze Results in Data Visualization UI
  34. 34. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Using Explain Feature to Automate Deriving Context 34 1. Right-Click on attribute or measure column to “explain” the drivers of its values 2. ML algorithm explains basic facts, drivers, anomalies and identifies segments of interest
  35. 35. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Display Selected Column Explanations on Dashboard 35
  36. 36. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Transform, Aggregate and Join Datasets 36 Multi-step dataset joins Aggregate Datasets Binning and Grouping
  37. 37. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Predictive Modeling and Forecasting 37 1. Select Prediction Model best suited to predicting Kudos from Strava bike rides 2. Select column whose values are to be predicted, and model parameter values 3. Train model and then test against remaining dataset
  38. 38. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Analyzing Data At-Scale Hosted on Big Data Cloud 38
  39. 39. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Oracle Analytics Cloud, Data Lake - Summary ● Edition of Oracle Analytics Cloud that extends Standard with ○ Essbase Cloud ○ Data Flows and integration with Big Data` ● Data Flow feature enables multi-step transform of ingested data ● Sentiment Analyze operator useful for social/text data enrichment ● Enables BI developers to train and build predictive models ● ML-driven Explain feature automates understanding of context ● Basic data engineering for BI developers ● Find out more at https://mjr-analytics.com or speak to us after the session
  40. 40. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com From BI Developer to Data Engineer with Oracle Analytics Cloud, Data Lake Mark Rittman, CEO and Founder, MJR Analytics Oracle Open World 2018, San Francisco

×