Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Data Warehouse Like a Tech Startup with Oracle Autonomous Data Warehouse

379 views

Published on

“Tech startups can't afford DBAs, and they don't have time to provision servers and scale them up and down or deal with patches or downtime. They've never heard of indexes and they need data loaded and ready for analysis in days, not months. In this session learn how Oracle Database developers can build data warehouses as a hip startup data engineer would—but using a proper database built on Oracle technology. Oracle Data Visualization Desktop provides analytics and data exploration with techniques explained in this session. Hear real-world development experiences from working on data and analytics projects at a tech startup in the UK.”

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Data Warehouse Like a Tech Startup with Oracle Autonomous Data Warehouse

  1. 1. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Data Warehouse Like a Tech Startup with Oracle Autonomous Data Warehouse Cloud Mark Rittman, CEO and Founder, MJR Analytics Oracle Open World 2018, San Francisco
  2. 2. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Introducing MJR Analytics ● Specialists in Oracle Cloud Analytics ● Founded by Mark Rittman in 2018 ● 100% Cloud focus + project delivery ○ Oracle Analytics Cloud ○ Oracle Autonomous DW Cloud ○ Oracle Data Integration Cloud ○ Oracle Big Data Cloud ● Speak to us now during OOW 2018 info@mjr-analytics.com +44 7866 568246 https://www.mjr-analytics.com
  3. 3. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com It’s Good to be back at Oracle Open World ● Former co-founder and CTO of an Oracle BI+DW consultancy ● Oracle ACE Director, Author of two Oracle Press books ● Consultant specialising in architecting, building and tuning Oracle BI+DW ● Regular speaker at every Oracle Openworld since 2006 ● … but then in 2016 I took two years out
  4. 4. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com I joined a tech startup
  5. 5. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Agile self-organizing teams Thinking product not consulting Real-world big data vs. PoCs and BigDataLite Monthly product releases
  6. 6. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com And everything I thought I knew, was wrong
  7. 7. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Product Manager for Platform Analytics Features Raw, Event-Level Tables optimized for fast ingest and processing Industry-Vertical Business Views Customer-Specific Semantic Model Materialized Historic View Rows Real-Time Data View (Today’s Data) Base Views Industry Extensions Base + Industry Explore + Extensions
  8. 8. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Startups need to move fast. “The battle between every startup and incumbent comes down to whether the startup gets distribution before the incumbent gets innovation.” - Alex Rampell, A16z, “Distribution vs. Innovation”
  9. 9. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com This startup was built on data Lots of data
  10. 10. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com 150,000+events/second >2PBdata in 70,000+ tables 1Bn+interactions/day
  11. 11. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com zero DBAs
  12. 12. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com They didn’t have time to take backups
  13. 13. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com They’d never heard of an index
  14. 14. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com They didn’t do server patchsets. Or servers.
  15. 15. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Or own any hardware
  16. 16. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com So what did they care about?
  17. 17. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Agility and time to provision Scaling their capacity up (and down) Costs that start small, and align with growth
  18. 18. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Scale-out solves one problem, but then you need to manage that cluster of servers Data Warehouse RBDMS > 3 nodes, >1 TB Hadoop / HBase cluster > 50 nodes 1000+ HBase regions per server
  19. 19. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Elastic Compute + Storage Elastic Pricing
  20. 20. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com MVP - “Adoption or fail fast” GA Release Initial Early Adopters Product / Market Fit Minimal cost Pay via CC and expense Costs linear with product adoption Can get expensive
  21. 21. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com No-Ops Autoscaling Black Friday New Year Sales De-provisioning of excess nodes -$$$ Provisioning of additional nodes +$$$
  22. 22. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Elastic Costs based on Planned Workload Black Friday New Year Sales Automatic provisioning of additional nodes +$$$ De-provisioning of excess nodes -$$$
  23. 23. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com No-Ops Autoscaling Black Friday New Year Sales Automatic provisioning of additional nodes +$$$ Automatic de-provisioning -$$$ So you don’t waste $$$ on capacity you don’t need
  24. 24. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Startups need to focus.
  25. 25. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Serving Customers Building Products Domain Knowledge Patching servers Doing backups Scaling capacity Things you are good at Things you suck at Do These Automate this
  26. 26. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Data Analysis within a Tech Startup
  27. 27. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Data Warehouse Analysis within Startups Provision a Cloud DW Instance Land Data in Object Store Analyze via BI Connector Deprovision Cloud DW Instance Create simple column-store tables
  28. 28. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com How do these database services work?
  29. 29. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com
  30. 30. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com However...
  31. 31. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com If you need to know, they’ve failed
  32. 32. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com But they do have limitations
  33. 33. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Limited or non-standard SQL Primarily INSERT-only Distributed joins + resource limits
  34. 34. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Coming to Disrupt Your IndustryNow you can do the disruptingOracle Autonomous Data Warehouse Cloud
  35. 35. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Oracle Autonomous Data Warehouse Cloud ● Fully-managed DW Platform-as-a-Service ● Based on Oracle 18c Exadata Database technology ● Near-instant provisioning ● Elastic scaling and pricing ● Simplified column-store table creation ● Automated provisioning, patching and upgrades ● Automated backups ● Includes data visualization + notebook apps
  36. 36. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Data Warehouse Analysis using ADWC
  37. 37. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Provisioning an Autonomous DW Cloud Instance ● Provision from https://cloud.oracle.com ● Choose DB name, display name ● Select CPU count (1 min) and storage (1TB min, 128TB max) ○ Creates tenant within multi-tenant Exadata instance ○ Scale-up and down within 128 CPU/TB instance limit ● Provisioning typically takes ~ 1 min ● Download client credentials as secure wallet ZIP file
  38. 38. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com
  39. 39. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com
  40. 40. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com
  41. 41. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com
  42. 42. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com
  43. 43. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Loading Data into Autonomous Data Warehouse ● Data for analysis can uploaded as part of table creation ● More typically, data is located in OCS/S3 object storage ○ Allows sharing of data with other cloud applications ○ Real-time analysis of data streamed via Event Hub
  44. 44. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Data Lake on Oracle Big Data Cloud Topology ● New DBMS_CLOUD package for object store operation ○ Analyze OCS/S3 data in-place as external tables ○ Or load into Autonomous Data Warehouse Cloud ● Extends scale of ADWC beyond limits of Exadata storage
  45. 45. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Simplified Table Creation & Accessing Data ● “Load and go” ○ Define tables, load data, run queries ■ Typically using free SQL*Developer GUI ○ Because it’s based on Exadata technology ■ No indexes ■ No need for partitioning ■ No need for materialized views ○ Because it’s Autonomous ■ Self-tuning and self-optimzing
  46. 46. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com
  47. 47. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Simplified Table Creation & Accessing Data ● Comes with entitlement to DV Desktop license ● Includes web notebook based on Apache Zeppelin ● Connect most popular BI + data integration tools, e.g. ○ Oracle Analytics Cloud ○ Looker
  48. 48. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com
  49. 49. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com
  50. 50. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Scale-Up and Scale-Down of Compute + Storage ● Ability to scale-up / scale-down with steps then automated ○ Pricing based upon CPU ($/CPU/hour) and Storage ($/TB/Month) ● Close your ADWC instance when not needed, restart instantly ○ Leave data in ADWC storage and set CPU to 0 ○ CPU and storage to 0 and data left in object store
  51. 51. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Scale-Up and Scale-Down of Compute + Storage ● Independently scale compute or storage ○ Resizing occurs instantly, fully online ○ Memory, IO bandwidth, concurrency scales linearly with CPU ● Automates resizing tasks - not autoscaling (yet)
  52. 52. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com So why Autonomous Data Warehouse?
  53. 53. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Move like a startup without introducing new technology to your mix.
  54. 54. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Elastic scaling and elastic pricing … but now using a real database
  55. 55. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com https://www.forbes.com/sites/oracle/2018/10/04/how-a-big -business-can-use-autonomous-it-to-move-like-a-startup/#15b 9df6e1752
  56. 56. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com MJR Analytics for ADWC ● Specialists in Oracle Cloud Analytics ● Founded by Mark Rittman in 2018 ● 100% Cloud focus + project delivery ○ Oracle Analytics Cloud ○ Oracle Autonomous DW Cloud ○ Oracle Data Integration Cloud ○ Oracle Big Data Cloud ● Speak to us now during OOW 2018 info@mjr-analytics.com +44 7866 568246 https://www.mjr-analytics.com
  57. 57. © MJR Analytics 2018, T: +44 01273 041134 (UK) 415-218-2161 (US) W: https;//mjr-analytics.com E: info@mjr-analytics.com Data Warehouse Like a Tech Startup with Oracle Autonomous Data Warehouse Cloud Mark Rittman, CEO and Founder, MJR Analytics Oracle Open World 2018, San Francisco

×