Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Autonomous database 100

329 views

Published on

In this course, you will learn the features of Autonomous Data Warehouse Cloud Service (ADWCS) and Autonomous Transaction Processing (ATP). You will also learn how to launch an ADW service and query a large data set.

Published in: Technology
  • Be the first to comment

Autonomous database 100

  1. 1. 1Copyright © 2018, Oracle and/or its affiliates. All rights reserved.Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Autonomous Database Level 100 Sanjay Narvekar December 2018
  2. 2. 2Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  3. 3. 3Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Autonomous Database Cloud Service - Objectives After completing this lesson, you should be able to: • Describe the features of Autonomous Data Warehouse Cloud Service (ADWCS) and Autonomous Transaction Processing (ATP) • Launch an ADW and ATP database on Oracle Cloud Infrastructure (OCI) • Connect to the ADW (or ATP) using SQL Developer • Load data into ADW (or ATP) and query data from ADW (or ATP) • Scale up/down the ADW (or ATP) • Monitor the ADW (or ATP) • Have an understanding of backup and recovery mechanism in ADW (or ATP)
  4. 4. 4Copyright © 2018, Oracle and/or its affiliates. All rights reserved. One Autonomous Database - Optimized by Workload Autonomous Transaction Processing (ATP) Best for all Analytic Workloads: • Data Warehouse, Data Mart • Data Lake, Machine Learning Autonomous Data Warehouse (ADW) Best for TP and Mixed Workloads: • Transactions, Batch, Reporting, IoT • Application Dev, Machine Learning ORACLE AUTONOMOUS DATABASE
  5. 5. 5Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Autonomous Optimizations - Specialized by Workload ADW ATP Row FormatColumnar Format Creates Indexes*Creates Data Summaries Memory for Caching to Avoid IOMemory Speeds Joins, Aggs Statistics updated in real-time while preventing plan regressions * Coming Soon in Oracle Database 19c
  6. 6. 6Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Autonomous Data Warehouse
  7. 7. 7Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Fully-managed Oracle automates end-to-end management of the data warehouse • Provisioning new databases • Growing/shrinking storage and/or compute • Patching and upgrades • Backup and recovery Full lifecycle managed using the service console • Alternatively, can be managed via command-line interface or REST API
  8. 8. 8Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Automated Tuning “Load and go” • Define tables, load data, run queries – No tuning required – No special database expertise required – No need to worry about tablespaces, partitioning, compression, in-memory, indexes, parallel execution • Fast performance out of the box with zero tuning • Simple web-based monitoring console • Built-in resource-management plans
  9. 9. 9Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Fully-elastic Size the DW to the exact compute and storage required • Not constrained by fixed building blocks, no predefined shapes Scale the DW on demand • Independently scale compute or storage • Resizing occurs instantly, fully online Shut off idle compute to save money • Restart instantly
  10. 10. 10Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Full Support of DW Ecosystem Autonomous Data Warehouse Cloud supports : • Existing tools, running on-premises or in the cloud – Third-party BI tools – Third-party data-integration tools – Oracle BI and data-integration tools: BIEE, ODI, etc. • Oracle cloud services: Analytics Cloud Service, Golden Gate Cloud Service, Integration Cloud Service, and others • Connectivity via SQL*Net, JDBC, ODBC
  11. 11. 11Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Autonomous Data Warehouse Cloud: Architecture Oracle Exadata Cloud Service Oracle Database Cloud Service Express Cloud Service Data Warehouse Services (EDWs, DW, departmental marts and sandboxes) Autonomous Data Warehouse Cloud Service Console Built-in Access Tools Oracle ML Service Management DW Database SQL Developer Developer Tools Data Integration Services Oracle Data Integration Platform Cloud 3rd Party DI on Oracle Cloud Compute 3rd Party DI On-premises
  12. 12. 12Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Getting Started with Autonomous Data Warehouse Cloud Provisioning an ADWC database requires only answers to 5 simple questions: • Database name? • Which data center (region)? • How many CPU cores? • How much storage capacity (in TBs)? • Admin password? New service created in a few minutes (regardless of size) • Database is open and ready for connections
  13. 13. 13Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Securing Autonomous Data Warehouse Cloud • Stores all data in encrypted format in the Oracle Database. Only authenticated users and applications can access the data when they connect to the database. • All connections to Autonomous Data Warehouse Cloud use certificate based authentication and Secure Sockets Layer (SSL). This ensures that there is no unauthorized access to the Autonomous Data Warehouse Cloud and that communications between the client and server are fully encrypted and cannot be intercepted or altered. • Certificate based authentication uses an encrypted key stored in a wallet on both the client (where the application is running) and the server (where your database service on the Autonomous Data Warehouse Cloud is running). The key on the client must match the key on the server to make a connection. A wallet contains a collection of files, including the key and other information needed to connect to your database service in the Autonomous Data Warehouse Cloud.
  14. 14. 14Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Connecting to the Autonomous Database Warehouse • Securely connect using credential wallets via SQL*Net, JDBC, ODBC • Wallet can be downloaded from the service console or using REST APIs
  15. 15. 15Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Database Service Names 3 pre-defined database services identifiable as high, medium and low • Choice of performance and concurrency for ADW HIGH • Highest resources, lowest concurrency • Queries run in parallel MEDIUM • Less resources, higher concurrency • Queries run in parallel LOW • Least resources, highest concurrency • Queries run serially No of concurrent queries Max idle time CPU shares HIGH 3 5 mins 4 MEDIUM 20 5 mins 2 LOW 32 1 hour 1 Example for a database with 16 OCPUs *When connecting for replication purposes, use the LOW database service name. For example, use this service with Oracle GoldenGate connections.
  16. 16. 16Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Creating users in Autonomous Data Warehouse Simplified user creation via the new database role • No need to specify anything except the password • Autonomous Data Warehouse Cloud requires strong passwords, the password you specify must meet the default password complexity rules. • DWROLE includes all necessary privileges for a DW developer/user >create user ocitest identified by P#ssw0rd12##; User OCITEST created. >Grant dwrole to ocitest; Grant succeeded.
  17. 17. 17Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Data Loading Options for Autonomous Data Warehouse Data loading via SQL*Net • Suitable for small volumes of data – SQL*Loader from local filesystem – ETL scripts that use DML to insert/update data Data loading from Oracle Object Storage • Preferred technique for large volumes of data – Additionally enables data-sharing with other cloud services • Stage data in Oracle Object Storage, then load into the database using new PL/SQL APIs
  18. 18. 18Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Loading Data from Object Stores Source data from files on object stores for data loading • OCI Object Storage, OCI Object Storage Classic, AWS S3, or Microsoft Azure • Any supported ORACLE_LOADER file format • Roadmap: any Hadoop file format ADWC OBJECT STORES
  19. 19. 19Copyright © 2018, Oracle and/or its affiliates. All rights reserved. New Cloud API to Access Object Stores, DBMS_CLOUD New PL/SQL package for accessing files in object stores No need to manually define external tables for loading files • Makes it easier to specify the format of the source data
  20. 20. 20Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Loading Data from the Oracle Object Store Define your credentials for the object store • Oracle Cloud Infrastructure Object Store username and Swift password required Credential stored in the database schema once and used for accessing the object store for all loads begin dbms_cloud.create_credential( credential_name => 'OBJ_STORE_CRED', username => 'tenant1', password => ’password' ); end; /
  21. 21. 21Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Loading Data from the Oracle Object Store Load data directly into the target table without any intermediate steps Data format in the source file easily specified as JSON begin dbms_cloud.copy_data( table_name =>'CHANNELS', credential_name =>'OBJ_STORE_CRED', file_uri_list =>'https://swiftobjectstorage.us-ashburn- 1.oraclecloud.com/v1/dwcsdemo/DEMO_DATA/chan_v3.dat', format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value 'true') ); end; /
  22. 22. 22Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Troubleshooting Loads Load operations logged for troubleshooting and historical load tracking • New table user/dba_load_operations Log and bad files accessible as tables select table_name,status,rows_loaded,logfile_table,badfile_table from user_load_operations; TABLE_NAME STATUS ROWS_LOADED LOGFILE_TABLE BADFILE_TABLE -------------------- --------- ----------- -------------------- -------------------- CHANNELS FAILED COPY$1_LOG COPY$1_BAD CHANNELS COMPLETED 5 COPY$2_LOG COPY$2_BAD
  23. 23. 23Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Querying Data on the Oracle Object Store • Define your credentials for the object store – Oracle Cloud Infrastructure Object Store username and Swift password required • Credential stored in the database schema once and used for accessing the object store for all queries • Call the dbms_cloud API for creating an external table on top of the source files
  24. 24. 24Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Querying Data on the Oracle Object Store begin dbms_cloud.create_external_table( table_name =>'CHANNELS_EXT', credential_name =>'OBJ_STORE_CRED', file_uri_list => 'https://swiftobjectstorage.us-ashburn- 1.oraclecloud.com/v1/dwcsdemo/DEMO_DATA/chan_v3.dat', format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value 'true'), column_list => 'CHANNEL_ID NUMBER, CHANNEL_DESC VARCHAR2(20), CHANNEL_CLASS VARCHAR2(20), CHANNEL_CLASS_ID NUMBER, CHANNEL_TOTAL VARCHAR2(13), CHANNEL_TOTAL_ID NUMBER' ); end; / select count(*) from channels_ext;
  25. 25. 25Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Built-in SQL Worksheet and Notebook – Oracle Machine Learning Quickly start running queries with built-in web- based notebooks • No need to install a client query tool Initially supports SQL and PL/SQL • More languages in the roadmap Based on Apache Zeppelin
  26. 26. 26Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Scaling Your Database Scale your database on demand without tedious manual steps • Independently scale compute or storage • Resizing occurs instantly, fully online • Memory, IO bandwidth, concurrency scales linearly with CPU Close your database to save money when not used • Restart instantly
  27. 27. 27Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Automated Database Configuration in ADWC Database initialization parameters • Parameters are optimized for DW workloads • Memory, parallelism, sessions configured based on number of CPUs • Users can modify a limited set of parameters, e.g. NLS settings Tablespace management • Pre-defined data and temporary tablespaces • Users cannot create/modify tablespaces Compression • All tables compressed using Hybrid Columnar Compression • Users cannot change compression method or disable compression
  28. 28. 28Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Automated Database Configuration in ADWC Optimizer stats gathering • Stats gathered automatically during direct load operations • Users can gather stats manually if they want Optimizer hints • Hints ignored by default • Users can enable hints explicitly Result cache configuration • Result cache enabled by default for all queries
  29. 29. 29Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Monitoring • Simplified monitoring using the web-based service console • Historical and real-time performance charts • Real-Time SQL Monitoring to monitor running and past SQL statements • Historical data load monitoring
  30. 30. 30Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Autonomous Data Warehouse Cloud – Backup and recovery • Autonomous Data Warehouse Cloud automatically backs up your database for you. The retention period for backups is 60 days. You can restore and recover your database to any point-in-time in this retention period. • Autonomous Data Warehouse Cloud automatic backups provide weekly full backups and daily incremental backups. • Manual backups for your ADW database is not needed. • But, you can do manual backups using the cloud console if you want to take backups before any major changes, for example before ETL processing, to make restore and recovery faster. The manual backups are put in your Cloud Object Storage bucket. When you initiate a point-in-time recovery Autonomous Data Warehouse Cloud decides which backup to use for faster recovery. • You can initiate recovery for your Autonomous Data Warehouse Cloud database using the cloud console. Autonomous Data Warehouse Cloud automatically restores and recovers your database to the point-in-time you specify.
  31. 31. 31Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Demo Autonomous Data Warehouse
  32. 32. 32Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Autonomous Transaction Processing
  33. 33. 33Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Provisioning ATP Provisioning requires only 6 simple questions: • Compartment? • Display Name? • Database name? • How many CPU’s? • How many TB’s? • Admin password? New service created in couple minutes (regardless of size) • Ready to connect via SQL*Net
  34. 34. 34Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Demo 1 Provisioning
  35. 35. 35Copyright © 2018, Oracle and/or its affiliates. All rights reserved. REST APIs available for all ATP Operations
  36. 36. 36Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Connecting ORACLE AUTONOMOUS DATABASE
  37. 37. 37Copyright © 2018, Oracle and/or its affiliates. All rights reserved. SERVICES NAME RESOURCE MANAGEMENT PLAN SHARES PARALELLISM HIGH 8 MANUAL MEDIUM 4 MANUAL LOW 1 MANUAL PARALLEL 1 CPU_COUNT Pre-defined Services for Autonomous Transaction Processing • Four pre-defined database services controlling priority and parallelism • Different services defined for Transactions and Reporting/Batch For Transaction Processing For Reporting or batch processing
  38. 38. 38Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Demo 2 Download Security Wallet
  39. 39. 39Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Demo 3 Connecting to ATP
  40. 40. 40Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Loading Data ORACLE AUTONOMOUS DATABASE
  41. 41. 41Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Data Loading Options Data loading via SQL*Net • Suitable for small volumes of data – SQL*Loader from local filesystem on client – ETL scripts that use DML to insert/update data Data loading from Oracle Object Storage • Preferred technique for large volumes of data – Additionally enables data-sharing with other cloud services • Stage data in Oracle Object Storage, then load into the database using new PL/SQL API or Data Pump Import
  42. 42. 42Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Loading Data from Object Stores Data can be loaded direct from any object store • Oracle Object Store, AWS S3 or Azure • Any supported ORACLE_LOADER file format • Short term Roadmap: any Hadoop file format Any Object Storage
  43. 43. 43Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Loading Data from the Oracle Object Store • Define your credentials for the object store • Oracle Cloud Infrastructure Object Store username and Swift password required • Credential stored in the database schema once and used for accessing the object store for all loads BEGIN dbms_cloud.create_credential( credential_name => 'OBJ_STORE_CRED', username => 'tenant1', password => 'password' ); END; /
  44. 44. 44Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Loading Data from the Oracle Object Store • Load data directly into the target table without any intermediate steps • Data format in the source file easily specified as JSON BEGIN dbms_cloud.copy_data( table_name =>'CHANNELS', credential_name =>'OBJ_STORE_CRED', file_uri_list =>'https://swiftobjectstorage.us-ashburn1.oraclecloud.com/v1 /dwcsdemo/DEMO_DATA/chan_v3.dat', format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value 'true') ); END; /
  45. 45. 45Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Loading data via SQLDev
  46. 46. 46Copyright © 2018, Oracle and/or its affiliates. All rights reserved. How It Scales ORACLE AUTONOMOUS DATABASE
  47. 47. 47Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Demo 4 Scaling ATP
  48. 48. 48Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Autonomous Database - Summary You should now be able to • Describe the features of Autonomous Data Warehouse (ADW) Cloud Service and Autonomous Transaction Processing (ATP) Cloud Service • Launch an ADW and ATP on Oracle Cloud Infrastructure • Connect to the ADW (or ATP) using SQL Developer • Load data into ADW (or ATP) and query data from ADW (or ATP) • Scale up/down the ADW (or ATP) • Monitor the ADW (or ATP) • Have an understanding of backup and recovery mechanism in ADW (or ATP)
  49. 49. 49Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Additional resources • Autonomous Data Warehouse Service Documentation https://docs.us-phoenix- 1.oraclecloud.com/Content/Database/Concepts/databaseoverview.htm • Autonomous Data Warehouse Cloud for Experienced Oracle Database Users https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/user/experienced- database-users.html - GUID-58EE6599-6DB4-4F8E-816D-0422377857E5 • Migrating Amazon Redshift to Autonomous Data Warehouse Cloud https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/user/migrating.html - GUID-A00E1C78-BCB1-46E9-97FA-DD1B377DF1F2
  50. 50. 50Copyright © 2018, Oracle and/or its affiliates. All rights reserved. cloud.oracle.com/iaas cloud.oracle.com/tryit

×