Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Ground Breakers Romania: Oracle Autonomous Database
Similar to Ground Breakers Romania: Oracle Autonomous Database (20)
More from Maria Colgan
More from Maria Colgan (17)
Recently uploaded
Recently uploaded (20)
Ground Breakers Romania: Oracle Autonomous Database
- 1. Maria Colgan Master Product Manager Oracle Database October 2019 @SQLMaria Oracle Autonomous Database Under the Hood 1
- 2. Safe harbor statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation. 2
- 4. Oracle Autonomous Database Key Attributes Self-Driving Automates all database and infrastructure management, monitoring, tuning Self-Securing Protects from both external attacks and malicious internal users Self-Repairing Protects from all downtime including planned maintenance Spend Less, Reduce Risk, Innovate More
- 5. Oracle Spent Last 20 Years Automating Database Technology Copyright © 2019 Oracle and/or its affiliates. • Automatic Query Rewrite • Automatic Undo Management • Autonomous Health Framework • Automatic Diagnostic Framework • Automatic Refresh of Clones • Automatic SQLTuning • AutomaticWorkload Capture/Replay • Automatic SQL Plan Management • Automatic Capture of SQL Monitor • Automatic Data Optimization • Automatic Memory Management • Automatic Segment Space Mgmt • Automatic StatisticsGathering • Automatic Storage Management • AutomaticWorkload Repository • Automatic Diagnostic Monitor • Automatic Columnar Flash • Automatic IM population • AutomaticApplication Continuity 9i 10g 11g 12c 18c • Automatic Indexes • SQL Quarantine • Real-Time Statistics 19c
- 6. Oracle Spent Last 10 Years Automating Database Infrastructure Copyright © 2019 Oracle and/or its affiliates. • Smart Scan • Infiniband Scale-Out • DatabaseAware Flash Cache • Storage Indexes • Hybrid Columnar Data • IO Priorities • Data MiningOffload • Offload Decryption • Direct-to-wire Protocol • JSON and XML offload • Instant failure detection • Network Resource Mgmt • Prioritized File Recovery • In-Memory Columnar in Flash • Smart Fusion BlockTransfer • Exadata Cloud Service 2008 2019
- 7. Eliminates All the Complexity of Mission Critical Databases Autonomous Database Completes the Job Copyright © 2019 Oracle and/or its affiliates. Automated Data Center Operations and Machine Learning Complete Infrastructure Automation Complete Database Automation ORACLE AUTONOMOUS DATABASE Oracle Cloud
- 8. One Autonomous Database – Two Deployment Choices ORACLE AUTONOMOUS DATABASE DedicatedServerless Ultra-Simple and Elastic Customizable Private Database Cloud in Public Cloud Copyright © 2019 Oracle and/or its affiliates.
- 9. Autonomous Database Serverless – Primary Benefits • Simple – Oracle automates and manages everything • Deployment, lifecycle, software updates, etc. – Customer just chooses database compute, storage, and region • Elastic – Low minimum size - 1 OCPU – Low minimum time commitment - 1 hour – Automatically scales online for true pay-per-use Copyright © 2019 Oracle and/or its affiliates.
- 10. Autonomous Database Dedicated – Primary Benefits • Provides your own Database Cloud running on dedicated Exadata Infrastructure – Runs all your databases - any size, scale, or criticality • Highest Isolation – Runs inside Secure Isolation Zone for highest protection from other tenants – Configure multiple Exadatas or Container Databases for intra-company isolation • Customizable Operational Policies – Control of provisioning, updates, availability, density Copyright © 2019 Oracle and/or its affiliates.
- 11. ORACLE AUTONOMOUS DATABASE DedicatedServerless Data-Driven Results Fast LOB, Dept, SMB Developer, Data Scientist Rethink Database I.T. CIO, CTO, Architect, DBA One Autonomous Database – Primary Use Cases
- 12. Copyright © 2019 Oracle and/or its affiliates. Part 2 How it Works
- 13. A Look Under the Hood Key Capabilities of Self-driving, Self-Securing, Self-Repairing Self Driving Rapid Provisioning Self Scaling Automatic Indexing Cloning Self Securing Self Patching Encryption by Default Separation of Duties Auditing Self Repairing Maximum Availability Architecture Self-Healing Hardware Self-Healing Software Copyright © 2019 Oracle and/or its affiliates.
- 14. Self-Driving | Database Provisioning • Quickly deploys a RAC Database on Exadata • Ensures seamless scale-out • Provides a HA configuration • Enables online patching • Autonomous Database decides where to place each database during provisioning • Fewer number of instances preferred • Databases may be open on one node • Still RAC enabled Copyright © 2019 Oracle and/or its affiliates.
- 15. One Autonomous Database – Optimized by Workload ORACLE AUTONOMOUS DATABASE AutonomousTransaction Processing - ATP Autonomous Data Warehouse - ADW Best for all AnalyticWorkloads: • Data Warehouse, Data Mart • Data Lake, Machine Learning Best forTP and MixedWorkloads: • Transactions, Batch, Reporting • IoT, Machine Learning Copyright © 2019 Oracle and/or its affiliates.
- 16. Self-Driving | Automatic Tuning 16 Row Format Optimizes Response Time Creates Indexes Columnar Format Optimizes Complex SQL Creates Data Summaries Autonomous Transaction Processing Autonomous Data Warehouse Memory for Caching, No IOMemory Speeds Joins, Aggs Statistics updated in real-time while preventing plan regressions
- 17. Capture Identify VerifyDecide Monitor An expert system that implements indexes based on what a skilled performance engineer would do Reinforcement Learning allows it to learn from its own actions as all candidate indexes are validated before being implementing The entire process is fully automatic Transparency is equally important as sophisticated automation All tuning activities are auditable Self-Driving | Automatic Indexing Copyright © 2019 Oracle and/or its affiliates.
- 18. Self-Driving | Optimizer Statistics • Stats are gathered automatically for direct load operations • CREATE TABLE … AS SELECT …; INSERT /*+ append */ INTO … SELECT …; • Data Pump Import loads • DBMS_CLOUD loads • Real-time Statistics Collection • Gathers a subset of optimizer statistics for conventional DML operations • Number of rows, MAX and MIN column values, etc. • High-frequency Statistics Collection • Gathers full optimizer statistics every 15 minutes if statistics are stale Copyright © 2019 Oracle and/or its affiliates.
- 19. SERVICE SHARE OF RESOURCES CONCURRENCY BEFORE QUEUING DEFAULT SQL PARALLELISM TPURGENT 12 100 X CPUs MANUAL TP 8 100 X CPUs 1 HIGH 4 3 CPUs MEDIUM 2 1.25 X CPUs 4 LOW 1 100 X CPUs 1 OLTP DW, Batch, Reporting Pre-defined Services to Control Workload Priority • Applications connect to a pre-defined database service to control: • Relative priority, max concurrently executing users, SQL parallelism • E.g. OLTP applications connect to “TP” service, most Batch to “LOW” service Copyright © 2019 Oracle and/or its affiliates.
- 20. Which Service to Use? • OLTP Applications • Connect to TP or TPURGENT based on priority requirements • DW or Reporting • Connect to HIGH, MEDIUM, LOW based on concurrency requirements Copyright © 2019 Oracle and/or its affiliates. HIGH • Low concurrency • More resources per Query MEDIUM • Higher concurrency • Fewer resources per Query LOW • Highest concurrency • Fewest resources per Query
- 21. • Instant scaling online for highest performance and lowest cost • Scale compute or storage completely independent of one another • All scaling operation occur online – while the application continuous to run • Scaling actions are exposed through Cloud UI and REST APIs 1 2 3 4 5 Self-Driving | Elastic Scaling Copyright © 2019 Oracle and/or its affiliates.
- 22. Self-Driving | Auto Scaling • Enables the database to use up to 3x CPU/IO resources immediately when needed by the workload • Pay for exactly what you use • Helps CPU or IO bound workloads • Does not scale up other resources • Number of sessions • Concurrency • PGA, SGA, etc. • Need to provision more OCPUs to scale up these resources Copyright © 2019 Oracle and/or its affiliates. 1 2 3 4 5
- 23. Copyright © 2019 Oracle and/or its affiliates. ADW2 with 4 OCPUs Auto scaling enabled Can use up to 12 OCPUs ADW1 with 4 OCPUs Auto scaling disabled Can use up to 4 OCPUs
- 24. Copyright © 2019 Oracle and/or its affiliates. 8 sessions on CPU 8 running queries 100% utilization 8 running queries 8 sessions on CPU 33% utilization ADW2 with 4 OCPUs Auto scaling enabled ADW1 with 4 OCPUs Auto scaling disabled
- 25. Copyright © 2019 Oracle and/or its affiliates. 8 sessions on CPU 16 sessions waiting for CPU 24 running queries 100% utilization 24 running queries 24 sessions on CPU 100% utilization ADW2 with 4 OCPUs Auto scaling enabled ADW1 with 4 OCPUs Auto scaling disabled
- 26. Self-Driving | Auto Scaling • Historical view of the number of OCPUs used each hour Copyright © 2019 Oracle and/or its affiliates.
- 27. • Cloning creates a point-in-time copy of an ADB • Two types of clone can be created: – A full database clone – A metadata clone (Schema but no data) • Easy and fast as user only has to decide: 1. Compartment for the clone 2. Name of the clone 3. CPU and storage 4. New ADMIN password Self-Driving | Cloning Clone DB for Development or testProduction DB Copyright © 2019 Oracle and/or its affiliates.
- 28. A Look Under the Hood Key Capabilities of Self-driving, Self-Securing, Self-Repairing Self Driving Rapid Provisioning Self Scaling Automatic Indexing Cloning Self Securing Encryption by Default Self Patching Separation of Duties Auditing Self Repairing Maximum Availability Architecture Self-Healing Hardware Self-Healing Software Copyright © 2019 Oracle and/or its affiliates.
- 29. Self-Securing | Encryption by Default 29 Encryption for Data at Rest • Automatically configured – all application data is encrypted within the database • Database Backups are also encrypted Encryption for Data in Motion • Automatically configured – all network access is encrypted to and from the database • Choice of two methods – Oracle Native Network Encryption – Transport Layer Security (TLS) v1.2 (default) • Oracle client credentials can be downloaded via encrypted wallet files Copyright © 2019 Oracle and/or its affiliates.
- 30. Self-Securing | Self Patching • Automatic Patching of all components (on-demand for critical security issue) • Firmware, OS, Hypervisor, Clusterware, Database • Patches applied in a rolling fashion across RAC nodes and Exadata storage servers • Database is continuously available to application • Applications using Application Continuity best practices, run without interruption • Patching is automatically scheduled • Customer can adjust patching window within a time range on Dedicated deployments* 30 Copyright © 2019 Oracle and/or its affiliates.
- 31. But Remember, In the Cloud… Security Is a Shared Responsibility Security Managed by Oracle • Network security and monitoring • OS and platform security • Database patches and upgrades • Administrative separation of duties • Data encryption by default 31 Security Managed by the Customer • Ongoing security assessments • Users & Privileges • Sensitive data discovery • Data protection • Activity auditing Self-Securing | Separation of Duties Copyright © 2019 Oracle and/or its affiliates.
- 32. 49% 19% 19% Security Assessment High Risk: 33 Medium Risk: 22 Low Risk: 13 68 Risks 49% 11% 32% User Assessment Critical Risk: 47 High Risk: 9 Medium Risk: 2 Low Risk: 26 84 Users 26% 21% 19% 17% 17% Data Discovery Employee Basic Data…27% Public Identifier: 37 Address:34 Compensation data… Oraganization Data: 30 179 Columns Unified Database Security Control Center • Security ConfigurationAssessment • User Risk Assessment • User Activity Auditing • Sensitive Data Discovery • Data Masking Saves time and mitigates security risks Defense in Depth for all customers No special security expertise needed Announcing: Oracle Data Safe Availability: OOW 2019 Available in the Oracle Cloud at no additional cost Copyright © 2019 Oracle and/or its affiliates.32
- 33. Self-Securing | Auditing • Autonomous Database leverages Oracle Unified Audit to capture security-relevant activity • Login failures • Changes to users, including creation of new accounts, grants of privileges or roles • Changes to database structures, including tables, procedures, and synonyms • Customers have access to all of the audit data via the UNIFIED_AUDIT_TRAIL view • The DBMS_FGA package can be used to add more polices 33Copyright © 2019 Oracle and/or its affiliates.
- 34. A Look Under the Hood Key Capabilities of Self-driving, Self-Securing, Self-Repairing Self Driving Rapid Provisioning Self Scaling Automatic Indexing Cloning Self Securing Self Patching Encryption by Default Separation of Duties Auditing Self Repairing Maximum Availability Architecture Self-Healing Hardware Self-Healing Software Copyright © 2019 Oracle and/or its affiliates.
- 35. • High Availability – Protection from hardware failures, software crashes, software updates • Uses RAC Database,App continuity, Flashback DB, redundant network, triple mirrored storage, and daily backup • Fully automated daily backup, or on-demand backup • Extreme Availability – Adds protection from site outages and data corruptions (coming soon) • Uses Active DataGuard Standby • Goal is less than 20 seconds application impact from any availability event Primary Database Site #1 Site #2 Standby Database Active Data Guard Backup Service Self-Repairing | Availability Policies Copyright © 2019 Oracle and/or its affiliates.
- 36. Self-Repairing | Self-Healing Hardware 36 • Database Infrastructure for Autonomous is provided by Exadata • Exadata Provides advanced predictive failure capabilities • Unique detection of server failures without a long timeout avoids system hangs • Unique sub-second redirection of IOs around sick devices avoid database hangs Continuously monitors for sick devices Failing disk is automatically taken offline and database activities directed to mirror DevOps notified and fail drive is replaced Automatically detects change in disk latency ASM automatically rebalances data off of failing disk to create additional mirror ASM automatically detects new disk and rebalances data to create additional mirror Copyright © 2019 Oracle and/or its affiliates.
- 37. Self-Repairing | Self-Healing Software Automatically detect problems/issues: 1. Collect diagnostics info to establish an anomaly timeline or signature 2. Use Pattern Recognition to determine if it’s a known problems 3. If known problem a. Explain what will be done to fix b. Save environment checkpoint c. Apply Fix and do root cause analysis d. Roll back fix if required 4. If a new problem a. Package up all diagnostic information b. Hand off to DevOps Collect Scrubbed Logs,Traces Problem Repository Determine if it’s a known Problems using pattern recognition Save environment check point Apply FixRoll back if necessary Package for DevOps Done! Unknown Known ResolvedUnresolved Problem Occurs Copyright © 2019 Oracle and/or its affiliates.
- 38. Copyright © 2019 Oracle and/or its affiliates. Part 3 App Dev with Autonomous Database
- 39. Developer Tools out of the box with Autonomous • Execute SQL and PL/SQL commands • Build Data Models, create diagrams, generate DDL statements • Monitor and manage the database • Ability to REST enable a schema and autogenerate REST endpoints for tables and views • Provides a create, retrieve, update, and delete API on your REST-enabled objects • Web-based Function rich, low code development env • No client software needed • App definitions are stored in the database as meta data • Page generation is efficient only 1 request / response APEX Oracle REST Data ServicesSQL DeveloperWeb Copyright © 2019 Oracle and/or its affiliates.
- 40. Copyright © 2019 Oracle and/or its affiliates. Summary
- 41. Benefits of Autonomous Database • Spend Less • ReduceAdmin Cost: Eliminates expensive, tedious, unsafe manual database management • Reduce Runtime Cost: Cloud pay-per-use, Exadata speed, and self-tuning cuts costs up to 90% • Reduce Risk • Prevent cyber-attacks: Secure configurations, fully automatic and online security updates • AlwaysAvailable: protects from all downtime including maintenance and DR • Proven: Runs all your enterprise workloads, easy migration of existing databases • Innovate More • Refocus talent: Focus on new projects, user experience, analytics, ML, instead of admin • Develop faster: Instant provisioning, self-tuning, integrated database app development
- 42. Copyright © 2019 Oracle and/or its affiliates. Learn More about Autonomous Database
- 43. NEW Always Free Services always free for as long as you use them FreeTrial $300 free credits for 30 days Highly reduced pricing $ + NEW Free Tier for Oracle Cloud Learn, explore, and build for free!
- 44. Always Free - What’s Included Autonomous Database 2 x Databases 20GB Storage Each Load Balancing 10 Mbps Bandwidth Shape Storage 2 x 50GB Block 10GB Object 10GB Archive Compute 2 xVM 1GB Memory Each Available to all new and existing cloud accounts
Editor's Notes
- The Oracle Autonomous Database is: Self-Driving – Lower Cost & Increased Productivity Eliminates human labor to provision, secure, monitor, backup, recover, troubleshoot, and tune the database Automatically upgrades and patches itself while running. Testing automation ensures changes are safe. Elastically grows and shrinks compute or storage without downtime. Pay only for what you use. Self-Securing – Lower Risk Protection from external attacks and from malicious internal users Automatically applies security updates with no downtime. Automatic encryption of all data. Self-Repairing – Higher Availability Automation eliminates administrator errors SLA Guarantees 99.995% availability. Less than 30 minutes downtime per year including planned maintenance
- Oracle Autonomous Database is actually a family of products with each member of the family optimized by workload. The first member of the family to become available in March of this year was the Autonomous Data Warehouse (ADW), which has been optimized for analytic workloads, such as data warehouse, data marts or as part of a data lake. The second member of the family that’s just become available is Autonomous Transaction Processing (ATP). ATP is optimized for transaction processing or mixed workload environments and makes an excellent platform for new application development. Change Control, Policy, Operation, Customizable Operational Policy, Customizable Policies, Control over Policies
- Lets’ deep dive into how the database automatically repairs itself.. This ensures better performance and availability of the database This section highlights the benefits of self repairing and the key Oracle technologies and capabilities that enable them. This is certainly not an exhaustive list. But these are the key capabilities that we will be diving into in this section and understanding how they work to make the database self repairing
- Provisioning Oracle quickly deploys a mission critical database on our Exadata infrastructure cloud, a platform optimized for Oracle Database. The database is created as a RAC cluster, so it can scale out seamlessly, but also protect the database in case of a server failure. In order to protect the system against a complete failures, we also create a standby system using Active Data Guard in a different region in the cloud to
- 16
- Oracle execution plans are like driving directions, they will change as the data distribution changes (data volumes and statistics). Indexes can be thought of as roads and bridges, with auto indexing new roads will be added as the workload changes. 1.Capture Periodically captures SQL statements into a SQL repository Include plans, bind values, execution statistics, etc. 2.Identify Candidate Indexes Identify candidate indexes that may benefit the newly captured SQL statements Creates index candidates as unusable, invisible indexes (just metadata) 3.Verify Ask the optimizer if index candidates will be used for these statements Index candidates not used by the optimizer are automatically dropped Complete creation of chosen indexes and run captured statements to verify that the indexes did improve performance 4.Decide If performance is better for all statements, the indexes are marked visible If performance is worse for all statements, the indexes are dropped If performance is worse for some, the indexes are marked visible except for statements that regressed 5.Monitor Index usage is continuously monitored Automatically created indexes that have not been used in a long time will be dropped Rebuilds decaying indexes
- With 16 OCPUs you would get the follow number of concurrent queries before queuing would kick in 3 concurrent queries for HIGH 20 concurrent queries for MEDIUM 32 concurrent queries for LOW
- Lets’ deep dive into how the database automatically repairs itself.. This ensures better performance and availability of the database This section highlights the benefits of self repairing and the key Oracle technologies and capabilities that enable them. This is certainly not an exhaustive list. But these are the key capabilities that we will be diving into in this section and understanding how they work to make the database self repairing
- Data Encryption Security Keys AWS/Aurora violates specific rules agreed upon by the Cloud Security Alliance (CSA). The core principle is that the key provider cannot also be the hoster of the application – the separation eliminates a key vulnerability Oracle allows separation of keys Encryption - At Rest Oracle allows you to enable and disable encryption. Oracle delivers it on by default Aurora can turn on encryption when they start a database. Once on, it can’t be turned off.
- Patches applied Weekly or biweekly with ATP-s and quarterly for ATP-d Gold image – just as we do in Exadata today pre built image is built quarterly and applied.. So it is not available for serverless because there patching is done more frequently and it doesn’t make sense to build gold patches if the patches have to be applied very frequently.. The benefit of gold is that if there is a problem you can always go back to the previous version. Platinum support for patching is additional but it will be built in ATP Installs prebuilt Gold Image of patched database executables rather than directly applying patches - Currently not available in ATP-S or ADW, This functionality will only come with ATP-D Patching is very expensive because it requires downtime and several man hours to patch all the databases in your environment. Also patches may be applied once in a quarter so its an on going effort ADB will patch your systems for you while the database is running. Needs no downtime and manual effort. So it can never happen that you forgot to apply the patch or didn’t have time to do so Ensures that you are always protected from know cyber attacks
- Oracle tools: Data masking, database vault 1. No Access to the database node or file system Oracle DBAs are separated from actual data. This is not true for Aurora Aurora DBAs are given 100% access to data – cannot be revoked. This is specific area of vulnerability. It is assumed that DBA access is the way into data theft. Therefore, AWS solutions are extremely vulnerable and unsuitable for any security exposed applications. Aurora is not configurable to further restrict rights the DBA role. So, their superuser role has administrative rights as well as data read/write rights. 2. Dynamic Redaction/Masking Oracle can apply security policies as data leaves the database E.g. convert SSN to XX-1234 3. Static Masking For test/dev databases, can simply convert sensitive fields 4. Metadata Tagging – Label Security Option Data can be marked as “sensitive” “confidential” etc. Included for free in DB CS 5. Full DEFENSE IN DEPTH Built over 30 years meeting the needs of the most demanding organizations: high threat environments, security services, financial institutions Most data protection regulations require several different things: 1) Encrypt the data 2) Control access to the data 3) Monitor access to the data 4) Anonymize data in non-production systems Aurora handles requirement #1 but doesn't do much (if anything) for the other three. Oracle takes care of all 4
- [ graphics on right too small to read text; is security assessment about configurations? Now Oracle goes the next step by securing configurations and users…a space traditionally left to the customer to handle. I’m proud to announce we’ve extended the security capabilities of Oracle’s Autonomous DB by adding Data Safe! Data Safe manages risk associated with configurations, users and sensitive data. It is one integrated security service to… Identify poor configurations through configuration assessments Flag risky users or behavior Activity auditing tracks risky actions and streamlines audits Discover and … Mask sensitive data (three out of four DB instances are copies for Dev/Test, this lets you remove that risk) Security shouldn’t be an choice to make and as your trusted technology provider we’ll never pass the buck. We’ll continue to invest and innovate to bring you the most secure cloud and platforms.
- Lets’ deep dive into how the database automatically repairs itself.. This ensures better performance and availability of the database This section highlights the benefits of self repairing and the key Oracle technologies and capabilities that enable them. This is certainly not an exhaustive list. But these are the key capabilities that we will be diving into in this section and understanding how they work to make the database self repairing
- Can the policy be set at the CBD or pod level???
- ASM- automatic storage management Automatically detect anomalies or hangs in the database behavior, compare these behaviors with know issues and fix them. Exadata’s uses advance prediction capabilities to detect any kinds of hardware failure and migrate customer data off that failing h/w so the customer doesn’t get impacted at all. (its an exdata native feature to make it highly available). Exadata proactively looks for hardware faults, for example failure of drives and moves the data of their failed drive before the drive actually comes down. By default exadata tries to balance itself across a whole bunch of disks. It is predictive in nature so before a fault is able to manifest itself, it has the ability to move the customer data and workload of that failing device so the customer doesn’t see these failures. It rebalances the data of these falling devices to other devices that are currently functioning. In other words properly valid mirrors, so I/Os don’t hang, any failures don’t impact the system and the DevOps guys get a notification that they need to replace And rebalances
- Package for devops – create an SR for the problem This slide highlights how we use ML in the previous process for Database Operations We use ML algorithms like anomaly detection, pattern recognition, problem signatures to detect and prevent issues and failures and fix the know issues or raise SRs on behalf of our customers.. Bug detection should be our job not yours.. It can correlate problems across different systems to give you a complete story of the fault that occurs.