• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Oracle Database In-Memory Option
 

Oracle Database In-Memory Option

on

  • 252 views

Oracle Database 12c Upgrade, 15.072014

Oracle Database 12c Upgrade, 15.072014

Statistics

Views

Total Views
252
Views on SlideShare
252
Embed Views
0

Actions

Likes
1
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • This is a Safe Harbor Front slide, one of two Safe Harbor Statement slides included in this template. <br /> <br /> One of the Safe Harbor slides must be used if your presentation covers material affected by Oracle’s Revenue Recognition Policy <br /> <br /> To learn more about this policy, e-mail: Revrec-americasiebc_us@oracle.com <br /> <br /> For internal communication, Safe Harbor Statements are not required. However, there is an applicable disclaimer (Exhibit E) that should be used, found in the Oracle Revenue Recognition Policy for Future Product Communications. Copy and paste this link into a web browser, to find out more information.   <br /> <br /> http://my.oracle.com/site/fin/gfo/GlobalProcesses/cnt452504.pdf <br /> <br /> For all external communications such as press release, roadmaps, PowerPoint presentations, Safe Harbor Statements are required. You can refer to the link mentioned above to find out additional information/disclaimers required depending on your audience.
  • With the introduction of the Oracle Database In-Memory Option it is now possible <br /> to run real-time, ad-hoc, analytic queries on your business data as it exists <br /> right at this moment and receive the results in sub-seconds. <br /> <br /> True real real-time analytics. <br /> <br /> Imagine being able to know the total sales you have made in the state of California as of right now. Not last week, or even last night but right now and have that query <br /> return in sub-second time.
  • Row format is optimized for OLTP workloads. <br /> OLTP operations tend to access only a few rows but touch all of the columns. A row format allows quick access to all of the columns in a record since all the data for a given record are kept together in-memory and on-storage. Since all data for a row is kept together, much of the row data will be brought into the CPU with a single memory reference. Row format is also much more efficient for row updates and inserts. <br /> <br /> Analytical workloads access few columns but scan the entire data set. They also typically require some sort of aggregation. A columnar format allows for much faster data retrieval when only a few columns in a table are selected because all the data for a column is kept together in-memory and a single memory access will load many column values into the CPU. It also lends itself to faster filtering and aggregation, making it the most optimzed format for analytics. <br /> <br /> Up until now you have been force to pick just one format and suffer the tradeoff of either sub-optimal OLTP or sub-optimal Analytics. <br />
  • Oracle’s In-Memory columnar technology is a pure in-memory format. The in-memory columnar format is not persisted on storage. <br /> <br /> Other vendors have stored representations of the columnar format which adds overhead to change operations. <br /> In other products, changes must be logged and the changes must be merged periodically into the stored representation of the data. <br /> This means the ENTIRE table must periodically be rewritten (checkpointed) to disk which adds much overhead and IO. <br /> <br /> Oracle’s pure in-memory columnar format never causes extra writes to disk. <br /> <br /> Users decide which objects should be loaded in the in-memory column store and when. It is possible to pick all of the table or just some tables or partitions. <br /> The user also gets to choose what specific columns that they want loaded or not loaded into memory. <br /> <br /> The In-Memory Column Store is populated (memory load) by background processes. The database is fully active / accessible while this occurs. This is quite different from a pure in-memory databases. With a pure in-memory database, the database cannot be accessed until all the data is loaded which causes severe availability issues.
  • The Oracle in-memory column format is designed to enable very fast SIMD processing (single instruction processing multiple data values). <br /> <br /> You can imagine SIMD or vector processing as array processing. <br /> SIMD was originally designed for accelerating computer generated animation and High Performance Scientific computing. <br /> It describes computers with multiple processing elements that perform the same operation on multiple data points simultaneously.  <br /> <br /> With a SIMD processor there are two improvements to this process. For one the data is understood to be in blocks, and a number of values can be loaded all at once. Instead of a series of instructions saying "retrieve this blok, now retrieve the next blok", a SIMD processor will have a single instruction that effectively says "retrieve tuple bloks". <br /> <br /> Lets assume we are looking for the total number of sales we have had in the region of California this year. <br /> The sales table is stored in the In-Memory Column Store so we simply have to scan the state column and count the number of occurrence of the state of California. With SIMD processing we can check 16 values or entries in the state column in a single CPU cycle. <br /> <br /> Columnar speed comes from <br /> Scanning only needed columns <br /> SIMD optimized format <br /> Column specific compression algorithms <br /> <br /> <br /> http://en.wikipedia.org/wiki/SIMD <br />
  • We have shown that we can scan and filter in-memory data extremely quickly. <br /> But with any new data format we need to be able to join and aggregate the data as well as scan it. <br /> <br /> With In-memory vector joins, the join between the stores and sales table can be converted into a scan of the sales <br /> table with the ‘where’ clause predicate on the store table being converted into a filter on the sales table. <br /> This allows us to take full advantage of what In-memory is best for, fast scans and filters. <br />
  • Analytic style queries require more than simple filters and joins. They often require aggregations and summaries. <br /> This is where the Oracle Optimizer comes in. The optimizer recognizes these aggregation queries and transforms or rewrites them to take advantage of vector joins and vector group-bys. This allow the joins and group by <br /> (or aggregation) to occur simultaneously with the scan of the sales table, rather than waiting for the table scans and join operations to complete <br /> before beginning the aggregation. <br /> <br /> Aggregation normally has to dynamically determine the format of the result and the contents of the result at the same time. <br /> Any new row can cause the format of the result to change. <br /> By precomputing the report outline, the aggregation can run much faster since the report format is known ahead of the fact table scan and fact rows can just be added into the known format. <br />
  • Up until now, the only way to run analytic queries with an acceptable response on an OLTP environment was to create specific indexes for these queries. <br /> <br /> The good thing about indexes is that they are extremely scalable. They work well in-memory and also are extremely efficient on-disk since they minimize disk IO needed to find the requested data. <br /> <br /> Each reporting index is an additional stored structure that must be maintained and logged on every change.
  • Up until now, the only way to run analytic queries with an acceptable response on an OLTP environment was to create specific indexes for these queries. <br /> <br /> The good thing about indexes is that they are extremely scalable. They work well in-memory and also are extremely efficient on-disk since they minimize disk IO needed to find the requested data. <br /> <br /> All of these additional indexes need to be maintained as the data changes, <br /> which increase the elapse time for each of these changes.
  • The In-Memory Column store can now remove the need for additional analytic indexes if tables fit in memory. <br /> Makes DML faster and reduces the overall storage space required for system. <br /> <br /> And unlike a pure in-memory database, if the system should crash and need to restart your business can still operate fully. <br /> <br /> OLTP queries and updates (the heart of any transaction based system) will perform just as they always do against the indexed row store. <br /> <br /> Analytical queries will execute slowly until the In-Memory Column Store is populated, but they will still run. <br /> You don’t have to wait for all of the data to be populated in memory before resuming your business. <br /> <br /> Removing the need for analytic indexes greatly simplifies tuning and reduces ongoing administration. <br /> <br /> Queries on large tables no longer need to be indexed to perform well. <br /> <br /> <br /> <br />
  • Not only do we scale out in a RAC environment, we can also scale up on very large SMP systems, such as the new M6 machine, which has 32TB of RAM. <br /> <br /> SMP scaling removes the overhead of distributing queries across a cluster and coordinating transactions (such as two phase commits that are needed by shared-nothing databases) and data updates. <br /> <br /> The inter-process bandwidth on an SMP system also far exceeds any network, allowing large queries to operate more efficiently.
  • Implementing the In-Memory column store is as simple as flipping a switch. All you have to do is specify how much memory the column store can use, <br /> Then decide which objects (tables, partitions, and columns) you want to be loaded into the column store and your done using Oracle’s advisor. Oracle will take care of the rest. <br /> The Optimizer will simply direct any analytic queries to the In-Memory Column store, allowing you to do drop any indexes that were specifically created to <br /> enable analytic query performance.
  • But that’s not where the performance benefits end. Remember the fastest operation you can do is the one you don’t do. <br /> Just like we have storage indexes on Exadata, the In-memory column store <br /> Records the min/max value for each of the in-memory Column Unit (CU). This <br /> allows for very effective data pruning because when the column is being scanned <br /> We can simple skip any CU where the values we are looking for do not fall with the min / max <br /> Value of that CU. <br /> <br /> Min / max for all column stored in the CU can be read in one pass so multiple predicates can evaluated at once.
  • So, how much compression should you expect for your particular data set? <br /> The easiest way to determine the expected compression ratio and therefore the required amount of memory is to use the DBMS_COMPRESSION package. <br /> <br /> The get_compression_ratio procedure in the DBMS_COMPRESSION package will sample a set of rows from a given table or partition and apply the specified compression technique to that sample. The result <br /> Is an estimate of the compression ratio that can be expected if that object was populated into the In-Memory Column Store. <br /> <br /> This is the same method that allows you to determine the compression ratio for basic, OLTP, and HCC compression today.
  • The segments that are compatible with the INMEMORY attribute are the tables, partitions, subpartitions, inline LOBs, materialized views, materialized join views, and materialized view logs. INMEMORY partitions setting is shown in the fourth example of the slide. <br /> Clustered tables and IOTs are not supported with the INMEMORY clause. <br />
  • Oracle Database 12c: New Features for Administrators Ed 2 17 - 28
  • Under the admin tab of Oracle Enterprise Manager you will find a new In-Memory Central menu option, which opens this screen. This screen allows you manage and monitor the In-Memory column store. Including now much of the active sessions are using CPU on in-memory queries. <br /> [CLICK] In the top left hand corner you can see how much of the SGA is allocated to the column store and how much of the column store is actually being used. <br /> [CLICK] The top right hand side of the screen is an object that indicates what objects are currently in the column store and if you have the Oracle Database 12c heatmap feature then the object map will be color coded to reflect how “hot” each of the objects are. This is an easy way to monitor which objects would be good candidates to be removed from the column store. The objects in the object map are different size to reflect the amount of space they take up in the column store. [CLICK] Under the object map is detailed information on each object that is populated in the column store pulled from v$IM_SEMENTS. <br /> There are also some pre-calculated ratios for you. <br /> [CLICK] For example the compression factor for each object in the column store is pre-calculated. <br /> [CLICK] If you have a lot objects in the column store and you want to find a specfic one you can click on “search” just above the table and it will open a diaglog where you can search on the object name, type etc.
  • As with any new functionality in the Oracle Database, a new set of wait events have been created for Oracle Database In-Memory. <br /> These new wait events will show up in the new ASH Analytics page under the performance tab in EM, as well as in its predecessor the top activities page. In this example
  • As with any new functionality in the Oracle Database, a new set of wait events have been created for Oracle Database In-Memory. <br /> These new wait events will show up in the new ASH Analytics page under the performance tab in EM, as well as in its predecessor the top activities page. In this example
  • Because the In-Memory Column Store is embedded in the Oracle Database it is fully compatible with ALL existing features, <br /> And requires absolutely no changes in the application layer. This means you can start taking full advantage of it on day one, regardless of what applications you run in your environment. <br /> <br /> Any application that runs against the Oracle database will transparently benefit from the in-memory column store. <br />
  • To get the most benefits from Oracle Database In-Memory some simple rules must be followed. <br /> <br /> Data processing must be done by the database. Applications that try to pull data from the database and process it themselves will be slow forever. <br /> <br /> Data must be processed using SQL set processing. Applications that process data one-row at a time will be slow forever. They will spend all their time in communication. Also, row at a time processing cannot be parallelized by the database. <br /> <br /> Enable Oracle Database In-Memory Column format to accelerate analytics. <br /> <br /> Enable Parallel SQL. In the past customers held back on enabling parallelism because highly parallel SQL could swamp the storage subsystem which would slow down every other user on the same system. Running in-memory eliminates the possiblity of swamping the storage subsystems. Also, high core count processors are now the norm, even at the lower end of the server market. Even laptops and phones now have multiple cores. <br />

Oracle Database In-Memory Option Oracle Database In-Memory Option Presentation Transcript

  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2
  • Oracle Database In-Memory Option Powering the Real-Time Enterprise
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Database 12c In-Memory Option Goals • 100x Faster Queries: Real-Time Analytics • Instantaneous Queries on OLTP Database or Data Warehouse • Faster Mixed Workload OLTP • Transparent: no application changes • Simple to Implement 4
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Row Based Access to Data SELECT COL4 FROM MYTABLE X X X X X RESULT
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Column Based Access to Data COL1 COL2 COL3 COL4 X X X X X X X X X X X X X X X X X X X X SELECT COL4 FROM MYTABLE RESULT X X X X X
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Until Now Must Choose One Format and Suffer Tradeoffs Row Format Databases vs. Column Format Databases Row  Transactions run faster on row format – Example: Insert or query a sales order – Fast processing few rows, many columns Column  Analytics run faster on column format – Example : Report on sales totals by region – Fast accessing few columns, many rows SALES SALES 7
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Breakthrough: Dual Format Database • BOTH row and column formats for same table • Simultaneously active and transactionally consistent • Analytics & reporting use new in-memory Column format • OLTP uses proven row format 8 Memory Memory SALES SALES Row Format Column Format
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle In-Memory Columnar Technology • Pure in-memory column format • Not persistent, and no logging • Quick to change data: fast OLTP • 2x to 20x compression • Enabled at table or partition level • Available on all hardware platforms 9 SALES Pure In-Memory Columnar
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Orders of Magnitude Faster Analytic Data Scans • Each CPU core scans local in-memory columns • Scans use super fast SIMD vector instructions • Billions of rows/sec scan rate per CPU core • Row format is millions/sec 10 VectorRegister Load multiple region values Vector Compare all values an 1 cycle CPU Memory REGION CA CA CA CA Example: Find all sales in region of CA > 100x Faster
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Joining and Combining Data Also Dramatically Faster • Converts joins of data in multiple tables into fast column scans • Joins tables 10x faster 11 Example: Find total sales in outlet stores SalesStores StoreID Amount Type=Outlet StoreID in 15, 38, 64 StoreID Type Sum
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Generates Reports Instantly • Dynamically creates in-memory report outline • Then report outline filled-in during fast fact scan • Reports run much faster without predefined cubes 12 Example: Report sales of footwear in outlet stores Sales Stores Products In-Memory Report Outline Footwear Outlets $ $$ $ $$$ Footwear Sales Outlets
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Complex OLTP is Slowed by Analytic Indexes • Most Indexes in complex OLTP (e.g. ERP) databases are only used for analytic queries • Inserting one row into a table requires updating 10-20 analytic indexes: Slow! • Indexes only speed up predictable queries & reports Table 1 – 3 OLTP Indexes 10 – 20 Analytic Indexes 13
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | OLTP is Slowed Down by Analytic Indexes Insert rate decreases as more # of indexes increases
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Column Store Replaces Analytic Indexes • Fast analytics on any columns • Better for unpredictable analytics • Less tuning & administration • Column Store not persistent so update cost is much lower • OLTP & batch run faster Table 1 – 3 OLTP Indexes In-Memory Column Store 15
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Scale-Up for Maximum In-Memory Performance M6-32 Big Memory Machine 32 TB DRAM 32 Socket, 384 Cores 3 Terabyte/sec Bandwidth • Scale-Up on large SMPs • SMP scaling removes overhead of distributing queries across servers • Memory interconnect far faster than any network 16
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle In-Memory: Simple to Implement 1. Configure Memory Capacity • inmemory_size = XXX GB 2. Configure tables or partitions to be in memory • alter table | partition … inmemory; 3. Drop analytic indexes to speed up OLTP 17
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Configuring In-Memory Column Store Oracle Memory Structures System Global Area SGA Buffer Cache Shared Pool Redo Buffer Large Pool Other shared Memory Components Column Store
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Determining if In-Memory is used Execution Plan
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Determining If In-Memory Is actually Used New session statistics Actual number of In- Memory Column Units scanned
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Pure Columnar In-Memory Column Store Column Min/Max automatically recorded for each in-memory column unit Only read regions that satisfy where clause predicates Behaves similarly to an Exadata Storage Index on every column Oracle In-Memory Column Store Storage Index DRAM s t o r e i d Min 1 Max 3 Min 4 Max 7 Min 8 Max 12 Min 13 Max 15 Select … From stores Where storeid > 8;
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Use v$mystat to check what min/max pruning is achieved by the storage indexes Session Statistics Showing min/max Pruning Number of CUs pruned Number of CUs accessed
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | In-Memory Compression Unit User Chosen Compression Level Optimize Mode Details BASIC Populate Speed No compression, fastest populate, slower queries FOR QUERY Throughput Lightweight compression, fastest queries FOR CAPACITY – LOW Balanced Fast decompress, balance between space and speed FOR CAPACITY - HIGH Space Heavier weight decompressor, optimizes for space
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle In-Memory Column Store Compression Mode Compression Speed Range Avg. Populate 1x 1x Good Throughput 2x to 5x 3.3x Blazing Balanced 3x to 10x 5.2x Excellent Space 5x to 20x 10x Good
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Compression Advisor And In-Memory  Easy way to determine memory requirements  Use DBMS_COMPRESSION  Applies memcompress to sample set of data from a table  Returns estimated compression ratio Determining your compression ratio In-Memory
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Objects eligible for In-Memory • The following segment types are eligible for In-Memory –Tables –Partitions –Subpartition –Materialized views
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Determining What Objects Are In Column Store • V$IM_SEGMENTS New v$ views
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Setting INMEMORY attributes on objects • Enable/disable objects to be populated into IM column store – Whole table – Part of the table on demand – Whole table on startup
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Monitoring IM Column Store via EM New In-Memory Central Screen Object map integrated with 12c heatmap feature In-Memory Query (CPU):0.16
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Monitoring In-Memory Activity in ASH Analytics New In-Memory Query wait events CPU used to complete In-Memory Column Store population
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Monitoring In-Memory Activity in ASH Analytics New In-Memory Populate wait event CPU used for queries accessing In- Memory objects
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle In-Memory Requires Zero Application Changes Full Functionality - No restrictions on SQL Easy to Implement - No migration of data Fully Compatible - All existing applications run unchanged Fully Multitenant - Oracle In-Memory is Cloud Ready Uniquely Achieves All In-Memory Benefits With No Application Changes 32
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Keys to Real-Time Data Processing • Process data in database not application • Process sets of rows using SQL • Row by row processing is slow and cannot be parallelized • Enable in-memory column format • Enable parallel SQL • Memory removes storage bottlenecks enabling highly parallel SQL 33
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Time for coffee!
  • Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 36