The document discusses best practices for using Oracle Database In-Memory. It provides an overview of In-Memory and describes how to configure and populate the In-Memory Column Store. It also discusses how the optimizer utilizes In-Memory statistics and hints to optimize queries for In-Memory. Several examples of queries that benefit from In-Memory, such as aggregation queries and queries with predicates, are also provided.
Agenda:
- Introduction to Optimizer Hint
- Why Optimizer
- Hint Query
- Hint Statistics
- Hint Data
- Hint Drawback
By Kumar Rajiv Rastogi at India PG Day at InMobi.
http://technology.inmobi.com/events/india-postgresql-usergroup-meetup-bangalore
New Features
● Developer and SQL Features
● DBA and Administration
● Replication
● Performance
By Amit Kapila at India PostgreSQL UserGroup Meetup, Bangalore at InMobi.
http://technology.inmobi.com/events/india-postgresql-usergroup-meetup-bangalore
Star Transformation, 12c Adaptive Bitmap Pruning and In-Memory optionFranck Pachot
Besides adaptive joins and adaptive parallel distribution, 12c comes with Adaptive Bitmap Pruning. I’ll describe the case it applies to and which is often not well known: the Star Transformation
Oracle Parallel Distribution and 12c Adaptive PlansFranck Pachot
Parallel Distribution and 12c Adaptive Plans
In the previous newsletter we have seen how 12c can defer the choice of the join method to the first execution. We considered only serial execution plans. But besides join method, the cardinality estimation is a key decision for parallel distribution when joining in parallel query. Ever seen a parallel query consuming huge tempfile space because a large table is broadcasted to lot of parallel processes? This is the point addressed by Adaptive Parallel Distribution.
Once again, that new feature is a good occasion to look at the different distribution methods.
Oracle Join Methods and 12c Adaptive PlansFranck Pachot
Join Methods and 12c Adaptive Plans
In its quest to improve cardinality estimation, 12c has introduced Adaptive Execution Plans which deals with the cardinalities that are difficult to estimate before execution. Ever seen a hanging query because a nested loop join is running on millions of rows?
This is the point addressed by Adaptive Joins. But that new feature is also a good occasion to look at the four possible join methods available for years.
Agenda:
- Introduction to Optimizer Hint
- Why Optimizer
- Hint Query
- Hint Statistics
- Hint Data
- Hint Drawback
By Kumar Rajiv Rastogi at India PG Day at InMobi.
http://technology.inmobi.com/events/india-postgresql-usergroup-meetup-bangalore
New Features
● Developer and SQL Features
● DBA and Administration
● Replication
● Performance
By Amit Kapila at India PostgreSQL UserGroup Meetup, Bangalore at InMobi.
http://technology.inmobi.com/events/india-postgresql-usergroup-meetup-bangalore
Star Transformation, 12c Adaptive Bitmap Pruning and In-Memory optionFranck Pachot
Besides adaptive joins and adaptive parallel distribution, 12c comes with Adaptive Bitmap Pruning. I’ll describe the case it applies to and which is often not well known: the Star Transformation
Oracle Parallel Distribution and 12c Adaptive PlansFranck Pachot
Parallel Distribution and 12c Adaptive Plans
In the previous newsletter we have seen how 12c can defer the choice of the join method to the first execution. We considered only serial execution plans. But besides join method, the cardinality estimation is a key decision for parallel distribution when joining in parallel query. Ever seen a parallel query consuming huge tempfile space because a large table is broadcasted to lot of parallel processes? This is the point addressed by Adaptive Parallel Distribution.
Once again, that new feature is a good occasion to look at the different distribution methods.
Oracle Join Methods and 12c Adaptive PlansFranck Pachot
Join Methods and 12c Adaptive Plans
In its quest to improve cardinality estimation, 12c has introduced Adaptive Execution Plans which deals with the cardinalities that are difficult to estimate before execution. Ever seen a hanging query because a nested loop join is running on millions of rows?
This is the point addressed by Adaptive Joins. But that new feature is also a good occasion to look at the four possible join methods available for years.
ALTER TABLE Improvements in MariaDB ServerMariaDB plc
OpenWorks 2019 Presentation
MariaDB Server 10.3 introduced ALGORITHM=NOCOPY, allowing columns to be added in an instant because adding columns no longer required rebuilding a table. In MariaDB Server 10.4, we’re taking this a step further with instant DROP COLUMN and instant ALTER TABLE to support many more instant schema changes. In this session, Marko Mäkelä explains how instant schema changes work and how MariaDB is eliminating table rebuilds.
CBO choice between Index and Full Scan: the good, the bad and the ugly param...Franck Pachot
Usually, the conclusion comes at the end. But here I will clearly show my goal: I wish I will never see the optimizer_index_cost_adj parameters again. Especially when going to 12c where Adaptive Join can be completely fooled because of it. Choosing between index access and full table scan is a key point when optimizing a query, and historically the CBO came with several ways to influence that choice. But on some system, the workarounds have accumulated one on top of the other – biasing completely the CBO estimations. And we see nested loops on huge number of rows because of those false estimations.
Performance improvements in PostgreSQL 9.5 and beyondTomas Vondra
Let's see what major performance improvements PostgreSQL 9.5 brings, measure the impact on simple examples and also briefly look at improvements likely to appear in PostgreSQL 9.6 or some of the following releases.
Memory Management for High-Performance ApplicationsEmery Berger
Fast and effective memory management is crucial for many applications, including web servers, database managers, and scientific codes. However, current memory managers do not provide adequate support for these applications on modern architectures, severely limiting their performance, scalability, and robustness.
In this talk, I describe how to design memory managers that support high-performance applications. I first address the software engineering challenges of building efficient memory managers. I then show how current general-purpose memory managers do not scale on multiprocessors, cause false sharing of heap objects, and systematically leak memory. I describe a fast, provably scalable general-purpose memory manager called Hoard (available at www.hoard.org) that solves these problems, improving performance by up to a factor of 60.
Operating System
Topic Memory Management
for Btech/Bsc (C.S)/BCA...
Memory management is the functionality of an operating system which handles or manages primary memory. Memory management keeps track of each and every memory location either it is allocated to some process or it is free. It checks how much memory is to be allocated to processes. It decides which process will get memory at what time. It tracks whenever some memory gets freed or unallocated and correspondingly it updates the status.
ALTER TABLE Improvements in MariaDB ServerMariaDB plc
OpenWorks 2019 Presentation
MariaDB Server 10.3 introduced ALGORITHM=NOCOPY, allowing columns to be added in an instant because adding columns no longer required rebuilding a table. In MariaDB Server 10.4, we’re taking this a step further with instant DROP COLUMN and instant ALTER TABLE to support many more instant schema changes. In this session, Marko Mäkelä explains how instant schema changes work and how MariaDB is eliminating table rebuilds.
CBO choice between Index and Full Scan: the good, the bad and the ugly param...Franck Pachot
Usually, the conclusion comes at the end. But here I will clearly show my goal: I wish I will never see the optimizer_index_cost_adj parameters again. Especially when going to 12c where Adaptive Join can be completely fooled because of it. Choosing between index access and full table scan is a key point when optimizing a query, and historically the CBO came with several ways to influence that choice. But on some system, the workarounds have accumulated one on top of the other – biasing completely the CBO estimations. And we see nested loops on huge number of rows because of those false estimations.
Performance improvements in PostgreSQL 9.5 and beyondTomas Vondra
Let's see what major performance improvements PostgreSQL 9.5 brings, measure the impact on simple examples and also briefly look at improvements likely to appear in PostgreSQL 9.6 or some of the following releases.
Memory Management for High-Performance ApplicationsEmery Berger
Fast and effective memory management is crucial for many applications, including web servers, database managers, and scientific codes. However, current memory managers do not provide adequate support for these applications on modern architectures, severely limiting their performance, scalability, and robustness.
In this talk, I describe how to design memory managers that support high-performance applications. I first address the software engineering challenges of building efficient memory managers. I then show how current general-purpose memory managers do not scale on multiprocessors, cause false sharing of heap objects, and systematically leak memory. I describe a fast, provably scalable general-purpose memory manager called Hoard (available at www.hoard.org) that solves these problems, improving performance by up to a factor of 60.
Operating System
Topic Memory Management
for Btech/Bsc (C.S)/BCA...
Memory management is the functionality of an operating system which handles or manages primary memory. Memory management keeps track of each and every memory location either it is allocated to some process or it is free. It checks how much memory is to be allocated to processes. It decides which process will get memory at what time. It tracks whenever some memory gets freed or unallocated and correspondingly it updates the status.
Oracle OpenWorld is the world’s largest gathering of Oracle technology experts and a prominent showcase for Accenture’s thought leadership and innovative, cross-industry solutions that can help an organization accelerate their digital transformation and become a high-velocity business. Accenture is proud to be a Premier sponsor of this year's event, which includes exclusive sponsorship of four Oracle Executive Summit sessions: the CIO Summit, the Internet of Things Summit, the Communications Summit and the Public Sector Summit, as well as exclusive sponsorship of the Communications and Public Sector tracks at Industry Central, and the Customer Experience track at CX Central.
Deep Dive: Memory Management in Apache SparkDatabricks
Memory management is at the heart of any data-intensive system. Spark, in particular, must arbitrate memory allocation between two main use cases: buffering intermediate data for processing (execution) and caching user data (storage). This talk will take a deep dive through the memory management designs adopted in Spark since its inception and discuss their performance and usability implications for the end user.
a glance on memory management in operating system.
this note is useful for those who are keen to know about how the OS works and a brief explanation regarding several terms such
-paging
segmentation
fragmentation
virtual memory
page table
to A Level A2 Computing students, this light note may be helpful for your revision
Oracle Database In-Memory Option for ILOUGZohar Elkayam
Oracle 12.1.0.2 introduced a new feature: the Oracle In Memory Option (Databases In Memory - DBIM).
This is the presentation which was given before the ILOUG DBA SIG where I introduced the technology and how to use it.
This presentation is an INTRODUCTION to intermediate MySQL query optimization for the Audience of PHP World 2017. It covers some of the more intricate features in a cursory overview.
How to make data available for analytics ASAPMariaDB plc
There are many ways to import data into MariaDB ColumnStore, including command-line tools for importing files. However, a combination of bulk and streaming data adapters makes it easy to import data on demand, without having to wait for a scheduled job. MariaDB's Jens Röwekamp and Markus Mäkelä show all of the ways to import data, from manual imports to more advanced options such as C++, Java and Python data adapters, Apache Spark, change-data-capture streams and Apache Kafka message queues – all of which can be used to import data on demand so it’s available for analytics as fast as possible.
Geek Sync I Need for Speed: In-Memory Databases in Oracle and SQL ServerIDERA Software
You can watch the replay for this Geek Sync webcast in the IDERA Resource Center: http://ow.ly/S6MG50A5ok5
Microsoft introduced IN-MEMORY OLTP, widely referred to as “Hekaton” in SQL Server 2014. Hekaton allows for the creation of fully transactionally consistent memory-resident tables designed for high concurrency and no blocking. With SQL 2016, many of the original restrictions and limitations of this feature have been reduced. IDERA’s Vicky Harp will give an overview of this feature, including how to compile T-SQL code into machine code for an even greater performance boost.
There’s also been a lot of buzz about Oracle 12c’s new IN-MEMORY COLUMN STORE. Oracle ACE Bert Scalzo will cover this new feature, how it works, it’s benefits, scripts to measure/monitor it and more. He will also touch on performance observations from benchmarking this new feature against more traditional SGA memory allocations plus Oracle 11g R2’s Database Smart Flash Cache. All findings, scripts and conclusions from this exercise will be shared. In addition, two very popular database benchmarking tools will be highlighted.
MySQL 8 -- A new beginning : Sunshine PHP/PHP UK (updated)Dave Stokes
MySQL 8 has many new features and this presentation covers the new data dictionary, improved JSON functions, roles, histograms, and much more. Updated after SunshinePHP 2018 after feedback
Optimizing the queries you send to the database can greatly increase the database's performance, but what do you know about all those strange MySQL variables that can be played with to get even more power from the database? Join me as we go over some of the basics of the various MySQL settings you can twitch, tweak and massage to get the most out of your MySQL server.
Antes de migrar de 10g a 11g o 12c, tome en cuenta las siguientes consideraciones. No es tan sencillo como simplemente cambiar de motor de base de datos, se necesita hacer consideraciones a nivel del aplicativo.
This paper describes how the optimizer uses statistics and determines plans for executing SQL statement. It explains how the 10053 trace file can be used to understand Oracle's decisions on execution plans.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Generating a custom Ruby SDK for your web service or Rails API using Smithy
Oracle Database InMemory
1. Do’s and Don’ts of
Oracle Database In-Memory
Jorge Barba
Infrastructure Principal at Accenture Enkitec Group
https://jorgebarbablog.wordpress.com
Email: jorge.barba@accenture.com
Mar 2016
4. 4
Database In-Memory
In-Memory Column Store is an optional,
static SGA pool that stores segments in
columnar format.
It is a supplement to the Buffer Cache.
The database keeps the columnar data
transactionally consistent with the Buffer
Cache.
5. 5
Row Format vs. Column Format
Transactions run faster on row format
Example: Query or Insert a sales order
Fast processing a few rows, many columns.
Analytics run faster on column format
Example: Report on sales totals by region
Fast accessing few columns, many rows
6. 6
Dual Format Database
Same table on both formats
Analytics use the new in-memory Column Format
OLTP uses the Row Format
7. 7
Scanning Memory
Buffer Cache
Have to walk along that row until we find
col4.
IM Column Store
Go directly to the col4 structure and scan
all the entries.
8. 8
Storage Index
Storage Indexes
Automatically created and maintained
for each column in the Column Store.
Allow data pruning based on filter
predicates in the SQL statement.
Keeps track of minimum and maximum
values for each column in an IMCU.
If the column value is outside the
minimum and maximum range for an
IMCU, the scan of that IMCU is
avoided.
9. 9
In-Memory Join and Bloom Filter (BF)
• A Bloom filter transforms a join into a filter that can be
applied as part of the scan of the larger table.
• Very efficiently applied to column format data via SIMD
vector processing.
• Appears in two places, at creation time and again
when it is applied.
SELECT count(*)
FROM lineorder lo, part p
WHERE lo.lo_partkey=p.p_partkey
AND lo.lo_shipmode='TRUCK'
AND lo.lo_ordtotalprice between
55000000 and 56000000
AND p.p_name='papaya burlywood’;
10. 10
Vector Group By
• New optimizer transformation introduced with Oracle 12.1.0.2.0.
• Find the total sales of footwear products in outlet stores.
• The combination of these two phases dramatically improves the efficiency of a
multiple table join with complex aggregations.
| 9 | KEY VECTOR (USE) | :KV0000
| 10 | KEY VECTOR (USE) | :KV0001
11. 11
SIMD Single Instruction Multiple Data
SIMD
A set of column values are evaluated together in a
single CPU instruction.
Designed to maximize the number of column
entries loaded and evaluated in a single CPU
instruction.
8 entries are loaded into the register for evaluation.
13. 13
Configuring the In-Memory Column Store
Configuration
• INMEMORY_SIZE=1520M
• Minimum of 100M
• Part of the SGA
• Fixed size
SQL> select * from v$sga;
NAME VALUE
-------------------- ----------
Fixed Size 2932632
Variable Size 587202664
Database Buffers 2097152000
Redo Buffers 13844480
In-Memory Area 1593835520
alter system set inmemory_size=1520M scope=spfile;
shutdown immediate;
startup
14. 14
Populating the In-Memory Column Store
ALTER TABLE lineorder INMEMORY;
ALTER TABLE lineorder NO INMEMORY;
CREATE TABLE customer …
PARTITION BY LIST
(PARTITION p1 …… INMEMORY,
PARTITION p2 …… NO INMEMORY);
Eligible segments are:
• Tables
• Partitions
• Subpartitions
• Materialized Views
Not supported:
• IOTs, Hash clusters, Out of line
LOBs.
15. 15
Composition of In-Memory Area and DML
SQL> select pool, alloc_bytes
2 from v$inmemory_area;
POOL ALLOC_BYTES
-------------- -----------
1MB POOL 1274019840
64KB POOL 301989888
16. 16
Composition of In-Memory Area and DML
• Bulk Data Loads
• Typically conducted as a direct path load.
• The size of the missing data will be visible in
the BYTES_NOT_POPULATED column (V
$IM_SEGMENTS).
• Partition Exchange Loads
• Partition big tables or fact tables.
• Transaction Processing
• Single row data change operations (DML)
execute via the Buffer Cache.
17. 17
Repopulation
• Oracle Database will repopulate an IMCU when the
number of entries reaches a threshold.
• Repopulation is more frequent for IMCUs that are
accessed frequently or have higher percentage of stale
rows.
• Also the IMCO background process may also
repopulate.
18. 18
Overhead
• Keeping the IM Column Store
transactionally consistent.
• Rate of change, compression level,
location of the changed rows, type of
operations being performed.
20. 20
Optimizer
1. In 12c the optimizer is fully aware of the In-
Memory Column Store. This means that the
optimizer will cost the read from the In-
memory Column Store.
2. Using the same statistics and also new In-
Memory statistics
21. 21
Optimizer 10053 trace
SELECT /* opt_trace_test */ sum(lo_revenue)
FROM lineorder lo, customer c
WHERE lo.lo_custkey=c.c_custkey
AND c_region='AFRICA';
column sql_text format a30
select sql_id, child_number,
sql_text from v$sql
where sql_text like
'%opt_trace_test%';
SQL_ID
-------------
5b8n5m6gtx71r
22. 22
Optimizer 10053 trace (cont)
alter session set max_dump_file_size = unlimited;
execute DBMS_SQLDIAG.DUMP_TRACE(-
p_sql_id='5b8n5m6gtx71r', -
p_child_number=0, -
p_component='Optimizer', -
p_file_id='TRACE_10053');
How to Obtain Tracing of Optimizer Computations (EVENT 10053) (Doc ID 225598.1)
col value format a90
SELECT value
FROM v$diag_info
WHERE name='Default Trace File';
VALUE
---------------------------------------------------------------------------------------
/u01/app/oracle/diag/rdbms/db_inst2/DBNAME/trace/DB_INST2_ora_130114_TRACE_10053.trc
24. 24
Disabling and Enabling In-Memory
To disable In-Memory scans set the parameter
INMEMORY_QUERY = DISABLE
Plans will get the cost based on disk statistics and scans will be from the IM Column Store.
Use hint INMEMORY to force In-Memory Scan even if INMEMORY_QUERY is disabled.
NO_INMEMORY disables In-Memory scan even if the table is in the In-Memory Column Store.
SELECT /*+ INMEMORY */ sum(lo_revenue)
FROM lineorder lo, customer c
WHERE lo.lo_custkey=c.c_custkey
AND c_region='AFRICA’;
SELECT /*+ NO_INMEMORY */ sum(lo_revenue)
FROM lineorder lo, customer c
WHERE lo.lo_custkey=c.c_custkey
AND c_region='AFRICA’;
25. 25
Optimizer Summary
• In 12.1.0.2 the Optimizer is In-Memory aware.
• In-Memory statistics automatically generated at parse.
• Cost model adjusted for costing In-Memory Scans
• New INMEMORY and NO_INMEMORY hints
27. 27
Queries
• We have chosen some of the
queries that are candidates to
benefit from Database In-
Memory.
28. 28
Function MAX(column)
select max(lo_ordtotalprice)
from lineorder;
MAX(LO_ORDTOTALPRICE)
---------------------
55903140
Elapsed time: 0.004
select /*+ NO_INMEMORY */
max(lo_ordtotalprice)
from lineorder;
MAX(LO_ORDTOTALPRICE)
---------------------
55903140
Elapsed time: 4.014
29. 29
Function MAX(column)
B-tree index on lo_ordtotalprice
create index ordtotalprice_ix on
lineorder(lo_ordtotalprice);
select /*+ NO_INMEMORY */
max(lo_ordtotalprice)
from lineorder;
MAX(LO_ORDTOTALPRICE)
---------------------
55903140
Elapsed time: 0.001
30. 30
Function MAX(column)
How about Result Cache?
SQL show parameter result_cache_max_size
big integer 22000K
select /*+ RESULT_CACHE */
max(lo_ordtotalprice)
from lineorder;
MAX(LO_ORDTOTALPRICE)
---------------------
55903140
Elapsed time: 0.001
31. 31
How do we know it used the In-Memory Column Store?
select n.name, s.value
from v$mystat s, v$statname n
where s.statistic#=n.statistic#
and n.name like 'IM scan CUs columns accessed';
-----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 3 | 2224 (12)| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 3 | | |
| 2 | TABLE ACCESS INMEMORY FULL| LINEORDER | 24M| 68M| 2224 (12)| 00:00:01 |
-----------------------------------------------------------------------------------------
NAME VALUE
---------------------------------------------------------------- ----------
IM scan CUs columns accessed 45
33. 33
One equality predicate
B-tree index on lo_orderkey
create index lo_orderkey_ix
on lineorder (lo_orderkey);
SELECT /*+ NO_INMEMORY */
lo_orderkey, lo_custkey,
lo_revenue
FROM lineorder
WHERE lo_orderkey = 4000000;
LO_ORDERKEY LO_CUSTKEY LO_REVENUE
----------- ----------- -----------
4000000 51832 6983797
4000000 51832 917952
4000000 51832 2501733
4000000 51832 7895007
Elapsed time: 0.001
34. 34
Three equality predicate
SELECT lo_orderkey, lo_custkey, lo_revenue
FROM lineorder
WHERE lo_custkey = 13286
AND lo_shipmode = 'TRUCK’
AND lo_orderpriority = '3-MEDIUM’;
LO_ORDERKEY LO_CUSTKEY LO_REVENUE
----------- ----------- -----------
8268262 13286 6777268
8268262 13286 6207689
9048868 13286 6394887
17521920 13286 2905822
19397281 13286 3573400
Elapsed time: 0.002
SELECT /*+ NO_INMEMORY */ lo_orderkey,
lo_custkey, lo_revenue
FROM lineorder
WHERE lo_custkey = 13286
AND lo_shipmode = 'TRUCK’
AND lo_orderpriority = '3-MEDIUM’;
LO_ORDERKEY LO_CUSTKEY LO_REVENUE
----------- ----------- -----------
8268262 13286 6777268
8268262 13286 6207689
9048868 13286 6394887
17521920 13286 2905822
19397281 13286 3573400
Elapsed time: 4.868
IM scan CUs columns accessed 3
IM scan segments minmax eligible 44
IM scan CUs pruned 43
35. 35
How about composite index?
create index cust_ship_pri_ix
on lineorder
(lo_custkey, lo_shipmode,
lo_orderpriority);
SELECT /*+ NO_INMEMORY */ lo_orderkey,
lo_custkey, lo_revenue
FROM lineorder
WHERE lo_custkey = 13286
AND lo_shipmode = 'TRUCK’
AND lo_orderpriority = '3-MEDIUM’;
Customer#000118324 JORDAN
Customer#000119371 ETHIOPIA
Customer#000119386 SAUDI ARABIA
Customer#000118412
VIETNAMLO_ORDERKEY LO_CUSTKEY LO_REVENUE
----------- ----------- -----------
8268262 13286 6777268
8268262 13286 6207689
9048868 13286 6394887
17521920 13286 2905822
19397281 13286 3573400
Elapsed time: 0.001
36. 36
Greater Than () Instead of Equality (=)
SELECT MAX(lo_ordtotalprice)
FROM lineorder
WHERE lo_quantity 74;
Elapsed time: 0.002
SELECT MAX(lo_ordtotalprice)
FROM lineorder
WHERE lo_quantity 74;
Elapsed time: 4.82
IM scan CUs columns accessed 3
IM scan segments minmax eligible 44
IM scan CUs pruned 43
37. 37
How about an index?
create index lo_quantity_ix on
lineorder(lo_quantity);
SELECT /*+ INDEX(a lo_quantity_ix) */
MAX(lo_ordtotalprice)
FROM lineorder a
WHERE lo_quantity 74;
Elapsed time: 0.001
38. 38
Bloom Filter Example
SELECT c.c_name, c.c_nation
FROM customer c, part p,
lineorder lo
WHERE lo.lo_custkey = c.c_custkey
AND lo.lo_partkey = p.p_partkey
AND p.p_name = 'white salmon';
C_NAME C_NATION
------------------------- ---------------
Customer#000118324 JORDAN
Customer#000119371 ETHIOPIA
Customer#000119386 SAUDI ARABIA
Customer#000118412 VIETNAM
...
...
39. 39
Vector Group By Example
SELECT /*+ VECTOR_TRANSFORM */
c.c_name, c.c_nation,
sum(lo_ordtotalprice)
FROM customer c, part p,
lineorder lo
WHERE lo.lo_custkey = c.c_custkey
AND lo.lo_partkey = p.p_partkey
AND p.p_name = 'white salmon'
group by c.c_name, c.c_nation;
...
...
C_NAME C_NATION SUM(LO_ORDTOTALPRICE)
------------------------- --------------- ---------------------
Customer#000118540 ROMANIA 12634439
Customer#000119051 ARGENTINA 28032775
Customer#000118129 JAPAN 22258279
Customer#000118198 UNITED STATES 17849279
41. 41
RAC
Each node in a RAC environment has its own IM
column store.
Objects populated into memory will be
distributed across all of the IM column stores in
the cluster.
ALTER TABLE lineorder INMEMORY
DISTRIBUTE BY PARTITION;
ALTER TABLE lineorder INMEMORY
DISTRIBUTE AUTO DUPLICATE ALL;
42. 42
M6-32
M6-32
SMP removes the overhead of distributing
queries across a cluster and coordinating
transactions.
Algorithms NUMA optimized.
Memory interconnect far faster than any
network.
43. 43
Exadata
Exadata
Complete fault-tolerant In-Memory
solution.
Exceed DRAM limits and transparently
scale across Memory, Flash and Disk.
Initial population of data into the In-
Memory column store from storage is very
fast.
The In-Memory Aggregation optimization,
can be offloaded to Exadata storage cells
44. 44
Oracle Technologies
• Data Guard
• Golden Gate
• Oracle Multitenant
• Partitioning
• Parallelism
• Resource Manager
• RMAN
• ALTER TABLE EXCHANGE
45. 45
Partition Exchange
1. Create external table for flat files
2. Use CTAS to create non part table and
gather table stats
3. Set INMEMORY attribute ON
4. Populate non part table in column store
5. Alter table table_name exchange partition
part name with table non part table;
47. 47
The benefits of IM Column Store
Speed up Scans, Joins, Aggregates.
Scan of large tables and using predicates like =, , , IN
Querying a subset of columns in a table, for example, selecting 5 of 100 columns.
Accelerating joins by converting predicates on small dimension tables into filters on a large
fact table.
50. 50
Do
Queries that scan a large number of rows and apply filters that use operators such as the
following: =, , , and IN.
Queries that select a small number of columns from a table or materialized view with a
large number of columns, such as a query that selects five columns from a table with 100
columns.
Queries that join a small table to a large table.
Queries that aggregate data.
51. 51
Don’t
Queries with complex predicates.
Queries that select a large number of columns.
Queries that return a large number of rows.
Queries with multiple large table joins.
52. 52
Don’t
In-memory speeds up analytic data access, not:
Network round trips, logon/logoff
Parsing, PL/SQL, complex functions
Data processing (as opposed to access)
Complex joins or aggregations where not much data is filtered before processing
Load and select once - Staging tables, ETL, temp tables