You can watch the replay for this Geek Sync webcast in the IDERA Resource Center: http://ow.ly/S6MG50A5ok5
Microsoft introduced IN-MEMORY OLTP, widely referred to as “Hekaton” in SQL Server 2014. Hekaton allows for the creation of fully transactionally consistent memory-resident tables designed for high concurrency and no blocking. With SQL 2016, many of the original restrictions and limitations of this feature have been reduced. IDERA’s Vicky Harp will give an overview of this feature, including how to compile T-SQL code into machine code for an even greater performance boost.
There’s also been a lot of buzz about Oracle 12c’s new IN-MEMORY COLUMN STORE. Oracle ACE Bert Scalzo will cover this new feature, how it works, it’s benefits, scripts to measure/monitor it and more. He will also touch on performance observations from benchmarking this new feature against more traditional SGA memory allocations plus Oracle 11g R2’s Database Smart Flash Cache. All findings, scripts and conclusions from this exercise will be shared. In addition, two very popular database benchmarking tools will be highlighted.
2. Who Are We
Bert Scalzo
• Senior PM with IDERA
• Author (10 books), Speaker,
Oracle subject matter expert
Vicky Harp
• Strategist with IDERA
• Developer, Speaker, SQL
Server subject matter expert
3. Agenda
• Introductions
• What are In Memory Databases
• In Memory Databases in Oracle
• In Memory Databases in SQL Server
• Wrap-Up
4. In Memory Databases
• Databases or table structures which reside
entirely or primarily in main memory
• Able to make efficient use of the huge
RAM storage available on modern systems
• Able to utilize structures and algorithms
which would be inefficient on disk
• May or may not be fully ACID compliant
• May be 100s of times faster
6. In Memory for Oracle
• In Memory Column Store (IMCS)
• New Feature Oracle 12c 2nd release
– 12.1.0.2.0 – July 214
– not in 12.1.0.1.0 – 2013
• The IMCS is a new static pool in the SGA.
• Data in IMCS does not reside in traditional row format, instead, in a
columnar format.
• Each column is stored as a separate structure.
• IMCS does not replace the buffer cache, but it acts as a supplement so data
can be stored in memory in both row & columnar formats.
• To enable IMCS, INMEMORY_SIZE init parameter must = a non-zero value.
• The IMCS is included with the Oracle Enterprise Edition
• IMCS is an option (i.e. not free, costs $$$ like partitioning).
7. IMCS Pros & Cons
IMCS can apply to:
• Column
• Table
• Materialized View
• Tablespace
• Partition
IMCS is good for:
• A query that scans a large number of rows
and applies filters that use operators such
as: =, <, >, and IN
• A query that selects a small number of
columns from a table or materialized view
with a large number of columns, such as a
query that selects five columns from a table
with 100 columns (e.g. DW Dimension)
• A query that joins a small table to a large
table (e.g. DW Dimension)
• A query that aggregates data (e.g. DW fact)
8. IMCS Pros & Cons
Availability:
• 12c 12.1.0.2
• Enterprise Edition
• EE Option $$$
(see next slide)
IMCS is not good for:
• IM is NOT good for:
• Queries with complex predicates
• Queries that selects lots of columns
• Queries that return lots of rows
• Queries with multiple large table joins
• Objects owned by SYS and stored in
SYSTEM or SYSAUX tablespaces
10. Oracle SGA
As Oracle Database populates
data in the IM column store and
converts this data to columnar
format, the columnar version of
the table gradually becomes
available.
If a query requires data that has
not yet been populated in the IM
column store, then the database
reads the requested data from
the buffer cache or disk while
background processes
asynchronously populate the
missing data in the IM column
store.
17. Loading Options
When a database is restarted, all of the data for database objects with a priority level other than NONE
is populated in the IM column store during startup.
18. Test Approach
The TPC Benchmark™ H (TPC-H) is a decision support
benchmark. It consists of a suite of business-oriented ad-
hoc queries and concurrent data modifications. The
queries and the data populating the database were
chosen to have broad industry-wide relevance, while
maintaining a sufficient degree of ease of implementation.
This benchmark illustrates decision support systems that
do the following:
– Examine large volumes of data;
– Execute queries with a high degree of complexity;
– Give answers to critical business questions.
http://www.tpc.org/tpc_documents_current_versions/pdf/tpch2.17.1.pdf
20. Test Results
Test #
Feature
Debut
Configuration Parameters
Run Time
(Mins:Secs)
Compress
Factor
% Run
Time
Reduction
1
SGA_TARGET = 4G
*** Nothing Else–Baseline *** 45:52
2 11g
SGA_TARGET = 4G
DB_FLASH_CACHE_SIZE = 16G
DB_FLASH_CACHE_FILE = '/flash/flash01.dbf'
23:19 49%
3
SGA_TARGET = 20G
*** Nothing Else – All memory buffer cache *** 19:50 57%
4 12c
SGA_TARGET = 20G
DB_BIG_TABLE_CACHE_PERCENT_TARGET = 80
PARALLEL_DEGREE_POLICY=auto
ORA-600
5 12c
SGA_TARGET = 20G
INMEMORY_SIZE = 16G
INMEMORY (default)
PRIORITY CRITICAL (restart instance & wait for object to load)
05:01 3.63X 89%
6 12c
SGA_TARGET = 20G
INMEMORY_SIZE = 16G
INMEMORY MEMCOMPRESS FOR QUERY HIGH
PRIORITY CRITICAL (restart instance & wait for object to load)
04:56 4.68X 89%
21. Script #1
-- im_tabs.sql
set linesize 256
set pagesize 999
set verify off
col OBJECT format a30
SELECT owner||'.'||table_name OBJECT,
inmemory INMEMORY,
inmemory_priority PRIORITY,
inmemory_distribute DISTRIBUTE,
inmemory_compression COMPRESSION,
inmemory_duplicate DUPLICATE
FROM all_tables
where owner like upper('%&1%')
ORDER BY inmemory, owner||'.'||table_name;
SQL> @im_tabs BMF1
OBJECT INMEMORY PRIORITY DISTRIBUTE COMPRESSION DUPLICATE
------------------------------ -------- -------- --------------- ----------------- ------------
BMF1.H_CUSTOMER ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE
BMF1.H_LINEITEM ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE
BMF1.H_NATION ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE
BMF1.H_ORDER ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE
BMF1.H_PART ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE
BMF1.H_PARTSUPP ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE
BMF1.H_REGION ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE
BMF1.H_SUPPLIER ENABLED CRITICAL AUTO FOR QUERY LOW NO DUPLICATE
First, we need to know which tables for our test
schema (BMF1) have been “in-memory” enabled
and with which options.
22. Script #2
-- im_segs.sql
set linesize 256
set pagesize 999
set verify off
col owner format a20
col segment_name format a30
select owner, segment_name, populate_status
from v$im_segments;
SQL> @im_segs
OWNER SEGMENT_NAME POP STATU
-------------------- ------------------------------ ---------
BMF1 H_LINEITEM STARTED
BMF1 H_SUPPLIER COMPLETED
BMF1 H_PARTSUPP COMPLETED
BMF1 H_ORDER COMPLETED
BMF1 H_PART COMPLETED
BMF1 H_CUSTOMER COMPLETED
Second, we need to know when our “in-memory”
enabled tables have been successfully loaded.
In my tests, I waited until all such enabled tables
were COMPLETED to more fully leverage the in-
memory feature.
23. Script #3
-- im_pct.sql
set linesize 256
set pagesize 999
set verify off
select segment_name,ROUND(SUM(BYTES)/1024/1024/1024,2) "ORIG GB",
ROUND(SUM(INMEMORY_SIZE)/1024/1024/1024,2) "IN-MEM GB",
ROUND(SUM(BYTES-BYTES_NOT_POPULATED)*100/SUM(BYTES),2) "% IN_MEM",
ROUND(SUM(BYTES-BYTES_NOT_POPULATED)/SUM(INMEMORY_SIZE),2) "COMP RATIO"
from V$IM_SEGMENTS
group by owner,segment_name
order by SUM(bytes) desc;
SQL> @im_pct -- Try #1 - INMEMORY (default)
SEGMENT_NAME ORIG GB IN-MEM GB % IN_MEM COMP RATIO
------------------------------ ---------- ---------- ---------- ----------
H_LINEITEM 8.42 2.32 100 3.63
H_ORDER 1.75 .89 100 1.97
H_PARTSUPP .6 .35 100 1.71
H_PART .29 .13 100 2.29
H_CUSTOMER .22 .19 100 1.19
H_SUPPLIER .01 .02 100 .91
SQL> @im_pct -- Try #2 - INMEMORY MEMCOMPRESS FOR QUERY HIGH
SEGMENT_NAME ORIG GB IN-MEM GB % IN_MEM COMP RATIO
------------------------------ ---------- ---------- ---------- ----------
H_LINEITEM 8.42 1.77 98.17 4.68
H_ORDER 1.75 .63 100 2.78
H_PARTSUPP .6 .32 100 1.86
H_PART .29 .09 100 3.16
H_CUSTOMER .22 .15 100 1.52
H_SUPPLIER .01 .01 100 1.31
wait
Third, we need to know how well our “in-
memory” enabled tables compressed to confirm
the Oracle statement that the in-memory feature
does not add much overhead (20%).
25. In Memory Databases in SQL
Server
• First appearance in SQL 2014 “Hekaton”
• Enterprise Edition feature
• Increased functionality in SQL 2016
• Called by Many Names
– In-Memory OLTP
– Memory Optimized Tables
– Hekaton (or just HK)
26. SQL Server In-Memory OLTP
• A distinct memory-optimized engine within
SQL Server
• Distinct storage and memory structures from
the rest of SQL Server
• Completely lock-less design eliminates
blocking
• ACID compliant with optional durability
• Interacts with traditional TSQL or new
natively-compiled stored procedures
28. Memory Optimized Tables
• Created on a MEMORY_OPTIMIZED_DATA
filegroup
• Completely distinct physical structure than
on-disk tables
• Tables are compiled to a DLL on creation
• Data and indexes fully resident in RAM
• Fully optimistic (lockless) concurrency
29. Memory Optimized Table
Structure
• Data is written to and read from hash
buckets in memory
• Table data is persisted to the transaction
log and periodically flushed to disk
• Indexes are maintained only in memory
and never persisted
30. Memory Optimized Table
Limitations
• Long list of miscellaneous unsupported
features and commands
– Computed columns, clustered indexes, sparse
columns, DDL triggers, full text indexing,
truncate, data compression, dbcc checkdb
• Refer to documentation
31. Durability Options
• SCHEMA_AND_DATA
– Changes are logged in transaction log
– Table structure and data are persisted on disk
– ACID compliant
• SCHEMA_ONLY
– Changes are not logged
– Table structure is persisted
– Data is not persisted
– Atomic, Consistent, and Isolated but not Durable
32. T-SQL Interop
• Access Memory-Optimized tables directly
using regular T-SQL
• Allows “cross-container” access
• Most (but not all) T-SQL language constructs
are supported
• Only certain isolation levels supported
– Snapshot (with caveats)
– Repeatable Read
– Serializable
33. Natively Compiled Stored
Procedures
• Stored procedures written in T-SQL and
compiled directly to DLLs
• Do not incur compilation overhead
• Much, much faster
• Many unsupported T-SQL features
– LIKE, CONTAINS, PIVOT, EXCEPT, common
table expressions, cursors, select into, temp
tables, etc
34. Memory Optimized Table Types
• May be used as a temporary table but
without making use of tempdb
• May be desirable for temporary objects
which can reside entirely in memory
Welcome can be done by Bert or Vicky. Recommend Bert to move smoothly to next slide.
Bert to introduce himself
Vicky to introduce herself
Bert or Vicky to go over agenda. Recommend Vicky as continuation of previous slide.
Bert AND Vicky go over this slide, extemporaneously
Recommendation: Vicky to read bullet points, add a little color, converse (like 1 line) with Bert on each.
Sample script:
Vicky: “Databases which reside in main memory” continues to “That is to say, rather than having all your data on disk and then read into memory as it’s being accessed, in-memory objects reside primarily in memory and if they’re written to disk at all it’s only for the purposes of durability through a server reboot.”
Bert: “Yeah, that’s a complete inversion of the way the enterprise RDBMS systems have traditionally worked. <etc>”
Vicky: “These in-memory databases are able to make more efficient use of the huge RAM storage…” continuing to “Until recently the limitations of RAM on even large servers made this sort of feature impractical because any database which could fit entirely in memory was also small enough that the overhead of disk activities just wasn’t very meaningful. But now that we have really huge servers that’s changing.”
Bert: (whatever)
(and so forth)
Let’s be sure to review what ACID stands for at the appropriate moment
(ending with Bert transitioning to his slides)
This may need to be 2 slides, one for each platform