SlideShare a Scribd company logo

SQL Server 2014 In-Memory OLTP

Deep dive deck given at SQL Saturday Manchester 2015 covering Checkpoint File Pairs, Hash Indexign etc.

1 of 41
Download to read offline
SQL Server 2014
In-Memory OLTP
Deep Dive
tonyrogerson@torver.net
@tonyrogerson
http://dataidol.com/tonyrogerson
Agenda
• Memory
• Storage
• MVCC
• Indexes
• Hash
• Range
Memory
SQL Server Memory Pools
Stable
Storage
(MDF/NDF
Files)
Buffer Pool
Table
Data Data in/out as required
Memory Internal Structures (proc/log
cache etc.)
SQL Server
Memory
Space
Memory Optimised Tables
Memory Optimised Tables
SQL Server Memory Pools – MOT aggression
Stable
Storage
(MDF/NDF
Files)
Buffer Pool
Table
Data
Memory Internal Structures (proc/log
cache etc.)
SQL Server
Memory
Space
Create Resource Pool
CREATE RESOURCE POOL mem_xtp_pool
WITH ( MAX_MEMORY_PERCENT = 50,
MIN_MEMORY_PERCENT = 50
);
ALTER RESOURCE GOVERNOR RECONFIGURE;
EXEC sp_xtp_bind_db_resource_pool 'xtp_demo', 'mem_xtp_pool';
ALTER DATABASE xtp_demo SET OFFLINE WITH ROLLBACK IMMEDIATE;
ALTER DATABASE xtp_demo SET ONLINE;
Best
Practice

Recommended

RocksDB compaction
RocksDB compactionRocksDB compaction
RocksDB compactionMIJIN AN
 
Off-heaping the Apache HBase Read Path
Off-heaping the Apache HBase Read Path Off-heaping the Apache HBase Read Path
Off-heaping the Apache HBase Read Path HBaseCon
 
Some key value stores using log-structure
Some key value stores using log-structureSome key value stores using log-structure
Some key value stores using log-structureZhichao Liang
 
RocksDB detail
RocksDB detailRocksDB detail
RocksDB detailMIJIN AN
 
Meet hbase 2.0
Meet hbase 2.0Meet hbase 2.0
Meet hbase 2.0enissoz
 

More Related Content

What's hot

HBaseCon 2013: Apache HBase at Pinterest - Scaling Our Feed Storage
HBaseCon 2013: Apache HBase at Pinterest - Scaling Our Feed StorageHBaseCon 2013: Apache HBase at Pinterest - Scaling Our Feed Storage
HBaseCon 2013: Apache HBase at Pinterest - Scaling Our Feed StorageCloudera, Inc.
 
HBaseCon 2012 | Learning HBase Internals - Lars Hofhansl, Salesforce
HBaseCon 2012 | Learning HBase Internals - Lars Hofhansl, SalesforceHBaseCon 2012 | Learning HBase Internals - Lars Hofhansl, Salesforce
HBaseCon 2012 | Learning HBase Internals - Lars Hofhansl, SalesforceCloudera, Inc.
 
HBaseCon 2015: HBase Operations at Xiaomi
HBaseCon 2015: HBase Operations at XiaomiHBaseCon 2015: HBase Operations at Xiaomi
HBaseCon 2015: HBase Operations at XiaomiHBaseCon
 
Optimizing RocksDB for Open-Channel SSDs
Optimizing RocksDB for Open-Channel SSDsOptimizing RocksDB for Open-Channel SSDs
Optimizing RocksDB for Open-Channel SSDsJavier González
 
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of FacebookTech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of FacebookThe Hive
 
8a. How To Setup HBase with Docker
8a. How To Setup HBase with Docker8a. How To Setup HBase with Docker
8a. How To Setup HBase with DockerFabio Fumarola
 
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, HerokuPostgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, HerokuRedis Labs
 
MySQL shell and It's utilities - Praveen GR (Mydbops Team)
MySQL shell and It's utilities - Praveen GR (Mydbops Team)MySQL shell and It's utilities - Praveen GR (Mydbops Team)
MySQL shell and It's utilities - Praveen GR (Mydbops Team)Mydbops
 
HBase: Where Online Meets Low Latency
HBase: Where Online Meets Low LatencyHBase: Where Online Meets Low Latency
HBase: Where Online Meets Low LatencyHBaseCon
 
Data analysis scala_spark
Data analysis scala_sparkData analysis scala_spark
Data analysis scala_sparkYiguang Hu
 
Exadata下的数据并行加载、并行卸载及性能监控
Exadata下的数据并行加载、并行卸载及性能监控Exadata下的数据并行加载、并行卸载及性能监控
Exadata下的数据并行加载、并行卸载及性能监控Kaiyao Huang
 
Managing terabytes: When Postgres gets big
Managing terabytes: When Postgres gets bigManaging terabytes: When Postgres gets big
Managing terabytes: When Postgres gets bigSelena Deckelmann
 
HBase Blockcache 101
HBase Blockcache 101HBase Blockcache 101
HBase Blockcache 101Nick Dimiduk
 
The Hive Think Tank: Rocking the Database World with RocksDB
The Hive Think Tank: Rocking the Database World with RocksDBThe Hive Think Tank: Rocking the Database World with RocksDB
The Hive Think Tank: Rocking the Database World with RocksDBThe Hive
 
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...Cloudera, Inc.
 
Meet HBase 1.0
Meet HBase 1.0Meet HBase 1.0
Meet HBase 1.0enissoz
 
Digital Library Collection Management using HBase
Digital Library Collection Management using HBaseDigital Library Collection Management using HBase
Digital Library Collection Management using HBaseHBaseCon
 
Evolution of MongoDB Replicaset and Its Best Practices
Evolution of MongoDB Replicaset and Its Best PracticesEvolution of MongoDB Replicaset and Its Best Practices
Evolution of MongoDB Replicaset and Its Best PracticesMydbops
 

What's hot (20)

HBaseCon 2013: Apache HBase at Pinterest - Scaling Our Feed Storage
HBaseCon 2013: Apache HBase at Pinterest - Scaling Our Feed StorageHBaseCon 2013: Apache HBase at Pinterest - Scaling Our Feed Storage
HBaseCon 2013: Apache HBase at Pinterest - Scaling Our Feed Storage
 
HBaseCon 2012 | Learning HBase Internals - Lars Hofhansl, Salesforce
HBaseCon 2012 | Learning HBase Internals - Lars Hofhansl, SalesforceHBaseCon 2012 | Learning HBase Internals - Lars Hofhansl, Salesforce
HBaseCon 2012 | Learning HBase Internals - Lars Hofhansl, Salesforce
 
HBaseCon 2015: HBase Operations at Xiaomi
HBaseCon 2015: HBase Operations at XiaomiHBaseCon 2015: HBase Operations at Xiaomi
HBaseCon 2015: HBase Operations at Xiaomi
 
Optimizing RocksDB for Open-Channel SSDs
Optimizing RocksDB for Open-Channel SSDsOptimizing RocksDB for Open-Channel SSDs
Optimizing RocksDB for Open-Channel SSDs
 
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of FacebookTech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
 
Hbase Nosql
Hbase NosqlHbase Nosql
Hbase Nosql
 
8a. How To Setup HBase with Docker
8a. How To Setup HBase with Docker8a. How To Setup HBase with Docker
8a. How To Setup HBase with Docker
 
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, HerokuPostgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
 
MySQL shell and It's utilities - Praveen GR (Mydbops Team)
MySQL shell and It's utilities - Praveen GR (Mydbops Team)MySQL shell and It's utilities - Praveen GR (Mydbops Team)
MySQL shell and It's utilities - Praveen GR (Mydbops Team)
 
HBase: Where Online Meets Low Latency
HBase: Where Online Meets Low LatencyHBase: Where Online Meets Low Latency
HBase: Where Online Meets Low Latency
 
Data analysis scala_spark
Data analysis scala_sparkData analysis scala_spark
Data analysis scala_spark
 
Exadata下的数据并行加载、并行卸载及性能监控
Exadata下的数据并行加载、并行卸载及性能监控Exadata下的数据并行加载、并行卸载及性能监控
Exadata下的数据并行加载、并行卸载及性能监控
 
Managing terabytes: When Postgres gets big
Managing terabytes: When Postgres gets bigManaging terabytes: When Postgres gets big
Managing terabytes: When Postgres gets big
 
HBase Blockcache 101
HBase Blockcache 101HBase Blockcache 101
HBase Blockcache 101
 
The Hive Think Tank: Rocking the Database World with RocksDB
The Hive Think Tank: Rocking the Database World with RocksDBThe Hive Think Tank: Rocking the Database World with RocksDB
The Hive Think Tank: Rocking the Database World with RocksDB
 
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
 
Meet HBase 1.0
Meet HBase 1.0Meet HBase 1.0
Meet HBase 1.0
 
Advanced Sqoop
Advanced Sqoop Advanced Sqoop
Advanced Sqoop
 
Digital Library Collection Management using HBase
Digital Library Collection Management using HBaseDigital Library Collection Management using HBase
Digital Library Collection Management using HBase
 
Evolution of MongoDB Replicaset and Its Best Practices
Evolution of MongoDB Replicaset and Its Best PracticesEvolution of MongoDB Replicaset and Its Best Practices
Evolution of MongoDB Replicaset and Its Best Practices
 

Viewers also liked

Database Performance
Database PerformanceDatabase Performance
Database PerformanceBoris Hristov
 
Sql disaster recovery
Sql disaster recoverySql disaster recovery
Sql disaster recoverySqlperfomance
 
Scaling sql server 2014 parallel insert
Scaling sql server 2014 parallel insertScaling sql server 2014 parallel insert
Scaling sql server 2014 parallel insertChris Adkin
 
Partitioning kendralittle
Partitioning kendralittlePartitioning kendralittle
Partitioning kendralittlengupt28
 
Sql server scalability fundamentals
Sql server scalability fundamentalsSql server scalability fundamentals
Sql server scalability fundamentalsChris Adkin
 
физическая структура хранения артемов Ready
физическая структура хранения артемов Readyфизическая структура хранения артемов Ready
физическая структура хранения артемов Readyrit2010
 
Sql server performance tuning
Sql server performance tuningSql server performance tuning
Sql server performance tuningngupt28
 
Sphinx new
Sphinx newSphinx new
Sphinx newrit2010
 
Индексы в MSSQL: принципы работы и способы оптимизации
Индексы в MSSQL: принципы работы и способы оптимизацииИндексы в MSSQL: принципы работы и способы оптимизации
Индексы в MSSQL: принципы работы и способы оптимизацииAlexander Byndyu
 
Customer satisfaction для программистов
Customer satisfaction для программистовCustomer satisfaction для программистов
Customer satisfaction для программистовAlexander Byndyu
 
Sql Server Performance Tuning
Sql Server Performance TuningSql Server Performance Tuning
Sql Server Performance TuningBala Subra
 
Ten query tuning techniques every SQL Server programmer should know
Ten query tuning techniques every SQL Server programmer should knowTen query tuning techniques every SQL Server programmer should know
Ten query tuning techniques every SQL Server programmer should knowKevin Kline
 
А.Левенчук -- корпоративный искусственный интеллект
А.Левенчук -- корпоративный искусственный интеллектА.Левенчук -- корпоративный искусственный интеллект
А.Левенчук -- корпоративный искусственный интеллектAnatoly Levenchuk
 
Performance tuning and optimization (ppt)
Performance tuning and optimization (ppt)Performance tuning and optimization (ppt)
Performance tuning and optimization (ppt)Harish Chand
 
Database Performance Tuning Introduction
Database  Performance Tuning IntroductionDatabase  Performance Tuning Introduction
Database Performance Tuning IntroductionMyOnlineITCourses
 

Viewers also liked (15)

Database Performance
Database PerformanceDatabase Performance
Database Performance
 
Sql disaster recovery
Sql disaster recoverySql disaster recovery
Sql disaster recovery
 
Scaling sql server 2014 parallel insert
Scaling sql server 2014 parallel insertScaling sql server 2014 parallel insert
Scaling sql server 2014 parallel insert
 
Partitioning kendralittle
Partitioning kendralittlePartitioning kendralittle
Partitioning kendralittle
 
Sql server scalability fundamentals
Sql server scalability fundamentalsSql server scalability fundamentals
Sql server scalability fundamentals
 
физическая структура хранения артемов Ready
физическая структура хранения артемов Readyфизическая структура хранения артемов Ready
физическая структура хранения артемов Ready
 
Sql server performance tuning
Sql server performance tuningSql server performance tuning
Sql server performance tuning
 
Sphinx new
Sphinx newSphinx new
Sphinx new
 
Индексы в MSSQL: принципы работы и способы оптимизации
Индексы в MSSQL: принципы работы и способы оптимизацииИндексы в MSSQL: принципы работы и способы оптимизации
Индексы в MSSQL: принципы работы и способы оптимизации
 
Customer satisfaction для программистов
Customer satisfaction для программистовCustomer satisfaction для программистов
Customer satisfaction для программистов
 
Sql Server Performance Tuning
Sql Server Performance TuningSql Server Performance Tuning
Sql Server Performance Tuning
 
Ten query tuning techniques every SQL Server programmer should know
Ten query tuning techniques every SQL Server programmer should knowTen query tuning techniques every SQL Server programmer should know
Ten query tuning techniques every SQL Server programmer should know
 
А.Левенчук -- корпоративный искусственный интеллект
А.Левенчук -- корпоративный искусственный интеллектА.Левенчук -- корпоративный искусственный интеллект
А.Левенчук -- корпоративный искусственный интеллект
 
Performance tuning and optimization (ppt)
Performance tuning and optimization (ppt)Performance tuning and optimization (ppt)
Performance tuning and optimization (ppt)
 
Database Performance Tuning Introduction
Database  Performance Tuning IntroductionDatabase  Performance Tuning Introduction
Database Performance Tuning Introduction
 

Similar to SQL Server 2014 In-Memory OLTP

SQL Server 2014 Memory Optimised Tables - Advanced
SQL Server 2014 Memory Optimised Tables - AdvancedSQL Server 2014 Memory Optimised Tables - Advanced
SQL Server 2014 Memory Optimised Tables - AdvancedTony Rogerson
 
SQL Server 2014 Extreme Transaction Processing (Hekaton) - Basics
SQL Server 2014 Extreme Transaction Processing (Hekaton) - BasicsSQL Server 2014 Extreme Transaction Processing (Hekaton) - Basics
SQL Server 2014 Extreme Transaction Processing (Hekaton) - BasicsTony Rogerson
 
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]Malin Weiss
 
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]Speedment, Inc.
 
AWS re:Invent 2016: Streaming ETL for RDS and DynamoDB (DAT315)
AWS re:Invent 2016: Streaming ETL for RDS and DynamoDB (DAT315)AWS re:Invent 2016: Streaming ETL for RDS and DynamoDB (DAT315)
AWS re:Invent 2016: Streaming ETL for RDS and DynamoDB (DAT315)Amazon Web Services
 
Inside sql server in memory oltp sql sat nyc 2017
Inside sql server in memory oltp sql sat nyc 2017Inside sql server in memory oltp sql sat nyc 2017
Inside sql server in memory oltp sql sat nyc 2017Bob Ward
 
Performance Tuning RocksDB for Kafka Streams’ State Stores
Performance Tuning RocksDB for Kafka Streams’ State StoresPerformance Tuning RocksDB for Kafka Streams’ State Stores
Performance Tuning RocksDB for Kafka Streams’ State Storesconfluent
 
Performance Tuning RocksDB for Kafka Streams' State Stores (Dhruba Borthakur,...
Performance Tuning RocksDB for Kafka Streams' State Stores (Dhruba Borthakur,...Performance Tuning RocksDB for Kafka Streams' State Stores (Dhruba Borthakur,...
Performance Tuning RocksDB for Kafka Streams' State Stores (Dhruba Borthakur,...confluent
 
MariaDB ColumnStore
MariaDB ColumnStoreMariaDB ColumnStore
MariaDB ColumnStoreMariaDB plc
 
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...Fwdays
 
Monitoring MongoDB’s Engines in the Wild
Monitoring MongoDB’s Engines in the WildMonitoring MongoDB’s Engines in the Wild
Monitoring MongoDB’s Engines in the WildTim Vaillancourt
 
Apache Drill talk ApacheCon 2018
Apache Drill talk ApacheCon 2018Apache Drill talk ApacheCon 2018
Apache Drill talk ApacheCon 2018Aman Sinha
 
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibabahbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at AlibabaMichael Stack
 
HBase in Practice
HBase in PracticeHBase in Practice
HBase in Practicelarsgeorge
 
Best Practices for Migrating Your Data Warehouse to Amazon Redshift
Best Practices for Migrating Your Data Warehouse to Amazon RedshiftBest Practices for Migrating Your Data Warehouse to Amazon Redshift
Best Practices for Migrating Your Data Warehouse to Amazon RedshiftAmazon Web Services
 

Similar to SQL Server 2014 In-Memory OLTP (20)

SQL Server 2014 Memory Optimised Tables - Advanced
SQL Server 2014 Memory Optimised Tables - AdvancedSQL Server 2014 Memory Optimised Tables - Advanced
SQL Server 2014 Memory Optimised Tables - Advanced
 
SQL Server 2014 Extreme Transaction Processing (Hekaton) - Basics
SQL Server 2014 Extreme Transaction Processing (Hekaton) - BasicsSQL Server 2014 Extreme Transaction Processing (Hekaton) - Basics
SQL Server 2014 Extreme Transaction Processing (Hekaton) - Basics
 
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
 
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
 
AWS re:Invent 2016: Streaming ETL for RDS and DynamoDB (DAT315)
AWS re:Invent 2016: Streaming ETL for RDS and DynamoDB (DAT315)AWS re:Invent 2016: Streaming ETL for RDS and DynamoDB (DAT315)
AWS re:Invent 2016: Streaming ETL for RDS and DynamoDB (DAT315)
 
L6.sp17.pptx
L6.sp17.pptxL6.sp17.pptx
L6.sp17.pptx
 
Inside sql server in memory oltp sql sat nyc 2017
Inside sql server in memory oltp sql sat nyc 2017Inside sql server in memory oltp sql sat nyc 2017
Inside sql server in memory oltp sql sat nyc 2017
 
Performance Tuning RocksDB for Kafka Streams’ State Stores
Performance Tuning RocksDB for Kafka Streams’ State StoresPerformance Tuning RocksDB for Kafka Streams’ State Stores
Performance Tuning RocksDB for Kafka Streams’ State Stores
 
Performance Tuning RocksDB for Kafka Streams' State Stores (Dhruba Borthakur,...
Performance Tuning RocksDB for Kafka Streams' State Stores (Dhruba Borthakur,...Performance Tuning RocksDB for Kafka Streams' State Stores (Dhruba Borthakur,...
Performance Tuning RocksDB for Kafka Streams' State Stores (Dhruba Borthakur,...
 
NoSQL: Cassadra vs. HBase
NoSQL: Cassadra vs. HBaseNoSQL: Cassadra vs. HBase
NoSQL: Cassadra vs. HBase
 
MariaDB ColumnStore
MariaDB ColumnStoreMariaDB ColumnStore
MariaDB ColumnStore
 
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...
 
Monitoring MongoDB’s Engines in the Wild
Monitoring MongoDB’s Engines in the WildMonitoring MongoDB’s Engines in the Wild
Monitoring MongoDB’s Engines in the Wild
 
Deep Dive on Amazon Redshift
Deep Dive on Amazon RedshiftDeep Dive on Amazon Redshift
Deep Dive on Amazon Redshift
 
Apache Drill talk ApacheCon 2018
Apache Drill talk ApacheCon 2018Apache Drill talk ApacheCon 2018
Apache Drill talk ApacheCon 2018
 
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibabahbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
hbaseconasia2019 Phoenix Improvements and Practices on Cloud HBase at Alibaba
 
Deep Dive on Amazon Redshift
Deep Dive on Amazon RedshiftDeep Dive on Amazon Redshift
Deep Dive on Amazon Redshift
 
HBase in Practice
HBase in PracticeHBase in Practice
HBase in Practice
 
HBase in Practice
HBase in Practice HBase in Practice
HBase in Practice
 
Best Practices for Migrating Your Data Warehouse to Amazon Redshift
Best Practices for Migrating Your Data Warehouse to Amazon RedshiftBest Practices for Migrating Your Data Warehouse to Amazon Redshift
Best Practices for Migrating Your Data Warehouse to Amazon Redshift
 

Recently uploaded

Artificial Intelligence for Vision: A walkthrough of recent breakthroughs
Artificial Intelligence for Vision:  A walkthrough of recent breakthroughsArtificial Intelligence for Vision:  A walkthrough of recent breakthroughs
Artificial Intelligence for Vision: A walkthrough of recent breakthroughsNikolas Markou
 
Choose your perfect jacket.pdf
Choose your perfect jacket.pdfChoose your perfect jacket.pdf
Choose your perfect jacket.pdfAlexia Trejo
 
HayleyDerby_Market_Research_Spotify.docx
HayleyDerby_Market_Research_Spotify.docxHayleyDerby_Market_Research_Spotify.docx
HayleyDerby_Market_Research_Spotify.docxHayleyDerby
 
What you need to know about Generative AI and Data Management?
What you need to know about Generative AI and Data Management?What you need to know about Generative AI and Data Management?
What you need to know about Generative AI and Data Management?Denodo
 
Introduction to data science.pdf-Definition,types and application of Data Sci...
Introduction to data science.pdf-Definition,types and application of Data Sci...Introduction to data science.pdf-Definition,types and application of Data Sci...
Introduction to data science.pdf-Definition,types and application of Data Sci...DrSumathyV
 
WOMEN IN TECH EVENT : Explore Salesforce Metadata.pptx
WOMEN IN TECH EVENT : Explore Salesforce Metadata.pptxWOMEN IN TECH EVENT : Explore Salesforce Metadata.pptx
WOMEN IN TECH EVENT : Explore Salesforce Metadata.pptxyosra Saidani
 
EXCEL-VLOOKUP-AND-HLOOKUP LECTURE NOTES ALL EXCEL VLOOKUP NOTES PDF
EXCEL-VLOOKUP-AND-HLOOKUP LECTURE NOTES ALL EXCEL VLOOKUP NOTES PDFEXCEL-VLOOKUP-AND-HLOOKUP LECTURE NOTES ALL EXCEL VLOOKUP NOTES PDF
EXCEL-VLOOKUP-AND-HLOOKUP LECTURE NOTES ALL EXCEL VLOOKUP NOTES PDFProject Cubicle
 
Customer Satisfaction Data - Multiple Linear Regression Model.pdf
Customer Satisfaction Data -  Multiple Linear Regression Model.pdfCustomer Satisfaction Data -  Multiple Linear Regression Model.pdf
Customer Satisfaction Data - Multiple Linear Regression Model.pdfruwanp2000
 
itc limited word file.pdf...............
itc limited word file.pdf...............itc limited word file.pdf...............
itc limited word file.pdf...............mahetamanav24
 
Cousera Cap Course Datasets containing datasets from a Fictional Fitness Trac...
Cousera Cap Course Datasets containing datasets from a Fictional Fitness Trac...Cousera Cap Course Datasets containing datasets from a Fictional Fitness Trac...
Cousera Cap Course Datasets containing datasets from a Fictional Fitness Trac...Samuel Chukwuma
 
Unlocking New Insights Into the World of European Soccer Through the European...
Unlocking New Insights Into the World of European Soccer Through the European...Unlocking New Insights Into the World of European Soccer Through the European...
Unlocking New Insights Into the World of European Soccer Through the European...ThinkInnovation
 
Ratio analysis, Formulas, Advantage PPt.pptx
Ratio analysis, Formulas, Advantage PPt.pptxRatio analysis, Formulas, Advantage PPt.pptx
Ratio analysis, Formulas, Advantage PPt.pptxSugumarVenkai
 
fundamentals of digital imaging - POONAM.pptx
fundamentals of digital imaging - POONAM.pptxfundamentals of digital imaging - POONAM.pptx
fundamentals of digital imaging - POONAM.pptxPoonamRijal
 
Operations Data On Mobile - inSis Mobile App - Sample Screens
Operations Data On Mobile - inSis Mobile App - Sample ScreensOperations Data On Mobile - inSis Mobile App - Sample Screens
Operations Data On Mobile - inSis Mobile App - Sample ScreensKondapi V Siva Rama Brahmam
 
introduction-to-crimean-congo-haemorrhagic-fever.pdf
introduction-to-crimean-congo-haemorrhagic-fever.pdfintroduction-to-crimean-congo-haemorrhagic-fever.pdf
introduction-to-crimean-congo-haemorrhagic-fever.pdfSalamaAdel
 
Basics of Creating Graphs / Charts using Microsoft Excel
Basics of Creating Graphs / Charts using Microsoft ExcelBasics of Creating Graphs / Charts using Microsoft Excel
Basics of Creating Graphs / Charts using Microsoft ExcelTope Osanyintuyi
 
Tips to Align with Your Salesforce Data Goals
Tips to Align with Your Salesforce Data GoalsTips to Align with Your Salesforce Data Goals
Tips to Align with Your Salesforce Data GoalsDataArchiva
 

Recently uploaded (18)

Artificial Intelligence for Vision: A walkthrough of recent breakthroughs
Artificial Intelligence for Vision:  A walkthrough of recent breakthroughsArtificial Intelligence for Vision:  A walkthrough of recent breakthroughs
Artificial Intelligence for Vision: A walkthrough of recent breakthroughs
 
Choose your perfect jacket.pdf
Choose your perfect jacket.pdfChoose your perfect jacket.pdf
Choose your perfect jacket.pdf
 
HayleyDerby_Market_Research_Spotify.docx
HayleyDerby_Market_Research_Spotify.docxHayleyDerby_Market_Research_Spotify.docx
HayleyDerby_Market_Research_Spotify.docx
 
What you need to know about Generative AI and Data Management?
What you need to know about Generative AI and Data Management?What you need to know about Generative AI and Data Management?
What you need to know about Generative AI and Data Management?
 
Introduction to data science.pdf-Definition,types and application of Data Sci...
Introduction to data science.pdf-Definition,types and application of Data Sci...Introduction to data science.pdf-Definition,types and application of Data Sci...
Introduction to data science.pdf-Definition,types and application of Data Sci...
 
Electricity Year 2023_updated_22022024.pptx
Electricity Year 2023_updated_22022024.pptxElectricity Year 2023_updated_22022024.pptx
Electricity Year 2023_updated_22022024.pptx
 
WOMEN IN TECH EVENT : Explore Salesforce Metadata.pptx
WOMEN IN TECH EVENT : Explore Salesforce Metadata.pptxWOMEN IN TECH EVENT : Explore Salesforce Metadata.pptx
WOMEN IN TECH EVENT : Explore Salesforce Metadata.pptx
 
EXCEL-VLOOKUP-AND-HLOOKUP LECTURE NOTES ALL EXCEL VLOOKUP NOTES PDF
EXCEL-VLOOKUP-AND-HLOOKUP LECTURE NOTES ALL EXCEL VLOOKUP NOTES PDFEXCEL-VLOOKUP-AND-HLOOKUP LECTURE NOTES ALL EXCEL VLOOKUP NOTES PDF
EXCEL-VLOOKUP-AND-HLOOKUP LECTURE NOTES ALL EXCEL VLOOKUP NOTES PDF
 
Customer Satisfaction Data - Multiple Linear Regression Model.pdf
Customer Satisfaction Data -  Multiple Linear Regression Model.pdfCustomer Satisfaction Data -  Multiple Linear Regression Model.pdf
Customer Satisfaction Data - Multiple Linear Regression Model.pdf
 
itc limited word file.pdf...............
itc limited word file.pdf...............itc limited word file.pdf...............
itc limited word file.pdf...............
 
Cousera Cap Course Datasets containing datasets from a Fictional Fitness Trac...
Cousera Cap Course Datasets containing datasets from a Fictional Fitness Trac...Cousera Cap Course Datasets containing datasets from a Fictional Fitness Trac...
Cousera Cap Course Datasets containing datasets from a Fictional Fitness Trac...
 
Unlocking New Insights Into the World of European Soccer Through the European...
Unlocking New Insights Into the World of European Soccer Through the European...Unlocking New Insights Into the World of European Soccer Through the European...
Unlocking New Insights Into the World of European Soccer Through the European...
 
Ratio analysis, Formulas, Advantage PPt.pptx
Ratio analysis, Formulas, Advantage PPt.pptxRatio analysis, Formulas, Advantage PPt.pptx
Ratio analysis, Formulas, Advantage PPt.pptx
 
fundamentals of digital imaging - POONAM.pptx
fundamentals of digital imaging - POONAM.pptxfundamentals of digital imaging - POONAM.pptx
fundamentals of digital imaging - POONAM.pptx
 
Operations Data On Mobile - inSis Mobile App - Sample Screens
Operations Data On Mobile - inSis Mobile App - Sample ScreensOperations Data On Mobile - inSis Mobile App - Sample Screens
Operations Data On Mobile - inSis Mobile App - Sample Screens
 
introduction-to-crimean-congo-haemorrhagic-fever.pdf
introduction-to-crimean-congo-haemorrhagic-fever.pdfintroduction-to-crimean-congo-haemorrhagic-fever.pdf
introduction-to-crimean-congo-haemorrhagic-fever.pdf
 
Basics of Creating Graphs / Charts using Microsoft Excel
Basics of Creating Graphs / Charts using Microsoft ExcelBasics of Creating Graphs / Charts using Microsoft Excel
Basics of Creating Graphs / Charts using Microsoft Excel
 
Tips to Align with Your Salesforce Data Goals
Tips to Align with Your Salesforce Data GoalsTips to Align with Your Salesforce Data Goals
Tips to Align with Your Salesforce Data Goals
 

SQL Server 2014 In-Memory OLTP

  • 1. SQL Server 2014 In-Memory OLTP Deep Dive tonyrogerson@torver.net @tonyrogerson http://dataidol.com/tonyrogerson
  • 2. Agenda • Memory • Storage • MVCC • Indexes • Hash • Range
  • 4. SQL Server Memory Pools Stable Storage (MDF/NDF Files) Buffer Pool Table Data Data in/out as required Memory Internal Structures (proc/log cache etc.) SQL Server Memory Space Memory Optimised Tables
  • 5. Memory Optimised Tables SQL Server Memory Pools – MOT aggression Stable Storage (MDF/NDF Files) Buffer Pool Table Data Memory Internal Structures (proc/log cache etc.) SQL Server Memory Space
  • 6. Create Resource Pool CREATE RESOURCE POOL mem_xtp_pool WITH ( MAX_MEMORY_PERCENT = 50, MIN_MEMORY_PERCENT = 50 ); ALTER RESOURCE GOVERNOR RECONFIGURE; EXEC sp_xtp_bind_db_resource_pool 'xtp_demo', 'mem_xtp_pool'; ALTER DATABASE xtp_demo SET OFFLINE WITH ROLLBACK IMMEDIATE; ALTER DATABASE xtp_demo SET ONLINE; Best Practice
  • 8. CREATE DATABASE ALTER DATABASE xtp_basics ADD FILEGROUP xtp_demo_mod CONTAINS MEMORY_OPTIMIZED_DATA; ALTER DATABASE xtp_basics ADD FILE ( NAME = N'xtp_basics_mod1', FILENAME = N'c:SQLDATAinmemxtp_basics_mod1' , MAXSIZE = 4GB), ( NAME = N'xtp_basics_mod2', FILENAME = N'e:SQLDATAinmemxtp_basics_mod2' , MAXSIZE = 4GB), ( NAME = N'xtp_basics_mod3', FILENAME = N'c:SQLDATAinmemxtp_basics_mod3' , MAXSIZE = 4GB) TO FILEGROUP xtp_demo_mod; go
  • 9. Multiple Containers – Load Balancing • Specify an odd number of Files in the File Group • CFP {Data and Delta files} allocated in round robin • If only two files – Data will always be on “1” and Delta on “2”
  • 10. Life of a Row Memory CFP (Data / Delta) CFP (Data / Delta ) No active rows 3. MERGE 4. GARBAGE COLLECT 2. CHECKPOINT 1. Write to storage LDF – offline checkpoint worker writes to CFP (512MiB written to log/tran is bigger than CFP size) 2. Close CFP and mark ACTIVE (Durable Checkpoint established) 3. ACTIVE CFP’s with >= 50% free space can be merged. 4. Files with no active rows can be deleted
  • 11. Offline Checkpoint Worker • After 512MiB data written to log (from all DB activity) or Manual CHECKPOINT • CFP will become ACTIVE if amount of data written in single transaction warrants it • CFP state: UNDER CONSTRUCTION (on recovery, data taken from transaction log) • On CHECKPOINT • UNDER CONSTRUCTION CTP closed, becomes ACTIVE • Now have a durable checkpoint (otherwise use the log) • Reiterate the need for Odd containers – read from container ‘A’ and write new CFP to container ‘B’
  • 12. CFP UNDER CONSTRUCTION LDF ACTIVE CHECKPOINT No new data except deletes to tombstone file ACTIVE ACTIVE MERGE TARGET ACTIVE MERGED SOURCE MERGED SOURCE Change status Change status
  • 13. CFP MERGED SOURCE IN TRANSITION TO TOMBSTONE TOMBSTONE Log backup LSN above file – file no longer required Waiting on the FILESTREAM garbage collector
  • 16. Multi-Version Concurrency Control (MVCC) • SNAPSHOT isolation • Update is actually INSERT and Tombstone • Row Versions are kept in memory • Compare And Swap replaces Latching • Each in-memory row has a Start and End timestamp (on row header) • Each database has: • xtp_transaction_id counter (increment on BEGIN TRAN, reset on SQL restart) • Global Transaction timestamp (used on COMMIT)
  • 17. Multi-Version Concurrency Control (MVCC) • SNAPSHOT isolation • Update is actually INSERT and Tombstone • Row Versions are kept in memory • Compare And Swap replaces Latching • Memory Garbage Collection cleans up versions no longer required (stale data rows) • Versions no longer required determined by active transactions – may be inter-connection dependencies • Your in-memory table can double, triple, x 100 etc. in size depending on access patterns (data mods + query durations) • Row chains can expand dramatically and cause really poor performance
  • 18. SNAPSHOT isolation Newest Oldest TS_START TS_END Connection 2 NULL 55 7 NULL 60 50 NULL 80
  • 19. SNAPSHOT isolation Newest Oldest TS_START TS_END Connection 2 10 55 7 NULL 60 50 NULL 80 Tombstone
  • 20. SNAPSHOT isolation Current TS_START TS_END Connection 2 NULL 55 7 NULL 60 50 NULL 80 Tombstone
  • 21. MVCC Summary • Scalability achieved because MVCC removes need for latching and logging (SNAPSHOT isolation) • Row Versions (BeginTS, EndTS) are the main stay • For UPDATES’s the biggest issue is because of MVCC • Row Versions! • Garbage collection can be slow to clean up stale rows • In-Memory is not for large UPDATE’s
  • 23. Hash Indexing bucket_position = f hash( data ) Hash is deterministic (same value in gives same value out)
  • 24. Hash is used as bucket position in array of Memory Pointers aka the Hash Index Bucket position
  • 25. Hash Collisions • Example:1 million unique values hashed into 5000 hash values? • Multiple unique values mush hash to the same hash value • Multiple key (real data) values hang off same hash bucket • Termed: Row Chain • Chain of connected rows • SQL Server 2014 Hash function is Balanced and follows a Poisson distribution – 1/3 empty, 1/3 at least one index key and 1/3 containing two index keys
  • 26. BUCKET_COUNT options for HASH index 1024 – 8 kilobytes 2048 – 16 kilobytes 4096 – 32 kilobytes 8192 – 64 kilobytes 16384 – 128 kilobytes 32768 – 256 kilobytes 65536 – 512 kilobytes 131072 – 1 megabytes 262144 – 2 megabytes 524288 – 4 megabytes 1048576 – 8 megabytes 2097152 – 16 megabytes 4194304 – 32 megabytes 8388608 – 64 megabytes 16777216 – 128 megabytes 33554432 – 256 megabytes 67108864 – 512 megabytes 134217728 – 1,024 megabytes 268435456 – 2,048 megabytes 536870912 – 4,096 megabytes 1073741824 – 8,192 megabytes
  • 27. Link from Hash Index to first row in Row Chain
  • 28. Memory Optimised Table Row Header Up-to 8 Index Memory Pointers • Above is a 3 row – row chain, all three rows hash to bucket #3 • Most recent row is first row in chain • Rows in stable storage • accessed using files, pages and extents • Rows in memory optimised • accessed using memory pointers • Timestamp required for MVCC Memory Location Hash Index
  • 30. Row Header – Multiple Index Pointers Data Rows (Index #‘x’ Pointer is next memory pointer in ‘row’ chain) Memory Pointers per Hash Bucket Row chain is defined through the Row Header
  • 31. Hash Index Scan (Table Scan) • Scan follows Hash Index (8 byte per bucket) • Jump to each first row in row chain • Read row chain (lots of bytes per row – header + data)
  • 32. Hash Index - thoughts • Base BUCKET_COUNT on cardinality of the column • Don’t use where low cardinality – high rows: gives large row chains – performance will suck • Equality queries only • Good candidates are FK on joins
  • 33. What makes a Row Chain grow? • Hash collisions • Row versions from updates • An update is a {delete, insert} • Late garbage collection because of long running transactions • Garbage collection can’t keep up because of box load and amount to do {active threads do garbage collection on completion of work} • Deleted rows may still be in memory for some time even though no other connections have a snapshot
  • 34. Affect of Row Chains on Memory • MEMORY RESOUCES.sql
  • 35. Range Index (Bw Tree ≈ B+Tree) 8KiB 1 row tracks a page in level below Root Leaf Leaf row per table row {index-col}{pointer} B+ Tree
  • 36. Range Index (Bw Tree ≈ B+Tree) 8KiB 1 row tracks a page in level below Root Leaf Leaf row per table row {index-col}{pointer} B+ Tree
  • 37. Range Index (Bw Tree ≈ B+Tree) Bw Tree 8KiB Root Leaf Leaf row per unique value {index-col}{memory-pointer} Row Chain • Nodes 1 row – 8KiB • Leaf is pointer to Row Chain • Nodes don’t change – add/merge
  • 38. Range Index (Bw Tree ≈ B+Tree) 8KiB 1 row tracks a page in level below Root Leaf Leaf row per table row {index-col}{pointer} B+Tree 8KiB Root Leaf Leaf row per unique value {index-col}{memory-pointer} Row Chain ≈ Bw Tree
  • 39. Index recommendations • Use HASH when • Mostly unique values • Query uses expressions “Equality” i.e. = or != • Set BUCKET_COUNT appropriately • Seriously think if you are considering composite index • Use RANGE when • Expressions are not Equality e.g. < > between • Composite Index • Low Cardinality Best Practice
  • 41. Q & A / Discuss – uses??? • Feedback Forms