Sql Server 2014 In Memory


Published on

Slides from my presentation at Code Camp NYC

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Sql Server 2014 In Memory

  1. 1. Ravi Okade
  2. 2. Agenda  Why do you need it ?  How it is done  Benefits  Limitations
  3. 3. Why do you need a new architecture ?  In-Memory  Memory is cheap  Servers have HUGE memory  Some data is hot  Traditional page based architecture has limitations, even when all pages are in memory  Native Compilation, Lock-less architecture  Individual cores are not getting any faster  While number of cores is increasing, Parallel processing has its limits due to lock contention  You cannot scale linearly by adding more cores
  4. 4. Comparable Products and Technologies  Similar Products  Oracle TimesTen  Sybase ASE  Other technologies with some similarities  Distributed Cache ○ Oracle Coherence is one of many distributed cache’s that support query language (CQL). It can also persist to Oracle. Another example with similar capabilities is Gigaspaces  NoSQL Databases ○ Most NoSQL databases support distributed querying and aggressively cache data (e.g. MongoDB)
  5. 5. What it all means..
  6. 6. Use Cases  Hot tables – frequent inserts and updates, but queried heavily e.g:  Stock Trading applications  Patient Vital Stats  Flight information (live)  Transient data e.g.:  Web Session data  Stock Market data  Need for High speed OLTP queries
  7. 7. In-memory Architecture  Main Concepts  In-memory  Lock free (row versions)  Native compiled stored procedures  Data storage optimizations  Memory optimized data structures  Transaction log optimization (block writes, no undo)  Data file Optimization (Sequential writes, Merging)  Index optimization (Only in-memory, Not persisted to disk, no tx logging)  No TempDB  Garbage collection optimization (huh?)  Non-durable option (Not persisted to disk)
  8. 8. In Memory – Myths and Realities  Myth: In-Memory OLTP is the same as DBCC PINTABLE  Reality: In-Memory OLTP uses a completely new design built from the ground up.  Myth: If SQL Server crashes all data is lost  Reality: No – it is persisted to disk and is fully recoverable.
  9. 9. SQL Server 2014 In-Memory Architecture SQL Server Integration • Same manageability, administration & development experience • Integrated queries & transactions • Integrated HA and backup/restore Main-Memory Optimized • Optimized for in-memory data • Indexes (hash and range) exist only in memory • No buffer pool, B-trees • Stream-based storage T-SQL Compiled to Machine Code • T-SQL compiled to machine code via C code generator and VC • Invoking a procedure is just a DLL entry-point • Aggressive optimizations @ compile-time High Concurrency • Multi-version optimistic concurrency control with full ACID support • Core engine uses lock-free algorithms • No lock manager, latches or spinlocks
  10. 10. In-Memory integration with SQL Server
  11. 11. Memory Management  Table data resides in memory at all times.  No paging.  Must configure SQL box with sufficient memory to store memory-optimized tables. Max supported 512GB  Failure to allocate memory will fail transactional workload at run-time
  12. 12. In-Memory Data Structures  Rows  New row format ○ Structure of the row is optimized for in-memory residency and access.  One copy of row ○ Indexes point to rows, they do not duplicate them.  Indexes  Hash Index for equality search  memory-optimized B-tree for range and equality search (In CTP2)  Do not exist on disk – recreated during recovery.
  13. 13. In Memory – Table DDL: Need filestream filegroup FILEGROUP [Hekaton_DB_hk_fs_fg] CONTAINS MEMORY_OPTIMIZED_DATA (NAME = [Hekaton_DB_hk_fs_dir], FILENAME = 'C:Hekaton_testsqlservrdataHekaton_DB_hk_fs_dir')
  14. 14. Table creation internals CREATE TABLE DDL Code generation and compilation Table DLL produced Table DLL loaded
  15. 15. Table Storage  Filestream is the underlying storage mechanism  Data files  Store inserted records  Data written only upon Tx commit  Delta files  Store deleted records (updates are an insert/delete pair)
  16. 16. Transaction Logging  Uses SQL transaction log to store content  All logging is logical  No log records for physical structure modifications.  No index-specific / index-maintenance log records.  No UNDO information is logged
  17. 17. Native Compiled Stored Procedures  Compiled to C language and compiled into a dll using VC  Optimized aggressively at compile time  Can only access In-memory tables  Not all T-SQL constructs and functions supported  No alter procedure – must drop and recreate
  18. 18. Procedure Creation CREATE PROC DDL Query optimization Code generation and compilation Procedure DLL produced Procedure DLL loaded
  19. 19. Using Row Versions for Lock free architecture  SQL Server 2014 In Memory DB has no locks (period).  Row versions are used to maintain updates  No TempDB  Row Versions which are no longer referenced are garbage collected.  Supports Snapshot, Repeatable Read and Serializable Isolation levels
  20. 20. Transaction Isolation Levels  SNAPSHOT  Reads are consistent as of start of the transaction  Writes are always consistent  REPEATABLE READ  Read operations yield same row versions if repeated at commit time  SERIALIZABLE  Transaction is executed as if there are no concurrent transactions – all actions happen at a single serialization point (commit time)
  21. 21. Garbage Collection
  22. 22. Benefits  Uses Commodity Hardware  Works seamlessly with current SQL Server objects  Yes, it is still ACID compliant  Yes, you can mix in-memory and disk based tables in the same database  Yes, your transactions can span in-memory and disk based tables  No, you cannot have partial in-memory tables  Yes, it has to fit in-memory 100%, forever.  Yes, you can limit how much memory is used by the in-memory tables (think resource pools) (not in CTP1)  Yes, you can have high availability
  23. 23. Limitations  Row Sizes can’t be larger than 8060 bytes (incl. Variable Length Columns)  LOB, XML Data Types are not supported  No Foreign Key and Check Constraints  No IDENTITY, SEQUENCE  No DML Triggers  No ALTER Table (Need to recreate table)  No Add/Remove Index (Need to recreate table)  No diff backups  Indexes are rebuilt (consider startup time)  Disk files are merged while being loaded  Running out of memory
  24. 24. CTP1 Limitations  No B-Tree index (used for range search)  No Resource Pools  No always on
  25. 25. Demo  Performance against disk based table  Performance with native sproc  Number of Tx records written disk vs memory  Northwind example  [TBD] Dealing with transactions  [TBD] Performance of T-SQL/Interop sprocs with In memory tables  [TBD] Disk files and merging ?  [TBD] Backup, Tx log backup; no diff backups
  26. 26. More information  Tech-ed SQL Server 2014 Videos  Microsoft SQL Server In-Memory OLTP: Overview of Project “Hekaton”  Microsoft SQL Server In-Memory OLTP Project “Hekaton”: App Dev Deep Dive  Microsoft SQL Server In-Memory OLTP Project “Hekaton”: Management Deep Dive  Microsoft SQL Server 2014 In-Memory OLTP: Overview  SQL Server In-Memory OLTP: DB Developer Deep Dive  SQL Server In-Memory OLTP: DBA Deep Dive  SIGMOD 2013 article by Microsoft  Hekaton Whitepaper by Kalen Delaney  High-Performance Concurrency Control Mechanisms for Main-Memory Databases, Microsoft Research  SQL Server 2014 - MSDN Online Documentation  Blog articles by Bob Beauchemin  Getting Started with SQL Server 2014 In-Memory OLTP (SQL Server Team blog)  Additional links  Oracle TimesTen FAQs  SQL Server In Memory Set up and Demos (My Blog)