0101 foundation - detailed view of hana architecture

SAP HANA
Detailed View of HANA Architecture

© 2011 SAP AG. All rights reserved. / Page 2
Disclaimer
This presentation outlines our general product direction and should not be relied on in
making a purchase decision. This presentation is not subject to your license
agreement or any other agreement with SAP.
SAP has no obligation to pursue any course of business outlined in this presentation
or to develop or release any functionality mentioned in this presentation. This
presentation and SAP's strategy and possible future developments are subject to
change and may be changed by SAP at any time for any reason without notice.
This document is provided without a warranty of any kind, either express or implied,
including but not limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement. SAP assumes no responsibility for errors or
omissions in this document, except if such damages were caused by SAP
intentionally or grossly negligent.

Agenda
1. Architecture Overview
2. Row Store
3. Column Store
4. Persistency Layer

ERP
Architecture Overview
In-Memory Computing Engine and Surroundings
LogERP DB
In-Memory Computing Engine
Clients (planned, e.g.) BI4 Explorer
Dashboard
Design
SAP BI4 universes
(WebI,...)
Request Processing / Execution Control
MS Excel
BI4 Analysis
SQL Parser MDX
SQL Script Calc Engine
Transaction
Manager
Session Management
Relational Engines
Row Store Column Store
Persistence LayerPage Management Logger
Disk Storage
Log VolumesData Volumes
Authorization
Manager
Metadata
Manager
In-Memory Computing Studio
Administration Modeling
Load
Controller
Replication
Agent
Replication
Server
SAP Business Objects BI4
Data
Services
Designer
SBO BI4
servers
( program
for client)
SBO BI4
Information
Design Tool
Other Source Systems
SAP
NetWeaver
BW
3rd Party
Data
Services

ERP
The Engine
LogERP DB
Clients (planned, e.g.) SBOP Explorer 4.0
Xcelsius SAP BI universes (WebI,...)
MS Excel
SBOP Analysis
IMC Studio
Load
Controller
Replication
Agent
Business Objects Enterprise
Data
Services
Designer
SBO server
programs
for clients
SBO
Information
Design Tool
SAP
NetWeaver
BW
3rd Party
Data
Services
SQL Parser MDX
Transaction
Manager
Session Management
Relational Engines
Disk Storage
Authorization
Manager
Metadata
Manager
Replication
Server

ERP
Loading Data into SAP HANA
LogERP DB
SQL Parser MDX
Transaction
Manager
Session Management
Relational Engines
Disk Storage
Authorization
Manager
Metadata
Manager
Load
Controller
Replication
Agent
Replication
Server
Data
Services
Designer
SBO BI4
servers
( program
for client)
SBO
Information
Design Tool
SAP
NetWeaver
BW
3rd Party
Data
Services
Dashboard
Design
SAP BI4 universes
(WebI,...)
MS Excel
BI4 Analysis

ERP
Data Modeling
LogERP DB
SQL Parser MDX
Transaction
Manager
Session Management
Relational Engines
Disk Storage
Authorization
Manager
Metadata
Manager
Load
Controller
Replication
Agent
Replication
Server
Data
Services
Designer
SBO BI4
servers
( program
for client)
SBO
Information
Design Tool
SAP
NetWeaver
BW
3rd Party
Data
Services
Dashboard
Design
SAP BI4 universes
(WebI,...)
MS Excel
BI4 Analysis

Clients (planned, e.g.)
ERP
Reporting
LogERP DB
SQL Parser MDX
Transaction
Manager
Session Management
Relational Engines
Disk Storage
Authorization
Manager
Metadata
Manager
Load
Controller
Replication
Agent
Replication
Server
Data
Services
Designer
SBO BI4
servers
( program
for client)
SBO
Information
Design Tool
SAP
NetWeaver
BW
3rd Party
Data
Services
BI4 Explorer
Dashboard
Design
SAP BI4 universes
(WebI,...)
MS Excel
BI4 Analysis

ERP
Administration
LogERP DB
SQL Parser MDX
Transaction
Manager
Session Management
Relational Engines
Disk Storage
Authorization
Manager
Metadata
Manager
Load
Controller
Replication
Agent
Replication
Server
Data
Services
Designer
SBO BI4
servers
( program
for client)
SBO
Information
Design Tool
SAP
NetWeaver
BW
3rd Party
Data
Services
Dashboard
Design
SAP BI4 universes
(WebI,...)
MS Excel
BI4 Analysis

DB Server
SAP High-Performance Analytic Appliance 1.0
SAP HANA
JDBC ODBC ODBO
SQL
DBC
SAP In-Memory
Computing Engine
Replication
Server
SAP In-Memory Computing Studio
SAP Business
Application
Replication
Agent
SAP BusinessObjects
Data Services 4.0
Any
source
SAP
BusinessObjects
BI 4.0
Repository
SAP BusinessObjects BI clients
SQL
MDX
BICS
Authentication
Contentmgmt
sync
Admin&model
load
(optional)
(optional)
(optional)
(existing)

Request Processing and Execution Control
Conceptual View
Standard SQL
Processed directly by DB engine
SQL Script, MDX and planning engine
interface
Domain-specific programming
languages or models
Converted into calculation models
Calc Engine
Create logical execution plan for
calculation models
Execute user defined functions
Relational Engine
DB optimizer produces physical
executing plan
Access to row and column store

Calc Engine for Dummies
The easiest way to think of Calculation Models is to see them as dataflow graphs,
where the modeler can define data sources as inputs and different operations (join,
aggregation, projection,…) on top of them for data manipulations.
The Calculation Engine will break up a model, for example some SQL Script, into
operations that can be processed in parallel (rule based model optimizer). Then these
operations will be passed to the database optimizer which will determine the best
plan for accessing row or column stores (algebraic transformations and cost based
optimizations based on database statistics).

Calc Engine for Dummies
Example

Agenda
2. Row Store
3. Column Store

High Level Architecture
Row Store
One of the
relational engines
Interfaced from
calculation /
execution layer
Pure in-memory
store
Persistence
managed in
persistence
layer
SAP in-memory
computing engine
HANA

Row Store Architecture
Row Store Block Diagram
Row Store Block Diagram
Transactional Version Memory
Contains temporary versions
Needed for Multi-Version
Concurrency Control (MVCC)
Segments
Contain the actual data (content of
row-store tables) in pages
Page Manager
Memory allocation
Keeping track of free/used pages
Version Memory Consolidation
Think ‘garbage collector for MVCC’
Persistence Layer
Invoked in write operations (log)
And in performing savepoints
checkpoint writer

Row Store Architecture
Highlights
Write Operations
Mainly go into “Transactional Version
Memory”
“INSERT” also writes to Persisted
Segment
Read Operations
Write Operations
Transactional
Version
Memory
Main Memory
Persisted
Segment
Data that
may be
seen by all
active
transactions
Recent
versions of
changed
records
Version Memory
Consolidation
Version Consolidation
Moves “visible version”
from Transaction Version
Memory into Persisted
Segment (based on
Commit ID)
Clears “outdated” record
versions from Transactional
Version Memory
Memory Handling
Row store tables are
linked list of memory
pages
Pages are grouped in
segments
Page size: 16 KB
Persisted Segment
Contains data that may be seen by any
ongoing transaction
Data that has been committed before
any active transaction was started)

Indexes for Row Store Tables
Primary Index / Row ID / Index Persistence
Each row-store table has a primary index
Primary index maps ROW ID primary key of table
ROW ID: a number specifying for each record its memory segment and page
How to find the memory page for a table record?
A structure called “ROW ID” contains the segment and the page for the record
The page can then be searched for the records based on primary key
ROW ID is part of the primary index of the table
Secondary indexes can be created if needed
Persistence of indexes in row store
Indexes in row store only exist in memory
No persistence of index data
Index definition stored with table metadata
Indexes filled on-the-fly when system loads tables into memory on system start-up

Agenda
2. Row Store
3. Column Store

High Level Architecture
Column Store
One of the relational
engines
Interfaced from
calculation / execution
layer
Pure in-memory store
Persistence
managed in
persistence layer
Optimized for high
performance of read
operation
Good performance of
write operations
Efficient data
compression
SAP in-memory
computing engine
HANA

Column Store Architecture
Column Store Block Diagram
Column Store Block Diagram
Optimizer and Executor
Handles queries and
execution plan
Main and Delta Storage
Compressed data for fast read
Delta data for fast write
Asynchronous delta merge
Consistent View Manager
Transaction Manager
Persistence Layer

Column Store
Highlights
Storage Separation (Main & Delta)
Enables high compression and high write
performance at the same time
Delta Merge Operation
See next slide
Read Operations
Write
Operations
Main
Main Memory
Delta
Write
optimized
Compressed
and
Read
optimized
Read Operations
Always have to read from both
main & delta storages and merge
the results.
Engine uses multi version
concurrency control (MVCC) to
ensure consistent read operations.
Data Compression in Main
Storage
Compression by creating
dictionary and applying further
compression methods
Speed up
Data load into CPU cache
Equality check Search
The compression is computed
during delta merge operation.
Write Operations
Only in delta storage because write optimized.
The update is performed by inserting a new
entry into the delta storage.

Column Store
Delta Management
Delta Merge Operation
Purpose
To move changes in delta storage into the compressed and read optimized main storage
Characteristics
Happens asynchronously
Even during merge operation the columnar table will be still available for read and write
operations
To fulfil this requirement, a second delta and main storage are used internally
Read Operations
Write
Operations
Main
Before Merge
Delta
Read Operations
Write
Operations
Main
New
After Merge
Delta
New
Read Operations
Write
Operations
Main
During Merge
Main
New
Delta
New
Delta
Merge Operations

Agenda
2. Row Store
3. Column Store
4. Persistence Layer

Persistence Layer
Purpose and Scope
Why Does An In-memory Database Need A Persistence Layer?
Main Memory is volatile. What happens upon…
Database restart?
Power outage?
...
Data needs to be stored in a non-volatile way
Backup and restore
SAP in-memory computing engine offers one persistence layer which is used by row store and
column store
Regular “savepoints”
full persisted image of DB at time of savepoint
Logs capturing all DB transactions since last savepoint (redo logs and undo logs written)
restore DB from latest savepoint onwards
Ability to create "snapshots"
used for backups

Persistence Layer
System Restart and Population of In-memory Stores
Actions During System Restart
Last savepoint must be restored plus…
Undo logs must be read for uncommitted transactions saved with last savepoint
Redo logs for committed transactions since last savepoint
Complete content of row store is loaded into memory
Column store tables may be marked for preload or not
Only tables marked for preload
are loaded into memory during
startup
If table is marked for loading
on demand, the restore
procedure is invoked on first
access

Thank you!

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein
may be changed without prior notice.
Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.
Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation.
IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries, zSeries,
eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390 Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6, POWER5+,
POWER5, POWER, OpenPower, PowerPC, BatchPipes, BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, Parallel Sysplex,
MVS/ESA, AIX, Intelligent Miner, WebSphere, Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM Corporation.
Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems Incorporated in the United States and/or other
countries.
Oracle is a registered trademark of Oracle Corporation.
UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.
Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.
HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.
Java is a registered trademark of Sun Microsystems, Inc.
JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.
SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, Clear Enterprise, SAP BusinessObjects Explorer and other SAP products and services mentioned herein as well
as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.
Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and
services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP France in the United States and in other countries.
All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only.
National product specifications may vary.
The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the
express prior written permission of SAP AG.
This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies,
developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or
development. Please note that this document is subject to change and may be changed by SAP at any time without notice.
SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other
items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of
merchantability, fitness for a particular purpose, or non-infringement.
SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these
materials. This limitation shall not apply in cases of intent or gross negligence.
The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links
contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages.
© 2011 SAP AG. All Rights Reserved

0101 foundation - detailed view of hana architecture

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (8)

Similar to 0101 foundation - detailed view of hana architecture

Similar to 0101 foundation - detailed view of hana architecture (20)

Recently uploaded

Recently uploaded (20)

0101 foundation - detailed view of hana architecture