EFFICIENT TRANSACTION
PROCESSING IN SAP HANA
Presented by Vijay Soppadandi
21363273
CONTENTS
• What is SAP HANA?
• Goal
• Agenda
• Framework
• Summary & discussion
WHAT IS SAP HANA?
• High-Performance Analytic Appliance
• In-memory computing
GOAL
• The overall goal of the SAP HANA database is to provide a generic but
powerful system for different query scenarios, both transactional and
analytical, on the same data representation within a highly scalable
execution environment
AGENDA
OLTP (Online Transaction Proccessing)
o large number of concurrent users and transactions
o high update load
o very selective point queries
o Row-stores
OLAP (Online Analytical Processing)
o aggregation queries over a huge volume of data
o compute statistical models from the data
o Column-stores
AGENDA
• Zoo of different systems with different capabilities for different application
scenarios
• However, workloads usually contain both
o transactional database needs statistical information to make on-the-fly
business decisions
o data-warehouses are required to capture transactions feeds for real-time
analytics
FRAMEWORK
• Layered Architecture of the SAP HANA
• Lifecycle Management of Database Records
• Types of Merging and optimization
• Summary & discussion
LAYERED ARCHITECTURE OF THE
SAP HANA
Reference 3
FRAMEWORK
• Layered Architecture of the SAP HANA
• Lifecycle Management of Database Records
• Types of Merging and optimization
• Summary & discussion
LIFECYCLE MANAGEMENT OF
DATABASE RECORDS
Reference 1
LIFECYCLE MANAGEMENT OF
DATABASE RECORDS
• It has three main stages
- L1-delta
-L2-delta
-Main
• SAP HANA theoretically escalate records through different stages of a
physical representation
L1-DELTA
• Accepts all incoming data requests
• No data compressions
• Stores them in write optimized manner
- Preserves in logical-row format
-Fast insert & deleat, field update and record projections
• Holds 10,000 to 100,000 rows per single-node
L2-DELTA
• The next stage of the Unified table structure
• Organized in column format
• Stores upto 10 millions of row
• It emplyoys dictionary encoding
- However, for performance reasons, the dictionary is unsorted requiring secondary index
structures to optimally support point query access patterns
MAIN STORE
• Represents core data
• Final data format
• Organized in column format
• Highest compression rate
-Positions in a sorted dictionary
-Stored in a bit-packet manner
-Dictionary is always comprised
UNIFIED TABLE ACCESS
• A common abstract interface to access different stores •
• Records are propagated asynchronously
– without interfering with running operations
• Two transformations (or merge steps)
– L1-deta to L2-delta
– L2-delta to main
L1-DETA TO L2-DELTA MERGE
• Rows format is converted into column format
- column-by-column inserted into the L2-delta
Steps for merging:
• Step 1 ( parallel)
- new entries are added (advance) to the dictionary
• Step 2 (parallel)
-using dictionary encoding technique new column values are
added
• Step 3
- added values are removed from the L1-delta
L2-DELTA TO MAIN MERGE
• New main structure created from L2-delta
• Resourse intensive
-Is carefully schedulled and highly optimized
• Current L2-delta is closed when its done
• Retries the merge on failure
PERSISTENCE MAPPING
Reference 1
CONT...
• Its fully supports ACID gurentees
• Uses REDO logs instead of UNDO mechanisms and saving point
• It has some complications in making merging process
FRAMWORK
• Layered Architecture of the SAP HANA
• Lifecycle Management of Database Records
• Types of Merging and optimization
• Summary & discussion
MERGE OPTIMIZATION
• Basic idea to provide transperent record propagation from write to read
optimization.
• The classic merge
• Re-sorting merge
• Partial merge
THE CLASSIC MERGE
• It has two phases
• In first phase generates the new sorted dictionary
• In secound phase, new main index generates on the new dictionary
THE CLASSIC MERGE
Reference 1
RE-SORTING MERGE
• Provides higher compression potential
• reorganizes the content of the full table to yield a data layout which
provides higher compression potential
• not easy because all records should have the same order in all columns
• uses a scheme discussed in another paper
PARTIAL MERGE
• It aims to reduce overall merge overhead
• Split the main into two independent main structures
- passive : Not part of the merge process
-Active : Takes part in the merge process with L2-delta
PARTIAL MERGE
Reference 1
CHARACTERISTICS OF RECORD
STAGES
Reference 1
REFERENCES
1.Efficient Transaction Processing in SAP HANA Database – The End of a Column Store Myth
-Vishal Sikka SAP 3412 Hillview Ave Palo Alto, CA 94304, USA vishal.sikka@sap.com
-Franz Färber SAP Dietmar-Hopp-Allee 16 69190, Walldorf, Germany franz.faerber@sap.com
-Wolfgang Lehner SAP Dietmar-Hopp-Allee 16 69190, Walldorf, Germany wolfgang.lehner@sap.com
-Sang Kyun Cha SAP 63-7 Banpo 4-dong, Seochoku 137-804, Seoul, Korea sang.k.cha@sap.com
-Thomas Peh SAP Dietmar-Hopp-Allee 16 69190, Walldorf, Germany thomas.peh@sap.com
-Christof Bornhövd SAP 3412 Hillview Ave Palo Alto, CA 94304, USA christof.bornhoevd@sap.com
2.Efficient Transaction Processing in SAP HANA Database
Presented by Henggang Cui 15-799b Talk
3. The SAP HANA Database–An Architecture Overview
-Franz F¨arber Norman
END
ANY QUESTIONS ?

Efficient transaction processing in sap hana

  • 1.
    EFFICIENT TRANSACTION PROCESSING INSAP HANA Presented by Vijay Soppadandi 21363273
  • 2.
    CONTENTS • What isSAP HANA? • Goal • Agenda • Framework • Summary & discussion
  • 3.
    WHAT IS SAPHANA? • High-Performance Analytic Appliance • In-memory computing
  • 4.
    GOAL • The overallgoal of the SAP HANA database is to provide a generic but powerful system for different query scenarios, both transactional and analytical, on the same data representation within a highly scalable execution environment
  • 5.
    AGENDA OLTP (Online TransactionProccessing) o large number of concurrent users and transactions o high update load o very selective point queries o Row-stores OLAP (Online Analytical Processing) o aggregation queries over a huge volume of data o compute statistical models from the data o Column-stores
  • 6.
    AGENDA • Zoo ofdifferent systems with different capabilities for different application scenarios • However, workloads usually contain both o transactional database needs statistical information to make on-the-fly business decisions o data-warehouses are required to capture transactions feeds for real-time analytics
  • 7.
    FRAMEWORK • Layered Architectureof the SAP HANA • Lifecycle Management of Database Records • Types of Merging and optimization • Summary & discussion
  • 8.
    LAYERED ARCHITECTURE OFTHE SAP HANA Reference 3
  • 9.
    FRAMEWORK • Layered Architectureof the SAP HANA • Lifecycle Management of Database Records • Types of Merging and optimization • Summary & discussion
  • 10.
  • 11.
    LIFECYCLE MANAGEMENT OF DATABASERECORDS • It has three main stages - L1-delta -L2-delta -Main • SAP HANA theoretically escalate records through different stages of a physical representation
  • 12.
    L1-DELTA • Accepts allincoming data requests • No data compressions • Stores them in write optimized manner - Preserves in logical-row format -Fast insert & deleat, field update and record projections • Holds 10,000 to 100,000 rows per single-node
  • 13.
    L2-DELTA • The nextstage of the Unified table structure • Organized in column format • Stores upto 10 millions of row • It emplyoys dictionary encoding - However, for performance reasons, the dictionary is unsorted requiring secondary index structures to optimally support point query access patterns
  • 14.
    MAIN STORE • Representscore data • Final data format • Organized in column format • Highest compression rate -Positions in a sorted dictionary -Stored in a bit-packet manner -Dictionary is always comprised
  • 15.
    UNIFIED TABLE ACCESS •A common abstract interface to access different stores • • Records are propagated asynchronously – without interfering with running operations • Two transformations (or merge steps) – L1-deta to L2-delta – L2-delta to main
  • 16.
    L1-DETA TO L2-DELTAMERGE • Rows format is converted into column format - column-by-column inserted into the L2-delta Steps for merging: • Step 1 ( parallel) - new entries are added (advance) to the dictionary • Step 2 (parallel) -using dictionary encoding technique new column values are added • Step 3 - added values are removed from the L1-delta
  • 17.
    L2-DELTA TO MAINMERGE • New main structure created from L2-delta • Resourse intensive -Is carefully schedulled and highly optimized • Current L2-delta is closed when its done • Retries the merge on failure
  • 18.
  • 19.
    CONT... • Its fullysupports ACID gurentees • Uses REDO logs instead of UNDO mechanisms and saving point • It has some complications in making merging process
  • 20.
    FRAMWORK • Layered Architectureof the SAP HANA • Lifecycle Management of Database Records • Types of Merging and optimization • Summary & discussion
  • 21.
    MERGE OPTIMIZATION • Basicidea to provide transperent record propagation from write to read optimization. • The classic merge • Re-sorting merge • Partial merge
  • 22.
    THE CLASSIC MERGE •It has two phases • In first phase generates the new sorted dictionary • In secound phase, new main index generates on the new dictionary
  • 23.
  • 24.
    RE-SORTING MERGE • Provideshigher compression potential • reorganizes the content of the full table to yield a data layout which provides higher compression potential • not easy because all records should have the same order in all columns • uses a scheme discussed in another paper
  • 25.
    PARTIAL MERGE • Itaims to reduce overall merge overhead • Split the main into two independent main structures - passive : Not part of the merge process -Active : Takes part in the merge process with L2-delta
  • 26.
  • 27.
  • 28.
    REFERENCES 1.Efficient Transaction Processingin SAP HANA Database – The End of a Column Store Myth -Vishal Sikka SAP 3412 Hillview Ave Palo Alto, CA 94304, USA vishal.sikka@sap.com -Franz Färber SAP Dietmar-Hopp-Allee 16 69190, Walldorf, Germany franz.faerber@sap.com -Wolfgang Lehner SAP Dietmar-Hopp-Allee 16 69190, Walldorf, Germany wolfgang.lehner@sap.com -Sang Kyun Cha SAP 63-7 Banpo 4-dong, Seochoku 137-804, Seoul, Korea sang.k.cha@sap.com -Thomas Peh SAP Dietmar-Hopp-Allee 16 69190, Walldorf, Germany thomas.peh@sap.com -Christof Bornhövd SAP 3412 Hillview Ave Palo Alto, CA 94304, USA christof.bornhoevd@sap.com 2.Efficient Transaction Processing in SAP HANA Database Presented by Henggang Cui 15-799b Talk 3. The SAP HANA Database–An Architecture Overview -Franz F¨arber Norman
  • 29.