Db2 analytics accelerator technical update

Db2 Analytics Accelerator
Technical Update
Cüneyt Göksu, IDAA Development
IBM Germany
2020 IBM Systems Technical University
7.2.2020 | Istanbul

Agenda
—Db2 Analytics Accelerator Overview
—Version 7.5 Functionality
—Db2 Analytics Accelerator - Deployment option details
IBM Systems Technical University © Copyright IBM Corporation 2020

DATA GRAVITY
HYBRID TRANSACTION &
ANALYTICAL PROCESSING1
Analyze data in place to improve
integrity and minimize cost & complexity
Enable simplified infrastructure and
embed insight to drive innovation
from real-time analytics
Data is at the core
of analytic and AI insights
1 Hybrid transactional and analytic processing (HTAP) source: https://www.gartner.com/doc/2657815/hybrid-
transactionanalytical-processing-foster-opportunities
IBM Systems Technical University © Copyright IBM Corporation 2020

Take the analysis to the data
• Avoid all the pitfalls of moving the data
Simplified infrastructure with more resiliency
• One copy of the data not dozens
Much more secure
• Z security built in
Lower cost
• Saves money (Infrastructure, SW, and people)
Much lower analytics latency
• Low to no latency with transactional data
A Data Gravity
approach performs analytics
where the majority of the
data originates
By far, the best place to analyze
Z data is on IBM Z
Data Gravity approach to analytics

and Db2 for z/OS
WHAT
An integrated, hybrid workload-optimized database
management system
HOW
Runs each query workload efficiently in its optimal
environment
WHY
To ensure the greatest performance and cost efficiency
Transaction
Processing
HTAP Analytical
Workload
WOW
Exploit IBM Z data in-place to improve efficiency, drive
smarter outcomes and gain competitive differentiation

Accelerator on IBM Integrated
Analytics System
• Pre-configured hardware and software for
easy deployment, management, and high
performance
• Secure, flexible and elastic data storage –
easy to deploy and manage
Accelerator on
IBM Z
• Deep integration with IBM Z offers a
unified homogeneity of service, support
and operations
• Flexible capacity to respond to peak
analytic workload requirements
Flexible, integrated deployment options
High-speed analysis of enterprise data for real-time insight
Uniform experience – transition easily between deployment options with one API and one database engine

Powered by Db2 with BLU Acceleration (Db2 Warehouse)
• Fast ingest for incremental updates, and thereby low HTAP query delay!
• IBM’s premier analytics engine across many products
• Latest analytics technology innovations
• SQL compatibility across all IBM products
• High degree of concurrent users and queries
In-memory column processing
with dynamic movement of data
from storage
Multi-core and SIMD
parallelism
(Single instruction
Multiple Data)
Patented compression technique
preserves order -- data can be
used without decompressing
Skips unnecessary
processing of irrelevant data

Db2 Analytics Accelerator Version 7.5
Db2 Analytics Accelerator Version 7.5 delivers:
• Integrated Synchronization a new advanced data
synchronization technique
• A wider range of scalability for Db2 Analytics Accelerator
on IBM Z deployments -- from very small to very large
General Availability: December 6, 2019

Integrated Synchronization
Integrated, low-latency data coherence protocol
between Db2 for z/OS and the Db2 Warehouse
• zIIP enabled
• Complete application transparency
• Enterprise-grade HTAP enabler
• Simplified administration, packaging, upgrades, support . . .
Deeper integration between Db2 for z/OS and Db2 Analytics Accelerator
to provide insight from the most current transactional data

Wider range of scalability for Accelerator on IBM Z deployments
Delivering a wide range of scalability, from very small to
very large deployments
• Reduced IFL and memory requirements enable
organizations with smaller deployments to take advantage
of the Accelerator’s capabilities
• Multi-node deployment delivers scalability for demanding
workloads, optimized for large workloads, provides flexible
adjustment of resources

Agenda
• Query Routing
• Data Synchronization and „True HTAP“
• High Performance Storage Saver
• In-database transformation (Accelerator-only tables)
• Enhance Accelerator functionality with Db2 Analytics Accelerator Loader
• Federation

Query execution process flow
AcceleratorDRDARequestor
Application
Interface
Heartbeat
(availability and performance indicators)
Application
Optimizer
Query execution run-time for queries
that cannot be or should not be
routed to Accelerator
Heartbeat
Queries executed
with Accelerator
Queries executed
without Accelerator

Routing criteria
 Dynamic and static queries can be
accelerated
 Db2 Optimizer decides if query should be
sent to Accelerator
• Dynamic: At execution time
• Static: At BIND time
 Whole query, not parts of query are
accelerated
 Only read queries are considered for
acceleration
 Queries within INSERT statements can be
accelerated
 Prerequisites for query routing:
• Accelerator is started
• All used tables are available on Accelerator
• Query routing option QUERY_ACCELRATION
is specified
 Via special register, BIND option or ZPARM
 ELIGIBLE, ENABLE, ENABLE WITH
FAILBACK, ALL

SQL functionality support and restrictions
—Improved Db2 for z/OS SQL support on Accelerator V7 (compared to V5)
• All data types supported except LOBs or XML
• Improved Db2 for z/OS function support on the Accelerator
o Still some not supported, e.g ACOS, ASIN, CLOB, ..
• Correlated subquery support
• Recursive SQL support
• Special register support
—Restrictions:
• No user defined functions (except inline SQL scalar UDF, compiled SQL scalar
UDF)
• No multiple encoding schemes in the same statement
Knowledge Center: Conditions for query routing to an accelerator
https://www.ibm.com/support/knowledgecenter/en/SS4LQ8_7.1.0/com.ibm.datatools.aqt.doc/gui/concepts/c_idaa_que
ry_offloading_criteria.html

Pass-through support for Db2 Warehouse built-in functions
Enhancing Db2’s native SQL Capabilities with the Accelerator
— Many built-in functions that are supported by the underlaying DBMS (Db2 Warehouse) in the
Accelerator are not supported natively by Db2 for z/OS (yet).
— Some of them can now be used in SQL queries routed to the Accelerator with the new Built-In-
Function (BIF) Pass-through support
• Db2 for z/OS is ”aware” of the Accelerator, when parsing the SQL statement.
• If a BIF is referenced, which is only available on the Accelerator, the Db2 for z/OS parser validates the signature
and allows its invocation within the rewritten SQL.
• Db2 for z/OS still needs to validate parameters, return types, …. Therefore the pass-through is limited to
commonly requested BIFs.
— Supported BIFs
• OLAP/Aggregate functions: CUME_DIST, FIRST_VALUE, LAG, LAST_VALUE, LEAD, NTH_VALUE,
NTILE, PERCENT_RANK, RATIO_TO_REPORT
• Scalar functions: REGEXP_COUNT, REGEXP_INSTR, REGEXP_LIKE, REGEXP_REPLACE,
REGEXP_SUBSTR
— Db2 12 only, FL504

Synchronization options Use cases, characteristics and requirements Technical aspects
Full table load/refresh
The entire content of a
database table is
loaded/refreshed
 Source table data is entirely replaced
 Smaller, un-partitioned tables
 Reporting based on consistent snapshot
 Scope: Table or Partition
 ACCEL_LOAD_TABLES stored procedure
 Data Studio provides options to
 Load/Refresh a table/partitions
 Indicate changed partitions
 Queries can be routed while load is in
progress
Table partition
load/refresh
For a partitioned database
table, selected partitions
can be loaded/refreshed
 More efficient than full table refresh for larger tables
 Reporting based on consistent snapshot
 Optionally: automatically load changed partitions only
Incremental Update
Log-based capturing of
changes and propagation
to Accelerator with low
latency (typically few
minutes)
 Scattered updates after “bulk” load
 Reporting on continuously updated data (e.g., an ODS),
considering most recent changes
 More efficient for smaller updates than full table
refresh
 Scope: Row
 Based on Integrated Synchronization or
Change Data Capture (CDC) of IBM
InfoSphere Data Replication
 Management integrated into stored
procedures and Data Studio to:
• Enable/Disable tables for replication
• Start/Stop replication
Data load and update options with Db2 Analytics Accelerator

Accelerator data load
Db2 Analytics
Accelerator
Studio
.
.
.
.
.
.
Db2AnalyticsAcceleratorAdministrative
StoredProcedures
Table B
.
.
.
Table A
Unload USS Pipe
Unload USS PipePart 2
Unload USS PipePart m
Table C
Part 1
Part 3
Part 2
Table D
Part 1
Data
Slices

Integrated Synchronization - Db2/Z to-Accelerator data synchronization
Applications executing
I/U/D Statements on replicated tables
Accelerator Users enabling
tables for replication
Table
T1
Log data
processor
Db2 Log
Table
T2
Table
T3
Table
T1
Table
T2
Table
T3
Accelerator
Server
Encrypted Log Data
Stored
Procedures
Log Data
Provider Staging
area
Process control

— Log data provider is a newly developed, internal Db2 for z/OS component
• Adheres to Db2 life-cycle management resulting in simplified installation, packaging, administration,
upgrade, support, … as compared to external data capture tools
• Fully zIIP enabled - MSU savings potential
• Streamlined design resulting in reduced CPU usage and higher throughput
— Log data processor is a newly developed, internal accelerator component
• Adheres to the accelerator life-cycle management resulting in simplified installation, packaging,
administration, upgrade, support, … as compared to external data capture tools
• Custom-built and optimized resulting in higher throughput and lower latency
• Significant enhancements in DB2 Warehouse insert/update/delete performance
Supports transactional consistency protocol that guarantees queries executed by IDAA
return most recently committed data: the cornerstone of application transparency and HTAP
Integrated Synchronization - Db2/Z to-Accelerator data synchronization

—Dynamic switch between „bulk“ and „trickle“ apply mode
• Bulk apply for mass updates in one table
• Trickle apply for small updates in many tables
—Presumed commit (early apply)
• feed (but not commit!) large changes as they arrive, not only after they are committed on
source
• When rollback on source, rollback on target
—Better handling of non-logged changes to Db2 tables
• Future item planned to be able to replicate selected non-logged utility actions, such as LOAD
with dummy input or REORG DISCARD of full partition
Optimized apply processing on accelerator side

— Db2 Analytics Accelerator V7.5
— Db2 12 for z/OS with APAR PH06628 PTF UI63356 installed
• In order to activate the new function, Db2 needs to be recycled
— Db2 running in function level V12R1M500
— Db2 12 for z/OS APAR PH19181 when available
• Fixes a problem when Db2 is highly loaded
Integrated Synchronization Pre-Reqs

“True HTAP” Overview
—Changes in Db2 z/OS data are propagated to the Accelerator using replication
technology
• On the Accelerator the incoming changes are applied
 This leads to a latency of a few seconds or even more (dependent on used replication technique)
—Consequence: Queries routed to the Accelerator may not see the latest changes
commited on the Db2 z/OS system
• For many use cases / applications this is absolutely acceptable
—Some use cases require, however, that the queries are guaranteed to return
results that are consistent with the latest committed data.
—“True HTAP” is a solution that, in general, maintains the efficiency of the
replication approach while delivering query results that are 100% up-to-date with
respect to the latest committed data in Db2 z/OS relative to SQL execution.
• Latency does not impact SQL result consistency
24

How does HTAP work?
Wait for committed data
from time of SQL request
25
Asynchronous
replication
Most
recent
committed
data
available?
no
Wait for
given
time
period
Most
recent
committed
data
required?
yes
no
Initiate
apply
Write
requests
OLTP
reads
OLAP
reads yes

How does HTAP work?
— Introducing new zParm QUERY_ACCEL_WAITFORDATA + Special
register + BIND option
• CURRENT QUERY ACCELERATION WAITFORDATA = n
o n = 0 - 3600 (seconds)
o Default: 0 = No wait
o Important: Can be set differently for each query
• WAITFORDATA = 0
o Immediately execute in accelerator (Current behavior, no delay)
• WAITFORDATA > 0
o Wait for committed changes to be applied via asynchronous replication
• If wait time is exceeded check CURRENT QUERY ACCELERATION special
register
 If “WITH FAILBACK” is specified, execute query in DB2
26

Agenda

Db2 Analytics Accelerator V7.5, deployment on IBM Integrated
Analytics System (IIAS)
• Hardware appliance
• A full solution that provides all components out of the box – including optimized hardware and software
• All components provided by IBM in a balanced, performance-optimized configuration
• HW, which includes the rack, the physical servers and the storage
• SW stack including the Linux operating system, the Docker software as well as the Docker container and
the infrastructure management
• IBM Power hardware for the appliance, balanced and optimized for price/performance

Db2 Analytics Accelerator Version 7.5, deployment on IBM Z
• A software appliance running on IBM Z
• Packages the SW stack into an IBM Secure Service Container to deliver a fully self-managed appliance running
in a SSC LPAR that can be deployed in minutes
• Integrates seamlessly into the customer’s Z environment and leverages known LPAR-, memory and CPU
management procedures, including call home support for enterprise hardware components.
• Uses customer-provided storage to hold the accelerator-side data
Existing
Components
SE / HCM
PR/SM LPAR CPU Memory
Storage (SAN)
Filesystem
IBM Secure Service Container
Customer’s
Storage
Management
Docker container
Db2 Warehouse
engine
Authentication
Accelerator
server
Workload
Monitoring
Systems Manager
Additional future
functionality
Docker supported OS + management
Deliveredaspartof
AcceleratorSolution

Db2 Analytics Accelerator on IBM Z
— Leverages IBM Secure Service Container
− SSC security features ensure that the appliance image cannot be tampered with
and the appliance code and data are protected and kept confidential both in
flight and at rest
— Accelerator on Z runs natively in an SSC LPAR on IFLs
— Customizable configuration and highly flexible scaling
− Single-Node: Minimum 2 IFLs / 64 GB memory, maximum 40 IFLs / 4,096 GB
− Multi-Node: Minimum 30 IFLs / 1.5 TB memory, maximum 190 IFLs / 20 TB
− Can utilize shared infrastructure such as network or storage adapters
— No additional licensed software required – no z/VM, no KVM,
no Linux on Z, no Docker, no …
− Accelerator not supported to run under z/VM or KVM control
— No operating system access or maintenance
− No system administrator access to appliance possible
− All required updates, e.g., security fixes, component updates, etc., are delivered
and installed as accelerator image updates
− All required configuration via administrative UI or configuration files

IBM Db2 Analytics Accelerator on IBM Z
Product components
IBM Z
Db2 code
including Stored Procedures
Accelerator Appliance
• Can be deployed on the
same CEC as Db2 or on a
different one
Appliance UI
• Data Studio with Db2
Analytics Accelerator
Studio Plug-in
• Data Server Manager
2.1.5 or higher
Dedicated highly available
network connection
OSA
OSA
OSA
OSA

Accelerator on IBM Z – Deployment Options
32

Multi-Node Deployment – Architecture
33

Multi-Node Deployment – IFLs & Memory
34
Storage
Db2 z/OS
Accelerator on Z
network
Head
IDAA server
Db2 WH
• Catalog
• No data
partitions
HiperSocket
Data 1
Db2 WH
• Data
partitions
Data 5
Db2 WH
• Data
partitions…
…
LPAR Group with absolute capping
SSC LPAR SSC LPAR SSC LPAR
OSA 30 IFLs
(shared)
256 GB
weight=high
14 IFLs
(shared)
512 GB
weight=low
14 IFLs
(shared)
512 GB
weight=low
Performance goal:
70-80 IFLs comparable to N3001-010
70 IFLs

Multi-Node Deployment – Advantages
35
—Scalability of the Accelerator on Z for the most demanding workloads
—Multi-node accelerator can grow from the entry level (30 IFLs) to the largest size
using all available IFLs on a system (190 IFLs on IBM z15) without ever reloading
the data
—Extremely flexible adjustment of resources (IFLs, memory, storage) to optimize for
the actual workload requirements
− Even dynamic adjustments (add/remove IFLs, add/remove memory, add storage) are
supported and require only short or even no downtime
− True “capacity-on-demand” without any disruption (for IFL capacity)
—Maintains all advantages of the deep integration into the Z platform

Learn more!
• What’s available?
• Product videos
• Guided demo
• Hands-on lab
Visit the Db2 Analytics Accelerator
on IBM Demos:
http://ibm.biz/Acceleratordemos

Notices and disclaimers
— © 2019 International Business Machines Corporation. No part of
this document may be reproduced or transmitted in any form
without written permission from IBM.
— U.S. Government Users Restricted Rights — use, duplication or
disclosure restricted by GSA ADP Schedule Contract with IBM.
— Information in these presentations (including information
relating to products that have not yet been announced by IBM)
has been reviewed for accuracy as of the date of
initial publication and could include unintentional technical or
typographical errors. IBM shall have no responsibility to update
this information. This document is distributed “as is” without
any warranty, either express or implied. In no event, shall IBM
be liable for any damage arising from the use of this
information, including but not limited to, loss of data, business
interruption, loss of profit or loss of opportunity.
IBM products and services are warranted per the terms and
conditions of the agreements under which they are provided.
— IBM products are manufactured from new parts or new and used
parts.
In some cases, a product may not be new and may have been
previously installed. Regardless, our warranty terms apply.”
— Any statements regarding IBM's future direction, intent or
product plans are subject to change or withdrawal without
notice.
— Performance data contained herein was generally obtained in a
controlled, isolated environments. Customer examples are
presented as illustrations of how those
— customers have used IBM products and the results they may have
achieved. Actual performance, cost, savings or other results in
other operating environments may vary.
— References in this document to IBM products, programs, or
services does not imply that IBM intends to make such products,
programs or services available in all countries in which
IBM operates or does business.
— Workshops, sessions and associated materials may have been
prepared by independent session speakers, and do not necessarily
reflect the views of IBM. All materials and discussions are provided
for informational purposes only, and are neither intended to, nor
shall constitute legal or other guidance or advice to any individual
participant or their specific situation.
— It is the customer’s responsibility to insure its own compliance
with legal requirements and to obtain advice of competent legal
counsel as to the identification and interpretation of any
relevant laws and regulatory requirements that may affect the
customer’s business and any actions the customer may need to
take to comply with such laws. IBM does not provide legal advice
or represent or warrant that its services or products will ensure that
the customer follows any law.

Notices and disclaimers continued
— Information concerning non-IBM products was obtained from the suppliers
of those products, their published announcements or other publicly
available sources. IBM has not tested those products about this publication
and cannot confirm the accuracy of performance, compatibility or any other
claims related to non-IBM products. Questions on the capabilities of non-
IBM products should be addressed to the suppliers of those products.
IBM does not warrant the quality of any third-party products, or the ability of
any such third-party products to interoperate with IBM’s products. IBM
expressly disclaims all warranties, expressed or implied, including but
not limited to, the implied warranties of merchantability and fitness for a
purpose.
— The provision of the information contained herein is not intended to, and
does not, grant any right or license under any IBM patents, copyrights,
trademarks or other intellectual property right.
— IBM, the IBM logo, ibm.com and [names of other referenced
IBM products and services used in the presentation] are
trademarks of International Business Machines Corporation,
registered in many jurisdictions worldwide. Other product and
service names might be trademarks of IBM or other
companies. A current list of IBM trademarks is available on
the Web at "Copyright and trademark information" at:
www.ibm.com/legal/copytrade.shtml

Db2 analytics accelerator technical update

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Db2 analytics accelerator technical update

Similar to Db2 analytics accelerator technical update (20)

More from Cuneyt Goksu

More from Cuneyt Goksu (20)

Recently uploaded

Recently uploaded (20)

Db2 analytics accelerator technical update