Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Db2 family and v11.1.4.4


Published on

Db2: Write once, run anywhere, from any source

Published in: Software
  • Be the first to comment

  • Be the first to like this

Db2 family and v11.1.4.4

  1. 1. Db2 family and update Jessica Rockwood IBM Monday October 1, 2018
  2. 2. Please note : • IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice and at IBM’s sole discretion. • Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. • The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. • The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. • Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 2
  3. 3. Db2: Write once, run anywhere, from any source 3 Db2 Warehouse on Cloud Db2 Warehouse Db2 Hosted Db2 on Cloud IBM Integrated Analytics SystemDb2 Big SQL Db2 Common SQL Engine
  4. 4. Db2 Common SQL Engine = Investment protection • The Db2 Common SQL Engine lets you leverage your existing investment in skills and applications across the different deployments • Application portability • Common programming model • Common client interfaces • “Inclusive” compatibility: Oracle, Netezza • Eco-system compatibility • One ISV product certification for all platforms – test once, certify all 4
  5. 5. DB2 Common SQL Engine = Support for different workload types • The Db2 Common SQL Engine brings with it the proven ability to support different types of workloads • Analytics • OLTP • Hybrid of Transactional and Analytic Processing (HTAP) OR +OR 5
  6. 6. Db2 Common SQL Engine = Consistent Technical Capabilities • All of Db2’s strong underlying capabilities are available • High capacity • High concurrency levels • High performance levels • Continuous/High availability • Consistent operations • Inherent security (including encryption) • Integrated workload management • Common monitoring interfaces 6
  7. 7. Db2: More intelligent analytics and insight • Db2 Common SQL Engine provides the strong foundation needed to take the next steps up the AI ladder • High performance, high capacity • Integrated analytic capabilities • Data virtualization technologies to provide a single point of access regardless of where data resides 7 AI Machine Learning Analytics Data Db2 Common SQL Engine The “AI Ladder”
  8. 8. Db2: Flexibility of deployment 8 On Premise On Cloud SOFTWARE docker APPLIANCE PRIVATE CLOUD HOSTED DBaaS MANAGED DBaaS CONTROL SIMPLICITY Db2 Warehouse Db2 Warehouse IBM Integrated Analytics System Db2 Hosted Db2 on Cloud Db2 Warehouse on Cloud
  9. 9. Agenda • A quick overview of Db2 11.1 • Looking at the latest: • Highlights from Db2 and • A look into the Future 9
  10. 10. A quick overview 10
  11. 11. Db2 11.1: A new beginning • Starting with DB2 11.1, we have begun to move towards a “continuous delivery” model • Explicitly plan to deliver function throughout the active development lifetime of a version • New versions will still happen occasionally for various reasons • What does this mean to you? • Introduction of Modification levels to the product signature • Function delivered continuously during a version’s lifecycle • No more “big gulps”, just a series of “little bites” 11
  12. 12. What is a Modification level? • Db2 product signature consists of 4 parts: VV.RR.MM.F (Version.Release.Modification.Fix) • Traditionally, updates for a Db2 version were fixpacks and would increment the last part • E.g. The ninth update for Db2 10.5 was a fixpack with the signature of Db2 • These contained fixes and sometimes new function • Starting with Db2 11.1, each update with planned function will be a new modification level • Also decided to increase fixpack level at same time to keep reference terms familiar (e.g. “Fixpack 2” is the same as “Mod 2”) • Db2 was the first update provided after Db2 11.1 general availability (GA) level • It is an update just like past fixpacks and is applied in the same way with the same expectations* 12 * Except that mod 1 was not pureScale rolling update compatible!! All subsequent 11.1 updates are!
  13. 13. V11.1.0.0 V11.1.1.1 V11.1.2.2 V11.1.4.mV11.1.4.4 Fixes + Function Fixes + Function Fixes + Function FixesGA V11.1.3.3 Fixes + Function 4Q ‘16Mid ‘16 2Q ‘17 1Q ‘18 2H ‘18 Regular fix/mod pack deliveries (every 6-9 months) ! With overriding focus on stability for immediate production deployment ! Modifications will contain select functional enhancements (often off by default) The Db2 11.1 lifecycle 13
  14. 14. 14 pureScale “for the Masses” • Up and running in hours • Zero data loss DR with HADR • Multi-switch GDPC (no SPOF) • Seamless HADR pureScale upgrades • Restore REBUILD • Online Index Create • Add/Drop CF Increased Serviceability • 16x increase in active log space capability • Enhanced monitoring for lock and logging events Additional Core Function • Workload Manager (WLM) multi-tenancy extensions • Federation simplification and integration Comprehensive Enterprise Security Mainframe Class Availability Significant Core Database Advances Enterprise Encryption GA, FP1, FP2, FP3 Highlights: Mission Critical Workloads • Centralized Key Managers (KMIP) • PKCS#11 HSM support • SSL Encryption for non-pS HADR (Linux x86, ALL platforms) Even Greater Availability • Online Crash Recovery • Faster Rollback of Multiple and Large Txn • ADMIN_MOVE_TABLE advances • Better tablespace state and replay window monitoring with HADR • Per partition OLR • Avoid Lock Escalation • Timeout support for vendor Archive Simpler, Faster, More Online Upgrades • Faster, no need for offline backup • Streamlined HADR upgrade • DB2 Version 9.7 direct to 11.1 Very Large Database Performance • Higher user throughput • Faster MCR by default • Improved sockets performance • Multiple hosts in maintenance mode • Recover HADR Standby w/ tbsp restore (now for pS) • Improved XA performance HADR ease of use: • Export HADR TSAMP configuration to XML file
  15. 15. 15 Massive Scale Warehousing at In-Memory Performance MPP BLU Scalability • PB scale in-memory warehousing Next Gen In-Memory Performance, Function and Workloads Continued Performance Improvements • Synopsis table enhancements • Additional SIMD exploitation • Aggregation enhancements, sort elimination • INSERT from sub-select – multi-core parallelism improvements • BLU Acceleration indexes SQL and Compatibility PDF manuals for DB2 LUW v11 Multi-Lingual SQL Advances • BINARY, EXTENDED ROWSIZE • CHAR(255) BLU Restriction Removal • Automatic Dictionary Creation improvements • ALTER VARCHAR/VARGRAPHIC length support • Codepage 819 GA, FP1, FP2, FP3 Highlights: SQL & Warehousing Workloads Faster ELT/ETL performance More Query Workloads Optimised More Function supported • Generated Columns • RCAC • User maintained temporal tables Richer and Even More Compatible SQL • JSON support • BOOLEAN data type • Common table expression (Db2 for z/OS compatibility) • WITH and SELECT INTO support • Alias for XML
  16. 16. What about getting critical fixes in between official updates? • To ensure that critical fixes are made available as soon as possible, we have introduced the concept of an interim fix (aka “iFix”) • An iFix is an update based on the last shipped update + a select set of critical updates that have occurred since the update was released • Critical updates = Hiper, Security, and “high impact” APARs • Cumulative (i.e. Db2 iFix002 contains Db2 iFix001) • Reset after each new official update (i.e. the first iFix after Db2 was Db2 iFix001) • To be released every 2-3 months until the next update occurs 16
  17. 17. How to stay current with Db2 • Passive • Browse the list of available fix packs • • Security Vulnerabilities, HIPER and Special Attention APARs fixed in DB2 for Linux, UNIX, and Windows Version 11.1 • • Proactive • Go to the IBM Support page and sign up for “My notifications” • 17
  18. 18. Getting to Db2 11.1 • Upgrade to Db2 11.1 is supported from Db2 9.7, Db2 10.1, and Db2 10.5 v New option to upgrade from Db2 10.5 without the need for an offline backup • For non-pureScale & non-DPF configurations, minimum origin level is Db2 • For pureScale configurations, minimum origin level is Db2 • Not available for DPF configurations at this time v New option for “seamless” upgrade for HADR configurations • For serial configurations, minimum origin level is Db2 • For pureScale configurations, minimum origin level is Db2 moving to at least Db2 18
  19. 19. UPGRADE without Offline Backup • No more need to take an offline backup to ensure recoverability across upgrade! • Example scenario : • Recovery Procedure Overview: • Revert back to original Db2 10.5 level • Restore online backup (A) • Begin to roll-forward to a desired point-in-time just before the error • Receive SQL2463N or SQL2464N indicating the start of upgrade encountered in the logs • Upgrade to Db2 11.1 and continue roll-forward recovery Online Backup (A) Online Backup …..UPGRADE Transactions Transactions …… Db2 10.5 FP7/9 Db2 11.1 19
  20. 20. Primary Replays through UPGRADE UPGRADE Primary Standby Backup Image UPGRADE + BACKUP Backup Image RESTORE COPY BACKUP IMAGE Standby UPGRADE Log Records Db2 11.1 pre-Db2 11.1 Seamless HADR Upgrades • HADR environments can now be upgraded without re- initializing the standby. • Standby database will ‘replay’ the upgrade done on the primary by rolling forward through the update log records • For non-pureScale: Db2 => Db2 • For pureScale: Db2 => Db2 • More details in the DB2 11.1 documentation in “Upgrading DB2 servers in HADR pureScale environments (without standby reinitialization)” • .1.0/ 20
  21. 21. Looking at the latest 21
  22. 22. Where to go to find out what is in each Db2 11.1 update? • “Mod Pack and Fix Pack Updates” section in the Db2 documents • c/c0061179.html • The following section describes only some of the highlights in Db2 and Db2 22
  23. 23. Just an FYI: RFEs that have been shipped so far in DB2 11.1 23 RFE
  24. 24. Db2 24
  25. 25. Online Index Create (concurrent write access) now on pureScale! • Prior to Db2, a CREATE INDEX on a pureScale system would prevent concurrent writes to the table • As of Db2, full online index creation is supported on pureScale! • Concurrent read and write access to the table now available • One exception is expression-based indexes • Off by default on pureScale db2set DB2_CREATE_INDEX_ALLOW_WRITE=YES db2 connect to MYDB db2 create index …. 25
  26. 26. Online ADD/DROP of a cluster caching facility (CF) • While Db2 already supports the ability to add a member online, the same was not possible for a cluster caching facility (CF) • As of Db2 you can now add or drop a CF online • Expected use cases: • Move a CF to a new host • Remove a failed or maintenance-mode CF • Improve HA of a single-CF cluster by adding a second CF 26 RFE
  27. 27. Greater flexibility in pureScale maintenance mode options • When you are applying updates to a host in a Db2 pureScale configuration, or to its underlying platform, you have to put it into maintenance mode to isolate it • Prior to Db2, your options were: • Single host in maintenance mode • All hosts (cluster outage) in maintenance mode • This can be a problem when you have a member in maintenance mode for a long time (e.g. hardware issue) as it prevents any other host from being updated until the first issue is resolved • As of Db2 you can now put multiple hosts into maintenance mode! • You can enter maintenance mode on each host one at a time as long until there is only one host (with at least a CF, and preferably a member) left in the cluster 27 RFE
  28. 28. V11.1.2.2 V11.1.3.3 Rollback time Seconds Faster Rollback Of Very Large Transactions • Internal changes made to use buffered I/O on filled log files which significantly reduces rollback time for very large transactions • Example from internal testing: • Single threaded rollback of a very large insert transaction is up to 3x faster! 28
  29. 29. HADR Read on Standby (RoS): Improved monitoring • Improve identification for cause of HADR ROS 'replay-only-windows’ • Additional diagnostic messages in db2diag.log on standby to report the operation that caused the replay-only window • When the LOG_DDL_STMTS database configuration parameter is set to YES on the primary, the DDL statement text is included • The frequency of these diagnostic messages can be configured using the DB2_HADR_REPLAY_ONLY_WINDOW_DIAGLEVEL registry variable. • db2fmtlog tool can also be used to display all log records (if any) that will cause a replay-only-window 29
  30. 30. User Maintained Temporal Tables for BLU • As of Db2, you can now: • Create column-organized user-maintained system-period temporal tables (STT) • Create column-organized user-maintained application-period temporal tables (ATT) • Create column-organized and user-maintained bi-temporal table (BTT) • Execute read-only temporal queries on column-organized temporal tables. • For more information, see “Temporal Table Support for Column-Organized Tables in DB2” technote • 30 RFE
  31. 31. Why BLU Acceleration indexes are needed • Db2 BLU acceleration provides superior performance for analytic workloads and is designed for querying very large amounts of data … but some queries only access a very small number of rows Ø An index is the most efficient way to do this! 31
  32. 32. BLU Acceleration indexes • DB2 11.1. currently implicitly creates unique indexes to support PRIMARY or UNIQUE KEY constraints • A unique index can be used to access data if at most 1 row qualifies and every key in the index must have an equality predicate OR the FETCH FIRST 1 ROW ONLY clause is specified • As of Db2, Db2 now supports CREATE INDEX for column-organized tables: • Unique and non-unique indexes • DROP and ALTER also supported • ALTER INDEX only supports COMPRESS [YES|NO] • These new indexes can be used to access column-organized data using most of the same methods as row-organized data and without restrictions on the number of qualifying rows 32
  33. 33. Who will benefit from BLU Acceleration indexes? • This new index support greatly improves Db2’s ability to execute queries that access only a ‘small’ amount of column-organized data • Extends Db2 BLU capabilities in the HTAP world • Will improve performance for a range of applications: • Data warehouse ETL with OLTP characteristics • Operational analytics • “Light” OLTP • For existing BLU analytics workloads, will only be used in limited situations • Analytic queries with transactional characteristics 33
  34. 34. Other notable content in Db2 • SSL Encryption for HADR communications now supported on all non-pureScale platforms • Improved performance for moderate to heavy write workloads on pureScale configurations using TCPIP sockets • Native encryption enhancement to significantly reduce keystore access for transaction logs • New value available for DB2AUTH registry variable to enable a warning diagnostic message to be output when SERVER_ENCRYPT is not being respected • Introduction of timeout support for Log-Archive/Retrieve Requests through Vendor API • New “-o” option for db2haiku to export the current TSAMP configuration to an XML output file 34 RFE
  35. 35. A look into the future 35
  36. 36. Sign up for early access: 36
  37. 37. Db2 37
  38. 38. Db2 • This is most likely the final modification level for Db2 11.1 • I.e. future Db2 11.1 updates after this will just contain fixes • Next updates would be DB2 followed by Db2 • Key content: • Extent reclaim support for pureScale • ISO JSON SQL functions • Preview of HADR Read on Standby (RoS) enhancements • Preview of 4K sector size support 38
  39. 39. Extent reclaim support for pureScale • When table objects are dropped, the storage may not be immediately available to be released from the tablespace • Only space above the tablespace high water mark (HWM) can be released • The ALTER TABLESPACE … REDUCE statement attempts to reduce the high water mark for the table space by moving live extents to unused ones lower in the tablespace • Supported with reclaimable storage DMS table spaces (created in Db2 9.7 or newer) • Not yet supported on pureScale….. but we’re working on it for Db2! 39
  40. 40. Extent reclaim 40 DROP TABLE 2 DROP TABLE 3 Internal table space metadata extents Table 1 Table 2 Table 3 Extent that is allocated to a table space, but not to a table ALTER TABLESPACE … REDUCE MAX
  41. 41. A brief history of Db2 & JSON • Db2 10.5 FP1 introduced JSON NoSQL support • Focused on allowing Db2 to participate in the NoSQL paradigm • Our customers began to ask for native SQL support of JSON in Db2 • Legacy applications accessing new JSON data • New JSON applications wanting to access legacy data Ø As a tactical response, we began to informally reveal the existence of our proprietary Db2 JSON SQL functions in 41
  42. 42. First wave of new built-in JSON SQL functions coming in Db2! • New implementations of JSON SQL functions based on recent ISO report on SQL support for JavaScript Object Notation (JSON) § The (proprietary) SYSTOOLS functions will be de-emphasized but will continue to be supported Schema Name Comments SYSIBM BSON_TO_JSON Convert BSON formatted document into JSON strings SYSIBM JSON_TO_BSON Convert JSON strings into a BSON document format SYSIBM JSON_ARRAY Creates JSON array from input key value pairs SYSIBM JSON_OBJECT Creates JSON object from input key value pairs SYSIBM JSON_VALUE Extract an SQL scalar value from a JSON object SYSIBM JSON_QUERY Extract a JSON object from a JSON object SYSIBM JSON_TABLE Creates relational output from a JSON object SYSIBM JSON_EXISTS Determine whether a JSON object contains the desired JSON value 42
  43. 43. What makes these JSON SQL functions different/better? • Following a public “standard” (ISO technical report) • Easier to use • No need to qualify or add SYSTOOLS to function path • No need to grant EXECUTE privilege • Simpler and more flexible storage options • You choose the stored format: JSON or BSON • You choose the table organization: row or column • You choose the column data type • BLOB, CHAR, CLOB, VARBINARY, VARCHAR • Normal Db2 mechanisms can be used to load JSON (or BSON) data into tables (e.g. INSERT, Load, etc..) • Conversion functions are now complimentary not mandatory 43
  44. 44. Preview: HADR Read on Standby (RoS) as it exists today… 44 T1 T2 Primary server T1 T2 Standby server HADR log flow • CREATE TABLE T1 • CREATE INDEX ON T1 • RUNSTATS ON T1 • Etc.
  45. 45. Preview: HADR Read on Standby (RoS) as it exists tomorrow… 45 T1 T2 Primary server T1 T2 Standby server HADR log flow • CREATE TABLE T1 • CREATE INDEX ON T1 • RUNSTATS ON T1 • Etc.
  46. 46. Preview: Support for storage devices using 4K sector size • This support can be enabled by setting the DB2_4K_DEVICE_SUPPORT registry variable to ON • This feature is not supported for production use at this time Ø Try it out and let us know! • Restrictions and limitations: • The use of DMS Raw containers is not supported • Backup and load copy files will be slightly larger • There may be a performance penalty accessing: • LOB data stored on 512-byte sector storage • Backup or load copy files created prior to the enablement of 4K device support 46
  47. 47. Db2 “VNext” 47
  48. 48. Db2 “VNext” • We plan to release a new version of Db2 in 2019 • This will get all our Db2 and Db2 Warehouse offerings on the same level of the Db2 common SQL engine • Db2 11.1 will be 3 years old in 2019!! • We will continue to follow the “continuous delivery” paradigm started with Db2 11.1 • Steady release of function through GA and subsequent modification updates • Db2 11.1 shipped 4 modification updates since GAing in 2016 • Version/Release number for Db2 VNext not decided yet 48
  49. 49. Examples of the enhancements planned for Db2 Vnext lifetime • Available in the Db2 Warehouse family today • External tables • Grooming for columnar tables • Reduced logging • Schema level authorization • Adaptive WLM • Used as the active default for the DB2 Warehouse family today • Advanced log space management • Enhanced support for Currently Committed (CC) isolation on pureScale 49
  50. 50. Continuous evolution: New technologies • Adaptation and integration of new technologies into the Db2 common SQL engine • E.g. Machine Learning (ML) • Exploration of new approaches to the challenges of data growth and real-time analytics • Db2 Event Store • IBM Queryplex ( 50
  51. 51. Further out? • We revisit development priorities frequently (e.g. every quarter) in response to customer and market demand/feedback • As a result: some items move up, some down, some in, and some out. • Public roadmap: • 51
  52. 52. How to learn more 52
  53. 53. Come hear more from Db2 community experts! 53 Db2 development lab speakers include:
  54. 54. Try out IBM Db2 Developer Community Edition • Free, fully-functional version of Db2 • Includes all Db2 features such as compression, BLU Acceleration • Supports all Db2 configurations including pureScale and DPF • No restrictions on where you can use it (e.g. production, development, test) • Find out more: • Restrictions • Unsupported edition (non-warranted) • Environment limited to: • 4 cores with 16GB of memory • 100GB per database • Excludes some “Supporting Programs”: • Cognos, IDA, WebSphere AS / MQ, DSM Enterprise • Provided as a docker-based install with Data Server Manager (DSM) and Data Studio • Also offered: Db2 Developer-C Community Edition which is a native Db2 (only) installation 54
  55. 55. Jessica Rockwood IBM Db2 family and update