Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
IBM Red Brick Warehouse Server Roadmap/Futures Fred Ho & Cindy Fung  Software Development Managers  Information Management...
Topics <ul><li>Red Brick Update </li></ul><ul><ul><li>Red Brick as part of IBM  </li></ul></ul><ul><ul><li>Red Brick Produ...
Quick update on Red Brick @ IBM <ul><li>Loyalty to Red Brick Warehouse Server </li></ul><ul><ul><li>Continued strong endor...
<ul><li>2002 </li></ul>2003 Integrated Analytics 2004 Interoperability Performance, Usability IBM Red Brick Warehouse Road...
<ul><li>SQL/OLAP </li></ul><ul><li>Vista Query Rewrite Extensions </li></ul><ul><li>Query Performance Monitor </li></ul><u...
<ul><li>Major Performance Improvements  </li></ul><ul><ul><li>Dynamic Smartscan  </li></ul></ul><ul><ul><li>Memory mapping...
IBM, Informix & Red Brick Database evolution C-ISAM Indexed Record access SE SQL Interface, Stored Procedures Online Avail...
<ul><li>Can I stay on Red Brick or will IBM make me move to DB2? </li></ul><ul><li>Our Philosophy </li></ul><ul><ul><li>&q...
Red Brick Customer Status <ul><li>IBM’s goal with Red Brick is to continue to sell to and support our customers via direct...
Customer Facing Events <ul><li>European Red Brick Users’ Group Meeting </li></ul><ul><ul><li>Sept. 5 th , 2003 in Milan, I...
Red Brick Integration into DB2 <ul><li>Introduce key Red Brick technologies over next few years into DB2 </li></ul><ul><ul...
GOLD BUNDLES <ul><li>WHAT IS A GOLD BUNDLE? </li></ul><ul><ul><li>Product offering designed to offer a sense of &quot;one&...
RED BRICK GOLD BUNDLES <ul><li>Red Brick Analytic Bundle - Enterprises contains: </li></ul><ul><ul><li>IBM Red Brick Wareh...
DB2 Information Integrator V8.1 & Red Brick Warehouse Server
DB2 Information Integrator <ul><li>Introducing new integration software : DB2 Information Integrator </li></ul><ul><ul><li...
DB2 Information Integrator : Value for Red Brick <ul><li>Key business value: </li></ul><ul><ul><li>Optimize IT investments...
Scenario: Integrate Red Brick with DB2 UDB <ul><li>Your Value  </li></ul><ul><ul><li>Integrate Red Brick data marts seamle...
Scenario: Enable Third Party App Support <ul><li>Your Value </li></ul><ul><ul><li>Want to use packaged applications (e.g. ...
IBM Red Brick Warehouse Server 6.3 Features
6.3 Features <ul><li>Major Performance Improvements </li></ul><ul><ul><li>Dynamic Smartscan </li></ul></ul><ul><ul><li>Mem...
6.3 Features <ul><li>Major Usability Improvements – cont’d </li></ul><ul><ul><li>Allow 3GB Address Space on Windows Platfo...
Dynamic SmartScan <ul><li>Additional queries could be considered for SmartScan segment elimination </li></ul><ul><li>Inclu...
Dynamic Segment Elimination Example <ul><li>Assume, “Sales” as a Fact table that is segmented by a foreign key “perkey”.  ...
MMAP Dimension Table/Index <ul><li>Reduce CPU overhead and I/O system calls by mmap dimension indexes and tables into shar...
TARGETjoin Performance <ul><li>Improve performance of TARGETjoin </li></ul><ul><ul><li>Particularly for local indexes </li...
Optimizer Hints <ul><li>STAR indexes can be specified for queries on a per table basis </li></ul><ul><ul><li>Must be caref...
Optimizer Hints <ul><li>STARjoin and TARGETjoin thresholds can be specified on a per table basis </li></ul><ul><li>SET STA...
TMU Memory Tuning <ul><li>Better control over TMU memory resource usage </li></ul><ul><li>Allows TMU buffer memory to be t...
TMU Memory Tuning <ul><li>Tuning rule of thumb: more logical I/O requires more buffers </li></ul><ul><li>Syntax: </li></ul...
More SQL/OLAP Functions <ul><li>Distribution Functions </li></ul><ul><li>CUME_DIST </li></ul><ul><li>PERCENT_RANK </li></u...
Distribution Functions <ul><li>CUME_DIST()  computes the position of specified row value relative to the set of values </l...
Inverse Distribution Functions <ul><li>Answers question such as “What is the median (50 th  percentile) value of my data?”...
ROUND() scalar function <ul><li>ROUND() returns rounded number to the integer places left or right of the decimal point </...
Expression Support in Loader <ul><li>Input data can now be modified while being loaded to a table </li></ul><ul><li>Basic ...
Expression Support in Loader <ul><li>Syntax: (snippets from TMU control file) </li></ul><ul><li>Arithmetic expressions: </...
Expression Support in Loader <ul><li>Pseudo column assignment </li></ul><ul><li>$A POSITION(2) INTEGER EXTERNAL(10), </li>...
XML Improvements in Loader <ul><li>Extends TMU and SQL Export functionality to provide additional XML support </li></ul><u...
Compact System Catalog <ul><li>Compacts free space within system catalog </li></ul><ul><ul><li>Occurs when objects are fre...
3GB Address Space on Windows <ul><li>Extend beyond 32-bit memory access limit </li></ul><ul><li>Increase virtual address s...
Loading with Multiple Characters Separator <ul><li>Apply to loading and exporting in delimited format </li></ul><ul><li>ne...
Export with String Delimiter Support <ul><li>Adds delimiter support to enclose a string </li></ul><ul><li>New export comma...
ALTER TABLE Using a Working Segment <ul><li>Provides more reliable recoverability of failed alter operations than existing...
ALTER TABLE enhancements <ul><li>Much requested feature </li></ul><ul><li>Combines  nominal space requirements  of  IN_PLA...
HP-Itanium Porting Project <ul><li>A native port, not architectural emulation  </li></ul><ul><ul><li>Yields high performan...
DB2 Products that support Red Brick currently <ul><li>DataJoiner Version 2.1.1 Works with ODBC wrapper. See  http://www7b....
Future DB2 Interoperability Options <ul><li>Information Integrator </li></ul><ul><ul><li>Planned for beta in late August <...
Features Being Considered Beyond v6.3
<ul><li>Features Being Considered </li></ul><ul><li>Windows-64 Support </li></ul><ul><li>Improved Query Parallelism </li><...
Red Brick Futures <ul><li>IBM committed to supporting Red Brick for the forseeable future </li></ul><ul><li>DB2 is the fla...
Questions ?
Upcoming SlideShare
Loading in …5
×

IBM Red Brick Warehouse Server Roadmap/Futures Fred Ho

1,359 views

Published on

  • Be the first to comment

  • Be the first to like this

IBM Red Brick Warehouse Server Roadmap/Futures Fred Ho

  1. 1. IBM Red Brick Warehouse Server Roadmap/Futures Fred Ho & Cindy Fung Software Development Managers Information Management Solutions
  2. 2. Topics <ul><li>Red Brick Update </li></ul><ul><ul><li>Red Brick as part of IBM </li></ul></ul><ul><ul><li>Red Brick Product Roadmap </li></ul></ul><ul><ul><li>Red Brick Versions 6.2 and 6.3 Overview </li></ul></ul><ul><ul><li>Red Brick Gold Bundles </li></ul></ul><ul><ul><li>Summary </li></ul></ul><ul><li>Red Brick and DB2 Information Integration </li></ul><ul><li>6.3 Features Detail </li></ul><ul><li>Future considerations </li></ul><ul><li>Summary </li></ul>
  3. 3. Quick update on Red Brick @ IBM <ul><li>Loyalty to Red Brick Warehouse Server </li></ul><ul><ul><li>Continued strong endorsement by customers </li></ul></ul><ul><ul><li>Continued investment & support by IBM </li></ul></ul><ul><li>Product roadmap </li></ul><ul><ul><li>Maintained release timelines as pre-acquisition days </li></ul></ul><ul><ul><li>Shipped one major enhancement release (v6.2) & several maintenance releases </li></ul></ul><ul><ul><li>Next feature release is on target </li></ul></ul><ul><li>Red Brick team </li></ul><ul><ul><li>Integrated into IBM Data Management and BI group at SVL </li></ul></ul><ul><ul><li>Retention still very high </li></ul></ul><ul><li>Red Brick partnerships </li></ul><ul><ul><li>Key part of the acquisition </li></ul></ul><ul><ul><li>Strong retention of partners </li></ul></ul>
  4. 4. <ul><li>2002 </li></ul>2003 Integrated Analytics 2004 Interoperability Performance, Usability IBM Red Brick Warehouse Roadmap High Performance, Easy to Administer Data Mart Engine 2005 Red Brick v6.2 Red Brick v6.3 <ul><li>Target: (1H2004) </li></ul><ul><li>DB2 BI tools integration </li></ul><ul><li>Performance Improvements to STARJoin and TARGETJoin </li></ul><ul><li>Dynamic Smart Scan </li></ul><ul><li>Self Tuning TMU Block Cache Surrogate Key Generation & </li></ul><ul><li>Expression support in TMU </li></ul><ul><li>Improve performance of aggregations,deletes, table scans, etc. </li></ul><ul><li>Others ..... </li></ul><ul><li>GA’ed Oct. ‘02 </li></ul><ul><li>ANSI SQL/OLAP functions </li></ul><ul><li>Integrated Backup/Restore </li></ul><ul><li>Query/Load Performance </li></ul><ul><li>Load/Export data in XML </li></ul><ul><li>Performance monitor </li></ul><ul><li>Locally segmented indexes </li></ul><ul><li>Others …. </li></ul>Red Brick v6.4 <ul><li>In Planning: (2H2005) </li></ul><ul><li>Increased parallelism for query performance and loader support </li></ul><ul><li>Additional platform support </li></ul><ul><li>Improved diagnostics </li></ul>
  5. 5. <ul><li>SQL/OLAP </li></ul><ul><li>Vista Query Rewrite Extensions </li></ul><ul><li>Query Performance Monitor </li></ul><ul><li>Locally Segmented Index </li></ul><ul><li>Backup & Restore Facility </li></ul><ul><li>XML Data Load/Unload </li></ul><ul><li>Remote Load TMU </li></ul><ul><li>System Catalog Improvements </li></ul><ul><li>Upgrade to 6.20 (32 bit to 64 bit) </li></ul><ul><li>Segment DDL Enhancements </li></ul><ul><li>Connectivity (ODBC/JDBC) Enhancements </li></ul>GA - 10/1/02 in Passport Advantage Red Brick v6.2 Content – Available today Majority of customers have migrated to v6.2
  6. 6. <ul><li>Major Performance Improvements </li></ul><ul><ul><li>Dynamic Smartscan </li></ul></ul><ul><ul><li>Memory mapping of dimension index/tables for STARjoin performance </li></ul></ul><ul><ul><li>Target Join improvements for local index </li></ul></ul><ul><ul><li>Optimizer hints to specify STARindex for fact-to-fact joins </li></ul></ul><ul><ul><li>Table Management Utility (TMU) Memory Tuning for non-optimized loads </li></ul></ul><ul><li>Major Usability Improvements </li></ul><ul><ul><li>Alter Table with Working Segment </li></ul></ul><ul><ul><li>Additional Expression support (e.g. conditional IF's) in the Loader </li></ul></ul><ul><ul><li>Additional SQL/OLAP functions, e.g. Round, Median </li></ul></ul><ul><ul><li>Shrink System Catalog Utility </li></ul></ul><ul><ul><li>Allow 3GB Address Space on Windows Platform </li></ul></ul><ul><ul><li>Delimiter enhancements in Loading and Exporting </li></ul></ul><ul><li>Port and OS Versions Upgrade </li></ul><ul><ul><li>Support HP Itanium Product Family (IPF) (64-bit) </li></ul></ul><ul><ul><li>Upgrades for AIX v5.2, SUN Solaris 9, DEC Tru64 5.1A </li></ul></ul>Planned v6.3 - Themes and Content GA – 1H2004 in Passport Advantage Red Brick v6.3 Themes and Content
  7. 7. IBM, Informix & Red Brick Database evolution C-ISAM Indexed Record access SE SQL Interface, Stored Procedures Online Available, Distributed, Performance IDS 7 Scalable, Parallel ’ 70s ’80 ’85 ’90 ’95 ’00 ’05 …… IMS – Hierarchical, Very Fast, Available DB2/390 - SQL Interface, Stored Procedures, Available DB2/UDB - Open, Parallel, Available, Federated, XPS – DW on MPP, Clusters Red Brick – Star Schema IDS 9 Extensible, Usable, Secure DB2 Arrowhead – SMART, Information Integration Includes best of IDS, XPS, Red Brick & DB2 30 years of Innovation and Investment protection
  8. 8. <ul><li>Can I stay on Red Brick or will IBM make me move to DB2? </li></ul><ul><li>Our Philosophy </li></ul><ul><ul><li>&quot;Moves anywhere are difficult. Move only if the result is a recognizable value advantage for you” </li></ul></ul><ul><li>Our Promise </li></ul><ul><ul><li>&quot;We'll continue to support the Red Brick product line for as long as customers require&quot; </li></ul></ul><ul><li>Our Proof </li></ul><ul><ul><li>&quot;100's of IMS* customers still receive regular updates to a product originally released in 1968&quot; (IMS is a mainframe, hierarchical database, the backbone of the world's financial systems) </li></ul></ul>
  9. 9. Red Brick Customer Status <ul><li>IBM’s goal with Red Brick is to continue to sell to and support our customers via direct sales and partner sales world-wide </li></ul><ul><li>We have maintained very strong customer loyalty </li></ul><ul><ul><li>Existing customers growing their Red Brick environments </li></ul></ul><ul><li>Our customers are happy with Red Brick, the IBM Data Management strategy and are looking into other DB2 products such as: DB2 Warehouse Manager, DB2 Intelligent Miner, DB2 Information Integrator, etc. </li></ul>
  10. 10. Customer Facing Events <ul><li>European Red Brick Users’ Group Meeting </li></ul><ul><ul><li>Sept. 5 th , 2003 in Milan, Italy </li></ul></ul><ul><li>IBM Data Management Technical Conference from </li></ul><ul><ul><li>Oct 27 – 31 2003 in Las Vegas </li></ul></ul><ul><ul><li>Numerous Technical sessions on Red Brick by senior members of the Red Brick team </li></ul></ul><ul><ul><li>Key Red Brick customer presentations </li></ul></ul><ul><ul><li>Joint DB2 and Red Brick sessions also included </li></ul></ul><ul><li>Red Brick v6.3 Beta Program to start in Oct 2003 </li></ul><ul><ul><li>Early access to the new features </li></ul></ul><ul><ul><li>Ability to work closely w/ R&D to help influence the quality of the release </li></ul></ul><ul><ul><li>Will work with other BI vendors to test and support Red Brick v6.3 </li></ul></ul>
  11. 11. Red Brick Integration into DB2 <ul><li>Introduce key Red Brick technologies over next few years into DB2 </li></ul><ul><ul><li>Lead development of next generation DB2 star join algorithm </li></ul></ul><ul><ul><li>Participate in the design of other core technology projects </li></ul></ul><ul><ul><ul><li>Data partitioning </li></ul></ul></ul><ul><ul><ul><li>Load performance and function </li></ul></ul></ul><ul><ul><ul><li>MQT (Materialized Query Table) enhancements </li></ul></ul></ul><ul><ul><li>Champion other DB2 SQL extensions </li></ul></ul><ul><ul><li>Assist definition of “ease of administration” requirements </li></ul></ul><ul><ul><ul><li>Synchronize with SMART team </li></ul></ul></ul><ul><li>Provide interoperability support with other DB2 products </li></ul><ul><ul><li>DB2 Warehouse Manager </li></ul></ul><ul><ul><li>DB2 Information Integrator </li></ul></ul><ul><ul><li>Query Monitoring Facility (QMF for Windows) </li></ul></ul><ul><li>Provide integration with System and Storage Management </li></ul><ul><ul><li>Tivoli TSM </li></ul></ul>
  12. 12. GOLD BUNDLES <ul><li>WHAT IS A GOLD BUNDLE? </li></ul><ul><ul><li>Product offering designed to offer a sense of &quot;one&quot; company </li></ul></ul><ul><ul><li>&quot;Customer Choice&quot; Database License </li></ul></ul><ul><ul><li>Customer can choose database platform as needs dictate, using same license for different products </li></ul></ul><ul><ul><li>Single database license offering customers a choice of IBM DM products (i.e. Informix, Red Brick, DB2) </li></ul></ul>
  13. 13. RED BRICK GOLD BUNDLES <ul><li>Red Brick Analytic Bundle - Enterprises contains: </li></ul><ul><ul><li>IBM Red Brick Warehouse Server v6.2 </li></ul></ul><ul><ul><li>IBM DB2 ESE v8.1 </li></ul></ul><ul><ul><li>IBM DB2 Developers Edition v8.1 (5 users) </li></ul></ul><ul><ul><li>IBM DB2 Warehouse Manager v8.1 </li></ul></ul><ul><li>Red Brick Analytic Bundle - SMB contains: </li></ul><ul><ul><li>IBM Red Brick Warehouse Server Work Group Edition v6.20 </li></ul></ul><ul><ul><li>DB2 WorkGroup Server Unlimited Edition WSE v8.1 </li></ul></ul><ul><ul><li>DB2 Universal Developers Edition v8.1 (One user only) </li></ul></ul>
  14. 14. DB2 Information Integrator V8.1 & Red Brick Warehouse Server
  15. 15. DB2 Information Integrator <ul><li>Introducing new integration software : DB2 Information Integrator </li></ul><ul><ul><li>DB2 Information Integrator 8.1 </li></ul></ul><ul><ul><li>DB2 Information Integrator for Content 8.2 </li></ul></ul><ul><li>Announcing beta availability </li></ul><ul><ul><li>Access beta through IBM client representative </li></ul></ul><ul><li>Key business value: </li></ul><ul><ul><li>Optimize IT investments given more choice in data access </li></ul></ul><ul><ul><li>Integrate data with better productivity and application efficiency </li></ul></ul><ul><ul><li>Gain more return from existing assets </li></ul></ul>
  16. 16. DB2 Information Integrator : Value for Red Brick <ul><li>Key business value: </li></ul><ul><ul><li>Optimize IT investments given more choice in data access </li></ul></ul><ul><ul><li>Integrate data with better productivity and application efficiency </li></ul></ul><ul><ul><li>Gain more return from existing assets </li></ul></ul><ul><li>Data Federation (leave your data where it is) </li></ul><ul><ul><li>Create new applications via DB2 that integrate data stored in Red Brick with data from other sources such as DB2, Oracle, Sybase, SQL Server, Teradata, XML files, Flat files, Message Queues, Web Services, ODBC databases… </li></ul></ul><ul><ul><li>Enable existing DB2 applications that use DB2/UDB APIs, e.g. ODBC/JDBC/ESQL to access data in Red Brick </li></ul></ul><ul><ul><ul><li>A single API for access to both Red Brick or Informix or DB2 or Oracle or… </li></ul></ul></ul>
  17. 17. Scenario: Integrate Red Brick with DB2 UDB <ul><li>Your Value </li></ul><ul><ul><li>Integrate Red Brick data marts seamlessly with DB2 UDB, Informix, Oracle or Flat files </li></ul></ul><ul><ul><li>Enable a unified view of data across multiple Red Brick marts combined with data in DB2 UDB </li></ul></ul><ul><li>What You can do </li></ul><ul><ul><li>Red Brick + DB2 II (Standard Edition) </li></ul></ul><ul><ul><ul><li>Includes a “limited use” license that allows you to federate Red Brick data marts with DB2 UDB and Informix databases (XPS and IDS) </li></ul></ul></ul><ul><ul><ul><li>Additional data sources, e.g. Oracle, require a full license </li></ul></ul></ul>
  18. 18. Scenario: Enable Third Party App Support <ul><li>Your Value </li></ul><ul><ul><li>Want to use packaged applications (e.g. SAS Campaign Manager) but application does not support Red Brick natively </li></ul></ul><ul><li>What You can do </li></ul><ul><ul><li>DB2 Information Integrator SE </li></ul></ul><ul><ul><li>Build the infrastructure to enable access of Red Brick data via DB2 II to the SAS Campaign Manager </li></ul></ul>
  19. 19. IBM Red Brick Warehouse Server 6.3 Features
  20. 20. 6.3 Features <ul><li>Major Performance Improvements </li></ul><ul><ul><li>Dynamic Smartscan </li></ul></ul><ul><ul><li>Memory mapping of dimension index/tables for STARjoin </li></ul></ul><ul><ul><li>TARGETjoin improvements for local index </li></ul></ul><ul><ul><li>Optimizer hints to specify STARindex for fact-to-fact STARjoin </li></ul></ul><ul><ul><li>Table Management Utility (TMU) Memory Tuning </li></ul></ul><ul><li>Major Usability Improvements </li></ul><ul><ul><li>Additional SQL/OLAP functions </li></ul></ul><ul><ul><li>Expression support in the Loader </li></ul></ul><ul><ul><li>XML Improvements in Loader </li></ul></ul><ul><ul><li>Compact System Catalog Utility </li></ul></ul>
  21. 21. 6.3 Features <ul><li>Major Usability Improvements – cont’d </li></ul><ul><ul><li>Allow 3GB Address Space on Windows Platform </li></ul></ul><ul><ul><li>Delimiter enhancements in Loading and Exporting </li></ul></ul><ul><ul><li>Alter Table with Working Segment </li></ul></ul><ul><ul><li>Interoperability with DB2 products </li></ul></ul><ul><li>System Port </li></ul><ul><ul><li>Support HP Itanium Product Family (IPF) </li></ul></ul><ul><li>OS Versions Upgrade </li></ul><ul><ul><li>AIX v5.2 </li></ul></ul><ul><ul><li>Sun Solaris 9 </li></ul></ul><ul><ul><li>HP-UX IPF 11i </li></ul></ul><ul><ul><li>Windows32 on Server 2003 </li></ul></ul>
  22. 22. Dynamic SmartScan <ul><li>Additional queries could be considered for SmartScan segment elimination </li></ul><ul><li>Include constraints not on the segment column and the fact table is segmented by the referenced foreign key </li></ul><ul><ul><li>Currently, could eliminate segments only with constraints on the segmenting column </li></ul></ul><ul><li>Dynamic segment elimination is possible: </li></ul><ul><ul><li>When we can evaluate the constraints in prelim plans on dimension-to-fact join, and </li></ul></ul><ul><ul><li>Strategizer chooses Table-Scan OR locally indexed Target-Join plan on the segmented fact table </li></ul></ul><ul><li>Improve selectivity estimates to consider outcome of segment elimination, both static and dynamic </li></ul><ul><ul><li>more accurate dynamic selection of STARjoin plan choices </li></ul></ul>
  23. 23. Dynamic Segment Elimination Example <ul><li>Assume, “Sales” as a Fact table that is segmented by a foreign key “perkey”. </li></ul><ul><li>RISQL> Select Sum(Dollars) From Sales, Period </li></ul><ul><ul><ul><li>Where Sales.perkey = Period.perkey And </li></ul></ul></ul><ul><ul><ul><li>Period.date >= ’01-01-01’ And </li></ul></ul></ul><ul><ul><ul><li>Period.date <= ’12-31-01’ ; </li></ul></ul></ul><ul><li>Above mentioned query could potentially eliminate unwanted segments from “Sales” table if it chooses Table Scan OR local target-join on “Sales”. </li></ul>
  24. 24. MMAP Dimension Table/Index <ul><li>Reduce CPU overhead and I/O system calls by mmap dimension indexes and tables into shared memory </li></ul><ul><li>Apply to STARjoin/TARGETjoin/tablescan plans </li></ul><ul><ul><li>Typically contain Btree-1-1-Match (B11M) and Functional Join operators to perform joins and row fetches to dimension tables </li></ul></ul><ul><li>High benefit for queries with large number of rows produced from join(s) below the B11M and Functional Join operators </li></ul><ul><ul><li>Mmap could potentially improve performance of </li></ul></ul><ul><ul><ul><li>B11M when mmap corresponding dimension primary index </li></ul></ul></ul><ul><ul><ul><li>Functional Join when mmap corresponding table </li></ul></ul></ul><ul><ul><ul><li>Multiple queries/users sharing the single copy in shared memory </li></ul></ul></ul>
  25. 25. TARGETjoin Performance <ul><li>Improve performance of TARGETjoin </li></ul><ul><ul><li>Particularly for local indexes </li></ul></ul><ul><ul><li>More consistent performance between tightly and loosely constraints </li></ul></ul><ul><ul><li>More efficient index access for TARGETjoin and Scan operators </li></ul></ul><ul><li>Local Index TARGETjoin improvement from 0 - 500% </li></ul><ul><ul><li>Biggest speedup on poorly performing joins </li></ul></ul><ul><ul><li>Particularly helps loose constraints on large dimensions </li></ul></ul><ul><ul><li>Preliminary test results approaching STARjoin performance in about 50% </li></ul></ul><ul><li>Allow single column B-Tree indexes in TARGETjoin </li></ul><ul><ul><li>Consider B-Tree index on foreign keys with very large dimensions </li></ul></ul><ul><ul><li>Not always a win, particularly with loosely constraints on large dimensions </li></ul></ul><ul><ul><li>Could be a big win with tight constraints </li></ul></ul>
  26. 26. Optimizer Hints <ul><li>STAR indexes can be specified for queries on a per table basis </li></ul><ul><ul><li>Must be careful when overriding optimizer selection </li></ul></ul><ul><li>A specific STAR index </li></ul><ul><li>SET STAR INDEX AVAILABILITY (TABLE1_STAR_IX1) FOR TABLE1; </li></ul><ul><li>Multiple STAR indexes </li></ul><ul><li>SET STAR INDEX AVAILABILITY (TABLE1_STAR_IX1, TABLE1_STAR_IX2) FOR TABLE1; </li></ul><ul><li>STAR indexes on a per multi-fact table STARjoin basis </li></ul><ul><li>SET STAR INDEX AVAILABILITY (TABLE1_STAR_IX1) FOR TABLE1 WHEN STARJOIN BETWEEN (TABLE1, TABLE2); </li></ul><ul><li>SET STAR INDEX AVAILABILITY (TABLE1_STAR_IX2) FOR TABLE1 WHEN STARJOIN BETWEEN (TABLE1, TABLE3); </li></ul>
  27. 27. Optimizer Hints <ul><li>STARjoin and TARGETjoin thresholds can be specified on a per table basis </li></ul><ul><li>SET STARJOIN THRESHOLD 20 FOR TABLE1; </li></ul><ul><li>SET STARJOIN THRESHOLD 5 FOR TABLE2; </li></ul><ul><li>SET TARGETJOIN THRESHOLD 40 FOR TABLE 1; </li></ul><ul><li>SET TARGETJOIN THRESHOLD 10 FOR TABLE2; </li></ul>
  28. 28. TMU Memory Tuning <ul><li>Better control over TMU memory resource usage </li></ul><ul><li>Allows TMU buffer memory to be tuned according to the load job </li></ul><ul><li>Introduces memory balancing between parallel loader tasks </li></ul><ul><li>Quickly allocate large amounts of buffer memory </li></ul><ul><li>Prevents excessive use of system memory by defining a maximum amount of buffer memory that could be used by the load job </li></ul><ul><li>Reports on TMU buffer usage: fine tuning for repetitive load jobs </li></ul>
  29. 29. TMU Memory Tuning <ul><li>Tuning rule of thumb: more logical I/O requires more buffers </li></ul><ul><li>Syntax: </li></ul><ul><ul><li>SET TMU MAX BUFFERS number_of_blocks </li></ul></ul><ul><ul><li>SET TMU CONVERSION BUFFER PERCENT p </li></ul></ul><ul><ul><li>SET TMU OUTPUT BUFFER PERCENT p </li></ul></ul><ul><ul><li>SET TMU INDEX BUFFER PERCENT p </li></ul></ul><ul><li>Recommend using new tunables over SET TMU BUFFERS approach </li></ul>
  30. 30. More SQL/OLAP Functions <ul><li>Distribution Functions </li></ul><ul><li>CUME_DIST </li></ul><ul><li>PERCENT_RANK </li></ul><ul><li>Inverse Distribution Functions (Median) </li></ul><ul><li>PERCENTILE_CONT </li></ul><ul><li>PERCENTILE_DISC </li></ul><ul><li>Scalar function </li></ul><ul><li>ROUND </li></ul>
  31. 31. Distribution Functions <ul><li>CUME_DIST() computes the position of specified row value relative to the set of values </li></ul><ul><ul><li>(# of values equal to or less than x) / (total # of values) </li></ul></ul><ul><li>PERCENT_RANK() returns the percent rank of a value relative to a group of values </li></ul><ul><ul><li>(rank of row in partition –1) / (# of rows in partition –1) </li></ul></ul><ul><li>Example </li></ul>0.75 0.33 220,000 1 1 230,000 0.75 0.33 220,000 0.25 0 100,000 Cume_dist Percent_rank Price
  32. 32. Inverse Distribution Functions <ul><li>Answers question such as “What is the median (50 th percentile) value of my data?” </li></ul><ul><li>Require a sort specification and a parameter that takes a value between 0 and 1 </li></ul><ul><li>Use the new WITHIN GROUP clause to specify the data ordering </li></ul><ul><li>Example </li></ul><ul><ul><li>Select Area, Price, </li></ul></ul><ul><ul><li>PERCENTILE_CONT(0.5) WITHIN GROUP (Order by Price) OVER (Partition by Area) as Median_cont, </li></ul></ul><ul><ul><li>PERCENTILE_DIST(0.5) WITHIN GROUP (Order by Price) OVER (Partition By Area) as Median_disc </li></ul></ul><ul><ul><li>From Homes; </li></ul></ul>
  33. 33. ROUND() scalar function <ul><li>ROUND() returns rounded number to the integer places left or right of the decimal point </li></ul><ul><li>Examples </li></ul><ul><ul><li>ROUND (864.827, 2) = 864.830 </li></ul></ul><ul><ul><li>ROUND (864.827, 1) = 864.800 </li></ul></ul><ul><ul><li>ROUND (864.827, 0) = 865.000 </li></ul></ul><ul><ul><li>ROUND (864.827, -1) = 860.000 </li></ul></ul><ul><ul><li>ROUND (864.827, -2) = 900.000 </li></ul></ul><ul><ul><li>ROUND (864.827, -3) = 1000.000 </li></ul></ul>
  34. 34. Expression Support in Loader <ul><li>Input data can now be modified while being loaded to a table </li></ul><ul><li>Basic arithmetic operations now supported </li></ul><ul><li>Modification also possible based on conditions </li></ul><ul><li>A pseudo column can now be assigned to a target column </li></ul><ul><li>Multiple conditions now possible in ACCEPT/REJECT clause with some limitations </li></ul><ul><li>Highly requested functionality </li></ul><ul><li>More integrated with the server than ETL tools </li></ul>
  35. 35. Expression Support in Loader <ul><li>Syntax: (snippets from TMU control file) </li></ul><ul><li>Arithmetic expressions: </li></ul><ul><li>$A POSITION(2) INTEGER EXTERNAL(10), </li></ul><ul><li>ColA ($A + 5)/2 </li></ul><ul><li>Conditions: </li></ul><ul><li>$A POSITION(2) INTEGER EXTERNAL(10), </li></ul><ul><li>ColB CASE WHEN $A > 5 THEN $A+3 </li></ul><ul><li> WHEN $A = 5 THEN $A-1 </li></ul><ul><li> ELSE $A+1 </li></ul>
  36. 36. Expression Support in Loader <ul><li>Pseudo column assignment </li></ul><ul><li>$A POSITION(2) INTEGER EXTERNAL(10), </li></ul><ul><li>ColC $A </li></ul><ul><li>ACCEPT/REJECT clause </li></ul><ul><li>ACCEPT ($A > 5 AND $B < 10) OR </li></ul><ul><li> ($C = 15) </li></ul><ul><li>Limitation: </li></ul><ul><li>If real columns are used, then just a single condition is allowed. With pseudo columns multiple conditions are allowed (example above) </li></ul>
  37. 37. XML Improvements in Loader <ul><li>Extends TMU and SQL Export functionality to provide additional XML support </li></ul><ul><ul><li>XML multiple namespaces support </li></ul></ul><ul><ul><li>Export generates default namespace </li></ul></ul><ul><li>Upgrade to the IBM XML4C v5.x parser </li></ul><ul><ul><li>key performance enhancements as well as critical fixes over 6.2 Xerces version </li></ul></ul><ul><ul><li>Seamless upgrade to new parser </li></ul></ul>
  38. 38. Compact System Catalog <ul><li>Compacts free space within system catalog </li></ul><ul><ul><li>Occurs when objects are freed but not at the end of the catalog </li></ul></ul><ul><li>Extension of System Catalog enhancement in 6.2 where free space is released at the end of the system catalog </li></ul><ul><li>Rb_syscompact </li></ul><ul><ul><li>Does the compaction </li></ul></ul><ul><ul><li>Requires DBA privilege </li></ul></ul><ul><ul><li>Creates a backup file </li></ul></ul><ul><ul><li>Checks for catalog sanity before compaction </li></ul></ul>
  39. 39. 3GB Address Space on Windows <ul><li>Extend beyond 32-bit memory access limit </li></ul><ul><li>Increase virtual address space from 2GB to 3GB </li></ul><ul><li>Feature Advantages </li></ul><ul><ul><li>More data can be cached in physical memory </li></ul></ul><ul><ul><li>Greater scalability and performance </li></ul></ul><ul><li>Supported on </li></ul><ul><ul><li>32-bit versions of the Windows® 2000 Advanced Server </li></ul></ul><ul><ul><li>32-bit versions of Windows.NET Server </li></ul></ul><ul><li>Enabled on executables: rb_tmu.exe, rb_ptmu.exe, rbw.exe, rbwtest.exe and risqltty.exe </li></ul>
  40. 40. Loading with Multiple Characters Separator <ul><li>Apply to loading and exporting in delimited format </li></ul><ul><li>new load format clause syntax: </li></ul><ul><ul><li>format separated by ‘ <separator> ’ [ enclosed by ‘<string delimiter>’ ] </li></ul></ul><ul><li>Separator may consist of 1 to 10 characters </li></ul><ul><ul><li>may be composed of single or multi-byte characters </li></ul></ul><ul><li>Feature Advantages: </li></ul><ul><ul><li>Data generated by other ETL tools that use multiple characters separator could be loaded directly without modification </li></ul></ul><ul><ul><li>Data containing separator string will be loaded correctly as long as the data is enclosed within the string delimiter </li></ul></ul>
  41. 41. Export with String Delimiter Support <ul><li>Adds delimiter support to enclose a string </li></ul><ul><li>New export command syntax: </li></ul><ul><li>export to ‘xxx’ format delimited [by ‘<export delimiter>’ ] [enclosed by ‘<string delimiter>’ ] (<select query>); </li></ul><ul><li>Export delimiter and string delimiter must be one character </li></ul><ul><ul><li>may be composed of single or multi-byte character </li></ul></ul><ul><li>Feature Advantages: </li></ul><ul><ul><li>May specify a different export delimiter for each export command </li></ul></ul><ul><ul><li>Export data may be directly loaded back into a database using the loader </li></ul></ul><ul><ul><li>Export delimiter can be part of the data content when string delimiter encloses the data </li></ul></ul>
  42. 42. ALTER TABLE Using a Working Segment <ul><li>Provides more reliable recoverability of failed alter operations than existing alter table IN_PLACE </li></ul><ul><li>Working segment can be reused after the alter operation is over </li></ul><ul><li>The table is still altered “in place” </li></ul><ul><li>Syntax: </li></ul><ul><ul><li>ALTER TABLE <table_name> [ADD | DROP] COLUMN </li></ul></ul><ul><ul><li>IN_PLACE [USING <segment_name>] </li></ul></ul><ul><li>New feature is strongly recommended over IN_PLACE alter </li></ul>
  43. 43. ALTER TABLE enhancements <ul><li>Much requested feature </li></ul><ul><li>Combines nominal space requirements of IN_PLACE alter with reliable recovery characteristic of alter in other segments </li></ul><ul><li>As a table segment is altered, its original contents are temporarily stored in a standard, user-defined segment (a ‘working segment’) </li></ul><ul><li>If the alter fails (e.g. due to a full disk), original contents of the table segment are available in the working segment for the alter to be resumed and completed successfully </li></ul><ul><li>Additional disk space required for the working segment is only as much as the largest segment of the table </li></ul>
  44. 44. HP-Itanium Porting Project <ul><li>A native port, not architectural emulation </li></ul><ul><ul><li>Yields high performance by directly taking advantage of Itanium’s architecture </li></ul></ul><ul><li>No need to convert data </li></ul><ul><ul><li>Red Brick databases created on PA-RISC will be fully compatible with Red Brick on HP-Itanium </li></ul></ul><ul><li>Currently, no vendors plan to support XBSA Backup/Restore interface on HP-Itanium </li></ul><ul><ul><li>BAR to files or UNIX tapes </li></ul></ul>
  45. 45. DB2 Products that support Red Brick currently <ul><li>DataJoiner Version 2.1.1 Works with ODBC wrapper. See http://www7b.boulder.ibm.com/dmdd/zones/informix/library/techarticle/0302rumsby/0302rumsby.html for more info on how to use DataJoiner with Red Brick </li></ul><ul><li>Warehouse Manager </li></ul><ul><ul><li>DB2 8.1 FP2 </li></ul></ul><ul><ul><li>See Warehouse Manager document for more information </li></ul></ul><ul><li>QMF for Windows (ODBC only) </li></ul><ul><li>Tivoli Storage Manager </li></ul>
  46. 46. Future DB2 Interoperability Options <ul><li>Information Integrator </li></ul><ul><ul><li>Planned for beta in late August </li></ul></ul><ul><ul><li>GA in November 2003 </li></ul></ul><ul><li>Intelligent Miner for Data </li></ul><ul><ul><li>Under consideration </li></ul></ul>
  47. 47. Features Being Considered Beyond v6.3
  48. 48. <ul><li>Features Being Considered </li></ul><ul><li>Windows-64 Support </li></ul><ul><li>Improved Query Parallelism </li></ul><ul><ul><li>Parallel Sort/Improved Order-By </li></ul></ul><ul><ul><li>Starjoin performance </li></ul></ul><ul><ul><li>Incremental Maintenance speedup for Vista </li></ul></ul><ul><li>Partition Parallel Loader </li></ul><ul><li>Backup/Restore Performance </li></ul><ul><ul><li>Compression </li></ul></ul><ul><ul><li>Pipeline parallelism </li></ul></ul><ul><li>Better Diagnostics </li></ul><ul><ul><li>Packaged diagnostics of memory, stack, version log dumps </li></ul></ul>
  49. 49. Red Brick Futures <ul><li>IBM committed to supporting Red Brick for the forseeable future </li></ul><ul><li>DB2 is the flagship DM product. </li></ul><ul><li>Red Brick features being integrated into DB2 over time. </li></ul><ul><li>Red Brick feature release approximately every 18 months </li></ul><ul><li>Channel/partners remains a key play for IBM </li></ul>Summary:
  50. 50. Questions ?

×