• Save
Adopt blu acceleration faster easier way
Upcoming SlideShare
Loading in...5
×
 

Adopt blu acceleration faster easier way

on

  • 907 views

IBM DB2 10.5 and BLU a new feature

IBM DB2 10.5 and BLU a new feature

Statistics

Views

Total Views
907
Views on SlideShare
707
Embed Views
200

Actions

Likes
1
Downloads
0
Comments
0

21 Embeds 200

http://feed.askmaclean.com 97
http://drsalbertspijkers.blogspot.com 43
http://drsalbertspijkers.blogspot.in 15
http://drsalbertspijkers.blogspot.nl 7
http://drsalbertspijkers.blogspot.fr 6
http://drsalbertspijkers.blogspot.com.br 4
http://drsalbertspijkers.blogspot.com.au 4
http://drsalbertspijkers.blogspot.sg 4
http://drsalbertspijkers.blogspot.co.il 3
http://drsalbertspijkers.blogspot.de 3
http://drsalbertspijkers.blogspot.kr 2
http://drsalbertspijkers.blogspot.co.uk 2
http://drsalbertspijkers.blogspot.ae 2
http://drsalbertspijkers.blogspot.hk 1
http://drsalbertspijkers.blogspot.jp 1
http://drsalbertspijkers.blogspot.se 1
http://drsalbertspijkers.blogspot.it 1
http://drsalbertspijkers.blogspot.be 1
http://cloud.feedly.com 1
http://drsalbertspijkers.blogspot.ru 1
http://drsalbertspijkers.blogspot.ca 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Adopt blu acceleration faster easier way Adopt blu acceleration faster easier way Presentation Transcript

  • © 2013 IBM Corporation 1 Adopt BLU Acceleration in DB2 10.5 The Faster, Easier Way July 25, 2013 Presented by: Daniel Zilio, Senior Optim Query Workload Tuner Developer , IBM Labs Marcia Miskimen, Integration QA Lead for Optim Data Studio Tools, IBM Labs
  • © 2013 IBM Corporation 2 Need webcast troubleshooting help? Click attachments Jessica Rockwood, Host IBM Warehouse developer and Performance Expert 1. The presentation for this Tech Talk: bit.ly/ttfilejuly13 2. Next steps and troubleshooting guide: click “Attachments“ in this webcast window A few details …. Daniel Zilio Senior Optim Query Workload Tuner Developer , IBM Labs Marcia Miskimen Integration QA Lead for Optim Data Studio Tools, IBM Labs Today’s technical presenters . . . DB2 Tech Talk series host and today‟s presenter: •Where’s Rick? •
  • © 2013 IBM Corporation 3 Need webcast troubleshooting help? Click attachments Disclaimer The information contained in this presentation is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided “as is”, without warranty of any kind, express or implied. In addition, this information is based on IBM‟s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other documentation. Nothing contained in this presentation is intended to, or shall have the effect of: • Creating any warranty or representation from IBM (or its affiliates or its or their suppliers and/or licensors); or • Altering the terms and conditions of the applicable license agreement governing the use of IBM software. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
  • © 2013 IBM Corporation Agenda  In this talk we assume you‟ve heard about BLU acceleration – New feature in DB2 LUW V10.5 for columnar tables – Super analytics, Super easy!  If you haven‟t heard about BLU , a VERY short review  How can you decide what to convert to BLU?  DB2 tooling supports your move to BLU acceleration - we‟ll talk about these today: – Data Studio – InfoSphere Optim Query Workload Tuner (OQWT) – InfoSphere Optim Performance Manager (OPM) – InfoSphere Optim Query Capture and Replay (OQCR)  Walk through a sample scenario  Now how can you try BLU and the tooling? – Some resources at end of this presentation and downloadable slides Need webcast troubleshooting help? Click attachments4
  • © 2013 IBM Corporation What is DB2 with BLU Acceleration?  New innovative technology for analytic queries • Columnar storage • New run-time engine with vector (aka SIMD) processing, deep multi-core optimizations and cache-aware memory management • “Active compression” - unique encoding for further storage reduction beyond DB2 10 levels, and run-time processing without decompression • Value : Order-of-magnitude benefits in … • Performance • Storage savings • Simplicity !  “Revolution by Evolution” • Built directly into the DB2 kernel • BLU tables coexist with traditional row tables, in same schema, tablespaces, bufferpools • Query any combination of BLU or row data • Memory-optimized (not “in-memory”) 5 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Moving to BLU - How to Get There?  Previously in DB2 LUW, tables were stored by rows: – Row Organization  New table organization is – Column Organization (BLU)  BLU implementation considerations: – Which tables to convert from to column organization to improve performance? – How do you convert a table from row to column organization? – How can you verify performance changes for some specific workload ? – How do you apply test methodology to ensure workload is same before and after BLU?  Next we‟ll look at a scenario using a fictional company wanting to try BLU in their shop Need webcast troubleshooting help? Click attachments6
  • © 2013 IBM Corporation Sample Scenario – Fictional Company Wants to Try BLU  Our fictional company has an application & DB they think might be good candidate for BLU.  They currently run on DB2 LUW V9.7 in production.  DB2 AESE V10.5 is installed in test, and they are ready for next steps  They want to carefully move tables to BLU, rather than the whole db at once  They‟ve devised a test plan for trying BLU – with these steps: 1. Run the existing V9.7 test system and capture its app behavior as the baseline • Using a “known good” state is always preferred 2. Run the app against 10.5 without BLU (iterate if necessary) 3. Determine candidate tables to convert to BLU 4. Convert the tables to columnar format 5. Run the app again, this time with the BLU tables in place 6. Review the performance after converting to BLU  DB2 AESE/AWSE includes additional tooling that can assist in execution of this test plan… Need webcast troubleshooting help? Click attachments7
  • © 2013 IBM Corporation Scenario – Fictional Company Wants to Try BLU - With Tooling  How can they use the tooling to execute the test plan?  Test plan steps: 1. Run the existing V9.7 test system and capture its app behavior as the baseline • Using a “known good” state is always preferred InfoSphere Optim Query Capture and Replay V1.1 • InfoSphere Optim Performance Manager V5.3 * 2. Run the app against 10.5 without BLU (iterate if necessary) • InfoSphere Optim Query Capture and Replay V1.1 • InfoSphere Optim Performance Manager V5.3 * 3. Determine candidate tables to convert to BLU • InfoSphere Optim Query Workload Tuner V4.1 * 4. Convert the tables to columnar format • InfoSphere Optim Query Workload Tuner V4.1 * • Data Studio V4.1 * 5. Run the app again, this time with the BLU tables in place. • InfoSphere Optim Query Capture and Replay V1.1 • InfoSphere Optim Performance Manager V5.3 * 6. Compare the results from before and after BLU • InfoSphere Optim Query Capture and Replay V1.1 • InfoSphere Optim Performance Manager V5.3 * * Included In DB2 AESE and AWSE Need webcast troubleshooting help? Click attachments8
  • © 2013 IBM Corporation InfoSphere Optim Query Capture and Replay (OQCR) Requirements Benefits Capture production workloads and replay them in testing environments • Minimize unexpected production problems • Shorten testing cycles • Develop more realistic database testing scenarios • Identify database problems sooner with validation reports and performance tuning • Use actual production workloads for testing rather than fabricated scenarios • Extend quality testing efforts to include the data layer Test Database Source DatabaseApplication InfoSphere Optim Query Capture and Replay Record Play Need webcast troubleshooting help? Click attachments9
  • © 2013 IBM Corporation10 Optim Performance Manager (OPM) Optim Performance Manager Server Monitor entire landscape of databases and clients Background operation User Interaction Performance Health History collection Configurable sampling Configurable details Data aggregation 15 minutes, 1 hour, 1 day Data pruning Configurable retention by aggregation Alerting Events, thresholds, health status, user scripts Notify via eMail & SNMP Administration  Monitoring configuration  User privileges Canned Reports  SQL analysis  Capacity planning  Database overview Interactive Dashboards Health summary Performance overview Detailed dashboards Real-time History DB2 Performance Repository Clients Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Optim Query Workload Tuner (OQWT) V4.1 Access Path Graph Visual Plan Hint Workload Control Center Query Report Index Advisor Query Advisor Statistical View Advisor Index Advisor OQWT User Interface Eclipse Statistics Advisor Access Path Advisor Formatter Annotation Query Based Tools and Advisors Workload Tools and Advisors Query AdvisorsQuery Tools DB2 LUW Statistics Advisor Test Index Candidates Compare Access Plans Collect Actuals Design Advisor Workload Access Plan Compare Access Plan Explorer Workload Test Candidate Indexes Workload Test Candidate Table Organization Table Organization Advisor new new 11 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Data Studio V4.1  Data Studio provides the tools that you need for: – developing database applications – administering databases – health and availability monitoring – run jobs – tuning queries.  IBM Data Studio consists of three components: – the Data Studio full client – the Data Studio administration client – and the Data Studio web console  Contains features for BLU Acceleration  http://pic.dhe.ibm.com/infocenter/dstudio/v4r1/index.jsp?topic=%2Fcom.ibm.datatools.ds.rel ease.doc%2Ftopics%2Fdsoverview.html  And it’s included with DB2 ! 12 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan – Scenario Review  Now that we‟ve introduced the tools, here‟s the test plan again:  Test plan steps: 1. Run the existing V9.7 test system and capture its app behavior as the baseline • Using a “known good” state is always preferred • InfoSphere Optim Query Capture and Replay V1.1 • InfoSphere Optim Performance Manager V5.3 * 2. Run the app against 10.5 without BLU (iterate if necessary) • InfoSphere Optim Query Capture and Replay V1.1 • InfoSphere Optim Performance Manager V5.3 * 3. Determine candidate tables to convert to BLU • InfoSphere Optim Query Workload Tuner V4.1 * 4. Convert the tables to columnar format • InfoSphere Optim Query Workload Tuner V4.1 * • Data Studio V4.1 * 5. Run the app again, this time with the BLU tables in place. • InfoSphere Optim Query Capture and Replay V1.1 • InfoSphere Optim Performance Manager V5.3 * 6. Compare the results from before and after BLU • InfoSphere Optim Query Capture and Replay V1.1 • InfoSphere Optim Performance Manager V5.3 * * Included In DB2 AESE and AWSE 13 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan Step 1: OQCR - Capturing the Workload on 9.7  Start capture (OQCR has simple-to-follow workflow) – Identify where to capture – Specify when to capture – can be „now‟ or some future time  OQCR Capture records all inbound local and remote database traffic – Filters are available ( what to capture) – RECOMMEND: Filter out the OPM traffic so it is not captured  Capture duration is specified, but can be stopped at any time 14 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation  A capture report provides aggregated workload information: – SQL count, unique SQL statements, transaction count, metrics, …  Good to check that right SQL was executed  Captured SQL can be exported as delimited text file to allow for custom analysis Test Plan Step 1: Reviewing the Captured Workload 15 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan Step 2: OQCR - Preparing the Replay  Transform workload: Map to replay database, map schema and user credentials  Transformed workload is replay-ready and can be replayed multiple times  Manually setup replay database 16 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan Step 2: OQCR - Replay Workload to 10.5 DB – ROW Organized  Replay the captured workload onto the 10.5 database  OQCR can replay a different rate than original (faster, slower)  Replay could be scheduled to run later, or run immediately  Replay progress is shown 17 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Sidebar - Will Your Workload Benefit From BLU? Probably:  Analytical workloads, data marts, etc.  Grouping, aggregation, range scans, joins  Queries touch only a subset of the columns in a table  Star Schema Probably not:  OLTP  Point access to 1 or few rows  Insert, Update, Delete of few rows per transaction  Queries touch many or all columns in a table  Heavy use of XML, Temporal, LOBs, etc. Need webcast troubleshooting help? Click attachments18 Understanding a workload‟s contents and performance are important OPM can be used to look at the detail and is a pathway to OQWT
  • © 2013 IBM Corporation Test Plan Step 2: OPM - Check Performance  OPM is monitoring the 10.5 database  Initial replays could be iterative process if you run into replay differences, or other issues  OPM‟s ability to slice-and-dice the view of perf data is valuable  Here we show the 20 slowest statements, filtered for certain statement text  We could drill into more detail for each statement (not shown here) 19 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan Step 2: OPM – Transfer Workload to OQWT for Analysis  Now you have identified and visualized your workload and its runtime actuals  You can transfer the statements as a set, directly from OPM into OQWT. – You could also export statements from OQCR and import into OQWT.  We‟re showing OPM-OQWT here: ** Query Workload Tuner is embedded in the Data Studio Client with license activation 20 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan Step 3: OQWT – Start Analysis Phase  Data Studio changes to Tuning perspective, and the Query Tuner Workflow Assistant  Save the statements as a named workload, which you can then tune  Runtime actuals collected by OPM are included when statements are sent to OQWT  Saved workloads can be re-tuned in the future to determine if new changes to the system require further performance tuning  Single query analysis and advisors can be invoked 21 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan Step 3: OQWT - Saving and Managing Workloads  Different workloads are saved under unique names  Workloads are just a set of SQL statements  The list of workloads are provided in the MANAGE tab  Users can invoke tuning features 22 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan Step 3: OQWT - Invoking the New Table Organization Advisor  For a workload defined in the MANAGE tab, select the advisor from the INVOKE view  Select “Table Organization” to execute the advisor  Outputs the result in the Review view 23 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan Step 3: OQWT - Review Summary  Once the advisor is executed, a summary of the recommendation is provided  Details can be found by selecting the Table Organization tab 24 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Estimated performance Improvement 47.5% Test Plan Step 3: OQWT - Review Details  Displays performance improvement  Lists all tables in the workload and ones that should be converted  Displays each table‟s current and recommended organization as well as other properties including: – The number of references of the table in the workload – Findings of why a table was or was not recommended, and any warnings if a conversion is made  Lists the statements in the workload affected by the table conversion Tables recommended for conversion Statements affected by recommended conversion 25 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan Step 3: OQWT - Review Details: Actions  Actions include: – Show DDL conversion script, Table filter, Test Candidate Column-Organized Tables, View Warning messages, Export into files 26 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan Step 3: OQWT - View Table Conversion Warnings  Warnings are provided when conversion will lose or change certain table properties  Warnings when converting a specific table include: – (Non-primary and non-unique) Indexes will be removed – ENFORCED RI or CHECK constraints will change to NOT ENFORCED – Data partitioning will be removed – Multi-dimensional clustering will be removed – MQTs will not exist for the converted table – Tablespace is changed  Reasons tables not recommended are provided – Table may not be eligible for conversion – Table may not improve performance – Table may not have statistics – No eligible tablespaces for BLU may exist 27 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation Test Plan Step 4: OQWT - Review Conversion DDL Script  Conversions made by using ADMIN_MOVE_TABLE  Tablespaces recommended are shown in the command  Users can save and execute the script later or in the SQL Editor to use the Job Scheduler 28 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation 29 Need webcast troubleshooting help? Click attachments Test Plan Step 4: OQWT - Conversion to BLU Acceleration (alternate)  Instead of using the DDL script from the WTOA advisor, another method can be used  In Data Studio 4.1 under database administration, select “conversion” feature  Conversion at the Table level: single/multi-selection  NOTE - Data Studio 4.1 has fully integrated BLU support throughout – – CREATE TABLE – LOAD – Properties – Configuration – Convert existing table – See resources section for more information
  • © 2013 IBM Corporation 30 Need webcast troubleshooting help? Click attachments Test Plan Step 4: OQWT - Conversion to BLU Acceleration (alternate)  Circular logging – The database won't be recoverable  Archive logging – An option to backup on migration will be proposed
  • © 2013 IBM Corporation Test Plan Step 4: OQWT Statistics Advising  Once the tables are converted, statistics should be collected  The conversion uses an API that copies row- organized statistics Need webcast troubleshooting help? Click attachments31
  • © 2013 IBM Corporation Test Plan Step 4: OQWT Improve Statistics Quality Using the Workload Statistics Advisor  Provides advice on: – Missing statistics – Conflicting statistics – Out-of-date statistics – Base tables and materialized query tables (MQTs)  Simplifies use – Provides Runstats statements – Run immediately or save View and run RUNSTATS recommendations Summary for each object used in the workload View details including distribution, column group and index statistics recommendations Need webcast troubleshooting help? Click attachments32
  • © 2013 IBM Corporation Test Plan Checkpoint: That‟s a lot of stuff to absorb – where are we in this scenario walkthrough?  STEP1 - Captured workload with OQCR  STEP2 – Replayed workload to a 10.5 row-organized database – Used OQCR and OPM to view the statement performance – Sent data to OQWT from OPM  STEP3 - Ran the OQWT Workload Table Organization Advisor  STEP4 - Applied advisor recommendations to convert some tables to column organized What‟s next:  STEP5 - Replay the original workload again, onto the DB2 V10.5 database with BLU tables  STEP6 - Review the performance after converting to BLU Need webcast troubleshooting help? Click attachments33
  • © 2013 IBM Corporation Test Plan Step 5: OQCR - Replay Workload to 10.5 DB – COLUMN Organized  Tables have been converted to COLUMN organization  Want to run the application workload again to see the change  Re-use the same workload previously captured with OQCR  Initiate the replay: Need webcast troubleshooting help? Click attachments34
  • © 2013 IBM Corporation Test Plan Step 6: OQCR - Replay Comparison Report Review – Post-BLU  When replay is finished, run report to compare the BLU performance to 9.7 baseline  Better than expected, worth doing more tests and moving forward with next workload! Need webcast troubleshooting help? Click attachments35
  • © 2013 IBM Corporation Test Plan Step 6: OPM - SQL Baseline Comparison Report  OQCR can show that a statement got better, or worse, but not why  Can use OPM predefined report to investigate – try SQL Baseline Comparison report  Compare two time intervals from the same database  Comparison available on many metrics – Avg. Elapsed time is good one  Comparison can show both improvements and regressions  Good place to start with more detailed investigations (Just one example here)   OPM reports on any SQL it finds in the reporting interval – the test workload SQL difference may not always bubble to the top, that‟s what was excluded here Need webcast troubleshooting help? Click attachments36
  • © 2013 IBM Corporation Test Plan Scenario Summary  The Fictional Company test plan has been executed successfully  The OQWT Table Organization Advisor (WTOA) suggested which tables should benefit from conversion to COLUMN org, based on evaluation of a set of SQLs, called a Workload  Having this advice was a help on the path to BLU acceleration  Using tools such as OQCR or OPM also facilitate the migration – Enables best test practices – known and repeatable workload – Analysis of performance deviations – OPM collects performance data along the way  The testers found some good results with BLU and plan to proceed with testing some more workloads and databases. Need webcast troubleshooting help? Click attachments37
  • © 2013 IBM Corporation 38 Need webcast troubleshooting help? Click attachments DB2 Tech Talk: Adopt BLU Acceleration The Faster, Easier Way Next Steps Roadmap Listen to other DB2 Tech Talks on BLU Acceleration • IDUG – DB2 Tech Channel: www.idug-db2.com • Intro to DB2 10.5, Berni Schiefer • BLU Deep Dive, Sam Lightstone Read the materials • developerWorks article: Optimize DB2 10.5 for LUW performance using InfoSphere OQWT with the DB2: ibm.co/1bNW4tD • Optim Tools Blog: Managing the Data Lifecycle ibm.co/13aWcds • developerWorks Article: What‟s new in DB2 10.5: ibm.co/17DNcBZ Leverage the Info Center • Main Data Studio Info Center: bit.ly/13b4NN7 • Click sections across top to access the tools, e.g. Click Tune for Optim Query Workload Tuner Access the Community Technical Forums • Data Studio: ibm.co/18DkGF6 • Optim Query Capture and Replay: ibm.co/1bi7V3H • Optim Query Workload Tuner: ibm.co/135INIb • Optim Performance Manager: ibm.co/16a1w3O Reference Call IBM to schedule a demo or learn more • 1 800 966-9875 (U.S) • 1-888-746-7426 (Canada) • 1800-425-3333 (India) • Or visit http://www.ibm.com/planetwide/ for contact information worldwide InfoSphere Optim page on ibm.com www.ibm.com/optim Optim Lifecycle Products on developerWorks ibm.co/RzWBqj SQL Tips Blog: www.sqltips4db2.com Step Three Step Two Step One Step Four
  • © 2013 IBM Corporation 39 Need webcast troubleshooting help? Click attachments Upcoming Tech Talks Don’t miss these in-depth DB2 10.1 feature talks! Dates and topics subject to change and modification. How to register : DB2 Tech Talks web site Next DB2 Tech Talk: SQL Refresher: What’s New in DB2 10.5, What’s new with the Standard, Tips and More! • August 29, 2013 at 12:30 PM • Rick Swagerman, DB2 Language Architect • Registration: bit.ly/tt2013aug IDUG DB2 Tech Conferences: Australia and Spain! • In-person conference sponsored by IDUG • Melbourne, Australia in September • Barcelona, Spain in October • Agenda and registration: www.idug.org, select events Information on Demand Conference • November 3 – 7, 2013 • Las Vegas, Nevada • Information: bit.ly/iod2013
  • © 2013 IBM Corporation DB2 10.5 with BLU Acceleration now electronically available! 40 Need webcast troubleshooting help? Click attachments http://ibm.co/db2download Download the free 90-day trial! • Test drive BLU Acceleration, as discussed in other Tech Talks • Try the new pureScale enhancements as discussed here • pureScale works on AIX and Linux 64 bit only • Supported RDMA network adapter required • Refer to the DB2 Info Center for pureScale requirements: http://bit.ly/DB2105Infocenter
  • © 2013 IBM Corporation DB2 Tech Talk 41 Need webcast troubleshooting help? Click attachments
  • © 2013 IBM Corporation 42 Need webcast troubleshooting help? Click attachments Thanks for attending! Please rate the session Presentation download: bit.ly/ttfilejuly13 click Attachments in this webcast environment