Accelerating Data Life Cycle ManagementWith Informix Warehouse Accelerator &Intel® Xeon® ProcessorsSession 3428Jantz Tran,...
Intel - Legal Disclaimers•   All products, computer systems, dates, and figures specified are preliminary based on current...
Intel - Legal Disclaimers Performance•   Performance tests and ratings are measured using specific computer systems and/or...
Call to Action:      • For more information on the content covered in this session,         o go to the Demo Room and see ...
Data Management Demo Room    DB2, Data Warehouse & Tools:             Stop by the Data Mgmt Demo Room and    Adaptive Comp...
Agenda• Overview of Informix Warehouse Accelerator.• Data Life Cycle scenarios and Performance• Using Informix high availa...
Overview of Informix Warehouse Accelerator.7                                         #ibmiod
Motivation• Data Warehouse query Performance without Perspiration• Consistent query performance without tuning efforts.• M...
Informix Ultimate Warehouse edition             IBM Smart Analytics                                                       ...
10     #ibmiod
Store_sales data mart11                        #ibmiod
Understanding Data Loading       From Informix to IWA12                            #ibmiod
Distributing data from IDS (Fact tables)               Fact Table                    IDS Stored Procedures            Data...
Distributing data from IDS (Dimension tables)                                           IDS Stored Procedure              ...
Data Transfer from Informix to IWA – First timeDesign the Data Mart        Deploy the Data Mart        LOAD the mart-- ISA...
Compression: Frequency Partitioning Trade Info (volume, product,                                                          ...
Data Life Cycle scenarios and Support17                                    #ibmiod
Case 1: Full refresh every timeDesign the Data Mart   Deploy the Data Mart   LOAD the mart               Ready-- ISAO Stud...
Time cyclic data management                     Partitioned fact table, partitioned by week        working window     week...
Time cyclic data management                         Partitioned fact table, partitioned by week                    working...
Case 2: Partition refresh: Updates to existing Partitions                                         IBM Smart AnalyticsStep ...
Time cyclic data management with IWA                  Partitioned fact table, partitioned by week              working win...
Case 3: Partition refresh: Time Cyclic data management                              IBM Smart Analytics Step 1. Create the...
dropPartMart() functiondropPartMart(accelerator name, mart name, table owner, table name, partition id or name)Execute fun...
loadPartMart() functionloadPartMart(accelerator name, mart name, table owner, table name, partition id or name)Execute fun...
create table cust(k1 int primary key, name varchar(32),   age int);                                                      -...
PerformanceWeb_sales data mart     • Fact table: web_sales with 4.1 billion rows     • Each partition has about 120 millio...
Keeping Informix and IWA data in sync  Fact Table: out of sync                                                            ...
Using Informix high availability to scale out29                                        #ibmiod
Informix Warehouse Accelerator – In 11.70.FC4                                                                           IB...
Background• Prior to 11.70.FC5, adding accelerator, create, deploy, load, enable, disable datamart,  accelerating queries ...
Informix Warehouse Accelerator – 11.70.FC5. MACH11 Support                                                                ...
Step 1: Install:• Informix and IWA are installed just like before.• Informix can be combination of standard, primary, SDS,...
Step 2: Configure• Informix and IWA are installed just like before.• Informix can be combination of standard, primary, SDS...
Step 2: Configure•The secondary servers should be updatablesecondary servers.•Set this in $ONCONFIG  UPDATABLE_SECONDARY 1...
Step 3: Connect• You can connect to IWA from Informix from any of the Informix servers using existing  method.      o Get ...
Step 3: Connect...continued• The name of the IWA will be used as the AQT site name in systables.sitename. So, it’s  import...
Step 4. Design, Validate and Deploy• The secondary servers should be updatable secondary servers.• Set this in $ONCONFIG  ...
IWA: Road so far…                                                                                    12.10: Now in Beta.  ...
Informix PublicationsBulletin of the Technical Committee on Data Engineering: March 2012Vol. 35 No. 1Real Time Business In...
Intel Inside® : Xeon® E7 Scaling41                                    #ibmiod
Intel® Xeon® E7-8870: One Terabyte Scaling     performance.• Hardware setup    o Intel® Xeon® E7-8870 processor – 4 socket...
Scaling in Westmere: Data Warehouse Setup.• TPC-DS Schema; web_sales           73,049                    66• Mart Data siz...
Scaling in Westmere: Results44                                    #ibmiod
Intel Inside® : Intel® Technology & Roadmap45                                    #ibmiod
INTEL/IWA: Breakthrough technologies for                          performance     7. Multi-core, multi-node environment   ...
Tick-Tock Development Model:     Sustained Microprocessor Leadership     Intel® Core™          ®     ™                Inte...
Intel Xeon Processor              ®              ®      Family for Business                                               ...
Intel® Xeon® Processor     E7-8800/4800/2800 Product Families     Building on Xeon® 7500 Leadership Capabilities     More ...
Intel® Xeon® 7500/E7 8 Socket Configuration           4+4 (8S)                        IBM® System                         ...
Advanced Reliability Starts With Silicon     Intel® Xeon® processor E7 family RAS Capabilities                    Memory  ...
Intel® Xeon® E5 and E7 Family Roadmap                                            2012                                     ...
Big Ridge* Introduction                Big Ridge Unleashes Platform and Application Performance with                      ...
IWA Resources     • IBM Informix Infocenter: http://ibm.co/fMcUDg     • Martin’s blog: http://ibm.co/Ts0cll     • Fred Ho’...
Upcoming SlideShare
Loading in …5
×

Informix IWA data life cycle mgmt & Performance on Intel.

549 views
468 views

Published on

IWA works with a snapshot of the data in Informix data mart. Once you have defined the data mart on IWA and loaded the data, you need to periodically refresh the data. You can choose to refresh all of the data or just the partitions that were added, dropped or modified. Whether you have hundreds of gigabytes or many terabytes, faster refresh will help you analyze recent data more rapidly and get closer to real-time business. This session will explain the options for data refresh, review their performance and explain how to correctly implement a refresh plan. IBM and Intel will demonstrate live data refresh on the Intel Xeon platform and examine the impact on performance.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
549
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • IBM IOD 2011 10/25/12 Prensenter name here.ppt
  • IBM IOD 2011 10/25/12 Prensenter name here.ppt 10/25/12 18:58
  • IBM IOD 2011 10/25/12 Prensenter name here.ppt 10/25/12 18:58
  • IWA uses a technique called Frequency Partitioning. In the diagram above, once sees that the table Trade Info contains columns Volume, Product, Orgin Country. Histograms are built for each column to determine frequency of data value occurrences, as shown with Origin and Product. Then the system looks for the most frequently occuring values in each of the columns, in the example, Top 64 Traded Goods. It then encodes those values with the least number of bits that can adequately represent the data (Approximate Huffman encoding) Idea being that most accessed values will require the least number of bits to be manipulated. Now, these values are then intersected with values in other columns, Top Traded Goods from China/USA. These encoded values are then placed in memory cells across all available memory in the system used for subsequest scan operations. The next slide shows an example of this and further encoding used for IWA.
  • Slide Purpose: Show full systems and use as chance to highlight the Energy Efficiency enhancements in Intel® Xeon® processor E7 family The Xeon E7 family is designed and built upon Intel’s 32nm Nehalem micro-architecture, which allows us to deliver 25% more cores and cache providing more performance within same maximum TDP as the Xeon 7500 series. It also supports 16 DIMMs per socket, which equates to 2TB of memory for the 4-socket E7-4800 product family – allowing for increased expandability. The Xeon E7 family features energy efficiency technologies including the Intel® Intelligent Power Technology (IPT) which is a shared technology from Intel’s Efficient Performance product line. IPT reduces partial active and idle power in the CPU and memory. Xeon E7 also supports lower power memory as well as memory buffers which support both standard and LV-DIMMs. The Xeon processor E7 family not only includes all of the reliability, availability and serviceability (RAS) features of the previous generation such as machine check architecture-recovery but also includes additional memory error correction features such as Enhanced DRAM Double Device Data Correction (DDDC) and Fine Grained Memory Mirroring. DDDC is an improved memory RAS feature which allows for a 2nd memory error & replacement of DIMMs w/o crashing . Fine Grained Memory Mirroring provides protection against uncorrectable memory errors that would otherwise result in a platform failure and allows for more flexible memory mirroring configurations (allows memory mirroring of just a critical portion of memory, leaving the rest of memory un-mirrored). This enables more cost-effective mirroring by mirroring just the critical portion of memory versus the entire memory space. New security features such as Intel® Advanced Encryption Standard New Instructions (AES-NI) and Intel® Trusted Execution Technology (TXT) are also supported. These advanced security features within the Xeon processor E7 family work to maintain data integrity, accelerate encrypted transactions, and maximize business continuity.
  • Intel Confidential
  • Informix IWA data life cycle mgmt & Performance on Intel.

    1. 1. Accelerating Data Life Cycle ManagementWith Informix Warehouse Accelerator &Intel® Xeon® ProcessorsSession 3428Jantz Tran, Intel®Keshava Murthy, IBM #ibmiod
    2. 2. Intel - Legal Disclaimers• All products, computer systems, dates, and figures specified are preliminary based on current expectations, and are subject to change without notice.• Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across different processor families. Go to: http://www.intel.com/products/processor_number• Intel, processors, chipsets, and desktop boards may contain design defects or errors known as errata, which may cause the product to deviate from published specifications. Current characterized errata are available on request.• Intel® Virtualization Technology requires a computer system with an enabled Intel® processor, BIOS, virtual machine monitor (VMM). Functionality, performance or other benefits will vary depending on hardware and software configurations. Software applications may not be compatible with all operating systems. Consult your PC manufacturer. For more information, visit http://www.intel.com/go/virtualization• No computer system can provide absolute security under all conditions. Intel® Trusted Execution Technology (Intel® TXT) requires a computer system with Intel® Virtualization Technology, an Intel TXT-enabled processor, chipset, BIOS, Authenticated Code Modules and an Intel TXT-compatible measured launched environment (MLE). Intel TXT also requires the system to contain a TPM v1.s. For more information, visit http://www.intel.com/technology/security• Requires a system with Intel® Turbo Boost Technology capability. Consult your PC manufacturer. Performance varies depending on hardware, software and system configuration. For more information, visit http://www.intel.com/technology/turboboost• Intel® AES-NI requires a computer system with an AES-NI enabled processor, as well as non-Intel software to execute the instructions in the correct sequence. AES-NI is available on select Intel® processors. For availability, consult your reseller or system manufacturer. For more information, see http://software.intel.com/en-us/articles/intel-advanced-encryption-standard-instructions-aes-ni/• Intel product is manufactured on a lead-free process. Lead is below 1000 PPM per EU RoHS directive (2002/95/EC, Annex A). No exemptions required• Halogen-free: Applies only to halogenated flame retardants and PVC in components. Halogens are below 900ppm bromine and 900ppm chlorine.• Intel, Intel Xeon, the Intel Xeon logo and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.• Copyright © 2012, Intel Corporation. All rights reserved. #ibmiod
    3. 3. Intel - Legal Disclaimers Performance• Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Intel products as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance. Buyers should consult other sources of information to evaluate the performance of systems or components they are considering purchasing. For more information on performance tests and on the performance of Intel products, Go to: http://www.intel.com/performance/resources/benchmark_limitations.htm.• Intel does not control or audit the design or implementation of third party benchmarks or Web sites referenced in this document. Intel encourages all of its customers to visit the referenced Web sites or others where similar performance benchmarks are reported and confirm whether the referenced benchmarks are accurate and reflect performance of systems available for purchase.• Relative performance is calculated by assigning a baseline value of 1.0 to one benchmark result, and then dividing the actual benchmark result for the baseline platform into each of the specific benchmark results of each of the other platforms, and assigning them a relative performance number that correlates with the performance improvements reported.• INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS”. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO THIS INFORMATION INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.• Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Intel products as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance. Buyers should consult other sources of information to evaluate the performance of systems or components they are considering purchasing. For more information on performance tests and on the performance of Intel products, reference www.intel.com/software/products. #ibmiod
    4. 4. Call to Action: • For more information on the content covered in this session, o go to the Demo Room and see demoes. o download this analyst paper, etc o go to this website, etc. • Stop by the Data Mgmt Demo Room and see deep dive demos on more than 25 products including DB2, Data Warehousing and Informix. The Data Mgmt Demo Room is located in the far back left corner of the Expo area. • Don’t miss this Special Analyst Session! In 2022, What Is a Database? Bloor, Forrester and IDC Analysts Discuss the Future- Philip Howard of Bloor, Noel Yuhanna of Forrester and Carl Olofson of IDC peer into the future of database software. Database software operates under many paradigms today, from relational and hierarchical to cloud to NoSQL and NewSQL to in-memory to columnar to the many aspects of Big Data and more. Where will this lead? Will databases be code-centric or data-centric? Dont miss this chance to hear where these veterans think we might be headed. (Session: IDB4230a, Wednesday, 12:00pm-1:15pm, South Pacific F)4 #ibmiod
    5. 5. Data Management Demo Room DB2, Data Warehouse & Tools: Stop by the Data Mgmt Demo Room and Adaptive Compression Continuous Data Ingest see deep dive demos on more than 25 Cubing and Dynamic Cubes products including DB2, Data Database Admin & Development Solutions Data Studio Warehousing & Informix. The Data Mgmt DB2 PureScale DB2 Merge Backup Demo Room is located in the far back left DB2 Recovery Expert corner of the Expo area. Ease of App. Migration HADR Multiple Standby InfoSphere Data Architect Informix: InfoSphere Federation Server Cloud solutions Multi-temperature Data Mgmt Flexible Grid Optim High Perf. Unload Genero Optim Performance Manager TimeSeries Optim Query Workload Tuner’ Informix clustering Optim Configuration Manager Informix Warehouse Accelerator Optim pureQuery Runtime OpenAdmin Tool Real-time Operational Warehousing PureSystems: Row & Column Access Control IBM Database Patterns Time Travel Query PureData System for Operational Analytics Workload Mgmt PureData System for Transactions Zig-Zag Join IBM Mobile Database5 #ibmiod
    6. 6. Agenda• Overview of Informix Warehouse Accelerator.• Data Life Cycle scenarios and Performance• Using Informix high availability to scale out• Intel Inside® : Intel® Technology & Roadmap o Scaling on Xeon® E7 Platform6 #ibmiod
    7. 7. Overview of Informix Warehouse Accelerator.7 #ibmiod
    8. 8. Motivation• Data Warehouse query Performance without Perspiration• Consistent query performance without tuning efforts.• More questions, faster answers, better data driven decisions & business insights• SKECHERS: Acceleration from 60x to 1400x – average acceleration of 450x8 #ibmiod
    9. 9. Informix Ultimate Warehouse edition IBM Smart Analytics StudioStep 1. Install, configure,start InformixStep 2. Install, configure, Step 3start Accelerator Step 1Step 3. Connect Studio toInformix & add accelerator Step 4 Informix Database ServerStep 4. Design, validate,Deploy Data mart Step 5Step 5. Load data toacceleratorReady for Queries BI Applications Step 2 Ready Informix warehouse Accelerator9 #ibmiod
    10. 10. 10 #ibmiod
    11. 11. Store_sales data mart11 #ibmiod
    12. 12. Understanding Data Loading From Informix to IWA12 #ibmiod
    13. 13. Distributing data from IDS (Fact tables) Fact Table IDS Stored Procedures Data Fragment Coordinator UNLOAD UNLOAD UNLOAD Process Data Fragment UNLOAD Data Fragment Data Fragment A copy of the IDS data is now transferred over to the Worker Copy process. The Worker process holds a subset of the data Worker Worker Worker (compressed) in main memory Process Process Process and is able to execute queries on this subset. The data is evenly distributed (no value based partitioning) across the CPUs. Compressed Data Compressed Data Compressed Data Compressed Data Compressed Data Compressed Data13 #ibmiod
    14. 14. Distributing data from IDS (Dimension tables) IDS Stored Procedure IDS Dimension Table Dimension Table Coordinator UNLOAD UNLOAD UNLOAD Process Dimension Table UNLOAD Dimension Table Worker Worker Worker All dimension tables are Process Process Process transferred to the worker process.14 #ibmiod
    15. 15. Data Transfer from Informix to IWA – First timeDesign the Data Mart Deploy the Data Mart LOAD the mart-- ISAO Studio Ready -- ISAO Studio -- ISAO Studio for Queries-- Workload Analysis -- Workload Analysis -- command line toolOptionally lock the table Informix Insert table data into external table Send data over to IWA Fact table – split into each worker Dim table – copy to each worker Compression frequency partitioning & encoding IWA Write the memory image to disk15 #ibmiod
    16. 16. Compression: Frequency Partitioning Trade Info (volume, product, Column Partitions origin country) Histogram Occurrences Number of Vol Prod Origin on Origin China GER, USA FRA, … Rest Common Rare Values values Origin Top 64 traded goods Cell Cell Cell 4 – 6 bit code 1 3 Product Cell Cell Cell 6 Rest 2 5 Histogram on Product Table partitioned into Cells • Field lengths vary between cells • Higher Frequencies  Shorter Codes (Approximate Huffman) • Field lengths fixed within cells16 #ibmiod
    17. 17. Data Life Cycle scenarios and Support17 #ibmiod
    18. 18. Case 1: Full refresh every timeDesign the Data Mart Deploy the Data Mart LOAD the mart Ready-- ISAO Studio -- ISAO Studio -- ISAO Studio (enabled)-- Workload Analysis -- Workload Analysis -- command line tool for Queries Disable the mart• During the data load, queries cannot be accelerated.• Work around: Create additional, duplicate mart.• Informix always picks up the data mart that was last loaded.18 #ibmiod
    19. 19. Time cyclic data management Partitioned fact table, partitioned by week working window week1 week2 week319 #ibmiod
    20. 20. Time cyclic data management Partitioned fact table, partitioned by week working window week1 week2 week3 week4 DETACH ATTACH week1 week4 partition partition20 #ibmiod
    21. 21. Case 2: Partition refresh: Updates to existing Partitions IBM Smart AnalyticsStep 1. Create the Sales-Mart Studio or storedand load it. Sales is the fact partitioned fact table procedures ortable -- range partitioned. command line toolStep 2. Load jobsupdate the fact table “sales” Step 1Only updates existing partition customer Step 2 salesStep 3. Identify the partition,execute dropPartMart(). Modified partition SQL Script: call Stored procedureStep 4. for same partition,execute loadPartMart(). stores Step 3 Step 4 Informix Database ServerReady for Queries INSERT, UPDATE, DELETE BI Applications IWA Sales-Mart Ready OLTP Apps 21 #ibmiod
    22. 22. Time cyclic data management with IWA Partitioned fact table, partitioned by week working window week1 week2 week3 week4 week6 DETACH ATTACH partition partition1. Execute dropPartMart()on IWA2. DETACH partition from the table a. ATTACH the partition b. Execute loadPartMart() on IWA22 #ibmiod
    23. 23. Case 3: Partition refresh: Time Cyclic data management IBM Smart Analytics Step 1. Create the Sales-Mart Studio or stored and load it. Sales is the fact partitioned fact table procedures or table -- range partitioned. command line tool Need to move the Time window to next range. Step 1 customerep 2. DETACH operation sales Execute dropPartMart() Move the window. DETACH the partitionep 3. ATTACH operationATTACH the partition storesExecute loadPartMart() Step 2 Step 3 Informix Database Server Ready for Queries BI Applications IWA Sales-Mart Ready OLTP Apps 23 #ibmiod
    24. 24. dropPartMart() functiondropPartMart(accelerator name, mart name, table owner, table name, partition id or name)Execute function dropPartMart(‘iwa’, ‘salesmart’, ‘dsusr’, ‘sales_fact’, ‘july_partition’);1. Uses the accelerator name, datamart name, table name and partition name.2. Partition name can be the name of the partition or partition number (sysfragments.partn) The partition name or number should be a valid partition for the table.3. Call dropPartMart() first before doing the DEATCH4. In IWA, the data is removed row by row; when all the rows in a block is freed, the block and memory is freed.5. No compression dictionary update on IWA.24 #ibmiod
    25. 25. loadPartMart() functionloadPartMart(accelerator name, mart name, table owner, table name, partition id or name)Execute function loadPartMart(‘iwa’, ‘salesmart’, ‘dsusr’, ‘sales_fact’, ‘july_partition’);1. Uses the accelerator name, datamart name, table name and partition name.2. Partition name can be the name of the partition or partition number (sysfragments.partn). The partition name or number should be a valid partition for the table.3. The partition should be in the table before loading to IWA. ATTACH the partition first, before calling loadPartMart().4. During every load, row is prefixed with the rowid and sent over to IWA.5. The data is encoded with existing compression dictionary and25 hence very fast. #ibmiod
    26. 26. create table cust(k1 int primary key, name varchar(32), age int); -- Simply update the data for existing partitionscreate table store(k2 int primary key, sname varchar(18)); execute function dropPartMart(my_acc, salesmart, o1, salesfact, p3);create table salesfact(sk1_cust int, sk2_store int, id int, val decimal(9,2)) partition by expression execute function loadPartMart(my_acc, salesmart, o1, salesfact, p3); partition p1 (id = 1) in rootdbs, partition p2 (id = 2) in rootdbs, -- drop partition from the table partition p3 (id = 3) in rootdbs; execute function dropPartMart(my_acc, salesmart, o1, salesfact, p1);insert into cust values(1, "John Smith", 32); alter fragment on table salesfact detach p1 salesfact_p1;insert into cust values(2, "Joe Smith", 32);insert into cust values(3, "John Doe", 32); -- attach a partition from the table. create table p4(sk1_cust int, sk2_store int, id int, val decimal(9,2), check (id = 4));insert into store values(101, "San Jose");insert into store values(102, "New Delhi"); alter fragment on table salesfact attach p4 as (id = 4) after p3;insert into store values(103, "Munich"); execute function loadPartMart(my_acc, salesmart, o1, salesfact, p4);insert into salesfact values(1, 101, 1, 20.22);insert into salesfact values(1, 101, 3, 80.24);insert into salesfact values(1, 102, 2, 34.34);insert into salesfact values(1, 103, 1, 23.28);insert into salesfact values(1, 101, 1, 20.22);insert into salesfact values(1, 102, 3, 80.24);insert into salesfact values(1, 103, 2, 34.34); 26insert into salesfact values(1, 101, 1, 23.28); #ibmiod
    27. 27. PerformanceWeb_sales data mart • Fact table: web_sales with 4.1 billion rows • Each partition has about 120 million rows• Full Refresh: 3 hours 29 minutes: 287 GB/hour• Dropping a partition from IWA: 46 seconds• Loading a partition from IWA: 115 secondsTotal operation time on IWA is less than 3 minutes.27 #ibmiod
    28. 28. Keeping Informix and IWA data in sync Fact Table: out of sync Fact Table: out of sync Dimensions: in-sync Dimensions: in-sync On IWA, drop and On IWA, drop and Reload fact partitions Reload fact partitions Load transactionsto existing partitions Attach new partition, in Fact table Fact Table: in-sync Dimensions: in-sync On IWA, drop and First, detach the partition on IWA Reload dimension partitionsThen detach the partition on Informix Fact Table: sync Update customer info, Dimensions: out of sync Intend to detach Store info, etc Fact Table: In-sync Full Reload of the data mart can be done any time. Dimensions: in-sync 28 #ibmiod
    29. 29. Using Informix high availability to scale out29 #ibmiod
    30. 30. Informix Warehouse Accelerator – In 11.70.FC4 IBM Smart AnalyticsStep 1. Install, configure, Studiostart InformixStep 2. Install, configure, Step 3start Accelerator Step 1Step 3. Connect Studio toInformix & add accelerator Step 4 Informix Database ServerStep 4. Design, validate,Deploy Data mart Step 5Step 5. Load data toacceleratorReady for Queries BI Applications Step 2 Ready Informix warehouse Accelerator3030 #ibmiod
    31. 31. Background• Prior to 11.70.FC5, adding accelerator, create, deploy, load, enable, disable datamart, accelerating queries – are all operations officially supported only on Standard server or Primary node of MACH11/HA environment.• We estimate about 50% of Informix customers use HDR secondary servers and growing number of customers use MACH11 (SDS secondary) configurations and RSS nodes. MACH11 is the Informix scale out solution.• IWA itself supports a scale out solution (on a cluster) starting with 11.70.FC4.• Reasons to support MACH11 and IWA together. o This feature will enable partitioning a cluster or HA group between OLTP and BI workload. o This feature will give help to off-load the expensive LOAD functionality to secondary servers o We have customers now requesting support for HDR secondary to IWA31 #ibmiod
    32. 32. Informix Warehouse Accelerator – 11.70.FC5. MACH11 Support IBM Smart AnalyticsStep 1. Install, configure, Studiostart InformixStep 2. Install, configure, Step 3 Informixstart Accelerator Informix Step 1 Informix HDR Informix RSS SecondaryStep 3. Connect Studio to SDS1Informix & add accelerator Informix Primary SDS2 Step 4Step 4. Design, validate,Deploy Data mart fromPrimary, SDS, HDR, RSS Step 5Step 5. Add IWA to sqlhostsLoad data toAccelerator from any node.Ready for Queries BI Applications Step 2 Ready Informix warehouse Accelerator3232 #ibmiod
    33. 33. Step 1: Install:• Informix and IWA are installed just like before.• Informix can be combination of standard, primary, SDS, HDR secondary and RSS nodes.• IWA can be installed on the same computer as any one of the nodes or on distinct computer.• IWA can also be installed on a cluster hardware with multiple worker nodes for scale out performance.33 #ibmiod
    34. 34. Step 2: Configure• Informix and IWA are installed just like before.• Informix can be combination of standard, primary, SDS, HDR secondary and RSS nodes.• IWA can be installed on the same computer as any one of the nodes or on distinct computer.• IWA can also be installed on a cluster hardware with multiple worker nodes for scale out performance.• Note: Informix MACH11 technology works with logged and ANSI databases only.34 #ibmiod
    35. 35. Step 2: Configure•The secondary servers should be updatablesecondary servers.•Set this in $ONCONFIG UPDATABLE_SECONDARY 1035 #ibmiod
    36. 36. Step 3: Connect• You can connect to IWA from Informix from any of the Informix servers using existing method. o Get the connection details via: # ondwa getpin o The output will be, ip address, port, pin for IWA connection. o Use that information to create the connection.• After successful connection from Informix to IWA, the SQLHOSTS will have something like thisFAST group - - c=1,a=484224232041684420473a283e612f74393e6025757159506a51344a6b4e2f2d2d47455e6b653f2f6c795f287d7b65 224d6c3c2f65722e6a2a4245397b3b447d572c3129696b306440FAST_1 dwsoctcp 162.34.42.188 21022 g=FAST• To use this connection on any of Informix nodes, copy these lines AS IS to the SQLHOSTS file of those servers.• Make sure copy ALL the lines within the FAST group.36 #ibmiod
    37. 37. Step 3: Connect...continued• The name of the IWA will be used as the AQT site name in systables.sitename. So, it’s important to have the right site name in SQLHOSTS entry for a successful connection.• Changing ANY of the details of this SQLHOSTS entry will result in connection, query matching and acceleration issues.37 #ibmiod
    38. 38. Step 4. Design, Validate and Deploy• The secondary servers should be updatable secondary servers.• Set this in $ONCONFIG UPDATABLE_SECONDARY 10• The Design, validate and deploy would be identical.• The partition refresh via dropPartMart() and loadPartMart() can be invoked from any node of the MACH11 cluster.38 #ibmiod
    39. 39. IWA: Road so far… 12.10: Now in Beta. Sign up! 2012 IOD 2012 IIUG 11.7xC6 Partition refresh 11.7xC5 (Dimension tables) Support TimeSeries 11.7xC4 Partition Refresh (fact) 11.7xC3 MACH11 support 11.7xC2 Solaris on Intel SMB: IGWE Scale out: IWA Workload Analysis Tool on Blade Server IWA 1 Release st More Locales On SMP Data Currency39 39 #ibmiod
    40. 40. Informix PublicationsBulletin of the Technical Committee on Data Engineering: March 2012Vol. 35 No. 1Real Time Business Intelligence. September 2, 2011 - Seattle, United States2012 Bloor Report: IBM Informix in hybrid workloadenvironments2012 Ovum Analyst report: Informix Accelerates Analytic Integrationinto OLTP http://youtu.be/xJd8M-fbMI0IBM Data management Magazine: Superchargingthedata wharehouse while keeping the costs down.DBTA Article: Empowering Business Analysts with Faster Insights 40 #ibmiod
    41. 41. Intel Inside® : Xeon® E7 Scaling41 #ibmiod
    42. 42. Intel® Xeon® E7-8870: One Terabyte Scaling performance.• Hardware setup o Intel® Xeon® E7-8870 processor – 4 socket (40C/80T) and 8 socket (80C/160T) configurations • 2.4 GHz, 30MB last level shared cache o 10 TB storage o 2 TB RAM• Software Setup o Informix and Informix Warehouse Accelerator: v11.70.FC5 o Both Informix and IWA on the same machine.42 #ibmiod
    43. 43. Scaling in Westmere: Data Warehouse Setup.• TPC-DS Schema; web_sales 73,049 66• Mart Data size: 1 terabytes 22• web_sales, 4.1 billion rows 360,000 o Fact with 34 partitions 86,400 4.1 billion• Dimensions: 13, non partitioned. 1,800 3600 20 1.9 million 15 million 7,200 66 30 million43 #ibmiod
    44. 44. Scaling in Westmere: Results44 #ibmiod
    45. 45. Intel Inside® : Intel® Technology & Roadmap45 #ibmiod
    46. 46. INTEL/IWA: Breakthrough technologies for performance 7. Multi-core, multi-node environment 1. Large memory support Nehalem has 8 cores and Westmere 10 cores. This trend is 64-bit computing; System X with MAX5 supports up expected to continue. IWA: Parallelize the scan, join, group to 6TB on a single SMP box; Up to 640GB on each operations. Keep copies of dimensions to avoid cross-node node of blade center. IWA: Compress large dataset synchronization. and keep it in memory; totally avoid IO. 6. Single Instruction Multiple Data Specialized instructions for manipulating 2. Large on-chip Cache 128-bit data simultaneously. IWA: L1 cache 64KB per core, L2 cache is 256KB per Compresses the data into deep columnar 7 1 core and L3 cache is about 4-12 MB. fashion optimized to exploit SIMD. Used in Additional Translation lookaside buffer (TLB). parallel predicate evaluation in scans. 6 2 IWA: New algorithms to avoid pipeline flushing and cache hash tables in L2/L3 cache 5 35. Hyperthreading 4 3. Frequency Partitioning2x logical processors; increases processor IWA: Enabler for the effective parallel accessthroughput and overall performance of threaded of the compressed data for scanning.software. IWA: Does not exploit this since the Horizontal and Vertical Partition Elimination.software is written to avoid pipeline flushing. 4. Virtualization Performance Lower overhead: Core micro-architecture enhancements, EPT, VPID, and End-to-End HW assist IWA: Helps informix and IWA to seemlessly run and perform in virtualized environment.4646 #ibmiod
    47. 47. Tick-Tock Development Model: Sustained Microprocessor Leadership Intel® Core™ ® ™ Intel® Microarchitecture Intel® Microarchitecture Intel® Microarchitecture Microarchitecture Codename Nehalem Codename Sandy Codename Haswell Bridge Merom Penryn Nehalem Westmere Sandy Ivy Haswell Future 65nm 45nm 45nm 32nm Bridge 32nm Bridge 22nm 22nm 14nm New New New New New New New New Micro- Process Micro- Process Micro- Process Micro- Process architecture Technology architecture Technology architecture Technology architecture Technology TOCK TICK TOCK TICK TOCK TICK TOCK TICK47 #ibmiod
    48. 48. Intel Xeon Processor ® ® Family for Business Scalable Enterprise Mainstream Top-of-the-line performance, Enterprise scalability, and reliability Best combination of performance, power efficiency, and cost Mission Critical Small Business Enterprise Server Performance and reliability for the most business critical workloads with outstanding Versatility for infrastructure apps (up to 4S) economics Economical and more Cloud Computing Cloud Computing dependable vs. desktop Efficient, secure, and open platforms for Highest virtualization density and advanced Internet datacenters and IAAS reliability for private cloud Entry Servers and High Performance Computing & High Performance Computing Workstations Workstations More features and performance than Bandwidth-optimized for high Greater scaling and memory capacity traditional desktop systems performance analytics & visualization Increasing capability 4848 #ibmiod
    49. 49. Intel® Xeon® Processor E7-8800/4800/2800 Product Families Building on Xeon® 7500 Leadership Capabilities More Performance More Expandable • 10 cores / 20 threads • Supports 32GB DDR3 DIMMs (2TB per 4- socket system)1 • 30MB of last level cache More Security & RAS More Efficient E7-4800 E7-4800 SECURITY • More performance within same • Intel® Advanced Encryption max CPU TDP as Xeon 7500 E7-4800 E7-4800 • Lower partial active & idle power Standard-New Instructions • Intel® Trusted Execution via Intel Intelligent Power Technology2 Technology (TXT) • Support for Low Voltage-DIMMs3 RELIABILITY, AVAILABILITY, SERVICEABILITY • Reduced power memory buffers4 • Enhanced DRAM Double Device Data Correction • Fine Grained Memory Mirroring Delivers more Performance, Expandability and RAS Delivers more Performance, while improving Energy Efficiency while improving Energy Efficiency 1. Up to 64 slots per standard 4 socket system x 32GB/DIMM = 2TB 2. Uses similar core and package C6 power states enabled on Intel Xeon 5500/5600 series processors. Requires OS support.49 3. 4. Savings dependent on workload and configuration. Memory buffer power savings of up to 1.3W active and 3W idle per buffer per Intel estimates. Slightly more savings when used with LV DIMMs #ibmiod
    50. 50. Intel® Xeon® 7500/E7 8 Socket Configuration 4+4 (8S) IBM® System x3850 X5 Up to 10 cores and 2.4 Ghz per CPU Support 8 socket mode by combining 2 systems via external QPI links Memory Configuration  4TB in 8 socket server  6TB in 8 socket + MAX5  Continued 1066MHz support50 #ibmiod
    51. 51. Advanced Reliability Starts With Silicon Intel® Xeon® processor E7 family RAS Capabilities Memory I/O Hub CPU/Socket • • Inter-socket Memory Mirroring Inter-socket Memory Mirroring •• Physical IOH Hot Add Physical IOH Hot Add • • Machine Check Architecture •Machine Check Architecture • Machine Check Architecture Machine Check Architecture • • Intel® ® Scalable Memory Intel Scalable Memory •• OS IOH On-lining* OS IOH On-lining* (MCA) recovery (MCA-R) (MCA) recovery (MCA-R) (MCA) recovery (MCA-R) (MCA) recovery (MCA-R) Interconnect (Intel® SMI) Lane Interconnect (Intel® SMI) Lane •• PCI-E Hot Plug PCI-E Hot Plug • • Corrected Machine Check Corrected Machine Check Failover Failover Interrupt (CMCI) Interrupt (CMCI) • • Intel® ® SMI Clock Fail Over Intel SMI Clock Fail Over • • Corrupt Data Containment Corrupt Data Containment • • Intel® ® SMIPacket Retry Intel SMI Packet Retry Mode Mode • • Memory Address Parity Memory Address Parity • • Viral Mode Viral Mode • • Failed DIMM Isolation Failed DIMM Isolation • • OS Assisted Processor Socket OS Assisted Processor Socket • • Memory Board Hot Add/Remove Memory Board Hot Add/Remove Migration* Migration* • • Dynamic Memory Migration* Dynamic Memory Migration* • • OS CPU on-lining ** OS CPU on-lining • • OS Memory On-lining ** OS Memory On-lining • • CPU Board Hot Add at QPI CPU Board Hot Add at QPI • • Recovery from Single DRAM Recovery from Single DRAM • • Electronically Isolated (Static) Electronically Isolated (Static) Device Failure (SDDC) plus Device Failure (SDDC) plus Partitioning Partitioning random bit error random bit error • • Single Core Disable for Fault Single Core Disable for Fault • • Memory Thermal Throttling Memory Thermal Throttling Resilient Boot Resilient Boot • • Demand and Patrol scrubbing Demand and Patrol scrubbing • • Fail Over from Single DRAM Fail Over from Single DRAM Intel® QuickPath Interconnect Device Failure (SDDC) Device Failure (SDDC) • • Intel QPI Packet Retry Intel QPI Packet Retry • • Enhanced DRAM Double Device Enhanced DRAM Double Device • • Intel QPI Protocol Protection via Intel QPI Protocol Protection via Data Correction Data Correction CRC (8bit or 16bit rolling) CRC (8bit or 16bit rolling) • • Fine Grained Memory Mirroring Fine Grained Memory Mirroring • • QPI Clock Fail Over QPI Clock Fail Over • • Memory DIMM and Rank Sparing Memory DIMM and Rank Sparing • • QPI Self-Healing QPI Self-Healing • • Intra-socket Memory Mirroring Intra-socket Memory Mirroring • • Mirrored Memory Board Hot Mirrored Memory Board Hot Add/Remove Add/Remove Advanced reliability features work to maintain data integrity reliability features51
    52. 52. Intel® Xeon® E5 and E7 Family Roadmap 2012 2013/Future Intel® Xeon® processor E7-8800/4800/2800 product families Expandable 2-8 sockets, up to 10C/20T per socket, up to 30MB shared cache, “Westmere” microarchitecture Future Intel® Micro- architecture Intel® Xeon® processor E5-4600 product family codename 4 sockets, up to 8C/16T per sockets, up to 20MB shared cache, “Sandy Bridge” microarchitecture Ivy Bridge 4S Efficient Performance Intel® Xeon® processor E5-2600 product family 2 sockets, up to 8C/16T per sockets, up to 20MB shared cache, “Sandy Bridge” microarchitecture 2S Efficient Performance5252 #ibmiod
    53. 53. Big Ridge* Introduction Big Ridge Unleashes Platform and Application Performance with Scalable, Predictable, Efficient I/O Performance Platform NVM Tier Storage Tier Memory Tier Compute Tier Big Ridge HDD SSD Intelligent Storage  Memory DRAM Extension Concurrently Supports NVM Direct Access and Fast Storage Usage Models Why Big Ridge: •NVM performance, cost/power create a significant inflection point for the platform/datacenter •CPU/Server performance has grown significantly, storage/memory has not kept pace •To unlock NVM potential, software optimization and new access methods are required •Big Ridge offers new levels of platform performance, power and overall TCO improvements •Architected for future NVM to further scale platform and application performance •We are building an extensive ecosystem of support from OEMs, ISVs and End Users * Big Ridge is Intel’s codename for its first generation of application optimized NVM technology, Safford Peak is Intel’s codename for its first product using this technology.53 #ibmiod
    54. 54. IWA Resources • IBM Informix Infocenter: http://ibm.co/fMcUDg • Martin’s blog: http://ibm.co/Ts0cll • Fred Ho’s blog: http://ibm.co/T9FaNy • Keshav’s blog: http://ibm.co/RQXExL Thank You54 #ibmiod

    ×