Your SlideShare is downloading. ×

v9.1.2 update

534

Published on

v9.1.2 update - Beate porst

v9.1.2 update - Beate porst

Published in: Data & Analytics, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
534
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. © 2013 IBM Corporation IBM InfoSphere Information Server v9.1.2 release update and roadmap Beate Porst (porst@us.ibm.com) InfoSphere Product Management
  • 2. © 2014 IBM Corporation IBM Information Server: Simplified Packaging for Information Integration and Quality Business Information Exchange Understanding & Collaboration • Information blueprints • Relationship discovery across data sources • IT-to-business mapping Data Quality Cleansing & Monitoring • Analysis & validation • Data cleansing • Data quality rules & management Data Integration Transformation • Massive scalability • Power for any complexity • Total traceability Delivery • Data capture at any time • Delivery anywhere • Big data readiness InfoSphere Information Server Enterprise Edition: Integrating and transforming data and content to deliver accurate, consistent, timely and complete information on a single platform unified by a common metadata layer
  • 3. © 2014 IBM Corporation Mapping Platform Components to Information Server Packages Platform Component Package Business Information Exchange Data Quality Data Integration Enterprise Edition Blueprinting and Best Practices (BPD) Governance Dashboard Data Discovery Metadata Management and Lineage (MWB) Logical and Physical Data Modeling (IDA) Business Glossary (BG) Data Cleansing and Enrichment (QS) Data Quality Validation & Monitoring (IA) Data Quality Exception Management (DQC) SOA Deployment (ISD) Data Specification Mapping (FT) Extraction, transformation, load (DS) Self-Service Data Integration (Data Click) Change Data Delivery (CDD)
  • 4. © 2014 IBM Corporation Information Server What’s new in v9.1.2
  • 5. © 2014 IBM Corporation TransformationProductivity Connectivity OperationsOverview Performance Admin 2010 2011 2012 2013 2014 Information Server Recent Activity 8.5 FP1 8.7 FP1 9.1 FP2 FP3 FP2 FP1 9.1.2 Data Integration Acceleration - Advanced transformation features (looping/v.pivot) - zOS File Stage - Integrated Balanced Optimizer capabilities Robust Enterprise Support - New Suite Installer - Active/Passive High Availabilitysupport - Source Code Control Integration Simple Data Quality - Standardization Quality Assessment - Match Specification Report - Match Designer Updates Stronger Governance - Operations Console - Business Glossary Workflow - Blueprint Task Management - Metadata Asset Manager Product Integration - Leverage Data Validation Rules in DataStage Jobs - Advanced Data Replication integration - Next Generation Netezza Connectivity & Optimization - HDFS Integration Advanced Admin & Productivity - Parallel Debugger - New Backup/restore tooling - Maintenance Mode - Stronger Encryption Agile integration - InfoSphere Data Click - Enhanced Workload Mgmt - ODM Integration - Hadoop Balanced Optimization - HDFS Extensions - InfoSphere Streams Integration Business Driven Governance - Policy and rules support for information governance - Web-based blueprints - Integrated metadata mgmt enhancements Sustainable Quality - Data Quality Console - Standardization Rules Designer - Data Rules Advancements - IDA 8.5 support Anywhere Integration - Big Data Features: * JSON support *BDFS REST API *JDBC connector - DB2 on z/OS load optimization - Data Click new data sources/targets Sustainable Quality - New QS standardization rulesets (Thailand , Ireland , update for India) - DQ Exception Mgt for DS/QS - Operational DQ Rules Business Driven Governance - Bulk metadata import - Governance Dashboard - IDA 8.5 support4
  • 6. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 5 Information Governance Dashboard What’s New in Information Server v9.1.2 Usage • Raises data confidence with visual governance status Value • Immediate insight into governance policy status • Interception of issues when they start, right at the source • Effectively measure results & compliance of policy enforcement 1000s of data points and policies visualized
  • 7. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 6 Support for Information Data Architect 8.5 • Builds on the new metabroker introduced at 9.1 for Information Data Architect which: • introduced better performance at lower resource cost • removed Windows only dependency • Certification of IDA v 8.5 added • Tolerance for orphaned and invalid objects (ability to ignore those that don’t impact rest of model) • Improved error/warning logging What’s New in Information Server v9.1.2
  • 8. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 7 Metadata Workbench Enhancements JDBC Connector support • Display details for JDBC Connector stage, including URL Definition, Schema, Table and SQL statements • Includes inclusion of JDBC (writes/reads) into data lineage flows XML/JSON Support • Browse, query and detail display for XML/JSON • Displays column level information within asset page • Can be linked via manual binding for lineage What’s New in Information Server v9.1.2
  • 9. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 8 Business Glossary Enhancements Integration with Data Rules • Data rule asset types (including unpublished rules) from InfoSphere Information Analyzer are now displayed in the Browse All Assets page, can be searched, assigned to terms, governance rules, business labels and data stewards • Drill down from a Governance Rule to a Data Rule to the Database column to which its applied What’s New in Information Server v9.1.2
  • 10. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 9 Business Glossary Enhancements Workflow Roles • Development Log now captures every history event including creation and reviewer comments • Security roles have been changed to provide a higher degree of granularity for existing roles: Author, Published and Reader • Two new workflow roles: • Reviewer: can review changes and make comments • Approver: can approve changes to a new or existing term (but no edit abilities themselves) • Can now add comments at every stage of the workflow process. Export Development Glossary • Can now export either development or published glossary What’s New in Information Server v9.1.2
  • 11. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 10 Data Quality Exception Management for DataStage and QualityStage What’s New in Information Server v9.1.2 Usage • Collect exception data from any Data Integration or Data Quality process • Support clerical review • Data Steward Dashboard Value • Promote consistency in the way data stewards and business analysts can investigate data issues • Insert good data quality controls and governance practices into each project • Support a variety of processing mechanisms at the point of greatest efficiency
  • 12. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 11 InfoSphere QualityStage Standardization Rules New Standardization Rules • Country specific rule sets for India, Ireland and Thailand • Provide for data standardization of names (individual and organizational), addresses, phone and locality (varies per country) • Delivered as archive files in the QSRules folder of the install directory • Client = ./InformationServer/Clients/Classic/QSRules • Server = ./InformationServer/Server/PXEngine/QSRules What’s New in Information Server v9.1.2
  • 13. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 12 JSON Document Support • Derive metadata format automatically from sample JSON documents… • Supports hierarchical formats with simple fields, objects and arrays • Schema views • New Parsing and Composing steps for provide for complex hierarchical data in JSON syntax; with value and structure validation options • Multiple options for reading/writing data: - files directly from disk - as part of a long string - passed in/out as a LOB What’s New in Information Server v9.1.2
  • 14. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 13 JDBC Connector • JDBC Connector provides Information Server products with access to JDBC data sources • Supports data read and write operations and metadata import operations • Certified in this release with Apache Derby and IBM Big Insights Big SQL drivers • Managed metadata import provided through new capabilities in InfoSphere Metadata Asset Manager (IMAM) • Filtering by asset type and name patterns What’s New in Information Server v9.1.2
  • 15. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 14 DB2 Z Bulk Load Optimization Huge Performance Gains For Load • Moved from a single load stream to parallel streaming via Z pipes. • Multiple LOAD utilities targeting separate partitions which performs faster than a single LOAD utility targeting all partitions. • 9.1.2 is 80 to 160% faster than 9.1.0 (depending on number of partitions.) • Performance scales almost linearly as you increase the number of partitions, regardless of load method. • Internal testing loading almost 1TB per hour using 16-way load Huge Performance Gains For Read • connector determines the number of partitions in the table and dynamically configures the number of DataStage nodes to match the number of partitions • Parallel read using the 9.1.2 DB2 connector is 40% faster than the 9.1.2 DB2Z stage, regardless of the number of partitions. Resilience • When Retry on connection failure is set to Yes the connector will try to establish an FTP connection again when the initial attempt to connect fails. What’s New in Information Server v9.1.2
  • 16. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 15 IBM CONFIDENTIAL What’s New in Information Server v9.1.2 Overview • Business users need quick and easy access to information to support their analytical projects. • Organizations need to avoid data sprawl, so governance best practices must be ensured • Originally only supported DB2 or Oracle to Netezza New in this release • Universal Connectivity via ODBC to now support DB2, Netezza, Oracle, Teradata, Sybase, SQL Server, Greenplum, and others…. as source or target • Automatic filtering of columns with data types not supported by the target data store • Leverages connector framework enhancement for data sampling via “row limits” http://www.youtube.com/watch?v=hUGGudh2iWI&feature=youtu.be InfoSphere Data Click - Self Service Data Integration
  • 17. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 16 Connector Enhancements Limit number of returned rows • New property to support database sampling(required for feature of Data Click) • Applies to the following Connectors: ODBC, DB2, Netezza, Oracle, Teradata and JDBC ODBC Connector expanded binary support • The ODBC Connector now supports automatically generated 'CREATE TABLE' statements for types Binary, VarBinary or LongVarBinary What’s New in Information Server v9.1.2 Name Label Description Default value LimitRows Limit number of returned rows. Select Yes to limit the number of rows that are returned by the connector. False (No) Limit Limit Enter the maximum number of rows that will be returned by the connector. 1000
  • 18. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 17 InfoSphere Metadata Asset Manager - Performance Optimization What’s New in Information Server v9.1.2 • Performance benefits of BI Simplification in 9.1 • 46% reduction in execution time of Express Import • Performance benefits of physical model import (Erwin) • 44-60% reduction in execution time of Express Import (Erwin) • IMAM Express Import in 9.1 is 7 - 1200% faster than in 8.7 for the following workloads • IDA: • Small workload (55K assets): +1200% (Throughput: 450 objects/s) • Large workload (119K – 430K): 9.1 succeeded (Throughput: 245 – 367 objects) whereas 8.7 failed due to Out of Memory • Erwin (124K assets): +50% (Throughput: 351 objects/s) • BO import (175K assets): +18% (Throughput: 318 objects/s) • DB2 PDR (108K assets): +7% (Throughput: 149 objects/s) • Cognos (141K assets): -20% (Throughput:120 objects/s) • MITI in 9.1 extracts more metadata (+45% reports &+27x more models, etc) than MITI in 8.7 Note: Performance results may vary in other environments
  • 19. © 2014 IBM Corporation Data IntegrationData Governance InfrastructureOverview Data Quality 18 Connectivity Accelerator What’s New in Information Server v9.1.2 • Growing number of pre-build Connectivity Sample • Cassandra • Hive • Hbase • MongoDB • Avro • Jaql • JMS • WTX • and more… https://www.ibm.com/developerworks/community/files/app?lang=en#/folder/4645e12a-7bdb-40ed-a103- f1160b707758?sort=collected
  • 20. © 2014 IBM Corporation Thank you

×