IBM Information Server 9.1 What's New - James LEBAS


Published on


Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

IBM Information Server 9.1 What's New - James LEBAS

  1. 1. IBM InfoSphere Information ServerWhat’s New in V9.1James LEBAS : Avant-Vente InfoSphere #ibmiod
  2. 2. Please noteIBM’s statements regarding its plans, directions, and intent are subject to changeor withdrawal without notice at IBM’s sole discretion.Information regarding potential future products is intended to outline our generalproduct direction and it should not be relied on in making a purchasing decision.The information mentioned regarding potential future products is not acommitment, promise, or legal obligation to deliver any material, code orfunctionality. Information about potential future products may not be incorporatedinto any contract. The development, release, and timing of any future features orfunctionality described for our products remains at our sole discretion.Performance is based on measurements and projections using standard IBMbenchmarks in a controlled environment. The actual throughput or performancethat any user will experience will vary depending upon many factors, includingconsiderations such as the amount of multiprogramming in the user’s job stream,the I/O configuration, the storage configuration, and the workload processed.Therefore, no assurance can be given that an individual user will achieve resultssimilar to those stated here. #ibmiod
  3. 3. Acknowledgements and Disclaimers: Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. © Copyright IBM Corporation 2011. All rights reserved. U.S. Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. IBM, the IBM logo,, InfoSphere, InfoSphere Business Glossary, InfoSphere Data Architect, InfoSphere Information Analyzer, InfoSphere Metadata Workbench, and InfoSphere QualityStage are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at Other company, product, or service names may be trademarks or service marks of others. #ibmiod
  4. 4. IBM is Investing to Lead #ibmiod
  5. 5. Across the Entire Information Supply Chain Transactional & Collaborative Analyze Business Analytics Applications Applications Integrate Content Master Big Data Data Manage Cubes Streams Data External Data Information Content Warehouses Sources Information Streaming Information Govern Governance Standards Security & Privacy Lifecycle Quality Gartner Magic Quadrant: “IBM is the only DBMS vendor that can offer an information architecture across the entire organization, covering information on all systems” #ibmiod
  6. 6. Information Server Release ReviewData Integration Features 2011 2012 2013 8.5 FP1 FP2 • New Suite Installer • Source Code Control Integration • Looping Transformer • Scalable XML • Vertical Pivot 8.7 FP1 • Connector Multilink Support • Distributed Transaction • Operations Console Stage • Big Data File Stage • zOS File Stage • Parallel Job Debugger • Connection Migration Tool • Job Log Designer • Audit Logging Integration • Balanced Optimization • Netezza Connector & • Data Masking Stage Balanced Optimization 9.1 • BluePrint Director • Real-time CDC Integration • ISA Lite • Data Rules Stage (with IA) • Designer & Engine Performance • Command Line Backup • Security Features • Metadata Asset Manager #ibmiod
  7. 7. New and exciting partner appsSeveral can be seen on the demo floor !!! iKnow Stream Quality Mgr Provides data integration automation, Enterprise DQM solution that provides a view monitoring and analysis features to enable into the quality of your data insuring the data trusted information in complex environments. is capable of meeting it’s intended purpose. IOD Booth #419 IOD Booth #G405 TCS Info ProcessWorks TWS for Apps 8.6 A tool for ETL Process: Automates and integrates - Automation DataStage jobs and sequencers - Standardization within the larger TWS environment for - Control maximum efficiency. - Monitoring IOD Booth #G512 Presentation in the http://www- Expo area Email Tues 6 to 6:30 pm ENUS212-178/ENUS212-178.PDF TestDrive iDirector complete testing framework to design Keep you test cases, create test data, specify expected informed of the status of your results, run test scripts, report test status and DataStage jobs and help you control manage all of the testing artifacts over time the runs of jobs remotely IOD Booth #321 #ibmiod
  8. 8. IBM InfoSphere Information Server 9.1 Agile Integration Wherever your integration resides, integrate it quickly and flexibly Business Driven Governance Make decisions with confidence using trusted data at the point of impact Sustainable Quality Ensure information accuracy and quickly adapt to strategic business changes #ibmiod
  9. 9. IBM InfoSphereBusiness Information Exchange 9.18 #ibmiod
  10. 10. New Information Governance Policies & Rules• Policies define areas of information governance• Information Governance Rules describe business expectations on information #ibmiod
  11. 11. Align business expectations (e.g. data quality)with IT implementation • Implemented By: Indicates the operational assets that implement this rule • Governs: Indicates the data sources should comply with this rule #ibmiod
  12. 12. Advanced Business Term Relationships• Significantly more advanced relationships: recursive and multi-hierarchy o is-a-type-of / has-types: o Has-a / of• Supports inheritance of Has relationships for subtypes• Supported in IDA, RSA, and other Eclipse #ibmiod
  13. 13. New: Publishing Blueprints Pro-active governance by defining, reviewing & managing the to-be architecture #ibmiod
  14. 14. And Many Others Enhancements • Common enhancements for Business Glossary & Metadata Workbench o Publishing blueprints to Metadata Server and viewing them in Business Glossary & Metadata Workbench o Single Sign-On • Business Glossary o Support for BPMN2 in RSA Eclipse • Metadata Workbench o Automated Services optimization o Additional DataStage Stages and Asset Types included in Lineage • Discovery o UNIX/LINUX and 64-bit support o Complete globalization13 #ibmiod
  15. 15. IBM InfoSphereInformation Server for Data Quality 9.114 #ibmiod
  16. 16. Key Data Quality Enhancements in 9.1 New Information Governance Rules & Extended Data Validation Rule Policies platform Impact Analysis support Data Validation Rule Sequencing define discover validate objectives cleanse & enrich monitor / track “master” New Standardization New Rules Data Quality Designer Console15 #ibmiod
  17. 17. Data Validation Rules Flexible Output Table Configuration, Sequencing & Impact Analysis • Flexible configuration of output tables for Data Validation Rules (naming, append/overwrite) • Registration & reuse of output tables • Sequencing of Data Validation Rules • Advanced web-based Data Validation Rule display incl. lineage and impact analysis16 #ibmiod
  18. 18. New Standardization Rules DesignerSimplifying & accelerating the speed of cleansing data Knowledge holders assess & looking at the data … define discover validate objectives cleanse & monitor / enrich track “master” what they want to see: what they will see in the new user interface #ibmiod
  19. 19. New Standardization Rules DesignerSimplifying & accelerating the speed of cleansing data #ibmiod
  20. 20. New Standardization Rules DesignerSimplifying & accelerating the speed of cleansing data #ibmiod
  21. 21. New Standardization Rules DesignerSimplifying & accelerating the speed of cleansing data #ibmiod
  22. 22. New Standardization Rules DesignerSimplifying & accelerating the speed of cleansing data #ibmiod
  23. 23. New Standardization Rules DesignerSimplifying & accelerating the speed of cleansing data #ibmiod
  24. 24. New Standardization Rules DesignerSimplifying & accelerating the speed of cleansing data #ibmiod
  25. 25. New Standardization Rules DesignerSimplifying & accelerating the speed of cleansing data #ibmiod
  26. 26. Key Data Quality Enhancements in 9.1 New Information Governance Rules & Extended Data Validation Rule Policies platform Impact Analysis support Data Validation Rule Sequencing define discover validate objectives cleanse & enrich monitor / track “master” New Standardization New Rules Data Quality Designer Console25 #ibmiod
  27. 27. New InfoSphere Data Quality ConsoleDashboard view displaying most critical information at a glance #ibmiod
  28. 28. New InfoSphere Data Quality ConsoleException summary display with advanced filtering options #ibmiod
  29. 29. IBM InfoSphereInformation Server for Data Integration 9.128 #ibmiod
  30. 30. Six Dimensions of Capability Investment Productivity Transformation Connectivity Rich user interface features Extensive set of pre-built Native access to common that simplify the design objects that act on data to industry databases and process and metadata satisfy both simple applications management & complex data exploiting key requirements integration tasks features of each Performance Operations Administration Runtime engine providing Simple management of the Intuitive and robust features unlimited scalability for all operational environment for installation, maintenance, integration patterns lending analytics configuration, batch/real-time, for understanding security and ETL/ELT/DV/SOA and investigation. resiliency #ibmiod
  31. 31. Information Server Home PageSingle starting place for access to Information Server Web UIs • Access new home page from your Start menu • Provides a central location for access to Information Server UIs #ibmiod
  32. 32. ProductivitySimplifying user experience to deliver business results faster XML Schema Views Focus on the business object New XML “schema views” narrow the scope of the schema tree to enhance the design experience and improve the efficiency of the parsing and composing processes. New features to achieve huge productivity gains. Rational Team Concert Integration Enterprise control of assets Extends the certification of our asset interchange integration with source code control systems to now include Rational Team Focus is on Concert “simplification” as a key theme for all portfolio investments! InfoSphere Data Click Load your data mart in 2 clicks New web-based experience allows novice users to move data in batch or real-time from databases or InfoSphere MDM via a few simple clicks with full set of governance and run-time optimization. #ibmiod
  33. 33. InfoSphere Data ClickAgility & Governance for Self Service Analytics  Built-in Governance: Define and control scope & architecture  Business Agility: get your data in just 2 clicks  Built-in Governance: Review and manage environment & compliance32 IBM CONFIDENTIAL #ibmiod
  34. 34. InfoSphere Data ClickSimple “two clicks” to data integration • Preselected sources and targets minimize user interaction • Checked tabs indicate that the required configuration is complete 1st Click • Review configuration 2nd Click Execution!!!33 #ibmiod
  35. 35. What did Data Click just do? •Running on the power of Information Server, Data Click Provides: Demo Floor Now Showing oAutomated creation of: –target database schema Data Click for –bulk extract and load (with DataStage) Netezza –real-time replication subscriptions (with CDC) And Managed workload for optimized performance of potentially 1000s of artifacts Data Click for Master Data Ensured governance around both data access as Management well as for metadata capture (to support data lineage and impact assessment) •Initial release support Oracle/DB2 sources and Netezza targets – others to follow #ibmiod
  36. 36. TransformationIncreased set of pre-built & extensibility features New Transformation Expressions Enhanced String Functions Includes the server functions for “ereplace” and “change” within Greatly simplifies the parallel transformer for better handling of string replacement. several complex use cases. IBM Operational Decision Mgmt Integration Support Dynamic Business Changes Allows the Direct integration with ODM (ILOG JRules) to provide unique rules management within the data integration framework. Where enterprise to business users need to directly control the logic, this combines further realize best of breed market capabilities to deliver the solution. the collaboration between business and IT Dynamic File Creation (seq & bdfs) directly within Drive job logic through data values the data New “Generate Multiple Files” property drives a dynamic nbr of integration layer output files based on data values, without needing multiple stages or recoding each time the output set number changes. when required. #ibmiod
  37. 37. Dynamic File Generation• Overview 1 • Create multiple files formed from a base name • Drive file roll over from keys value or file size • Include the key value as part of the file name • Combine with a sequential source of file pattern to create an output file for every input file • Avoids job updates when a new output file is required• Patterns to Generate Multiple Files 1. w/ Root File String (default) 2 2. w/ Root & Maximum File Size 3. w/ Root & Key Column 4. w/ Root & Key Column in File Name 5. w/ Root, Key Column in File Name & Exclude Part String 3 5 4 #ibmiod
  38. 38. ConnectivityReaching new sources of information and exploiting new options Java Integration Stage Second generation extensibility Offers simple user paradigm for integrating Java code and creates baseline for upcoming big data source support. MQ 7.1.x, 7.5.x Upcoming developerWorks articles/assets will accelerate how DB2 10.1.x customers can reach new NoSQL and other big data sources. Netezza 6.0.x, 6.1.x Oracle 11gR2 Teradata 13.10 &14.0 InfoSphere Streams Integration SQL Server 2012 Informix 11.7 Open up Real-time Big Data ODBC Datadirect 7.0.x Direct data flow integration between InfoSphere DataStage and BDFS Stage BigInsight InfoSphere Streams to combine the power and reach of both 1.4, Cloudera CH4.0 platforms Sybase IQ 15.4, ASE 15.7 JRules 7.1.x, 7.5.x, 8.0.x Native Excel Access InfoSphere Streams 3.0 Simple Worksheet Access New stage provides direct Excel read capabilities on all platforms with rich features to support ranges, multiple worksheets and more #ibmiod
  39. 39. Native Excel AccessNew Unstructured Stage provides…. • 3-Step Configuration – enter file name, select data range and setup column name mapping • Native Excel File Support – reads data from any supported OS (Windows, Linux, Unix) • XLS Version Variants - includes both Excel 97-2003 OLE2 (.xls) and 2007 OOXML (.xlsx) file format • Simplified Access - no need to define ODBC DSN for each excel file • Password Protected Files – read data from password encrypted files • Multiple File Support – ability to simultaneously process multiple files in different nodes • Multiple Sheet Support - extract data from multiple sheets at a time • Runtime Column Propagation – define column metadata • Excel Field Extraction – ability to extract row numbers, comments, hyperlinks, formulas, document Properties, etc... #ibmiod
  40. 40. Excel Access Filename Excel Column Header Row Number Sheet name HyperlinkDataStage Comment #ibmiod
  41. 41. PerformanceFaster….. Oh yeah!!! DBMS Connector Boost Deliver job results faster Performance optimizations including new “big buffer” optimizations which has increased bulk load performance in DB2 and Oracle Designer Client Connector by more than 30% in many cases. performance is further optimized Balanced Optimization for Hadoop in this release Harness the power of Hadoop with DataStage with gains of Allows a Balanced Optimizer customer to push levels of processing 60% or more for into Hadoop where locality of processing will boost performance. some most common Balanced Optimization for DB2 z/OS operations. Exploits the power of Z Extends the data locality/ELT paradigm to facilitate z/OS based New warehouses; aligns with and facilitates IBM IDAA strategy optimizations extend the lead of InfoSphere Streamlined Windows Processes DataStage Boosts performance and reduces overhead Move from MKS emulation to native process forking apis to increase as the fastest scalability on Windows and removing resource layers in that delivery. ETL platform #ibmiod
  42. 42. New Hadoop-based Big Data SupportBig Data Integration Common Use Cases Any to Big Data Any ETL “I need to mix in traditional sources into Hadoop so that I can run the analytical models I need.” Big Data to Any ETL Any “Now that I know something new, how do I move this back into my applications and warehouses so that it is easily consumable.” Big Data Hub ELT “I need to transform and cleanse information to make it (re)usable for analytics but can’t afford to move TBs across the network frequently.”41 #ibmiod
  43. 43. New Hadoop-based Big Data SupportIntegrating Extreme Velocity, Variety and Volumes• Continue Big Data investments within Information Server that began with Hadoop data file system (HDFS) integration• Programmatic invocation of Big Insights and other MapReduce jobs through the Job Sequencer, combining DataStage and Hadoop for end-to-end workflow.• Leverage the Designer User Interface and standard stage constructs to generate MapReduce jobs, giving users the capability to deal with BigData sources that would be more efficiently processed using Hadoop.• Direct data flow integration between InfoSphere DataStage and InfoSphere Streams to combine the power and reach of both platforms• Rich metadata support through all use cases. #ibmiod
  44. 44. OperationsOptimize and Manage the runtime environment Enhanced Operations Console Features Accelerate investigation & remediation Full log display included in Job Run Detail dialogue so users can perform complete job analysis and adds run, stop, and reset Information actions for Operators to take action when required. Server is truly unique in its Pre-built Operations Analytical Reports ability to manage Gain New Insight system resources Pre-built reports in Cognos BI which showcase the rich operation proactively and in metadata being gathered and whose schema was documented in providing release 8.7. Provides a great synergy story for Cognos operational customers & illustrates analytics to anyone using other BI tools. insight about the runtime environment. Workload Management Maximize performance in shared environment Empower the data integration admin to set the policies in a shared services framework for how jobs should be run and resource thresholds should be honored to optimize system perf #ibmiod
  45. 45. Workload ManagementPolicy driven control of system resources in a shared service environment • Allows the proactive mgmt of system resources where multiple teams share a common hardware infrastructure. • Optimize hardware utilization and prioritize mission critical tasks. • Throttle job activity where system resources exceed admin-specified thresholds. • Assign the priority of any submitted job. Configurable across various Projects as well as at the job level. • Manual overrides by privileged user to promote specific jobs to the top of the queue44 #ibmiod
  46. 46. WLM Performance Benefits•Greater % of CPU applied to data integration workload –Overloaded Systems consume disproportionate amount of “system cpu” –WLM minimizes “thrashing” and thereby maximizes “user cpu”•Our performance tests brought a job stream that ran in 55 minutesdown to 16 mins! (your mileage may vary) Original 54.75 Runtime With WLM 16.1 0 10 20 30 40 50 60 #ibmiod
  47. 47. AdministrationEnterprise class support for the data integration and governance platform PureSystems Private Cloud Rapid deployment of optimized environments New system pattern for Information Server Enterprise Edition provides unparalleled Private Cloud offering facilitating leaps in Major emphasis time to value on improving our upgrade experience. Migration User Interface Installable as a Reducing cost of upgrade New Migration Assistant Export Wizard guides the user through fixpack if already an interview and collects information the environment to on 8.7 (feature automatically create the response file for driving the migration. planned for Q1’13) New Support Features DS/QS 8.7 jobs will not need Expediting Investigation FFDC – generating stack traces for windows, linux and AIX re-compilation in platforms when a job fails; New Data Stage Runtime Trouble order to run. shooting guide for easy diagnosis and resolution of common issues #ibmiod
  48. 48. IBM Information Server – Migration Tool • Export47 #ibmiod
  49. 49. IBM Information Server – Migration Tool • Export: Getting Started48 #ibmiod
  50. 50. IBM Information Server – Migration Tool • Export: Archive and Work Directories49 #ibmiod
  51. 51. IBM Information Server – Migration Tool • Import50 #ibmiod
  52. 52. IBM Information Server – Migration Tool • Import: Getting Started51 #ibmiod
  53. 53. IBM Information Server – Backup/Restore Tool • Back Up / Restore52 #ibmiod
  54. 54. Expert Integrated SystemsFaster time to value on optimized systems  Enables companies who are interested in cloud computing to achieve faster time to value for data intensive projects.  Provides Data Integration and Data Quality capabilities to satisfy requirements for users developing solutions in dev/test as well as production environments.  Simple provisioning capabilities to get started and scale the environment quickly. #ibmiod
  55. 55. Head to the Demo Room to learn more! #20 - Big Data Integration with DataStage #44 – DataStage Managing and Monitoring Balanced Optimization for Hadoop Workload Management Seq File & BDFS Dynamic File Creation Enhanced Operations Console Features InfoSphere Streams Integration Operation Analytical Reports #28 - Information Server Serviceability #55 – Information Server Connectivity New Support Features DBMS Connector Boost #29 - Getting up and running Operational Decision Mgmt / ILOG Migration User Interface Native Excel Access Java Integration Stage #32 - Information Server Data Click InfoSphere Data Click XML Schema Views #33 - Information Server Hypervisor Edition #42 – Whats New in Information Server 9.1 PureSystems Info Server Private Cloud For anything else, ask here for the right engineer who can help #ibmiod
  56. 56. Thank You! #ibmiod