Ipedo XIP Overview for {Customer Name} July 2006 {Name}
Agenda Introductions EII Opportunities for <customer> Ipedo XIP Overview
{Customer Name} {Issues/Fits/Needs} Summarize issues or needs of the prospect
Ipedo Background Focus on Enterprise Information Integration (EII) software Customers and partners worldwide Based in Redwood City, CA  Founded in 2000; Venture backed by Draper  Fisher Jurvetson and AsiaTech Management  50+ Customers Quick Facts Influential Partners
EII Market Opportunity Growing Batch Realtime Process Data EAI EII ETL Sources: * Forrester Research;   TDWI/BI Research 10/05 Target is a user Supports ad hoc applications Results moved in realtime at query time Supports relational + XML Data warehouse can be source EII EII Target is a database Supports scheduled applications Batch; data moved ahead of time Supports relational + multidimensional Data warehouse is the target ETL EII Use Plans 
EII Brings Advantages to New Projects Solution Key Advantages of EII Dashboards Intelligently combine company-wide information Provide view customization per context or user Maintain or enhance enterprise security model Risk Management  BI / Reporting Enable better reports with up-to-date information - Access diverse data sources via queries, not code - Mix real-time data with historical information Extract key information across company data silos Reduce cost of compliance Manage complex XML schemas such as FpML
EII Enables Integration Infrastructure Solution Key Advantages of EII Metadata  Management Consolidate metadata across the enterprise Exchange data models with other modeling tools Append business definitions to integrated data Data Service Layer  for Service-Oriented Architectures (SOA) Master Data Management (MDM) Maintain consistent view of core business entities Avoid political issues around data ownership Reduce data replication Create data services layer that simplifies SOA Allow universal info exchange as Web Services Manage both HTTP and SOAP Web Services
Use Case: Real-Time Reporting Reporting Trade Data Market Data Risk Data Oracle OLTP Sybase OLTP Web Service EII Server Virtual DB
Use Case: Data Warehouse Prototyping Relational Databases Sales Ops Applications/Marts EII Server Virtual DB Data Warehouse Reporting & BI Tools Migrate ETL Prototype Permanent
Use Case: Database Migration - Insulation from Change Reporting Trade Data Trade Data Sybase DB2 EII Server Virtual DB Migrate
Use Case: Migrating to SOA Relational Databases Sales Ops Applications/Marts EII Server Virtual DB EII and SOA: Make relational models available as services Protect against a performance hit Maintain adequate governance Let DBAs be DBAs, and developers be developers View 1 View 2 Registry Application 1 Application 2
Ipedo Creates a Virtual Data Services Layer  from Multiple Remote Data Sources Ipedo XIP Sales Business Unit A Business Unit B Customer  System Finance System Operations System Finance System Operations System Mfg. System Finance Marketing Subset Data Distributed Data Mixed Data Query Query Query Query Cross business boundaries No data replication Mix data types in real time TIBCO SQL Server SAP Oracle SQL BAPI SOAP WSDL JMS SOAP WSDL SQL View 2 Operations View 1 Sales by Region View 3 Financials by Account View 4 Financials Quarterly
Ipedo Provides a Data Services Layer
Views Virtualize Data Access Custom Metadata Data Source  Connections Data Mappings Transformations Output Schema View Caching Policy Validation Security View Data Warehouse Relational Database Web Services Packaged Apps XML Documents Message Queues Ipedo Views A View is a metadata construct. It contains no data, unless explicitly cached.  When a query is executed against a View, Ipedo returns the data to fulfill the request.
Deploying Ipedo XIP Ipedo XIP Development/QA Ipedo XIP Monitor & Manage Production App Server Reporting Dashboards Web Apps SOA End Users Views are designed, developed and tested Views are then deployed to production server Additional production servers can be added to support more users View 1 View 2 View 3 View 1 View 2 View 3 Design Time Runtime Deploy
Ipedo XIP Solves Integration Challenges    Faster Development   $   Cheaper Deployment      Lower Risk $   Data discovery tools and wizards for RDB, XML  and SOA  Data source introspection with statistic and key gathering  Metadata collection and management $   Drag and Drop View creation $   Automatic SQL and XQuery generation  Rule and aggregate tools  Automatic XML to RDB bridging $   Automatic query plan generation  Cost-based query optimization  Policy-driven caching for SQL and XML result sets $   Virtualized data access without replication  Customizable result set presentation  Adjustable configurations for scaling up, down or out  Detailed data access controls  Performance thresholds and limits  Data lineage and query forensics reporting Automated Discovery Powerful Modeling Intelligent Optimization Flexible Deployment Comprehensive Management Where does our data reside and in what format? How can I quickly combine all the data I need for my application? How can I meet my users’ performance needs and not break existing systems? How can I tailor data to suit the various and changing user needs? How can I make sure we adhere to our security and data governance policies?
Ipedo Speeds Discovery and Data Combination Finance Sales  Ops Source Introspection Data Discovery Metadata Collection Tables, Keys, Statistics Stored Proc,  Keys, Statistics Services,  Methods Catalog Read Catalog Read UDDI Lookup Design Time Automated Discovery Powerful Modeling GUI View Creation Auto-Query Generation Intelligence SQL Statement XQuery Statement Metadata Lookup Metadata Addition Metadata Catalog Rules Aggregates
Auto-Discovery Simplifies Data Integration XIP > Automated Discovery Lists all tables Displays column names Shows data types Generates SQL DDL Ipedo automatically Introspects remote data sources.
Custom Metadata Annotates Data XIP > Automated Discovery Ipedo Metadata Catalog Add custom metadata  to annotate: * data sources  * remote tables  * columns  * views Retrieve metadata from Ipedo system tables using standard SQL system functions
GUI View Builder Simplifies Development XIP > Powerful Modeling ‘ Drag and Drop’ tables into the design pane Build logic using intuitive graphical tools: TotalPrice>$50,000 Ipedo automatically generates SQL, which can be modified
Ipedo Ideal for SOA Consuming Web Services Blending multiple web services Conditional invocation of web services in a specific order Combining web services with other data sources Processing Web Services Business rules checking on web services data Caching of web services data for reuse Ipedo uses a query-based approach, preserving the format of the original data
‘ Web Services’ Tables Allows SQL on XML Customer Records (Oracle) Market Exposure (SOAP Web Service) Credit Record (HTTP Web Service) XML XML Rel. Rel. XMLQuery XMLQuery XML Core SQL Core BI/Reporting Client ODBC / JDBC SQL SQL SQL SQL Data Sources Remote Tables Relational View Use familiar SQL syntax with Web Services:  Join, Group By, hints XIP > Powerful Modeling
Map Web Services to Relational Tables XIP > Powerful Modeling Select a  Web Service Map results to relational columns Modify data types to fit your application’s needs
Ipedo Application Adapters  Integrate Data Into Ipedo Framework Pre-built templates and objects Accessible via SQL, Web Services, XQuery Visual drag and drop, API/script access Broadest range of applications (Librados) Simple pricing structure ($15K/CPU per adapter) Application View Templates Application Data Objects Oracle Integration Tables  Integration Objects API BAPI SOQL Oracle Siebel SAP R/3 Salesforce.com Common Reports Common Objects
Ipedo Approach to SAP BW Data Librados adapters for relational data Simba OLAP adapters for cube data Best-in-class vendors of standards-based advanced analytical data adaptors Perform more complex analyses using broader range of data sources Integrate SAP BW with other data sources
Ipedo Optimizes Data Access Data Virtualization Customizable Presentation Deployment Time Flexible Deployment Intelligent Optimization Query Optimization Query Execution Cache Policy Cost-Based Optimizer Query Plan Advanced Tuning XML View Cache SQL View Cache Schedule Refresh Policy Result Set Streaming Query Throttling Concurrency Hybrid Intermediate Result Processing Virtual View Virtual View Virtual View SQL/JDBC SQL/ODBC XML/WS XML/WS JSP/HTTP Order View Risk View Leads View
Ipedo Appears as a Database External applications see tables within Ipedo Each table can contain data from disparate sources (relational or Web Services) XIP > Flexible Deployment
Performance Optimization
Cost-Based Query Optimization CBO means knowing how a query will perform; Rules-based optimization means guessing Examines statistics in remote data sources Size of database (cardinality) Indexes - presence, type, selectivity Allows user to assign cost function to remote procedure Determines join order and join algorithm based on “cost” of performing query Minimizes intermediate results to speed performance CBO greatly enhances EII performance XIP > Intelligent Optimization
Query Plan Shows Optimization Approach Automatic Query Plan Generation Join Order Join Algorithms Pushdowns Joins Predicates Aggregates XIP > Intelligent Optimization
Advanced Tuning Provides Ultimate Control Hints override Ipedo’s query engine with domain-specific knowledge SQL pass-through Ipedo sends data source specific SQL to the data source for execution Polymorphic remote  function mapping XIP > Intelligent Optimization pass sql &quot;dbcc show statistics (customers, zipcode_stat)&quot; to mssqlsvr select a.*  from  properties joinOrder=fixed orders  o, customers  c  properties joinStrategy=NESTEDLOOP where c.c_customerid = o.o_customerid
Query Execution Result Set Streaming Immediate response to user Query Throttling Control load on back-end systems Concurrency Support large numbers of users with optimal performance Hybrid memory-disk intermediate result set processing Manage memory to handle extremely large data sets  Flexible Security Model Common user credential across data sources  Dynamic session-based user credentials for data sources View-based access control XIP > Intelligent Optimization
Intelligent Caching Cuts Response Time, System Load Speeds Response to Repeated Queries Reduces Load on Back-End Data Sources Cache relational or XML data natively Sub-queries on cache Scheduled or on-demand cache loading Automatic cache invalidation XIP > Intelligent Optimization
Top-Down Modeling Tools  Can Create Tables and Views in Ipedo 1. Build Logical Views In enterprise modeling software  2. Export Data Definitions Using standard ODBC 3. Select Options  For schemas, tables, columns, indexes 4. Customize DDL for Ipedo Syntax reflects virtual table construct 5. View Model in Ipedo All metadata appears, available for integration
Reuse Ipedo Views in Modeling Tools  1. Build Views in Ipedo Map tables and create relationships across disparate data sources  2. Reverse Engineer Start with blank template inside ERwin 3. Select Options  Choose what information to import 4. Model Appears in ERwin Tables, metadata, joins, etc., available to share and reuse
Comprehensive Monitoring and Reporting  Simplifies System Management Partners Finance Marketing Management Dashboard Users Session Details Session Controls Query Controls Data Sources Source Status Source Impact Thresholds Limits on Views Limits on Users Data Lineage View Source Search Historical Access Reports and Search Ipedo XIP Users Administrators Ipedo Billing View Order View Risk View Leads View
Detailed User and Source Management Session Status XIP > Comprehensive Management Database Availability See Query Plan Kill Session or Cancel Query
Ipedo Facilitates Data Governance Where is the information in this view coming from? How does change to data source affect downstream views?
Dynamic Failover Redirection  Many large customers have database replicas worldwide - want EII Views to failover to replica database Ipedo XIP will failover after it reaches retry limit SOA Sybase 1 Oracle 2 Oracle 1 New York London New York Tokyo Sybase 2
Enterprise Class Product Clustering and Failover - support for standard clustering environment Scales to manage millions of documents and  terabytes of storage Industry-standard management APIs Robust archival & backup capabilities Automated schema migration Integrated document versioning and journaling
Ipedo XIP Components  SQL Engine •   Query Optimization XQuery Engine •  Query Optimization   Federation Engine •   Query Orchestration •   Statistical Analysis Data Services Manager •   Web Services  • SQL Data Services  • XQuery Data Services  •  APIs  • Web Tag Library  • Outbound Pipelines Rules / Intelligence Engine •  Visual Rules Wizard   •   Rules Processing  •  Compound Aggregate Indexing Integration / Transformation Engine •  Relational Views  • XML Views  • Graphical View Builder •  Feeds   • Inbound Pipelines Metadata Manager   •   Sources  •   Mappings   Cache Manager   •   Relational  •   XML   XML Store •   Schema Management
HP - Finance Dashboard Application Financial Dashboard Challenges Accessing data from disparate sources Frequently changing data needs Requirement to keep interface simple Granular security model Scalability for 25,000 users Solution Use Ipedo to aggregate data from multiple sources, filter using XML,  and transform for delivery into  HP’s internal portal,  Results Makes new information available for real-time business decisions My Dashboard Financial Databases (Oracle) Data Warehouse (Hyperion  Essbase) MS Word Analysis Reports Corporate Portal Framework Ipedo XIP Product View Region View Account View Channel View
Assurant - Insurance Policy Management Application Insurance Company: next-generation policy management system Challenges Manage large transaction volumes of policies in ACORD XML schema Integrate with complex workflow processes using a service-oriented architecture (SOA) Maintain uptime in high-availability environment Solution Use Ipedo to manage and integrate information from underwriting, accounting, claims processing and billing systems,  Results Streamlines operations, improves response time to inquiries Billing Database Ipedo XIP Profitable Accounts Claims Status Expensive Claims Arrears Sales View Workflow View Claims View Acct. View Claims System Acct. System Web Services Policies ACORD Schema
Sun - Customer Service Application Customer Service Challenges Combining XML messages, Web Services, databases, other data Connecting with external risk engine Providing rapid response to inquiries Managing huge data volumes Solution Embed Ipedo into Sun application to deliver risk analysis as a service Results Improves system uptime and availability and reduces incident volume and cost Customer Reference Data (Oracle) Risk  Engine (Fair Isaac) System Information (XML) Preventive Services Ipedo XIP Hardware View Software View Account View Risk View Risk  Assessment Advanced Diagnostics System Configurations Web Services JDBC
The Ipedo Difference High Performance Cost-based Query Optimization Intelligent Caching Investment Protection Choice of data model: Relational (SQL) or XML (XQuery) Choice of SOA deployment model – WSDL/SOAP or REST One cohesive product Seamless Deployment Fully scriptable with complete SQL DDL support Integrated Data Lineage and Impact Analysis reporting Comprehensive APIs: Java, Web Services, .NET Highly Scalable Concurrency for hundreds of simultaneous users Easily accommodates additional data sources
Additional Resources 2005 EII Survey Guide to EII ROI Guide
Background Slides
Enterprise Information Integration is… …  a new kind of virtual data integration technology that makes several remote data sources appear as one local database.
Ipedo XIP: Integration + Intelligence Vision: Intelligence without boundaries:   Give business decision makers “information on demand, from anywhere, at any time” Product: Ipedo XIP integrates and manages information from disparate and complex data sources  on demand
EII Generates Real Financial Benefits Developer Productivity EII allows developers to write to one, common interface to access multiple data sources, reducing the development cost for new applications What % of your IT budget is spent on data integration development? Analyst Utilization EII allows analysts to simplify the data collection process and spend more time producing information and less time gathering data How much time do your analysts spend collecting data? Increased Flexibility EII allows companies to respond quickly to changes in market conditions without incurring a major IT investment What effort is involved to link new data sources to existing reports or applications?  Data Consolidation Using EII to create a virtual centralized database can cut the cost of data maintenance, and reduce the need for data replication How much does it cost to deploy each new data mart?
Integration Platform Built for Developers/Integrators Data and Cache Management •  TTL • Performance • Versioning   Publishing and API Access •  XML  •  JSP Tag Lib • JDBC/ODBC   • .NET  • WS/SOA Transformation & Virtualization •  XML Schema   •  CSV • SQL <-> XML • Pipelines  Connectivity •  Relational Views  • XML Views  • SOA/WS Views •   JMS Feeds   • HTTP Views Query Management •  XQuery  •  SQL • Cost-based Optimizer   Rules Processor •  SQL SP • XQuery Modules • Java Functions   Security •  ACLs • Roles   Management •  JMX • JSP • APIs   GUI Tools •  View Builder • Query Builder • Rules Builder
EII Brings Advantages to New Projects -  Create data services layer that simplifies SOA MDM -  Maintain consistent view of core business entities -  Avoid political issues around data ownership SOA -  Allow universal info exchange as Web Services Solution Key Advantages of EII Dashboards -  Intelligently combine company-wide information -  Provide view customization per context or user Risk Management  -  Extract key information across company data silos -  Reduce cost of compliance  BI / Reporting -  Enable better reports with up-to-date information -  Access diverse data sources via queries, not code
Metadata Slides
Views Use Metadata Extensively Ipedo XIP View Output Schema Transformations Data Mappings Data Source  Connections View Definitions  of Fields Business  Definitions Caching Policies Security Primary /  Foreign Keys Security Data Source Metadata Source Repository Ipedo
Metadata Flows Throughout the Framework Metadata Manager Enterprise Data Sources Introspection Ipedo XIP Metadata Repositories Metadata Mgmt/Mod Tools Databases Web Services XML Doc Collections Other Data Sources Supported by Ipedo Modeling Tools BI/Reporting Tools SOA Tools App Dev Tools JDBC/ODBC/SQL WSDL/UDDI JDBC/ODBC/SQL WSDL/UDDI XMI JDBC/ODBC/SQL JDBC/ODBC/SQL XMI/Proprietary
Additional Customer Case Studies
Anhui Electric - Fraud and Risk Management Application Power Utility Billing Risk Management Dashboard Challenges Forecasting business risk No aggregated meter monitoring data to prevent rampant theft of power Lack of credit information on new customers Solution Use Ipedo to integrate energy meter collection system, customer databases and financial systems across multiple sites Results Significant reduction in fraud and theft; better management reporting Billing Database Customer Database Energy Meter Collection System Ipedo Credit View Theft View Risk View Forecast View
Application Ring tone, music, and mobile game  digital rights management platform Challenges Ability to address diverse content and transaction data types Adapting rapidly to changing market and customer requirements Scalability for millions of buyers and millions of content assets Solution Use Ipedo to integrate user, payment, and ring tone/game metadata for presentation through Web interface Results Allows rapid response to customers and easy integration of new content libraries into client Web sites  Navio Card Navio Merchant Navio Store ASP Applications Report Server Title Manager  Content Lockbox Digital Commerce Engine Ipedo XIP Ipedo XIP Ipedo XIP Navio - Rights-Based Commerce Web Storefront P2P Net
HP OpenView - Customer Service Portal Application Customer Service Portal Challenges Flexible search interface Frequently changing content Scalability for global users High Availability Solution Use Ipedo to manage granular content and deliver in response to Web-based user inquiries  Results Improve customer experience by directing users to relevant software downloads or reference materials Customer Service Marketing Databases (RDBMS) OpenView  Web Site Ipedo XIP Product Info Customer Info Marketing Info
Pharma R&D Data Integration   Ipedo Cuts Costs, Speeds Discovery BEFORE   IPEDO AFTER   IPEDO Too many data sources Unable to consolidate genomic data Data sources in proprietary formats Unable to analyze all genomic data Scientists manually research genes Scientists’ observations are not shared Ipedo   XIP Observations Annotations Deliver  one-stop shop for research  data Capture scientists’ annotations Keep scientists’ information synchronized  Views Integration Query Analysis Pipelining Presentation “ 10x researcher productivity gain.” Internal Data Warehouse GenBank BLAST MedLine Internal Data Warehouse GenBank BLAST MedLine Scientists Scientists Research Application Web Site Web-Based Queries Subscription Web Site ROI in Time to Market Research Portal
British Telecom Price Book October 2003
Vertical Market Unique Needs
Financial Services Has Unique Needs 360 degree view of the customer See across systems in subsidiaries/acquisitions Integrate data from different databases Combine live and historical data Satisfy regulatory requirements Cross-divisional reporting Granular security / data ownership
Insurance Has Unique Needs Integrate underwriting, policy management, billing, claims systems Perform ad-hoc reports Leverage industry standard schemas Produce reports for regulators 360 degree view of the customer Cross-divisional reporting Post-merger/acquisition consolidation Granular security / data ownership
Health Care Has Unique Needs HIPAA compliance Protect patient privacy - granular security model Mix historical and real-time information Customize reports for different uses:  administration, insurance, treatment 360 degree view of the patient
Retail Has Unique Needs Generate real-time sales information Track inventory across multiple locations Share information with supply chain Spot trends as they occur  360 degree view of the customer Cross-divisional reporting Post-merger/acquisition consolidation Granular security / data ownership
Business Intelligence
Business Intelligence and Reporting Use Business Objects, Cognos, MicroStrategy, or any BI tools to analyze data from: Relational databases Unstructured data sources Enterprise applications Web Services Combine historical information with real-time information from message queues or Web Services
Ipedo Supports BOBJ Three Ways Ipedo XIP BusinessObjects Universe Business Objects BI Platform Data Sources Data Mart
Ipedo Bridges Data Partitions Application Ad-hoc reporting across separate databases using BI tools Challenges Overcoming size limitation on Sybase servers Shielding reports from changes in underlying database partitions Combining data from disparate sources into a single “Universe” Maintaining high-performance for complex queries Solution Use Ipedo to create virtual union views BI tools connect to Ipedo views Ipedo optimizes and federates queries to all databases Business Intelligence / Enterprise Reporting Tools Ipedo XIP All Customers View All Orders View All Sellers View ... Sybase1 Sybase2 Sybase10
Ipedo Appears as a Database to  Business Intelligence Applications In this example, Business Objects Web Intelligence sees Ipedo as a single relational database. Ipedo is really pulling data from three different relational sources.
Ipedo Complements BOBJ Data Integration Real-time, query-based access to data sources Combining web services with other data sources Query optimization Validation of data in real-time Web Services Blending multiple web services Conditional invocation of web services in a specific order Business rules checking on web services data Caching of web services data for reuse
Master Data Management
Master Data Management Create a master index that leverages Ipedo’s high performance engine Cost-based query optimization Union join pushdown Reduce data replication Customize departmental subsets of customer data to avoid data politics and ownership issues Perform data cleansing (through Ipedo partnership) Help improve the performance of the denormalization process and boost the flexibility in handling underlying schema changes
Ipedo XIP for MDM BI/Reporting Client ODBC / JDBC SQL SQL SQL Data Sources Remote Tables Relational View Join data from multiple RDBMS using Master Customer Key in MDM Ipedo Customer View Join View Union View Master Customer Key SQL SQL CRM Data (DB2) Trading Data (Sybase) Master Data (DB2)
Data Quality
Data Quality Enhances EII Data Source  Mappings Virtual View Mappings User Query on View EII Step: Join DQ Call: Matching Join Key EII Step: Filter DQ Call: Matching Other Columns Data Sources Live Views (seconds of latency) User Query on View EII Step: Filter, Other DQ Call: Cleanse or Match Data Cached Views (minutes to hours of latency) Caching Options: Scheduled Cache (Push) – Include cleanse/match option for admin User-driven Cache (Pull) – Include cleanse/match option after initial view Aligning data between different sources for efficient join operations Reconciling differences between data from different sources Cleaning up cached information for subsequent reuse
Risk Management
Risk Management Present complete set of information for analysis Real-time feeds from message queues (trading data, interest rates, etc.) Analyze complex XML documents (such as FpML or other trading schemas) and combine seamlessly with relational data Cache XML for subsequent query, analysis, and integration Compound Aggregate Indexes (CAI) track key data fields and automatically compute aggregates across different data feeds Maintain audit trail for compliance and SOX
Ipedo for Risk Analysis Financial  View Ipedo XIP JDBC ODBC SQL Convert to  Relational Tables Calculate Aggregates Message  Queue Message  Queue XBRL FpML XML Caching BI/Reporting Client Trading Data In XML Format Order View Risk View
Enterprise Readiness
Ipedo Delivers Enterprise Readiness Performance Cost-based Query Optimization Result-set Streaming Policy-based Caching Advanced Tuning Scalability Clustering Concurrency Security Data Governance
Ipedo Joins Huge Data Sets… SELECT  CUSTOMER.C_NAME,  CUSTOMER.C_CUSTKEY, CUSTOMER.C_ADDRESS, COUNT(ORDERS.O_ORDERKEY),  SUM(ORDERS.O_TOTALPRICE ) FROM   CUSTOMER, ORDERS  WHERE   (CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY)  AND  CUSTOMER.C_NAME  =  'Customer#000000250' GROUP BY  CUSTOMER.C_NAME,  CUSTOMER.C_CUSTKEY,  CUSTOMER.C_ADDRESS Select all ORDERS of a given CUSTOMER ORDERS and TIME_DIM tables from Oracle (Orders Management App) 750,000 rows CUSTOMER and NATION tables from SQL Server (CRM)  10 0 , 0 0 0   r o w s
…And Delivers Sub-Second Performance Query Optimization: Search predicate is pushed down to remote RDBMS. Join predicate will be pushed into remote inner table if nested loop join is cost-effective. Number of rows of the where clause is taken into consideration in Cost-Based Optimization (CBO). Performance: average query time = 0.1 sec Query Plan:
Ipedo Security Protects Data and Views User and Group Level Hierarchical  Access Control List Integration with external LDAP directories Document/Attribute-level filtering (node-level) JAAS-compliant encryption algorithm flexibility for 2-way secure authentication (from client, as well as server)
Low Overhead in IT Environment Connection Pooling allows you to protect back-end data sources from excessive load Intuitive setup and administration means no DBA required to run Ipedo Runs on Windows, Linux, Solaris, HP-UX, AIX - any environment with a JVM Runs on Tomcat (included), WebLogic, WebSphere, JBoss, JRocket
Professional Services Simplifies Implementation Experienced Team SQL, XML, Java, Web Services Flexible Engagement Model Off-site development, non-invasive setup (1-5 days) On-site training for your team Sample Deployments Portal for global computer hardware company Trading application for commodity exchange Claims management system for insurance company Risk management system for power utility
Ipedo Performance Testing
Ipedo XIP SQL Performance Testing ORDERS and TIME_DIM tables from Oracle (Orders Management App) CUSTOMER and NATION tables from SQL Server (CRM) 100,000 rows 750,000 rows Testing conducted in July 2005 on dual-CPU Intel XEON 2.2GHz with 2.0Gb memory
SQL Query #1 SELECT  CUSTOMER.C_NAME,  CUSTOMER.C_CUSTKEY, CUSTOMER.C_ADDRESS, COUNT(ORDERS.O_ORDERKEY),  SUM(ORDERS.O_TOTALPRICE ) FROM   CUSTOMER, ORDERS  WHERE   (CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY)  AND  CUSTOMER.C_NAME  =  'Customer#000000250' GROUP BY  CUSTOMER.C_NAME,  CUSTOMER.C_CUSTKEY,  CUSTOMER.C_ADDRESS Select all ORDERS of a given CUSTOMER
Query Plan #1 and Performance Query Optimization: Search predicate is pushed down to remote RDBMS. Join predicate will be pushed into remote inner table if nested loop join is cost-effective. Number of rows of the where clause is taken into consideration in Cost-Based Optimization (CBO). Performance: average query time = 0.1 sec Query Plan:
SQL Query #2 SELECT   TIME_DIM.YEAR,  COUNT(ORDERS.O_ORDERKEY),  SUM(ORDERS.O_TOTALPRICE ) FROM   TIME_DIM,  ORDERS, CUSTOMER WHERE  (ORDERS.O_ORDERDATE=TIME_DIM.NATIVEDATE)  AND  CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY) AND   ( ( CUSTOMER.C_CUSTKEY ) = CASE WHEN  char(hour(current_timestamp)) = '24'  THEN  250  ELSE  251  END   )) GROUP BY   TIME_DIM.YEAR Conditionally select a customer’s total order count and sum
Query Plan #2 and Performance Query Optimization ANSI SQL standard case-when-else conditional expression is pushed down to remote RDBMS.  Column transitivity is supported to allow better query plan. e.g. (t1.c1 = t2.c2 and t2.c2 = constant)   (t1.c1=constant and t2.c2 = constant)  Performance: average query time = 0.2 sec Query plan:
SQL Query #3 SELECT   V_MERGED_CUSTOMER.C_NATIONKEY, sum(V_MERGED_ORDERS.O_TOTALPRICE) FROM  V_MERGED_CUSTOMER, V_MERGED_ORDERS WHERE  (V_MERGED_CUSTOMER.C_CUSTKEY=V_MERGED_ORDERS.O_CUSTKEY)  AND (V_MERGED_CUSTOMER.C_MKTSEGMENT  =  'MACHINERY ‘) GROUP BY V_MERGED_CUSTOMER.C_NATIONKEY Query efficiently across data partitions in Oracle and SQLServer: View v_merged_customer unions customer tables in both partitions. View v_merged_order unions order tables in both partitions.
Query Plan #3 and Performance Query Optimization Search predicates on views are pushed down to data sources Joins across union partitions are regrouped and pushed down to data sources, so that smaller joined result sets are combined in XIP Performance: average query time = 2 sec Query plan:
SQL Query #4 Query Optimization Shows stability and scalability in handling large result set with high concurrency without running out of memory. The first record is returned immediately without waiting for the entire result set to be processed. Ipedo support cursor fetching all the way through from client to Ipedo and to remote RDBMS. Query  can be cancelled without fetching the entire result set, and resources are released immediately. SELECT  ORDERS.O_ORDERKEY,  ORDERS.O_ORDERDATE,  ORDERS.O_CLERK FROM  ORDERS Select large result set (750,000 rows)
Query Performance #4 Cursor is used throughout the fetch chain to achieve high responsiveness: Cursor is used from client to Ipedo: Client obtains first row : 0.174s Cursor is used from Ipedo to remote RDBMS: Database returns first row in 0.171 s. This shows cursor is used in back end when Ipedo fetches data from remote RDBMS.
Ipedo XIP Reference Architectures
Ipedo XIP Sizing Considerations Number of Concurrent Users Query Load Number of queries Complexity of queries (number of joins) Amount of information being returned (row count) Additional Processing Data transformations (relational - XML) Rules processing (number and complexity of rules) High-Availability Requirements  Clustering (configuration) Replication (writing or reading transaction logs)
Basic EII Server Load Concurrent users: < 75 Queries: < 50/sec Server Configuration CPUs: 1-4 RAM: 2 G Typical Usage Enable departmental reporting across multiple sources
Basic EII with Failover Ipedo XIP Primary Application-aware session pooling Ipedo XIP Secondary Server Load Concurrent users: < 75 Queries: < 50/sec Server Configuration CPUs: 1-4 RAM: 2 G Typical Usage Enable reporting across multiple sources for a business unit Ipedo can also use a clustering server for failover
HA Clustered - “Shared-Disk” Architecture Shared  Disk Ipedo XIP Primary Ipedo XIP Secondary LAN Hub Private Network Windows Advanced Server Clustering Server Load Concurrent users: < 75/node Queries: < 50/sec/node Server Configuration CPUs: 1-4 per node RAM: 2 G per node Typical Usage Enable reporting across multiple sources for an enterprise or build an SOA data services layer Ipedo also supports clustering software on Linux, Solaris, HP-UX (HA = High Availability)
HA Clustered - “Shared-Nothing” Architecture SCSI  Disk Ipedo XIP Primary Ipedo XIP Secondary LAN Hub Private Network Windows Advanced Server Clustering SCSI  Disk Server Load Concurrent users: < 75/node Queries: < 50/sec/node Server Configuration CPUs: 1-4 per node RAM: 2 G per node Typical Usage Enable reporting across multiple sources for an enterprise or region, or build an SOA data layer Ipedo also supports clustering software on Linux, Solaris, HP-UX (HA = High Availability)
EII Server Farm (Clustered for Load Balancing) Ipedo XIP Ipedo XIP LAN Load Balancer Ipedo XIP Server Load Concurrent users: < 75/node Queries: < 50/sec/node Server Configuration CPUs: 1-4 per node RAM: 2 G per node Typical Usage Enable reporting across multiple sources for an enterprise or build an SOA data services layer

Ipedo Company Overview

  • 1.
    Ipedo XIP Overviewfor {Customer Name} July 2006 {Name}
  • 2.
    Agenda Introductions EIIOpportunities for <customer> Ipedo XIP Overview
  • 3.
    {Customer Name} {Issues/Fits/Needs}Summarize issues or needs of the prospect
  • 4.
    Ipedo Background Focuson Enterprise Information Integration (EII) software Customers and partners worldwide Based in Redwood City, CA Founded in 2000; Venture backed by Draper Fisher Jurvetson and AsiaTech Management 50+ Customers Quick Facts Influential Partners
  • 5.
    EII Market OpportunityGrowing Batch Realtime Process Data EAI EII ETL Sources: * Forrester Research;  TDWI/BI Research 10/05 Target is a user Supports ad hoc applications Results moved in realtime at query time Supports relational + XML Data warehouse can be source EII EII Target is a database Supports scheduled applications Batch; data moved ahead of time Supports relational + multidimensional Data warehouse is the target ETL EII Use Plans 
  • 6.
    EII Brings Advantagesto New Projects Solution Key Advantages of EII Dashboards Intelligently combine company-wide information Provide view customization per context or user Maintain or enhance enterprise security model Risk Management BI / Reporting Enable better reports with up-to-date information - Access diverse data sources via queries, not code - Mix real-time data with historical information Extract key information across company data silos Reduce cost of compliance Manage complex XML schemas such as FpML
  • 7.
    EII Enables IntegrationInfrastructure Solution Key Advantages of EII Metadata Management Consolidate metadata across the enterprise Exchange data models with other modeling tools Append business definitions to integrated data Data Service Layer for Service-Oriented Architectures (SOA) Master Data Management (MDM) Maintain consistent view of core business entities Avoid political issues around data ownership Reduce data replication Create data services layer that simplifies SOA Allow universal info exchange as Web Services Manage both HTTP and SOAP Web Services
  • 8.
    Use Case: Real-TimeReporting Reporting Trade Data Market Data Risk Data Oracle OLTP Sybase OLTP Web Service EII Server Virtual DB
  • 9.
    Use Case: DataWarehouse Prototyping Relational Databases Sales Ops Applications/Marts EII Server Virtual DB Data Warehouse Reporting & BI Tools Migrate ETL Prototype Permanent
  • 10.
    Use Case: DatabaseMigration - Insulation from Change Reporting Trade Data Trade Data Sybase DB2 EII Server Virtual DB Migrate
  • 11.
    Use Case: Migratingto SOA Relational Databases Sales Ops Applications/Marts EII Server Virtual DB EII and SOA: Make relational models available as services Protect against a performance hit Maintain adequate governance Let DBAs be DBAs, and developers be developers View 1 View 2 Registry Application 1 Application 2
  • 12.
    Ipedo Creates aVirtual Data Services Layer from Multiple Remote Data Sources Ipedo XIP Sales Business Unit A Business Unit B Customer System Finance System Operations System Finance System Operations System Mfg. System Finance Marketing Subset Data Distributed Data Mixed Data Query Query Query Query Cross business boundaries No data replication Mix data types in real time TIBCO SQL Server SAP Oracle SQL BAPI SOAP WSDL JMS SOAP WSDL SQL View 2 Operations View 1 Sales by Region View 3 Financials by Account View 4 Financials Quarterly
  • 13.
    Ipedo Provides aData Services Layer
  • 14.
    Views Virtualize DataAccess Custom Metadata Data Source Connections Data Mappings Transformations Output Schema View Caching Policy Validation Security View Data Warehouse Relational Database Web Services Packaged Apps XML Documents Message Queues Ipedo Views A View is a metadata construct. It contains no data, unless explicitly cached. When a query is executed against a View, Ipedo returns the data to fulfill the request.
  • 15.
    Deploying Ipedo XIPIpedo XIP Development/QA Ipedo XIP Monitor & Manage Production App Server Reporting Dashboards Web Apps SOA End Users Views are designed, developed and tested Views are then deployed to production server Additional production servers can be added to support more users View 1 View 2 View 3 View 1 View 2 View 3 Design Time Runtime Deploy
  • 16.
    Ipedo XIP SolvesIntegration Challenges  Faster Development $ Cheaper Deployment  Lower Risk $ Data discovery tools and wizards for RDB, XML and SOA  Data source introspection with statistic and key gathering  Metadata collection and management $ Drag and Drop View creation $ Automatic SQL and XQuery generation  Rule and aggregate tools  Automatic XML to RDB bridging $ Automatic query plan generation  Cost-based query optimization  Policy-driven caching for SQL and XML result sets $ Virtualized data access without replication  Customizable result set presentation  Adjustable configurations for scaling up, down or out  Detailed data access controls  Performance thresholds and limits  Data lineage and query forensics reporting Automated Discovery Powerful Modeling Intelligent Optimization Flexible Deployment Comprehensive Management Where does our data reside and in what format? How can I quickly combine all the data I need for my application? How can I meet my users’ performance needs and not break existing systems? How can I tailor data to suit the various and changing user needs? How can I make sure we adhere to our security and data governance policies?
  • 17.
    Ipedo Speeds Discoveryand Data Combination Finance Sales Ops Source Introspection Data Discovery Metadata Collection Tables, Keys, Statistics Stored Proc, Keys, Statistics Services, Methods Catalog Read Catalog Read UDDI Lookup Design Time Automated Discovery Powerful Modeling GUI View Creation Auto-Query Generation Intelligence SQL Statement XQuery Statement Metadata Lookup Metadata Addition Metadata Catalog Rules Aggregates
  • 18.
    Auto-Discovery Simplifies DataIntegration XIP > Automated Discovery Lists all tables Displays column names Shows data types Generates SQL DDL Ipedo automatically Introspects remote data sources.
  • 19.
    Custom Metadata AnnotatesData XIP > Automated Discovery Ipedo Metadata Catalog Add custom metadata to annotate: * data sources * remote tables * columns * views Retrieve metadata from Ipedo system tables using standard SQL system functions
  • 20.
    GUI View BuilderSimplifies Development XIP > Powerful Modeling ‘ Drag and Drop’ tables into the design pane Build logic using intuitive graphical tools: TotalPrice>$50,000 Ipedo automatically generates SQL, which can be modified
  • 21.
    Ipedo Ideal forSOA Consuming Web Services Blending multiple web services Conditional invocation of web services in a specific order Combining web services with other data sources Processing Web Services Business rules checking on web services data Caching of web services data for reuse Ipedo uses a query-based approach, preserving the format of the original data
  • 22.
    ‘ Web Services’Tables Allows SQL on XML Customer Records (Oracle) Market Exposure (SOAP Web Service) Credit Record (HTTP Web Service) XML XML Rel. Rel. XMLQuery XMLQuery XML Core SQL Core BI/Reporting Client ODBC / JDBC SQL SQL SQL SQL Data Sources Remote Tables Relational View Use familiar SQL syntax with Web Services: Join, Group By, hints XIP > Powerful Modeling
  • 23.
    Map Web Servicesto Relational Tables XIP > Powerful Modeling Select a Web Service Map results to relational columns Modify data types to fit your application’s needs
  • 24.
    Ipedo Application Adapters Integrate Data Into Ipedo Framework Pre-built templates and objects Accessible via SQL, Web Services, XQuery Visual drag and drop, API/script access Broadest range of applications (Librados) Simple pricing structure ($15K/CPU per adapter) Application View Templates Application Data Objects Oracle Integration Tables Integration Objects API BAPI SOQL Oracle Siebel SAP R/3 Salesforce.com Common Reports Common Objects
  • 25.
    Ipedo Approach toSAP BW Data Librados adapters for relational data Simba OLAP adapters for cube data Best-in-class vendors of standards-based advanced analytical data adaptors Perform more complex analyses using broader range of data sources Integrate SAP BW with other data sources
  • 26.
    Ipedo Optimizes DataAccess Data Virtualization Customizable Presentation Deployment Time Flexible Deployment Intelligent Optimization Query Optimization Query Execution Cache Policy Cost-Based Optimizer Query Plan Advanced Tuning XML View Cache SQL View Cache Schedule Refresh Policy Result Set Streaming Query Throttling Concurrency Hybrid Intermediate Result Processing Virtual View Virtual View Virtual View SQL/JDBC SQL/ODBC XML/WS XML/WS JSP/HTTP Order View Risk View Leads View
  • 27.
    Ipedo Appears asa Database External applications see tables within Ipedo Each table can contain data from disparate sources (relational or Web Services) XIP > Flexible Deployment
  • 28.
  • 29.
    Cost-Based Query OptimizationCBO means knowing how a query will perform; Rules-based optimization means guessing Examines statistics in remote data sources Size of database (cardinality) Indexes - presence, type, selectivity Allows user to assign cost function to remote procedure Determines join order and join algorithm based on “cost” of performing query Minimizes intermediate results to speed performance CBO greatly enhances EII performance XIP > Intelligent Optimization
  • 30.
    Query Plan ShowsOptimization Approach Automatic Query Plan Generation Join Order Join Algorithms Pushdowns Joins Predicates Aggregates XIP > Intelligent Optimization
  • 31.
    Advanced Tuning ProvidesUltimate Control Hints override Ipedo’s query engine with domain-specific knowledge SQL pass-through Ipedo sends data source specific SQL to the data source for execution Polymorphic remote function mapping XIP > Intelligent Optimization pass sql &quot;dbcc show statistics (customers, zipcode_stat)&quot; to mssqlsvr select a.* from properties joinOrder=fixed orders o, customers c properties joinStrategy=NESTEDLOOP where c.c_customerid = o.o_customerid
  • 32.
    Query Execution ResultSet Streaming Immediate response to user Query Throttling Control load on back-end systems Concurrency Support large numbers of users with optimal performance Hybrid memory-disk intermediate result set processing Manage memory to handle extremely large data sets Flexible Security Model Common user credential across data sources Dynamic session-based user credentials for data sources View-based access control XIP > Intelligent Optimization
  • 33.
    Intelligent Caching CutsResponse Time, System Load Speeds Response to Repeated Queries Reduces Load on Back-End Data Sources Cache relational or XML data natively Sub-queries on cache Scheduled or on-demand cache loading Automatic cache invalidation XIP > Intelligent Optimization
  • 34.
    Top-Down Modeling Tools Can Create Tables and Views in Ipedo 1. Build Logical Views In enterprise modeling software 2. Export Data Definitions Using standard ODBC 3. Select Options For schemas, tables, columns, indexes 4. Customize DDL for Ipedo Syntax reflects virtual table construct 5. View Model in Ipedo All metadata appears, available for integration
  • 35.
    Reuse Ipedo Viewsin Modeling Tools 1. Build Views in Ipedo Map tables and create relationships across disparate data sources 2. Reverse Engineer Start with blank template inside ERwin 3. Select Options Choose what information to import 4. Model Appears in ERwin Tables, metadata, joins, etc., available to share and reuse
  • 36.
    Comprehensive Monitoring andReporting Simplifies System Management Partners Finance Marketing Management Dashboard Users Session Details Session Controls Query Controls Data Sources Source Status Source Impact Thresholds Limits on Views Limits on Users Data Lineage View Source Search Historical Access Reports and Search Ipedo XIP Users Administrators Ipedo Billing View Order View Risk View Leads View
  • 37.
    Detailed User andSource Management Session Status XIP > Comprehensive Management Database Availability See Query Plan Kill Session or Cancel Query
  • 38.
    Ipedo Facilitates DataGovernance Where is the information in this view coming from? How does change to data source affect downstream views?
  • 39.
    Dynamic Failover Redirection Many large customers have database replicas worldwide - want EII Views to failover to replica database Ipedo XIP will failover after it reaches retry limit SOA Sybase 1 Oracle 2 Oracle 1 New York London New York Tokyo Sybase 2
  • 40.
    Enterprise Class ProductClustering and Failover - support for standard clustering environment Scales to manage millions of documents and terabytes of storage Industry-standard management APIs Robust archival & backup capabilities Automated schema migration Integrated document versioning and journaling
  • 41.
    Ipedo XIP Components SQL Engine • Query Optimization XQuery Engine • Query Optimization Federation Engine • Query Orchestration • Statistical Analysis Data Services Manager • Web Services • SQL Data Services • XQuery Data Services • APIs • Web Tag Library • Outbound Pipelines Rules / Intelligence Engine • Visual Rules Wizard • Rules Processing • Compound Aggregate Indexing Integration / Transformation Engine • Relational Views • XML Views • Graphical View Builder • Feeds • Inbound Pipelines Metadata Manager • Sources • Mappings Cache Manager • Relational • XML XML Store • Schema Management
  • 42.
    HP - FinanceDashboard Application Financial Dashboard Challenges Accessing data from disparate sources Frequently changing data needs Requirement to keep interface simple Granular security model Scalability for 25,000 users Solution Use Ipedo to aggregate data from multiple sources, filter using XML, and transform for delivery into HP’s internal portal, Results Makes new information available for real-time business decisions My Dashboard Financial Databases (Oracle) Data Warehouse (Hyperion Essbase) MS Word Analysis Reports Corporate Portal Framework Ipedo XIP Product View Region View Account View Channel View
  • 43.
    Assurant - InsurancePolicy Management Application Insurance Company: next-generation policy management system Challenges Manage large transaction volumes of policies in ACORD XML schema Integrate with complex workflow processes using a service-oriented architecture (SOA) Maintain uptime in high-availability environment Solution Use Ipedo to manage and integrate information from underwriting, accounting, claims processing and billing systems, Results Streamlines operations, improves response time to inquiries Billing Database Ipedo XIP Profitable Accounts Claims Status Expensive Claims Arrears Sales View Workflow View Claims View Acct. View Claims System Acct. System Web Services Policies ACORD Schema
  • 44.
    Sun - CustomerService Application Customer Service Challenges Combining XML messages, Web Services, databases, other data Connecting with external risk engine Providing rapid response to inquiries Managing huge data volumes Solution Embed Ipedo into Sun application to deliver risk analysis as a service Results Improves system uptime and availability and reduces incident volume and cost Customer Reference Data (Oracle) Risk Engine (Fair Isaac) System Information (XML) Preventive Services Ipedo XIP Hardware View Software View Account View Risk View Risk Assessment Advanced Diagnostics System Configurations Web Services JDBC
  • 45.
    The Ipedo DifferenceHigh Performance Cost-based Query Optimization Intelligent Caching Investment Protection Choice of data model: Relational (SQL) or XML (XQuery) Choice of SOA deployment model – WSDL/SOAP or REST One cohesive product Seamless Deployment Fully scriptable with complete SQL DDL support Integrated Data Lineage and Impact Analysis reporting Comprehensive APIs: Java, Web Services, .NET Highly Scalable Concurrency for hundreds of simultaneous users Easily accommodates additional data sources
  • 46.
    Additional Resources 2005EII Survey Guide to EII ROI Guide
  • 47.
  • 48.
    Enterprise Information Integrationis… … a new kind of virtual data integration technology that makes several remote data sources appear as one local database.
  • 49.
    Ipedo XIP: Integration+ Intelligence Vision: Intelligence without boundaries: Give business decision makers “information on demand, from anywhere, at any time” Product: Ipedo XIP integrates and manages information from disparate and complex data sources on demand
  • 50.
    EII Generates RealFinancial Benefits Developer Productivity EII allows developers to write to one, common interface to access multiple data sources, reducing the development cost for new applications What % of your IT budget is spent on data integration development? Analyst Utilization EII allows analysts to simplify the data collection process and spend more time producing information and less time gathering data How much time do your analysts spend collecting data? Increased Flexibility EII allows companies to respond quickly to changes in market conditions without incurring a major IT investment What effort is involved to link new data sources to existing reports or applications? Data Consolidation Using EII to create a virtual centralized database can cut the cost of data maintenance, and reduce the need for data replication How much does it cost to deploy each new data mart?
  • 51.
    Integration Platform Builtfor Developers/Integrators Data and Cache Management • TTL • Performance • Versioning Publishing and API Access • XML • JSP Tag Lib • JDBC/ODBC • .NET • WS/SOA Transformation & Virtualization • XML Schema • CSV • SQL <-> XML • Pipelines Connectivity • Relational Views • XML Views • SOA/WS Views • JMS Feeds • HTTP Views Query Management • XQuery • SQL • Cost-based Optimizer Rules Processor • SQL SP • XQuery Modules • Java Functions Security • ACLs • Roles Management • JMX • JSP • APIs GUI Tools • View Builder • Query Builder • Rules Builder
  • 52.
    EII Brings Advantagesto New Projects - Create data services layer that simplifies SOA MDM - Maintain consistent view of core business entities - Avoid political issues around data ownership SOA - Allow universal info exchange as Web Services Solution Key Advantages of EII Dashboards - Intelligently combine company-wide information - Provide view customization per context or user Risk Management - Extract key information across company data silos - Reduce cost of compliance BI / Reporting - Enable better reports with up-to-date information - Access diverse data sources via queries, not code
  • 53.
  • 54.
    Views Use MetadataExtensively Ipedo XIP View Output Schema Transformations Data Mappings Data Source Connections View Definitions of Fields Business Definitions Caching Policies Security Primary / Foreign Keys Security Data Source Metadata Source Repository Ipedo
  • 55.
    Metadata Flows Throughoutthe Framework Metadata Manager Enterprise Data Sources Introspection Ipedo XIP Metadata Repositories Metadata Mgmt/Mod Tools Databases Web Services XML Doc Collections Other Data Sources Supported by Ipedo Modeling Tools BI/Reporting Tools SOA Tools App Dev Tools JDBC/ODBC/SQL WSDL/UDDI JDBC/ODBC/SQL WSDL/UDDI XMI JDBC/ODBC/SQL JDBC/ODBC/SQL XMI/Proprietary
  • 56.
  • 57.
    Anhui Electric -Fraud and Risk Management Application Power Utility Billing Risk Management Dashboard Challenges Forecasting business risk No aggregated meter monitoring data to prevent rampant theft of power Lack of credit information on new customers Solution Use Ipedo to integrate energy meter collection system, customer databases and financial systems across multiple sites Results Significant reduction in fraud and theft; better management reporting Billing Database Customer Database Energy Meter Collection System Ipedo Credit View Theft View Risk View Forecast View
  • 58.
    Application Ring tone,music, and mobile game digital rights management platform Challenges Ability to address diverse content and transaction data types Adapting rapidly to changing market and customer requirements Scalability for millions of buyers and millions of content assets Solution Use Ipedo to integrate user, payment, and ring tone/game metadata for presentation through Web interface Results Allows rapid response to customers and easy integration of new content libraries into client Web sites Navio Card Navio Merchant Navio Store ASP Applications Report Server Title Manager Content Lockbox Digital Commerce Engine Ipedo XIP Ipedo XIP Ipedo XIP Navio - Rights-Based Commerce Web Storefront P2P Net
  • 59.
    HP OpenView -Customer Service Portal Application Customer Service Portal Challenges Flexible search interface Frequently changing content Scalability for global users High Availability Solution Use Ipedo to manage granular content and deliver in response to Web-based user inquiries Results Improve customer experience by directing users to relevant software downloads or reference materials Customer Service Marketing Databases (RDBMS) OpenView Web Site Ipedo XIP Product Info Customer Info Marketing Info
  • 60.
    Pharma R&D DataIntegration Ipedo Cuts Costs, Speeds Discovery BEFORE IPEDO AFTER IPEDO Too many data sources Unable to consolidate genomic data Data sources in proprietary formats Unable to analyze all genomic data Scientists manually research genes Scientists’ observations are not shared Ipedo XIP Observations Annotations Deliver one-stop shop for research data Capture scientists’ annotations Keep scientists’ information synchronized Views Integration Query Analysis Pipelining Presentation “ 10x researcher productivity gain.” Internal Data Warehouse GenBank BLAST MedLine Internal Data Warehouse GenBank BLAST MedLine Scientists Scientists Research Application Web Site Web-Based Queries Subscription Web Site ROI in Time to Market Research Portal
  • 61.
    British Telecom PriceBook October 2003
  • 62.
  • 63.
    Financial Services HasUnique Needs 360 degree view of the customer See across systems in subsidiaries/acquisitions Integrate data from different databases Combine live and historical data Satisfy regulatory requirements Cross-divisional reporting Granular security / data ownership
  • 64.
    Insurance Has UniqueNeeds Integrate underwriting, policy management, billing, claims systems Perform ad-hoc reports Leverage industry standard schemas Produce reports for regulators 360 degree view of the customer Cross-divisional reporting Post-merger/acquisition consolidation Granular security / data ownership
  • 65.
    Health Care HasUnique Needs HIPAA compliance Protect patient privacy - granular security model Mix historical and real-time information Customize reports for different uses: administration, insurance, treatment 360 degree view of the patient
  • 66.
    Retail Has UniqueNeeds Generate real-time sales information Track inventory across multiple locations Share information with supply chain Spot trends as they occur 360 degree view of the customer Cross-divisional reporting Post-merger/acquisition consolidation Granular security / data ownership
  • 67.
  • 68.
    Business Intelligence andReporting Use Business Objects, Cognos, MicroStrategy, or any BI tools to analyze data from: Relational databases Unstructured data sources Enterprise applications Web Services Combine historical information with real-time information from message queues or Web Services
  • 69.
    Ipedo Supports BOBJThree Ways Ipedo XIP BusinessObjects Universe Business Objects BI Platform Data Sources Data Mart
  • 70.
    Ipedo Bridges DataPartitions Application Ad-hoc reporting across separate databases using BI tools Challenges Overcoming size limitation on Sybase servers Shielding reports from changes in underlying database partitions Combining data from disparate sources into a single “Universe” Maintaining high-performance for complex queries Solution Use Ipedo to create virtual union views BI tools connect to Ipedo views Ipedo optimizes and federates queries to all databases Business Intelligence / Enterprise Reporting Tools Ipedo XIP All Customers View All Orders View All Sellers View ... Sybase1 Sybase2 Sybase10
  • 71.
    Ipedo Appears asa Database to Business Intelligence Applications In this example, Business Objects Web Intelligence sees Ipedo as a single relational database. Ipedo is really pulling data from three different relational sources.
  • 72.
    Ipedo Complements BOBJData Integration Real-time, query-based access to data sources Combining web services with other data sources Query optimization Validation of data in real-time Web Services Blending multiple web services Conditional invocation of web services in a specific order Business rules checking on web services data Caching of web services data for reuse
  • 73.
  • 74.
    Master Data ManagementCreate a master index that leverages Ipedo’s high performance engine Cost-based query optimization Union join pushdown Reduce data replication Customize departmental subsets of customer data to avoid data politics and ownership issues Perform data cleansing (through Ipedo partnership) Help improve the performance of the denormalization process and boost the flexibility in handling underlying schema changes
  • 75.
    Ipedo XIP forMDM BI/Reporting Client ODBC / JDBC SQL SQL SQL Data Sources Remote Tables Relational View Join data from multiple RDBMS using Master Customer Key in MDM Ipedo Customer View Join View Union View Master Customer Key SQL SQL CRM Data (DB2) Trading Data (Sybase) Master Data (DB2)
  • 76.
  • 77.
    Data Quality EnhancesEII Data Source Mappings Virtual View Mappings User Query on View EII Step: Join DQ Call: Matching Join Key EII Step: Filter DQ Call: Matching Other Columns Data Sources Live Views (seconds of latency) User Query on View EII Step: Filter, Other DQ Call: Cleanse or Match Data Cached Views (minutes to hours of latency) Caching Options: Scheduled Cache (Push) – Include cleanse/match option for admin User-driven Cache (Pull) – Include cleanse/match option after initial view Aligning data between different sources for efficient join operations Reconciling differences between data from different sources Cleaning up cached information for subsequent reuse
  • 78.
  • 79.
    Risk Management Presentcomplete set of information for analysis Real-time feeds from message queues (trading data, interest rates, etc.) Analyze complex XML documents (such as FpML or other trading schemas) and combine seamlessly with relational data Cache XML for subsequent query, analysis, and integration Compound Aggregate Indexes (CAI) track key data fields and automatically compute aggregates across different data feeds Maintain audit trail for compliance and SOX
  • 80.
    Ipedo for RiskAnalysis Financial View Ipedo XIP JDBC ODBC SQL Convert to Relational Tables Calculate Aggregates Message Queue Message Queue XBRL FpML XML Caching BI/Reporting Client Trading Data In XML Format Order View Risk View
  • 81.
  • 82.
    Ipedo Delivers EnterpriseReadiness Performance Cost-based Query Optimization Result-set Streaming Policy-based Caching Advanced Tuning Scalability Clustering Concurrency Security Data Governance
  • 83.
    Ipedo Joins HugeData Sets… SELECT CUSTOMER.C_NAME, CUSTOMER.C_CUSTKEY, CUSTOMER.C_ADDRESS, COUNT(ORDERS.O_ORDERKEY), SUM(ORDERS.O_TOTALPRICE ) FROM CUSTOMER, ORDERS WHERE (CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY) AND CUSTOMER.C_NAME = 'Customer#000000250' GROUP BY CUSTOMER.C_NAME, CUSTOMER.C_CUSTKEY, CUSTOMER.C_ADDRESS Select all ORDERS of a given CUSTOMER ORDERS and TIME_DIM tables from Oracle (Orders Management App) 750,000 rows CUSTOMER and NATION tables from SQL Server (CRM) 10 0 , 0 0 0 r o w s
  • 84.
    …And Delivers Sub-SecondPerformance Query Optimization: Search predicate is pushed down to remote RDBMS. Join predicate will be pushed into remote inner table if nested loop join is cost-effective. Number of rows of the where clause is taken into consideration in Cost-Based Optimization (CBO). Performance: average query time = 0.1 sec Query Plan:
  • 85.
    Ipedo Security ProtectsData and Views User and Group Level Hierarchical Access Control List Integration with external LDAP directories Document/Attribute-level filtering (node-level) JAAS-compliant encryption algorithm flexibility for 2-way secure authentication (from client, as well as server)
  • 86.
    Low Overhead inIT Environment Connection Pooling allows you to protect back-end data sources from excessive load Intuitive setup and administration means no DBA required to run Ipedo Runs on Windows, Linux, Solaris, HP-UX, AIX - any environment with a JVM Runs on Tomcat (included), WebLogic, WebSphere, JBoss, JRocket
  • 87.
    Professional Services SimplifiesImplementation Experienced Team SQL, XML, Java, Web Services Flexible Engagement Model Off-site development, non-invasive setup (1-5 days) On-site training for your team Sample Deployments Portal for global computer hardware company Trading application for commodity exchange Claims management system for insurance company Risk management system for power utility
  • 88.
  • 89.
    Ipedo XIP SQLPerformance Testing ORDERS and TIME_DIM tables from Oracle (Orders Management App) CUSTOMER and NATION tables from SQL Server (CRM) 100,000 rows 750,000 rows Testing conducted in July 2005 on dual-CPU Intel XEON 2.2GHz with 2.0Gb memory
  • 90.
    SQL Query #1SELECT CUSTOMER.C_NAME, CUSTOMER.C_CUSTKEY, CUSTOMER.C_ADDRESS, COUNT(ORDERS.O_ORDERKEY), SUM(ORDERS.O_TOTALPRICE ) FROM CUSTOMER, ORDERS WHERE (CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY) AND CUSTOMER.C_NAME = 'Customer#000000250' GROUP BY CUSTOMER.C_NAME, CUSTOMER.C_CUSTKEY, CUSTOMER.C_ADDRESS Select all ORDERS of a given CUSTOMER
  • 91.
    Query Plan #1and Performance Query Optimization: Search predicate is pushed down to remote RDBMS. Join predicate will be pushed into remote inner table if nested loop join is cost-effective. Number of rows of the where clause is taken into consideration in Cost-Based Optimization (CBO). Performance: average query time = 0.1 sec Query Plan:
  • 92.
    SQL Query #2SELECT TIME_DIM.YEAR, COUNT(ORDERS.O_ORDERKEY), SUM(ORDERS.O_TOTALPRICE ) FROM TIME_DIM, ORDERS, CUSTOMER WHERE (ORDERS.O_ORDERDATE=TIME_DIM.NATIVEDATE) AND CUSTOMER.C_CUSTKEY=ORDERS.O_CUSTKEY) AND ( ( CUSTOMER.C_CUSTKEY ) = CASE WHEN char(hour(current_timestamp)) = '24' THEN 250 ELSE 251 END )) GROUP BY TIME_DIM.YEAR Conditionally select a customer’s total order count and sum
  • 93.
    Query Plan #2and Performance Query Optimization ANSI SQL standard case-when-else conditional expression is pushed down to remote RDBMS. Column transitivity is supported to allow better query plan. e.g. (t1.c1 = t2.c2 and t2.c2 = constant)  (t1.c1=constant and t2.c2 = constant) Performance: average query time = 0.2 sec Query plan:
  • 94.
    SQL Query #3SELECT V_MERGED_CUSTOMER.C_NATIONKEY, sum(V_MERGED_ORDERS.O_TOTALPRICE) FROM V_MERGED_CUSTOMER, V_MERGED_ORDERS WHERE (V_MERGED_CUSTOMER.C_CUSTKEY=V_MERGED_ORDERS.O_CUSTKEY) AND (V_MERGED_CUSTOMER.C_MKTSEGMENT = 'MACHINERY ‘) GROUP BY V_MERGED_CUSTOMER.C_NATIONKEY Query efficiently across data partitions in Oracle and SQLServer: View v_merged_customer unions customer tables in both partitions. View v_merged_order unions order tables in both partitions.
  • 95.
    Query Plan #3and Performance Query Optimization Search predicates on views are pushed down to data sources Joins across union partitions are regrouped and pushed down to data sources, so that smaller joined result sets are combined in XIP Performance: average query time = 2 sec Query plan:
  • 96.
    SQL Query #4Query Optimization Shows stability and scalability in handling large result set with high concurrency without running out of memory. The first record is returned immediately without waiting for the entire result set to be processed. Ipedo support cursor fetching all the way through from client to Ipedo and to remote RDBMS. Query can be cancelled without fetching the entire result set, and resources are released immediately. SELECT ORDERS.O_ORDERKEY, ORDERS.O_ORDERDATE, ORDERS.O_CLERK FROM ORDERS Select large result set (750,000 rows)
  • 97.
    Query Performance #4Cursor is used throughout the fetch chain to achieve high responsiveness: Cursor is used from client to Ipedo: Client obtains first row : 0.174s Cursor is used from Ipedo to remote RDBMS: Database returns first row in 0.171 s. This shows cursor is used in back end when Ipedo fetches data from remote RDBMS.
  • 98.
    Ipedo XIP ReferenceArchitectures
  • 99.
    Ipedo XIP SizingConsiderations Number of Concurrent Users Query Load Number of queries Complexity of queries (number of joins) Amount of information being returned (row count) Additional Processing Data transformations (relational - XML) Rules processing (number and complexity of rules) High-Availability Requirements Clustering (configuration) Replication (writing or reading transaction logs)
  • 100.
    Basic EII ServerLoad Concurrent users: < 75 Queries: < 50/sec Server Configuration CPUs: 1-4 RAM: 2 G Typical Usage Enable departmental reporting across multiple sources
  • 101.
    Basic EII withFailover Ipedo XIP Primary Application-aware session pooling Ipedo XIP Secondary Server Load Concurrent users: < 75 Queries: < 50/sec Server Configuration CPUs: 1-4 RAM: 2 G Typical Usage Enable reporting across multiple sources for a business unit Ipedo can also use a clustering server for failover
  • 102.
    HA Clustered -“Shared-Disk” Architecture Shared Disk Ipedo XIP Primary Ipedo XIP Secondary LAN Hub Private Network Windows Advanced Server Clustering Server Load Concurrent users: < 75/node Queries: < 50/sec/node Server Configuration CPUs: 1-4 per node RAM: 2 G per node Typical Usage Enable reporting across multiple sources for an enterprise or build an SOA data services layer Ipedo also supports clustering software on Linux, Solaris, HP-UX (HA = High Availability)
  • 103.
    HA Clustered -“Shared-Nothing” Architecture SCSI Disk Ipedo XIP Primary Ipedo XIP Secondary LAN Hub Private Network Windows Advanced Server Clustering SCSI Disk Server Load Concurrent users: < 75/node Queries: < 50/sec/node Server Configuration CPUs: 1-4 per node RAM: 2 G per node Typical Usage Enable reporting across multiple sources for an enterprise or region, or build an SOA data layer Ipedo also supports clustering software on Linux, Solaris, HP-UX (HA = High Availability)
  • 104.
    EII Server Farm(Clustered for Load Balancing) Ipedo XIP Ipedo XIP LAN Load Balancer Ipedo XIP Server Load Concurrent users: < 75/node Queries: < 50/sec/node Server Configuration CPUs: 1-4 per node RAM: 2 G per node Typical Usage Enable reporting across multiple sources for an enterprise or build an SOA data services layer