Partner Webcast – Oracle Enterprise Data Quality for ODI: The importance of Quality in Data Integration - 30 May 2013
Upcoming SlideShare
Loading in...5
×
 

Partner Webcast – Oracle Enterprise Data Quality for ODI: The importance of Quality in Data Integration - 30 May 2013

on

  • 1,511 views

The importance of good quality data is not anymore referred only to customers, addresses, products. Beside these important data assets, there’s a growing interest in exploiting the new ‘wave’ of ...

The importance of good quality data is not anymore referred only to customers, addresses, products. Beside these important data assets, there’s a growing interest in exploiting the new ‘wave’ of data, like social, machine generated, unstructured, Big Data. Consequently also these data must be of High Quality.
The key to success in many Data Integration projects is simply including a Data Quality tool that demystifies DQ as a complex task. >Put the trust back in your data! With Oracle Enterprise Data Quality (OEDQ) collaborative and platform approach, we link the IT department and the Business, providing an easy and intuitive tool with the flexibility and productivity required by the IT department.
Along the updated Oracle Data Integration specialization, this webcast intends to help you to know how you could use next generation Data Quality with ETL/ELT technology in projects of migrations and consolidation, Business Intelligence & Data Warehouse.

Read More

Statistics

Views

Total Views
1,511
Views on SlideShare
1,508
Embed Views
3

Actions

Likes
0
Downloads
56
Comments
0

2 Embeds 3

http://digg.com 2
https://translate.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Partner Webcast – Oracle Enterprise Data Quality for ODI: The importance of Quality in Data Integration - 30 May 2013 Partner Webcast – Oracle Enterprise Data Quality for ODI: The importance of Quality in Data Integration - 30 May 2013 Presentation Transcript

    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration1CUSTOMER LOGO“This slide format serves to call attention to a quote froma prominent customer, executive, or thought leader inregards to a particular topic.”NameTitle, Company Nameblogs.oracle.com/IMC
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration2
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration3Oracle Data IntegrationMoving Datato TransformBusinessOracle Data Integration PlatformEnterprise Data QualityWebcast 30th of May 2013Ugo PollioBusiness Development Oracle EMEA
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration4Agenda Why Data Quality Why Oracle Competition Use cases Conversation with customers
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration5Why Data Quality?
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration6New Requirements in Data IntegrationReal-timeAnalyticsAny Data,Any SourceZero Downtime,High Availability
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration7Data DelugeWhat Analysts are saying about Growing Data Volumes & Complexity“External data sources areproliferating - On average,organizations are integrating 14 externaldata sources, up from 11 a year ago.- Aberdeen Group“New data stored by enterprisesexceeded 7 exabytes of data globally in2010 and new data stored by consumersexceeded an additional 6 exabytes..”- McKinsey Global Institute“As data growth and complexityaccelerates, companies should focus onquality assured data exchange (ensuredata consistency and accuracy from thepoint of entry.”- Aberdeen Group“40% projected growth in global datagenerated per year vs 5% growth inglobal IT spending.”- McKinsey Global Institute7
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration8CompaniesYour Data is Changing• 240 businesses will changeaddresses• 150 business telephone numbers willchange or be disconnected• 112 directorship (CEO, CFO, etc.)changes will occur• 20 corporations will fail• 12 new businesses will open theirdoors• 4 companies will change their nameSource: D&B, US Census Bureau, US Department of Health and Human Services, Administrative Office of the US Courts,Bureau of Labor Statistics, Gartner, A.T Kearney, GMA Invoice Accuracy Study• 5,769 individuals in the US willchange jobs• 2,748 individuals will changeaddress• 515 individuals will get married• 263 individuals will get divorced• 186 individuals will declare apersonal bankruptcyIndividualsMaster data changes at rate of 2% per monthProducts• On average 20% duplicates inproduct data• 90% product introductions fail• Retailers lost 40 billion or 3.5% oftotal sales lost each year due toitem info inefficiencies• 60% error rate for all invoicesgenerated• Global Data Sync will realize 30%lower IT costsIn one hour… In one hour… In one year…Compounded, 2% monthly change is 27% per year, 61% in two years, 104% in three years!!!
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration9Data Quality market outlook Gartner says (6-SEPT-2012):– Demand for data quality software is rising fast as moreorganizations seek to support data governance initiatives,application modernization projects and master datamanagement (…).– Organizations are also using data quality tools to support amuch wider range of use cases than in previous years,said Ted Friedman, vice president at the Stamford, Conn.-based analyst firm and author of the report.http://searchdatamanagement.techtarget.com/news/2240162796/New-Gartner-Magic-Quadrant-finds-demand-rising-for-data-quality-toolsGrowing market727800950145,4 160 190020040060080010002009 2010 2011DQ market worldwide (USD mil)DQ market(USD mil)DQ in EMEA(20% estim.)121618051015202009 2010 2011CAGR % 5yrsCAGR% 5yrs
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration10Business Impact of Data QualityWith Bad Data With Good Data• Reduced ROI• Increased project risk, time and cost• Expensive downstream consequences– wrong shipment, wrong invoices,incorrect parts…• Increased ROI on existing systems• Increased agility• Increased efficiency• Increased customer satisfaction• Increased scalability“Only 30% of BI/DWimplementations fully succeed.The top two reasons for failure?Budget constraints and dataquality.”“Data integration and data quality arefundamental prerequisites for thesuccessful implementation of enterpriseapplications, such as CRM, SCM, andERP.” ”“#1 reason CRM projects fail:Data Quality”
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration11Your Data Contains Errors and InconsistenciesVariation orErrorExampleVariation orErrorExampleSequence errors • Mark Douglas or Douglas MarkTranscriptionmistakes• Hannah, HamahInvoluntarycorrections• Browne – BrownMissing or extratokens• George W Smith, George Smith, SmithConcatenatednames• Mary Anne, MaryanneForeign sourceddata• Khader AL Ghamdi, Khadir A.AlGamdeyNicknames andaliases• Chris – Christine, Christopher, TinaUnpredictableuse of initials• John Alan Smith, J A SmithNoise• Full stops, dashes, slashes, titles,apostrophesTransposedcharacters• Johnson, JhonsonAbbreviations• Wlm/William, Mfg/Manufacturing Localization • Stanislav Milosovich – Stan MiloTruncations • Credit Suisse First Bost Inaccurate dates• 12/10/1915, 21/10/1951, 10121951,00001951Prefix/suffixerrors• MacDonald/McDonald/DonaldTransliterationdifferences• Gang, Kang, KwangSpelling & typingerrors• P0rter, Beht Phonetic errors • Graeme – Graham
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration12An example of a different use caseAttributeName.1 OC_NameAttributeValue.1 at_ns:.oc.ERoss3g1AttributeName.2 IdentifierAttributeValue.2 1546863AttributeName.3 Target_EntityAttributeValue.3RCROOT at_ns:.oss.3g1RCROOT SNW NISTE05RNC NISTE05AttributeName.4 Event_TypeAttributeValue.4 QualityofServiceAlarmAttributeName.5 Managed_ObjectAttributeValue.5RCROOT at_ns:.oss.3g1RCROOT SNW NISTE05RNC NISTE05AttributeName.6 Probable_CauseAttributeValue.6 ThresholdCrossedAttributeName.7 SeverityAttributeValue.7 WarningAttributeName.8 Event_TimeAttributeValue.8 18/12/2012 19:10:16AttributeName.9 StateAttributeValue.9 OutstandingAttributeName.11 Notification_IDAttributeValue.11 3589640175"Value.NmsTags.AlarmId 38444174nValue.NmsTags.ProposedRepair nValue.NmsTags.ManagedObjectkalkan,SubNetwork=ONRM_ROOT_MO,SubNetwork=NISTE05,MeContext=NISTE05,ManagedElement=1,RncFunction=1,UtranCell=WIS04296nValue.NmsTags.SpecificProblem UtranCell_RrcEarlyRejectnValue.NmsTags.Class RCROOTn{"OC_Name": "at_ns:.oc.ERoss3g1","Identifier": "1546863","Target_Entity": "RCROOTat_ns:.oss.3g1RCROOT SNW NISTE05 RNC NISTE05","Event_Type":"QualityofServiceAlarm","Managed_Object": "RCROOT at_ns:.oss.3g1RCROOT SNWNISTE05 RNC NISTE05","Probable_Cause": "ThresholdCrossed","Severity":"Warning","Event_Time": "18/12/2012 19:10:16","State":"Outstanding","Additional_Text":"UtranCell_RrcEarlyRejectnnstart_nms_tagsn@AlarmId=38444174n@ManagedObject=kalkan,SubNetwork=ONRM_ROOT_MO,SubNetwork=NISTE05,MeContext=NISTE05,ManagedElement=1,RncFunction=1,UtranCell=WIS04296n@SpecificProblem=UtranCell_RrcEarlyRejectn@ProposedRepairAction=n@Class=RCROOTnend_nms_tags nnSource:OSSRC_FM","Notification_ID": "3589640175"}Parse & classify complex unstructured,semi-structured dataTransform in structured data
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration13Why Oracle?A product overview
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration14 Very easy to use Intuitive Modular Great productivity Robust FlexibleMost common users’ commentsData Quality demistified
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration15What do you need to know about EDQ?Integrated Solution for All Data Quality ProblemsBroadest DQ offeringIntuitive GUI and easy to use toolProfiling, standardization, advanced parsing, matching, casemanagement, remediation, governanceMost usable DQ offeringCompletely integrated offering – designed to work togetherDesigned for business and technical usersTransparent operation and results – no black boxesLeverages best practices, high productivity, solutionpackaging for full reusabilityPervasive operation for enterprise data qualitygovernanceScalable and flexible platform, java basedWithin legacy systems and MDM HubsAs part of migration/system loadAs part of data movement/transferProfileStandardizeMatchGovernQuickly understand data contentDrive conformance to standardsIdentify & merge duplicatesMonitor effectiveness & resolve problemsCommonAccess/UI
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration16Introducing Enterprise Data QualityDQ-Based SolutionsDomain KnowledgeBusiness Solutions• Customer-delivered• Partner-delivered• Oracle-deliveredApplication ConnectorsData Quality Platform• Complete range of DQ capabilities• Best-of-breed capabilities for party and productdata• Easy to use, intuitive• Open, tunable, flexiblePre-Built Solutions• Any scope – components to end-to-end solutions• Any pre-built/reusable item– Processes, methods– Knowledge, reference data– Application integrationEnterprise Data QualityDashboardsParty DataExtensionsMatch/MergeGovernanceProduct DataExtensionsStandardizationProfile and Audit
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration17Collaboration Across User CommunitiesData AnalystsBusiness AnalystsExecutives & StakeholdersDirector UsersDirectorData StewardsDirectorExecutivesDirectorReviewers
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration18Build-out Full DQ Process
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration19Data Improvement & CleansingUse profiling results to create your own dataimprovement rulesUse provided processors for common taskssuch as address standardization• Fully configurable data transformation rules• Operates in both Batch and Real-Time• Full control over data updates• Original data always preserved (and all steps in between)• Source data may either be staged and processed or ‘streamed’ through the process
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration20Matching – Duplicate Identification and Prevention• Designed for business users• Flexible matching engine for any data with many comparison algorithms• Provided template match processors for individual, entity and address matching• Easy reuse of configured match processors• Fully configurable outputs (Links, Groups, Master and Slaves, Best Record)• Operates in both Batch and Real-Time• See Match Essentials deck for more information on MatchingPre-built rulescan beswitched onand offand/orcustomized
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration21EDQ Address VerificationEDQ Address Verification Server• Verify – Get the address correct• Worldwide address cleansing – over 240 countries – all populated countries on earth• The most advanced error-tolerant parsing algorithms• Geocode – Attach a location to a correct address• Generates a latitude/longitude coordinate for any address worldwide• Leverages the most comprehensive multi-source geographical reference dataGlobal Knowledge Repository Data Packs• Parse• Transliterate• Validate• FormatVerifyAddlatitude/longitudecoordinatesGeocodeEDQ Parse andStandardizeEDQ Profile andAuditEDQ Match andMerge
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration22The Oracle Information Platform Vision Simplify the delivery of TrustedBusiness Information Collaborate across differentareas of the business Reduce the risk of deployingseparate components Best of Breed and Integrated Accelerate time to value A scalable platform from smallbusiness to the enterpriseA Complete, Open and Integrated Information PlatformEnterprise Data QualityOracle Data IntegrationOracle BusinessIntelligenceOracleEPMOracle MDMOracle RDBMSOracle Master DataManagementVisualiseDeliverTransformCleanUnderstandDiscover
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration23EDQ Competitive Product AnalysisCapabilities, Features Oracle Enterprise Data QualityProduct Environment, GUI, Look & Feel • EDQ GUI extremely intuitive and easy to use.• Collaborative environment for different roles and users.• EDQ is codeless. No need of code implementation. For instance even Regular Expressions, canbe directly written without any function usage or interpretation.Profiling • EDQ has seamless profiling capabilities. For instance, you can create a matching process thatincludes profiling capabilities in order to profile matched/unmatched data and derive furtherinsights.Auditing • Flexible rules management from Basic to Complex.• No specific language (sql or others) required• Tens of rules provides equivalence to hundreds of rules in other productsMatching • EDQ matching and parsing capabilities are quite more flexible and configurable than Infa andIBM, where you can’t easily extend the rule set for matching and parsing• Multiple clustering capabilities in a single pass• Graphical summary of matching rules, gives at a glance clear understanding of all matchingcriteriaAddress Validation • Frequent “240 countries” statement: the number of countries in not that meaningful. Most ofour competitors claim it, but more importantly, we cover all populated countries on Earth with agreater level of detail than our nearest competitors• Provide Geocode data for 240 populated countries, much more compared to our competitors.• Provide out of the box statistical capabilities for the address validation process, useful forclassifying validated address and summarize results.Architecture • EDQ is java based, you can install it almost everywhere• EDQ doesn’t require client installation. Just an URL where download a WebStart App.• EDQ relies on Oracle Technology (DB, Weblogic), competitors don’t.
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration24DIY Versus Pre-Integrated – It’s Your Choice...Engineered to Work TogetherModelsDataQualityETL MDMDataWarehouseBI+ + + + ++ + + + +– You invest in building integration between each component OR– Oracle invests in integration between each component
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration25Investment in Innovation and IntegrationMORE THAN$24B IN R&DSINCE 2004$1.3B$1.5B$1.9B$2.2B$2.7B$2.8B$3.3B$4.5B $4.5BFY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY12Figures in GAAP
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration26Use Cases
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration27Data Quality Use Cases: Cross - Industry• Single view of high quality customer data drives accurate customer insight andimproved marketing effectiveness• Supports compliance and reporting KYC requirements• Single view of citizen for better internal information sharing, service delivery,licensing, provision of child care, and fraud detection• Reduce costs through system rationalisation• Harmonizes customer data from multiple channels to improve sales andmarketing effectiveness• Enhance online product search for ECommerceRetail• Improves customer insight for revenue optimization and targeted customerretention• Effective compliance and risk mitigation for next generation servicesTelco• Expands understanding of network assets and customer delivery points• Improves management of regulatory compliance and reporting requirementsUtilitiesHealthcareGovernmentFinancialServices• Delivers a comprehensive view of patient for care and billing• Manages patient, epidemiology, diagnosis and treatment data quality acrosssystems and organizations
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration28• Reduce ODI Implementation Time and Risk– 50% of data warehouse/BI projects have limited acceptance or are outright failures asa result of lack of attention to data quality issues– ETL mappings should not be solely developed based on specifications– Data Profiling helps uncover defects, patterns, formats early in the ETL developmentprocess– Use EDQ Profiling to analyze and understand your data and required mappings• Populate a Data Warehouse with High Quality Data– Avoid making poor decisions based on poor data (avoid garbage-in, garbage-out)– Platform for Data Governance/Data Stewardship and ongoing quality improvement– Engage business users in defining and implementing appropriate business rules– Use EDQ Batch Processing to deliver accurate, consistent and complete dataCore Use Cases with ODI
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration29 High reusability, Faster ROI, thanks to ability todeploy common solutions and best practices,enforce Data Monitoring Better scalability, thanks to high re-usage of bestpractices, standardized developments, betterreadability of deployed solutions Improved collaboration between IT and Business,using a collaborative platform Core functionalities (matching, standardization)significantly improved thanks to data remediation,matching review process initiativeBenefitsData Consolidation, Data MigrationExploit the potential of your data assetEDQ ODIReal-Time
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration30Data ConsolidationHow to spot•Customers who urge to have aconsolidated view of theircustomers, products, referencedata.•Business impacts•Useless or ineffective, not intime reporting•Applications integrationchallenges•poor customer services•ineffective marketingcampaigns•Symptoms•Many sources•IT projects slow down or taketime to start•Evident inconsistencies andgaps of information acrossLoBsWhy Oracle•Position ODI+EDQ as best toolfor heterogeneousenvironment•EDQ ease of use, GUI VERYfriendly•ODI best in ETL/ETL forscalability and productivity•EDQ, ODI great reusability(ODI KMs, EDQ Processorsand packaging)•EDQ, ODI modern platformjava based•EDQ, ODI leverage end 2 endsolutions, Engineered Sytems,Oracle Database, DB Options,WeblogicBenefits•Fast ROI: as soon as data areconsolidated and cleanedthere’s a direct positive effecton insisting applications,reporting, etc.•IT projects are faster and risk,contingency are under control
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration31 Increased data accuracy and consistency, sharpervision of the business No more costs for manual data cleansing Lowering risks in production due to stops andissues Faster time to value and go live when using datafrom DWH for new marketing, sales initiatives, ITprojects Complement Reporting and Dashboards with DQmetrics, trends, KPIs. Entrust your insights,discover new onesSolution & BenefitsTrustable DWH and BIGet consistent measures to your business and decisionsEDQ ODIODIEDQEMPDEPTDIMFACTDIMDIMDIMODSSchemaDWSchemaGoldengateODI
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration32Next Gen Data WarehouseChange data into valuable Information The Business Issue– BI Reports are not trustable, because of thestate of source data Reduce risks– Improve data quality by integrating cleansingas part of the process– Eliminate data redundancies Improve Business Insights– Improved business insight with improveddata quality– Better profiling of data to eliminate gaps ininsightProfiling• Investigate, Analyze, AuditCleasning• Standardize, Enrich, DeduplicateControl • Govern over timeDo not trust thisinformation!
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration33Datawarehouse and Business IntelligenceHow to spot•Business users don’t trust ITbecause of issues with BIreporting•IT and/or business claims BIplatform isn’t valuable or useful•Business impacts•Useless or ineffective reporting•Inconsistent views on forecast,sales, supply chains, etc.•Lack of insights, businessmodernization•Symptoms•IT struggles to respect SLAs•IT spend a lot of time in planningrollbacks because of bad data,ending up sometime with abackup restore (huge impacts)•IT projects slow down or taketime to start•Evident inconsistencies and gapsof information across LoBsWhy Oracle•Position ODI+EDQ as best tool forheterogeneous environment andfor Datawareouses (Oracle isleader in DWH)•ODI and EDQ as strategic toolsfor Oracle, embedded in Oracleecosystem•EDQ ease of use, low learningcurve, easy to be adopted byBusiness Users•Doesn’t require a client installation•EDQ provide stats, KPIs, metricstaht can be embedded in any BIplatform•EDQ, ODI leverage end 2 endsolutions Engineered Sytems,Oracle Database, DB Options,WeblogicBenefits•Business side•Better decisions•Time to value•Faster start-up newmarketing/sales initiatives•Faster response to businessevents•Ensure Company good reputation•Cost savings•Cross sell – Up sell•IT Dept side•Ensure SLA•Faster start-up for IT projects•System Reliability, thenconfidence in IT•Resource utilization effectiveness•Meet Business User’sexpectations
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration34Bad data awareness• Customer is not completely aware about bad data impacts within theOrganization. Let show them how to answer these 3 question:How do I know I have bad data?What is the business impact?What should I do about it?How to spot• Profiling day• Prove EDQ by analyzing customer data with the help of a business user• Show them all findings and relate them with business issuesWhy Oracle• Falls down into previous use caseBenefits
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration35Q&A
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration36CUSTOMER LOGO“This slide format serves to call attention to a quote froma prominent customer, executive, or thought leader inregards to a particular topic.”NameTitle, Company Nametwitter.com/oracleimc
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration37EDQLive demo
    • 38 Copyright © 2012, Oracle and/or its affiliates. All rightsreserved.Confidential – Oracle RestrictedEDQ With Oracle Data Integrator: Use CasesSourcesTarget(s)E.g. DataWarehousesuch as ExadataOracle DataIntegratorData ProfilingAnalyze and understand datato build ODI mappingsAutomated ProcessesDe-duplication, complexcleansing and parsinginvoked in ODI workflowMeasure Ongoing Data QualityAssess quality of datain target system. How wellis ETL working?Enterprise DataQuality
    • 39 Copyright © 2011, Oracle and/or its affiliates. All rightsreserved.Confidential – Oracle RestrictedEDQ and ODI: Complimentary Features• EDQ complements ODI in the following areas– Data Profiling– Semantic/Contextual Data Parsing and Standardization– Complex Matching and Merging of various entities: individuals, households,products etc.– Data Deduplication– Address Validation & Geolocation
    • 40 Copyright © 2011, Oracle and/or its affiliates. All rightsreserved.Confidential – Oracle RestrictedEDQ and ODI: Comprehensive Data QualityProcessSourcesOracle Enterprise Data QualityParsing Standardization Cleansing Matching MergingTargetsOracle Data IntegratorE-LT/ETL Process- Continuous QualityMonitoring- Quality Alerts4Create newData Quality Rules2- Add Data Qualityto E-LT/ETL Flow3Profile Data1
    • 41DesktopRepositoriesInformation Management infrastructureShared Infrastructure for ODI & EDQODI StudioOperatorDesigner TopologySecuritySources and TargetsLegacy ApplicationsERP/CRM/PLM/SCMFiles / XML DBMS DW / BI / EPMJVMJava EEApplicationODI SDKWebLogic 11g / Application ServerData Sources Connection PoolWeb Service ContainerODIPublic WSDataServicesFMW ConsoleODI Plug-inServlet ContainerODI ConsoleJava EEApplicationODI SDKRuntime WSJava EE AgentJVMRuntime WSStandaloneAgentEDQ Repository EDQ ResultSchemaEDQ EngineEDQ WSEDQ MatchReviewEDQ Case MgmtService BusEDQ LaunchpadDirectorAdministrationConsoleMatch ReviewODI MasterRepositoryODI WorkRepository #nODI WorkRepository #1Case Mgmt…ODI Server Mgmt EM MonitoringEDQ Server Mgmt
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration42EDQ system sizing The key is to understand the use case, or purpose, of the system and the tasks it is to perform. If youcan answer the following questions, PM can give a ballpark answer to processor sizing: What is the basic use case?– Business purpose?– Technical functions – profile, parse, standardize, transform, match, merge, real-time or batch operation etc.?– How will the results be used? How many tables & columns are involved? How many rows? Is there a time window for the operation(s)? Is there a requirement that there will be multiple environments for production, development, QA etc.
    • 43How ODI and EDQ work togetherExample: Loading a Slowly Changing DimensionStagingSources TargetCustomers ProspectsStg_CustomersDIM_CustomersODIextractsStg_Valid_CustomersODIloads123E-LTEDQStandardize, Parse& Match w reference data2
    • 44SRC_PROSPECTSNAME ADDRESS CITY STATE ZIP PHONE COUNTRY LEAD_CREATEDMr Norm G Desmond 1052 Ala Moana Blvd Honolulu HI 96814808 555-1127 USA 2000-12-15Timothy Johnson 1020 NW 63rd St Oklahoma City OK 73116405 555-1175 USA 2000-12-15Dr Phillip O Oxenberg1710 287 Business WSuite 150 Waxahachie TX 75165972 555-2877 USA 2001-10-20Dr Sheila T Bergin 103 N 50th Road Omaha NE 68132402 555-3141 USA 2002-05-06Maxx Zaphrey 3828 South First St Austin TX 78704512 443-1311 USA 2007-12-20Lawrence Getty825 E. Rundberg LaneSte B1 Austin TX 78753512 836-5472 USA 2007-12-20Step 1ODI extracts from source & stageSTG_CUSTOMERSName Address City Postcode Country Account_num Acct_rep TerritoryDr SimonBrennan 11 Abotsford Street London N153BT UK UK02306 1-DDE UK-LONMiss KylieBrennan 11 Abottsford Street Londn N153BT UK UK02307 1-FMM UK-LONSimon andKaren Brennan 11 Abottsfurd Ave London N153BT UK UK02308 1-FMM UK-LON... ... ... ... ... ... ... ...Extracts data from heterogenoeus sourcesCUSTOMERS.XLSTotal_Orders Name Phone Address1 City Postcode Country Account_N Acct_rep Territory5500.36Dr SimonBrennan 01249 44287811 AbotsfordStreet London N153BT UK UK02306 1-DDE UK-LON5500.36Karen KBrennan 01249 44287911 AbotsfordAvenue London N153BT UK UK02307 1-GEB UK-LON5500.36Miss KylieBrennan 01249 44287911 AbottsfurdStreet Londn N153BT UK UK02307 1-FMM UK-LON5500.36Dr SimonBrennan 01249 442878 11 Abottsford St Lodnon N153BT UK UK02306 1-GEB UK-LON5500.36Simon andKaren Brennan 01249 44287311 AbottsfurdAve London N153BT UK UK02308 1-FMM UK-LONODI
    • 45Step 2EDQ Cleanse Staged DataDIM_CUSTOMERSName Address City Postcode Country Account_num Acct_rep Acct_Status Territory ClrecidDr SimonBrennan11 AbottsfurdAvenue London N153BT UK UK02306 1-DDE ACTIVE UK-LON U21015STG_CUSTOMERSName Address City Postcode Country Account_num Acct_rep TerritoryDr SimonBrennan 11 Abotsford Street London N153BT UK UK02306 1-DDE UKMiss KylieBrennan 11 Abottsford Street Londn N153BT UK UK02307 1-FMM UK-LonSimon andKaren Brennan 11 Abottsfurd Ave London N153BT GB UK02308 1-FMMUKlondonSTG_VALID_CUSTOMERSName Address City Postcode Country Account_num Acct_rep TerritoryDr SimonBrennan 11 Abottsford Ave London N153BT UK UK02306 1-DDE UK-LONMiss KylieBrennan 11 Abottsford Ave London N153BT UK UK02307 1-FMM UK-LONSimonBrennan 11 Abottsford Ave London N153BT UK UK02308 1-FMM UK-LONKaren Brennan 11 Abottsford Ave London N153BT UK UK02308 1-FMM UK-LONStandardize, Parse Match w reference data, Address Ver.EDQThis record contains two different customersEDQ can generate a new record from the original oneCleansed dataBad data
    • 46Step 3ODI Loads DIM_CUSTOMERS(Slowly Changing Dimension)DIM_CUSTOMERSName Address City Postcode Country Account_num Acct_rep Territory Acct_Status ClrecidDr SimonBrennan11 AbottsfordAvenue London N153BT UK UK02306 1-DDE UK-LON CLOSED U21015Miss KylieBrennan11 AbottsfordStreet London N153BT UK UK02307 1-FMM UK-LON ACTIVE U21018Dr SimonBrennan11 AbottsfordAve London N153BT UK UK02306 1-FMM UK-LON ACTIVE U21016KarenBrennan11 AbottsfordAve London N153BT UK UK02308 1-FMM UK-LON ACTIVE U21017STG_VALID_CUSTOMERSName Address City Postcode Country Account_num Acct_rep TerritoryDr SimonBrennan 11 Abottsford Ave London N153BT UK UK02306 1-DDE UK-LONMiss KylieBrennan 11 Abottsford Ave London N153BT UK UK02307 1-FMM UK-LONDr SimonBrennan 11 Abottsford Ave London N153BT UK UK02308 1-FMM UK-LONKaren Brennan 11 Abottsford Ave London N153BT UK UK02308 1-FMM UK-LONCleansed dataInserted dataUpdated dataODI Loads using SCD Type2 IKM:1) update & close old record2) create new one ACTIVE
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration47
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration48CUSTOMER LOGO“This slide format serves to call attention to a quote froma prominent customer, executive, or thought leader inregards to a particular topic.”NameTitle, Company Nameblogs.oracle.com/IMC
    • Copyright © 2012, Oracle and/or its affiliates. All rights reserved. #OracleDataIntegration49