Sound Customer Data Quality for CRM Manoj Tahiliani, Senior Manager, Customer Hub Strategy
The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Data Quality – Pains, Drivers and ROI
Siebel Data Management Solution
Data Quality Products
<Insert Picture Here>
“ Data quality is a Business Issue”
Virtually all enterprises are experiencing a significant amount of pain directly attributed to data quality issues .
Significant amounts of wasted labor and lost productivity translate into direct financial losses to the business.
Some enterprises that have measured the impact have found they are losing multiple millions of dollars each year as a result of poor data quality.
Velocity of Data Change is Staggering
240 businesses will change addresses
150 business telephone numbers will change or be disconnected
112 directorship (CEO, CFO, etc.) changes will occur
20 corporations will fail
12 new businesses will open their doors
4 companies will change their name
Companies Source: D&B, US Census Bureau, US Department of Health and Human Services, Administrative Office of the US Courts, Bureau of Labor Statistics, Gartner, A.T Kearney, GMA Invoice Accuracy Study; 2 Data: An Unfolding Quality Disaster, Thomas C Redman, DM Review Magazine August 2004, Mintel Global New Products Database (GNPD), 2007. CNNMoney.com 2006, 3 Quality is Free, Philip Crosby
5,769 individuals in the US will change jobs
2,748 individuals will change address
515 individuals will get married
263 individuals will get divorced
186 individuals will declare a personal bankruptcy
Individuals “ If bad data impacts an operation only 5% of the time, it adds a staggering 45% to the cost of operations.” 2 “ Poor data quality cost business’ 10% to 20% of revenue!” 3 Change of Circumstances
4.7 Million Marriages
1.53 Million First Births
2.04 Million First-time Home Buyers
1.9 Million Divorces
43 Million Residential Moves
1.4 Million Work Retirements
In one hour… In one hour… In one year… Master data changes at rate of 2% per month.
7 Questions About Your Data
Have data initiatives failed or been delayed due to unreliable data?
Do you always deliver the right product to the right customer?
How many marketing pieces are un-delivered or un-answered?
How much time is spent in reworking inaccurate data?
Do you face difficulties with regulatory compliance?
Is customer satisfaction going down?
Do you distrust your data to take critical decisions?
Poor Data Quality is the #1 enemy of CRM Solutions Out of Date Rapid changes in a dynamic society: marriages, divorces, births, deaths, moves Garbage Typos, misspellings, transposed numbers, etc. Fraud Purposeful misrepresentation of data: identity theft, wrong information (bankruptcies, occupation, education, etc) Missed Opportunities Information that we do not know about (customer relationships, up-sells, cross-sells)
Increased the time to bring new projects and services to market
Proliferation of data problems from silos to more applications
Heightened credit risk costs
Potential non-compliance risk
Increased report generation costs
Measuring actual ROI achieved
Example of Customer Data Quality Issue A Simple Customer Table Sample Name Address City State Zip Phone Email Bob Williams 36 Jones Avenue Newton MA 02106 617 555 000 [email_address] Robert Williams 36 Jones Av. MA 02106 617555000 Burkes, Mike and Ilda 38 Jones av. Nweton MA 02106 617-532-9550 [email_address] Jason Bourne, Bourne & Cie. 76 East 51 st Newton MA 617-536-5480 6175541329 … … … … … … … Mis-fielded data Matching Records Typos Mixed business and contact names Multiple Names Non Standard formats Missing Data
20 Common Errors & Variation (1) Variation or Error Example Sequence errors
Mark Douglas or Douglas Mark
Browne – Brown
Mary Anne, Maryanne
Nicknames and aliases
Chris – Christine, Christopher, Tina
Full stops, dashes, slashes, titles, apostrophes
Credit Suisse First Bost
Spelling & typing errors
20 Common Errors & Variation (2) Variation or Error Example Transcription mistakes
Missing or extra tokens
George W Smith, George Smith, Smith
Foreign sourced data
Khader AL Ghamdi, Khadir A. AlGamdey
Unpredictable use of initials
John Alan Smith, J A Smith
Stanislav Milosovich – Stan Milo
12/10/1915, 21/10/1951, 10121951, 00001951
Gang, Kang, Kwang
Graeme – Graham
Two Facts about Data Quality
The Data Quality Challenge is an iceberg
The biggest DQ threats are the ones we do not see.
Data Profiling lowers the water line and draws a clear view of the quality issues
Data value decays
Data is an asset which value decays over time
Business events can make this worse
M&A, new applications, new products, new contact files, etc
Quality is not a one shot process but a constant effort in the enterprise processes.
Data Quality needs to be pervasive and continuous .
<Insert Picture Here> Siebel Data Management Solution
Data Management - Deployment Options Middleware Application Integration Architecture Middleware Application Integration Architecture CRM Web site Call Center SFA Partner Fusion App Fusion App Call SCM ERP2 Legacy ERP 1 MDM Fusion App Call SCM ERP2 Legacy ERP 1 Partner Data Mgmt Layer
Components of Siebel Data Management Trusted Customer Data Web Services Library Publish & Subscribe Transports & Connectors Authorization Registry Profile & Correct History & Audit Privacy Mgmt Events & Policies Import Workbench Identification & Cross-Reference Source Data History Survivorship Parse Cleanse & Standardize Enrich Manage Decay Match & Merge / Unmerge Roles & Relationships Party Vertical Variants Related Data Entities Hierarchy Management
<Insert Picture Here> Data Quality Products
Data Quality Functionality in a Glance Profiling Cleansing Matching Enrichment Understand data status, deduce patterns Tel# is null 30% LName + FName (Asian Countries); FN+MN+PN+LN (Latin); Addr = #, street, city, state, zip, country; St, Str = Street (ENU/DEU); Spot and correct data errors; transform to std format/phrase Identify and eliminate duplicates Haidong Song = 宋海东 = Attach additional attributes and categorizations Haidong Song: “single, 1 child, Summit Estate, DoNot Mail” Functionality Customer Data example Comprehensive data quality Feature Batch and Real-time
New Data Quality Products
Introducing New Products to provide full spectrum of information quality functions:
Oracle Data Watch & Repair
Ongoing Discovery of Actual state of Master Data
Oracle DQ Cleansing Server :
ASM (Address Standardization Module)
Integrated single engine– supports all countries
Oracle DQ Matching Server :
Full Administration Access and increased level of support
Improved performance and enhanced tuning capability
New Data Quality Products Matching Engine Data Quality Matching Server Data Quality Cleansing Server Administration UI / Rules Editor Improved performance 18 Languages 52 Languages Address Standardization Module 240 Languages support Data Quality Profiling Profiling Console & Engine Old Offering (SSA) New Offering
Profiling Cleansing Matching Enrichment Comprehensive data quality
Oracle Data Watch and Repair
Ongoing auditing prevents data decay, ensures continuous quality
Non intrusive profiling across existing applications/databases
Quickly narrow in on anomalies
Generate rules to repair problems
Edge Application (no Upgrade impact)
Out of the box connector to Siebel CRM
Profiling Ongoing Discovery of State of your Data
Advanced Validation and Standardization of addresses in more than 240 countries
Scalable high performance
Integrated single engine– supports all countries
Edge application (no upgrade impact)
Profiling Cleansing Matching Enrichment Comprehensive data quality >240 Countries One API Oracle DQ Cleansing Server Standardize & Validate against References
Not just number of records but also volumes of Txns
In use on systems with > 800 million records
> 250,000 txn/hour on large credit systems
> 1.5 million txn/hour on screening app
11,000 million index entries on one database
30,000,000 real time transactions in an hour
Flexible & Adaptive
Smart indexing & fuzzy logic to emulate expert reasoning
Edge application (no upgrade impact)
Unprecedented Global Coverage
Cross script matching
Oracle DQ Matching Server Records linked to Same or Related Entity Profiling Cleansing Matching Enrichment Comprehensive data quality
Hybrid Algorithm Industry’s Best Matching Technology Heuristic Probabilistic Deterministic Phonetic Linguistic Empirical
Best Solution: Hybrid
“ Which algorithm is the best in solving my searching and matching needs?”
The answer is “No single algorithm is capable of compensating for all the classes of error and variation present in identity data.”.
In order to achieve a consolidated view of your identity data, you will need a combination of these algorithms, and more, each one addressing a particular class of problem,
Oracle Matching Server uses a variety of techniques, including the six mentioned here and many more, to address different classes of error and variation in identities
Oracle Data Quality Matching Server Siebel UCM / CRM Application Object Manager User Interface Data Admin Oracle DQ Matching Server Loader & Utilities Rule Manager Key & Search Strategies Match Purposes Search Server Update Synchronizer Console Server Console Administrative Clients Population Override Mgr Edit Rule Wizard Indexes Rules Base
Acxiom, D&B Integration
Data Enrichment Add Details from External Sources Profiling Cleansing Matching Enrichment Comprehensive data quality
Prospect Mastering with Knowledge-Based MDM
Perform segmentation within Siebel Marketing application
Generate prospect selection criteria
Load selected prospect records into Oracle MDM-CDI solution
Consolidate existing customer info with prospects from other sources
Oracle EBS Acxiom/D&B Data Products MDM-CDI Siebel Marketing Load Loading & Matching Siebel CRM On Demand
Plug & Play Market Campaign Execution
Send criteria and list of existing cust/prospect to Acxiom/D&B etc
Acxiom/D&B produces the net new prospect list and send to customer
Next Generation Data Quality
Best of Breed Data Quality
Matching – uses “fuzzy” logic and a unique two-stage approach to overcome the limitations of traditional techniques for 52 languages
Cleansing – Contains postal address information for 240 countries and territories
Profiling - discovers the quality, characteristics and potential problems of source data
Enrichment – integrate with 3 rd party content providers for business & consumer data
Embedded best in class Data Quality Open framework & connectors
Universal DQ Connector
End to end connector available for selected vendors
<Insert Picture Here> Best Practices
Formalize a Governance Framework Leadership Policy Definition Planning and Coordination Execution and Decision-Making Compliance Monitoring and Enforcement Master Data Data Management Governance Record Definition Data Quality Assessment Initial Data Quality and Load Ongoing Data Cleansing and Conversion Data Management Processes
Central executive leadership
Enterprise steering committee to arbitrate issues and enforce the rules
Coordination and compliance
Define & communicate data quality expectations
Establish policies, procedures, success metrics and processes to maintain quality data
Identify all business and application stakeholders across the enterprise – data owners
Conduct audit and control
Communication and change management
Closed Looped DQ
A Day in the Life of a Data Steward Data Stewardship is a critical component of DQ Process
Runs profiling routines to monitor overall DQ within application
Inspects most crucial or known problem areas
Gains deep-level understanding of data (e.g. min, max, # nulls..)
Creates and applies new data rule based on profiling results
Resolves duplicates and creates links
Reviews history and audit trail
Defines compliance rules and policies
Defines event and policies for ongoing monitoring and management
Executes corrective action: recover, unmerge, etc.
Performs ongoing monitoring of data quality
Do we have complete profiling information for our accounts / contacts?
Where are the information holes?
Does the customer have valid address, phone number and email?
Have we been able to communicate to the customer using stored contact point information?
Information Uniqueness (Duplication)
What is the duplicate rate in our accounts and contacts? What is the trend over time?
Which systems creates the most duplicates?
Is the information still up to date
Does the information have the proper integrity based on available sources and/or defined business rules?
Data Quality Scorecard
<Insert Picture Here> Credentials
Case Study - Lead Telco
CHALLENGES / OPPORTUNITIES
Drive improved customer experience & satisfaction
Consolidate customer information from disparate systems and multiple lines of businesses
Improve customer data quality
Complete understanding of customer hierarchies and relationships
SOLUTIONS – Oracle MDM & Data Quality
Enterprise wide customer master to provide a single view of customer
Match, deduplicate, and consolidated customer information from multiple systems into the customer master
Built out customer hierarchies and relationships
Consolidated ~30 mil customer records from 10+ applications into customer master
Improved customer data accuracy and completeness
Provided consistency and integrity of data across multiple operational systems
Selected Oracle Data Quality Customers Human Resources