Adding a Data Reservoir to your Oracle Data Warehouse for Customer 360-Degree Analysis
1. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Adding a Data Reservoir to Your
Oracle Data Warehouse for
Customer 360-Degree Analysis
Mark Rittman, CTO, Rittman Mead
UKOUG Tech’15, Birmingham, December 2015
2. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
What Is This Presentation About…?
•What is a Hadoop-based data reservoir, and why might you add one to a data warehouse?
•How do you load, process and integrate one with your data warehouse using Oracle tools?
•How can you use it for what’s termed “Customer 360-degree insight?”
schema-on-read vs schema on write
real-time data ingestion
agile data provisioning vs. curated data
combining Hadoop, NoSQL and Oracle
omni-channel marketing
machine learning & decision engines
attitudinal vs behavioural data
3. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
About Me
•Mark Rittman, Oracle ACE Director, Oracle BI, DW & Big Data
•14 Years Experience with Oracle Technology
•Regular columnist for Oracle Magazine
•Author of two Oracle Press Oracle BI books
•Oracle Business Intelligence Developers Guide
•Oracle Exalytics Revealed
•Writer for Rittman Mead Blog :
http://www.rittmanmead.com/blog
•Past Editor of Oracle Scene Magazine,
BIRT SIG Chair, ODTUG Board Member
•Co-founder and CTO for Rittman Mead
4. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
… Or as I say at Parties…
5. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
15+ Years in Oracle BI and Data Warehousing
•Started back in 1997 on a bank Oracle DW project
•Our tools were Oracle 7.3.4, SQL*Plus, PL/SQL
and shell scripts
•Went on to use Oracle Developer/2000 and Designer/2000
•Our initial users queried the DW using SQL*Plus
•And later on, we rolled-out Discoverer/2000 to everyone else
•And life was fun…
6. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
The Oracle-Centric DW Architecture
•Over time, this data warehouse architecture developed
•Added Oracle Warehouse Builder to
automate and model the DW build
•Oracle 9i Application Server (yay!)
to deliver reports and web portals
•Data Mining and OLAP in the database
•Oracle 9i for in-database ETL (and RAC)
•Data was typically loaded from
Oracle RBDMS and EBS
•It was turtles Oracle all the way down…
7. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Traditional Three-Layer Relational Data Warehouses
Staging Foundation /
ODS
Performance /
Dimensional
ETL ETL
BI Tool (OBIEE)
with metadata
layer
OLAP / In-Memory
Tool with data load
into own database
Direct
Read
Data
Load
Traditional structured
data sources
Data
Load
Data
Load
Data
Load
Traditional Relational Data Warehouse
•Three-layer architecture - staging, foundation and access/performance
•All three layers stored in a relational database (Oracle)
•ETL used to move data from layer-to-layer
8. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
ETL Largely Batch-Based and with Single Route through DW
•All data lands in Staging layer, processed and then thrown-away
‣Too expensive to store all incoming granular data online - selected data stored as summary
•Processed through Foundation layer and then Access and Performance
•ETL development an expensive, manual task
•But this approach provided accurate numbers
that every could trust, and navigate around
9. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
And Now … Everyone’s Talking About Big Data
•Explosion in volume and variety of data that’s now available
•New, cheap and open-source technology
makes it economic to store + process it
•Users want more data stored in the DW,
but budgets for IT are getting smaller
•Analytics and analysis has gone beyond
tabular reports and dashboards, and requires
new platforms to enable new approaches
•Which is actually rather scary…
10. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Meanwhile, in the real world…
11. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Who is my customer?
12. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Who is my customer?
13. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
14. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
More Complete Data About
Your Customers
Advanced analytics and
machine learning
More Attributes and
Activities Stored at Scale
True 360°Customer Profile
Connect disparate data
Targeted, personalized
customer treatment
Customer 360-Degree Insight
15. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
16. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Combines transactions + master data with granular behavioural & attitudinal data
Adding “Who” and “Why” to Customer Datasets
Single Customer View
Enriched
Customer Profile
Correlatin
g
Modeling
Machine
Learning
Scoring
“How”
Interaction Data
Voice + Chat
Transcripts In-person
dialogs
Webserver
logs
Blogs
Surveys
Social Media
“Why”
Attitudinal Data
“What”
Behavioural Data
Transaction
History
Retail
Activity
Payment
History
Basket Analysis
Attributes
Segments
Relationships
“Who”
Descriptive Data
Demographics
17. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
But Wait … Isn’t This Just Data Warehousing & Data Mining?
•Data warehouses were conceived as a single source of reporting truth
•Formally accept, model and integrate data to provide analytical reporting platform
•Well-established design patterns for long-term data storage
•Stored in structured, indexed, optimised “schema on write” storage
•Data moved through layers via formal ETL
•Extreme Performance, Highly Secure
•Analytic SQL, In-Database Analytics
‣So why not use for this Customer 360 data?
18. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Back to the real world again…
19. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Consider How Consumers Used to Be Marketed To…
• Marketing used to be generic,
one-way “broadcasting” to public
• Then Web 2.0 gave customers
a voice, they could talk back…
• But they expected an immediate answer
• More work, but more intimate relationship
• Big data, smart technology + complex algorithms
makes a “360-degree view of customers
now possible
• Customers volunteer much data themselves
• But equilibrium of relationship now moved
irrevocably to the customer
20. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Customer Touch-points Now Cover Many More Channels
•The days of a single, high-street retail channel are long gone
•Prospects often now find you via web searches, social media connections
•Shopping and browsing “on the go”
using mobile devices, wearables
•Web increasingly the main sales channel
•“Order and go” collection at stores
•Call centre helplines,
•Customer service desks
•Forums, blogs, product reviews
and other user-generated content
1980s 1990s 2000s 2010s
Empowered
Employees
Digital is
Humanized
Knowledge
Everywhere
Internet of
Things
Mobile as
Primary Channel
Cross-Channel
Service
WHAT’S NEXT
21. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Consumers Now Drive Their Own Purchase Decisions
22. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Datasets for Marketing Need to Reflect Today’s Consumer
23. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Customers Share Data with You .. With Expectations
•Customers now share huge amounts of data willingly, and perhaps unknowingly
‣Through your channels and applications - with potential privacy issue
‣Through tweeting, posting on Facebook and other social networks
‣But they also want to be in control
-Ability to delete their data
-Understand what data you hold
-For what purposes
-And how it was collected
24. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
25. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
What is Customer 360-Degree Analysis?
•Gather together all meaningful information about the customer (“360-degree view”)
•Organizing, matching, profiling & storing every interaction in real time
•Matched and combined; factual, interpreted, learned
‣Across all channels, and on public forums and social media
•Captures interactions across all-touch points and all channels
‣Including activity on social networks, forums, blogs
•Typically stored and processed in a Hadoop “data reservoir”
•Dynamic customer profiles with segmentation,
behavioural analysis “at scale”
•Downstream feeds into DW, CRM and other systems
26. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
27. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
28. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
29. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Connect the Silos, Understand Customers, Drive Decisions
execute smarterlisten better
consumption logs,
clickstream & devices
demographic, user and
credit data
customer contacts and
service cases
transactions and
subscriptions
content metadata,
ratings, comments
marketing campaign
response
social media
activity
programmatic
advertising
audience
acquisition, retention
multi-channel
marketing
targeted
promotions
next best
offer
personalized
content
product & service
strategy
content acquisition
learn faster
Enriched
Customer Profile
Correlating
Modeling
Scoring
Micro-Segments
History
Preferences
30. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
But … Isn’t This Just CRM?
•Typically built for call centres, sales automation
•Core data is customer service activity
•Supplemented by purchase history
•CRM system typically system of record for
service activity, with links to transactions
‣LoB application focused on particular tasks
31. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Customer 360-Degree view typically used as central data store for digital marketing
•Provides key data for real-time decision engines, next-best offer, personalisation
Customer 360-Degree View as Driver of Digital Marketing
?
?
?
32. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Customer 360-Degree view typically used as central data store for digital marketing
•Provides key data for real-time decision engines, next-best offer, personalisation
Customer 360-Degree View Powering Marketing + Offers
Data Transfer Data Access
Real-Time Context
Environmental
User Journey
Offer Feedback
Single Customer View
Enriched
Customer Profile
Correlatin
g
Modeling
Machine
Learning
Scoring
Real Time
Offers &
Suggestions
Up-Sell / Cross-
Sell
Decisioning
Service
Self-Learning
Predictive
Models
33. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Typically Stored on Flexible, Scalable Hadoop + NoSQL
Voice + Chat
Transcripts
Call Center LogsChat Logs iBeacon Logs Website LogsCRM Data Transactions Social FeedsDemographics
Real-time Feeds,
batch and API
$50k
Hadoop
Node
$50k
Hadoop
Node
$50k
Hadoop
Node
Hadoop
Node
Hadoop
Node
$50k$50k
Hadoop
Node
Hadoop
Node
$50k
Enriched
Customer
Profile Modeling
Scoring
Hadoop Data
Reservoir
Raw customer data stored at detail
Enriched and processed for insights
$50k
34. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Architected using “Data Reservoir” Design Pattern
•Data for customer 360 system typically landed into a Hadoop & NoSQL-based
•Applies aggregation, joining and machine-learning processes to extract insights
Data Transfer Data Access
Data Factory
Data Reservoir
Business
Intelligence Tools
Hadoop Platform
File Based
Integration
Stream
Based
Integration
Data streams
Discovery & Development Labs
Safe & secure Discovery and Development
environment
Data sets and
samples
Models and
programs
Marketing /
Sales Applications
Models
Machine
Learning
Segments
Operational Data
Transactions
Customer
Master ata
Unstructured Data
Voice + Chat
Transcripts
ETL Based
Integration
Raw
Customer Data
Data stored in
the original
format (usually
files) such as
SS7, ASN.1,
JSON etc.
Mapped
Customer Data
Data sets
produced by
mapping and
transforming
raw data
35. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Data Transfer Data Access
Data Factory
Data Reservoir
Business
Intelligence Tools
Hadoop Platform
File Based
Integration
Stream
Based
Integration
Data streams
Discovery & Development Labs
Safe & secure Discovery and Development
environment
Data sets and
samples
Models and
programs
Marketing /
Sales Applications
Models
Machine
Learning
Segments
Operational Data
Transactions
Customer
Master ata
Unstructured Data
Voice + Chat
Transcripts
ETL Based
Integration
Raw
Customer Data
Data stored in
the original
format (usually
files) such as
SS7, ASN.1,
JSON etc.
Mapped
Customer Data
Data sets
produced by
mapping and
transforming
raw data
36. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
So What is a Data Reservoir?
37. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
What Does it Do?
38. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
And Does it Replace My Data Warehouse?
39. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
A technical digression…
40. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Data from Real-Time, Social & Internet Sources is Strange
Single Customer View
Enriched
Customer Profile
Correlatin
g
Modeling
Machine
Learning
Scoring
•Typically comes in non-tabular form
•JSON, log files, key/value pairs
•Users often want it speculatively
‣Haven’t though through final
purpose
•Schema can change over time
‣Or maybe there isn’t even one
•But the end-users want it now
‣Not when your ETL team are next
free
41. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Data Warehouse Loading Requires Formal ETL and Modeling
$1m
Analytic
DBMS Node
ETL
Data Model
ETL
Developer
Data Modeller
Curated Data
ETL Development takes time, is fragile, but results in well-curated data
But what about data whose schema is now known?
Or final use has not yet been determined?
Dimensional data modelling gives structure to the data for business users
But also restricts how that data can be analysed
What if the end-user is better placed to apply that schema?
42. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
… And Are Limited in What They Can Store (Economically)
$1m
Analytic
DBMS Node
DB
Instance
Compute
ETL
Data Model
ETL
Developer
Data Modeller
$1m
Analytic
DBMS Node
Compute
$1m
Analytic
DBMS Node
Compute
$1m
Analytic
DBMS Node
Single DB Instance
Compute
43. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Analytic
DBMS Node
Analytic
DBMS Node
Sharded Databases Can Scale Further - At Even More Cost
$1m
Analytic
DBMS Node
Compute
Data Model
ComputeCompute
DB Shard DB Shard DB Shard
Complex Shard-Aware ETL
A-F O-R S-T
$1m $1m
Analytic
DBMS Node
Compute
DB Shard
Analytic
DBMS Node
Compute
DB Shard
Analytic
DBMS Node
Compute
DB Shard
Analytic
DBMS Node
Compute
DB Shard
$1m$1m $1m $1m
G-J K-N U-W X-Z
.. and adding more nodes means re-sharding the dataset
Also rules out mixed-workload DBs with OLTP
44. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Exadata Combines Best of Both … Again At Cost
Data Model
Compute
DBMS Node
Compute
Storage Cell
Storage
Compute Offload
Query offloading Filtered, projected columns only returned
Storage Cell
Storage
Compute Offload
Storage Cell
Storage
Compute Offload
Storage Cell
Storage
Compute Offload
Storage Cell
Storage
Compute Offload
Storage Cell
Storage
Compute Offload
Storage Cell
Storage
Compute Offload
Storage Cell
Storage
Compute Offload
ETL
Compute
DBMS Node
Compute
Compute
DBMS Node
Compute
Single DB Instance
45. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Introducing Hadoop - Cheap, Flexible Storage + Compute
•A new approach to data processing and data storage
•Rather than a small number of large, powerful servers, it spreads processing over
large numbers of small, cheap, redundant servers
•Spreads the data you’re processing over
lots of distributed nodes
•Has scheduling/workload process that sends
parts of a job to each of the nodes
•And does the processing where the data sits
•Shared-nothing architecture
•Low-cost and highly horizontal scalable
Job Tracker
Task Tracker Task Tracker Task Tracker Task Tracker
Data Node Data Node Task Tracker Task Tracker
46. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Introducing Hadoop - Cheap, Flexible Storage + Compute
•Hadoop & NoSQL better suited to exploratory analysis
of newly-arrived data
‣Flexible schema - applied by user rather than ETL
‣Cheap expandable storage for detail-level data
‣Better native support for machine-learning and
data discovery tools and processes
‣Potentially a great fit for our new and emerging
customer 360 datasets, and great platform for analysis
47. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Hadoop Designed for Real-Time Storage of Raw Data Feeds
$50k
Hadoop
Node
Voice + Chat
Transcripts
Call Center LogsChat Logs iBeacon Logs Website Logs
Real-time Feeds
Raw Data
48. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Supplement with Batch + API Loads of ERP + 3rd Party Data
$50k
Hadoop
Node
Voice + Chat
Transcripts
Call Center LogsChat Logs iBeacon Logs Website Logs
Real-time Feeds
CRM Data Transactions Social FeedsDemographics
Batch Loads APIs, Web Service Calls
Raw Data
49. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Supplement with Batch + API Loads of ERP + 3rd Party Data
$50k
Hadoop
Node
Voice + Chat
Transcripts
Call Center LogsChat Logs iBeacon Logs Website LogsCRM Data Transactions Social FeedsDemographics
Raw Data
Customer 360 Apps
Predictive
Models
SQL-on-Hadoop
Business analytics
Real-time Feeds,
batch and API
50. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Hadoop
Node
Hadoop
Node
Hadoop
Node
Hadoop
Node
Supplement with Batch + API Loads of ERP + 3rd Party Data
Voice + Chat
Transcripts
Call Center LogsChat Logs iBeacon Logs Website LogsCRM Data Transactions Social FeedsDemographics
Real-time Feeds,
batch and API
Hadoop
Node
Compute
Hadoop
Node
Compute ComputeCompute
$5k
Compute Compute
$50k
Hadoop
Node
Raw Data across Cluster Filesystem
Compute
51. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Hadoop-Based Storage & Compute : A Better Logical Fit
Voice + Chat
Transcripts
Call Center LogsChat Logs iBeacon Logs Website LogsCRM Data Transactions Social FeedsDemographics
Real-time Feeds,
batch and API
$50k
Hadoop
Node
$50k
Hadoop
Node
$50k
Hadoop
Node
Hadoop
Node
Hadoop
Node
$50k$50k
Hadoop
Node
Hadoop
Node
$50k
Enriched
Customer
Profile Modeling
Scoring
Hadoop Data
Reservoir
Raw customer data stored at detail
Enriched and processed for insights
$50k
52. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Typically Stored on Flexible, Scalable Hadoop + NoSQL
Voice + Chat
Transcripts
Call Center LogsChat Logs iBeacon Logs Website LogsCRM Data Transactions Social FeedsDemographics
Real-time Feeds,
batch and API
$50k
Hadoop
Node
$50k
Hadoop
Node
$50k
Hadoop
Node
Hadoop
Node
Hadoop
Node
$50k$50k
Hadoop
Node
Hadoop
Node
$50k
Enriched
Customer
Profile Modeling
Scoring
Hadoop Data
Reservoir
Raw customer data stored at detail
Enriched and processed for insights
$50k
53. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Oracle Engineered system for big data processing and analysis
•Start with Oracle Big Data Appliance Starter Rack - expand up to 18 nodes per rack
•Cluster racks together for horizontal scale-out using enterprise-quality infrastructure
Oracle Big Data Appliance
Starter Rack + Expansion
• Cloudera CDH + Oracle software
• 18 High-spec Hadoop Nodes with
InfiniBand switches for internal
Hadoop traffic, optimised for network
throughput
• 1 Cisco Management Switch
• Single place for support for H/W + S/
W
Deployed on Oracle Big Data Appliance Engineered System
Oracle Big Data Appliance
Starter Rack + Expansion
• Cloudera CDH + Oracle software
• 18 High-spec Hadoop Nodes with
InfiniBand switches for internal
Hadoop traffic, optimised for network
throughput
• 1 Cisco Management Switch
• Single place for support for H/W + S/
W
Enriched
Customer Profile
Modeling
Scoring
Infiniband
54. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Flexible, Low-Cost Resilient Storage : Hadoop Distributed FS
•The filesystem behind Hadoop, used to store data for Hadoop analysis
‣Unix-like, uses commands such as ls, mkdir, chown, chmod
•Fault-tolerant, with rapid fault detection and recovery
•High-throughput, with streaming data access and large block sizes
•Designed for data-locality, placing data closed to where it is processed
•Accessed from the command-line, via internet (hdfs://), GUI tools etc
[oracle@bigdatalite mapreduce]$ hadoop fs -mkdir /user/oracle/my_stuff
[oracle@bigdatalite mapreduce]$ hadoop fs -ls /user/oracle
Found 5 items
drwx------ - oracle hadoop 0 2013-04-27 16:48 /user/oracle/.staging
drwxrwxrwx - oracle hadoop 0 2012-09-18 17:02 /user/oracle/moviedemo
drwxrwxrwx - oracle hadoop 0 2012-10-17 15:58 /user/oracle/moviework
drwxrwxrwx - oracle hadoop 0 2013-05-03 17:49 /user/oracle/my_stuff
drwxrwxrwx - oracle hadoop 0 2012-08-10 16:08 /user/oracle/stage
55. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Apache Hive : SQL Access + Table Metadata Over HDFS
•Apache Hive provides a SQL layer over Hadoop, once we understand the structure (schema)
of the data we’re working with
•Exposes HDFS and other Hadoop data as tables and columns
•Provides a simple SQL dialect for queries called HiveQL
•SQL queries are turned into MapReduce jobs under-the-covers
•JDBC and ODBC drivers provide
access to BI and ETL tools
•Hive metastore (data dictionary)
leveraged by many other Hadoop tools
‣Apache Pig
‣Cloudera Impala
‣etc
SELECT a, sum(b)
FROM myTable
WHERE a<100
GROUP BY a
Map
Task
Map
Task
Map
Task
Reduce
Task
Reduce
Task
Result
56. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
NoSQL Databases
•Family of database types that reject tabular storage,
SQL access and ACID compliance
•Focus is on scalability, speed and schema-on-read
‣Oracle NoSQL Database - speed and scalability
‣Apache HBase - speed, scalability and Hadoop
‣MongoDB - native storage of JSON documents
•May or may not run on Hadoop, but associated with it
•Great choice for high-velocity data capture
•CRUD approach vs write-once/read many in HDFS
57. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Best Data Store for Customer 360 Data : Hadoop & NoSQL
•Data for customer 360 system typically landed into a Hadoop & NoSQL-based
•Applies aggregation, joining and machine-learning processes to extract insights
Data Transfer Data Access
Data Factory
Data Reservoir
Business
Intelligence Tools
Hadoop Platform
File Based
Integration
Stream
Based
Integration
Data streams
Discovery & Development Labs
Safe & secure Discovery and Development
environment
Data sets and
samples
Models and
programs
Marketing /
Sales Applications
Models
Machine
Learning
Segments
Operational Data
Transactions
Customer
Master ata
Unstructured Data
Voice + Chat
Transcripts
ETL Based
Integration
Raw
Customer Data
Data stored in
the original
format (usually
files) such as
SS7, ASN.1,
JSON etc.
Mapped
Customer Data
Data sets
produced by
mapping and
transforming
raw data
58. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Interfaces with CRM Tools, and Accessible
‣CRM can be a consumer of data from the Hadoop-based Customer 360
‣And provide key customer attributes and sales events from CRM activity
‣Allows CRM tools to focus on their core strengths
‣With ability to interface with the Customer 360 system as appropriate
Data Reservoir
Business
Intelligence
Tools
CRM System
Models
Machine
Learning
Segments
Raw
Customer
Data
Data stored in
the original
format
(usually files)
such as SS7,
ASN.1, JSON
etc.
Mapped
Customer
Data
Data sets
produced by
mapping and
transforming
raw data
Data Transfer Data Access
59. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Combine with DW for Big Data Management Platform
60. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Combining Oracle RDBMS with Hadoop + NoSQL
•High-value, high-density data goes into Oracle RDBMS
•Better support for fast queries, summaries, referential integrity etc
•Lower-value, lower-density data goes into Hadoop + NoSQL
‣Also provides flexible schema, more agile development
•Successful next-generation BI+DW projects combine both - neither on their own is sufficient
61. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Today’s Oracle Information Management Ref Architecture
Actionable
Events
Event Engine Data
Reservoir
Data Factory Enterprise
Information Store
Reporting
Discovery Lab
Actionable
Information
Actionable
Insights
Input
Events
Execution
Innovation
Discovery
Output
Events
& Data
Structured
Enterprise
Data
Other
Data
62. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Next-Generation Layered Data Warehouse Architecture
Virtualization&
QueryFederation
Enterprise
Performance
Management
Pre-built &
Ad-hoc
BI Assets
Information
Services
Data Ingestion
Information Interpretation
Access & Performance Layer
Foundation Data Layer
Raw Data Reservoir
Data
Science
Data Engines &
Poly-structured
sources
Content
Docs Web & Social Media
SMS
Structured
Data
Sources
•Operational Data
•COTS Data
•Master & Ref. Data
•Streaming & BAM
Immutable raw data reservoir
Raw data at rest is not interpreted
Immutable modelled data. Business
Process Neutral form. Abstracted from
business process changes
Past, current and future interpretation of
enterprise data. Structured to support agile
access & navigation
Discovery Lab Sandboxes Rapid Development Sandboxes
Project based data stores to
support specific discovery
objectives
Project based data stored to
facilitate rapid content /
presentation delivery
Data Sources
63. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Oracle Engineered system for big data processing and analysis
•Start with Oracle Big Data Appliance Starter Rack - expand up to 18 nodes per rack
•Cluster racks together for horizontal scale-out using enterprise-quality infrastructure
Oracle Big Data Appliance
Starter Rack + Expansion
• Cloudera CDH + Oracle software
• 18 High-spec Hadoop Nodes with
InfiniBand switches for internal
Hadoop traffic, optimised for network
throughput
• 1 Cisco Management Switch
• Single place for support for H/W + S/
W
Deployed on Oracle Big Data Appliance Engineered System
Oracle Big Data Appliance
Starter Rack + Expansion
• Cloudera CDH + Oracle software
• 18 High-spec Hadoop Nodes with
InfiniBand switches for internal
Hadoop traffic, optimised for network
throughput
• 1 Cisco Management Switch
• Single place for support for H/W + S/
W
Enriched
Customer Profile
Modeling
Scoring
Infiniband
64. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Oracle Big Data Discovery for Cataloging Customer 360 Data
•Access a rich, interactive catalog of all
data in Customer 360 data reservoir
•Familiar search and guided navigation
for ease of use
•See data set summaries, user annotation
and recommendations
•Add personal and enterprise data to
Customer 360 datasets via self-service
‣Make sense of the wider customer
data now loaded into data reservoir
65. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Back to the real world again…
66. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
What is Customer 360-Degree Analysis?
•Gather together all meaningful information about the customer (“360-degree view”)
•Organizing, matching, profiling & storing every interaction in real time
•Matched and combined; factual, interpreted, learned
‣Across all channels, and on public forums and social media
•Captures interactions across all-touch points and all channels
‣Including activity on social networks, forums, blogs
•Typically stored and processed in a Hadoop “data reservoir”
•Dynamic customer profiles with segmentation,
behavioural analysis “at scale”
•Downstream feeds into DW, CRM and other systems
67. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
The Data Integration Challenge
•Data from all the sources will need to be integrated to create the single customer view
‣Hadoop technologies (Flume, Kafka, Storm) can be used to ingest events, log data
‣Files can be loaded “as is” into the HDFS filesystem
‣Oracle/DB data can be bulk-loaded using Sqoop
‣GoldenGate for trickle-feeding transactional data
•But nature of new data sources brings challenges
•May be semi-structured or unknown schema
‣Joining schema-free datasets
•Need to consider quality and resolve incorrect,
incomplete, and inconsistent customer data
Voice + Chat
Transcripts
Batch Load
from files,
DB:
Easy
Stream from
APIs, HTTP:
Moderate
Load raw text
from files:
Easy
Data Reservoir
Raw
Customer
Data
Data stored in
the original
format
(usually files)
such as SS7,
ASN.1, JSON
etc.
Mapped
Customer
Data
Data sets
produced by
mapping and
transforming
raw data
Cleanse,
enrich and
obfuscate raw
files:
Lots of work..!
Join
structured+
semi/
unstructured:
How..?
68. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Landing raw data is easy; then the real work needs to be done - can be > 90% of project
•Four main tasks to take raw data and apply schema and combine together
1. Apply Schema to Raw and Semi-Structured Data
2. Remove Sensitive Data from Any Input Files
3. Identify joins, further enrichments and transforms
4. Store as “mapped” data in data reservoir
Ingesting Raw Customer Data : Two Key Challenges
69. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Data enrichment tool aimed at domain experts, not programmers
•Uses machine-learning to automate
data classification + profiling steps
•Automatically highlight sensitive data,
and offer to redact or obfuscate
•Dramatically reduce the time required
to onboard new data sources
•Hosted in Oracle Cloud for zero-install
‣File upload and download from browser
‣Automate for production data loads
Raw Data
Data stored in the
original format (usually
files) such as SS7, ASN.
1, JSON etc.
Mapped Data
Data sets produced by
mapping and
transforming raw data
Voice + Chat
Transcripts
Oracle Big Data Preparation Cloud Service
70. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Step 1: Apply Schema to Raw and Semi-Structured Data
NLP
Embedded Information
in
Entities
Embedded Information
No reliable patterns
Invalid and missing data
Sensitive data
Invalid
emails
Stream from
APIs, HTTP:
Moderate
Batch Load
from files, DB:
Easy
Load raw text
from blog
entries,
reviews
71. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Step 2: Remove Sensitive Data from Any Input Files
•Automatically profile and analyse datasets
•Use Machine Learning to spot and obfuscate sensitive data automatically
72. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Step 3 : Identify Common Keys and Joins using BDD
•Data ingest process automatically applies some enrichments - geocoding etc
•Can apply others from Transformation page - simple transformations & Groovy expressions
73. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Step 4 : Process Joined and Enriched Data Back to Hadoop
•Data joined and transformed within Big Data Discovery can be saved back to Hadoop
•Export to HDFS, register with Hive (optional)
•Supports creation of
Data Reservoir
Raw
Customer Data
Data stored in the
original format
(usually files) such
as SS7, ASN.1,
JSON etc.
Mapped Customer
Data
Data sets produced
by mapping and
transforming raw
data
Cleanse,
enrich and
obfuscate raw
files:
Lots of work..!
Join structured+
semi/unstructured:
How..?
74. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Further Transforming & Managing Customer 360 Data
•Oracle Data Integration Suite offers a wider set of products for managing Customer 360 data
‣Oracle GoldenGate
‣Oracle Enterprise Data Quality
‣Oracle Data Integrator
‣Oracle Enterprise Metadata
Management
-All Hadoop enabled
-Works across Big Data,
Relational and Cloud
75. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Solution Component Logical View
•Data Factory for file, event and DB transaction batch and streaming ingestion
•Big Data Management Platform for combined Hadoop + RDBMS data storage
•Discovery Labs for innovation and sandboxing
76. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Discovery Labs and Sandboxes : Key to Innovation
•For Customer 360 systems to innovate beyond the initial roll-out, these are critical
•Allows innovation and new subject areas to be developed separate from IT governance
‣Then migrated to production when appropriate
• Support agile development of BI
• Demand management through BICC
• Automated provisioning / de-provisioning
• Data sourced from any data layer or off-platform
• Standardised use of BI tooling
• Careful governance required once work is
complete
• Measure: Engineering backlog and size of
shadow-IT
• Support agile discovery in data
• Demand management through Analytical CC
• Automated provisioning / de-provisioning
• Data sourced from any data layer or off-platform
• Broad range of analytical tools
• Governance step required to operationalise
insights
• Measure: value and rate of new insights to
business
77. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
And now the clever bit…
78. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
79. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
I’m too sexy…
80. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
81. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
• Complete view of customers
• Micro-segmented customer profiles
• Predictive models
82. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Customer 360-Degree view typically used as central data store for digital marketing
•Provides key data for real-time decision engines, next-best offer, personalisation
Customer 360-Degree View as Driver of Digital Marketing
?
?
?
83. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Customer 360-Degree view typically used as central data store for digital marketing
•Provides key data for real-time decision engines, next-best offer, personalisation
Customer 360-Degree View Powering Marketing + Offers
Data Transfer Data Access
Real-Time Context
Environmental
User Journey
Offer Feedback
Single Customer View
Enriched
Customer Profile
Correlatin
g
Modeling
Machine
Learning
Scoring
Real Time
Offers &
Suggestions
Up-Sell / Cross-
Sell
Decisioning
Service
Self-Learning
Predictive
Models
84. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Customer 360-Degree view typically used as central data store for digital marketing
•Provides key data for real-time decision engines, next-best offer, personalisation
Customer 360-Degree View as Driver of Digital Marketing
Data Transfer Data Access
Real-Time Context
Environmental
User Journey
Offer Feedback
Single Customer View
Enriched
Customer Profile
Correlatin
g
Modeling
Machine
Learning
Scoring
Real Time
Offers &
Suggestions
Up-Sell / Cross-
Sell
Decisioning
Service
Self-Learning
Predictive
Models
85. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Customer 360 systems can use machine learning across all data to build predictive models
•Decision engines (for example, Oracle RTD) can use its data as source
•Wider range of attributes and faster velocity
•Aim to process data and not just to store it
-Identify customers likely to defect,
work proactively to retain
-foster enhanced engagement
•increase revenue
•pass events and contextual data to
real-time decisioning engines
Use Machine Learning and Real-Time Decisions to add Value
Real Time
Offers & Suggestions
Up-Sell / Cross-Sell
Decisioning Service
Self-Learning
Predictive
Models
Real-Time Context
Environmental
User Journey
Offer Feedback
Single Customer View
Enriched
Customer Profile
Correlatin
g
Modeling
Machine
Learning
Scoring
Operational Data
Transactions
Customer
Master Data
Unstructured
Data
Voice + Chat
Transcripts
Touch Points
Store
Web
Service
86. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Real-time decisioning engine from Oracle, part of Oracle BI product family
•Predictive and Personalised Real-Time Recommendations
‣Behaviour-based models that take into
account activity across all channels
‣At very low levels of granularity
‣Micro-segmentation to individual customer
‣Multi-contextual recommendations
based on predicted customer needs
•Real-time offers, delivered to any channel
•Feedback loop to improve recommendation
Oracle Real-Time Decisions
87. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Decision engines and business processes use models to predict customer behavoir
•Traditional CRM-driven decision engines only consider what happened
•Big Data + real-time feeds can dramatically improve model performance
•Model with “Big Data” and potentially thousands of input variables:
•Customer sentient data
•Competitors data
•Environmental data
•Spatial location data
•Long term vs. recent historical behavior
•Sensor data
More Data + Variety Data -> Better Predictive Models
1980s 1990s 2000s 2010s
Empowered
Employees
Digital is
Humanized
Knowledge
Everywhere
Internet of
Things
Mobile as
Primary Channel
Cross-Channel
Service
WHAT’S NEXT
88. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Event-Driven Personalised Marketing & Offers
•Vastly increased size, time-relevance and scope of customer data into decision models
Touch Points
Store
Web
Service
Operational Data
Events &
Workflow
Transactions
Customer
Master Data
Profile-Based (Demographic)
Prioritization
Offers & Suggestions
Guided Search Guided Search
Pricing Marketing
Engagement
Real Time
Offers & Suggestions
Up-Sell / Cross-Sell
Decisioning Service
Self-Learning
Predictive
Models
Single Customer View
Enriched
Customer Profile
Correlatin
g
Modeling
Machine
Learning
Scoring
Real-Time Context
Environmental
User Journey
Offer Feedback
Relevant
Personalized
Experiences
•Then close the loop with real-time context back into decision engine
89. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Connect the Silos, Understand Customers, Drive Decisions
execute smarter
listen better
consumption logs,
clickstream & devices
demographic, user and
credit data
customer contacts and
service cases
transactions and
subscriptions
content metadata,
ratings, comments
marketing campaign
response
social media
activity
programmatic
advertising
audience
acquisition, retention
multi-channel
marketing
targeted
promotions
next best
offer
personalized
content
product & service
strategy
content acquisition
Single Customer View
Enriched
Customer Profile
Correlatin
g
Modeling
Machine
Learning
Scoring
90. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
•Start with pilot for area of the business that needs a single view of customers
•Then, over time, iterate and build out the Customer 360-degree view
Delivering a Successful Customer 360-Degree View
Start with a business
area that
needs a single
customer view
Obtain clear
understanding of
customer online &
offline behaviour
Build out
Predictive Models
and Decision Engines
to deliver value now
Build out Hadoop Data
Reservoir, Feeds
and link to DW + CRM
Iterate and Build-out,
add new integrations,
incrementally building
capability
Develop and Implement Strategy, Deliver Business
Value
Build DevOps Capability
Pilot & Quick Win
Create Full Production InfrastructurePilot (Virtualised / Commodity) Hadoop Infrastructure
91. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Packaged Customer 360 Applications
•Packaged Customer 360 applications seem a good way to start…?
•Risks around degree of fit and integration limits, but can be useful for mature projects
See everything
together –
comparisons with a
Set defined by you,
and evolving trend
scores for each
customer
From Data to DNA
– 1000s of metrics
determine
individual DNA –
common, industry
and customer
metrics
92. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Rittman Mead Customer 360 + Real-Time Decisions Projects
Start with a business
area that
needs a single
customer view
Obtain clear
understanding of
customer online &
offline behaviour
Build out
Predictive Models
and Decision Engines
to deliver value now
Build out Hadoop Data
Reservoir, Feeds
and link to DW + CRM
Iterate and Build-out,
add new integrations,
incrementally building
capability
93. T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or
+61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)
E : info@rittmanmead.com
W : www.rittmanmead.com
Adding a Data Reservoir to Your Oracle Data Warehouse for
Customer 360-Degree Analysis
Mark Rittman, CTO, Rittman Mead
UKOUG Tech’15, Birmingham, December 2015