Rethinking The Data Warehouse: Emerging Practices and Technologies to Meet Today’s Requirements

Emerging Practices and Technologies to Meet Today’s
Requirements
RETHINKING THE DATA WAREHOUSE

• Presenters Introduction
• Senturus Overview
• Rethinking the Data Warehouse
• Special Offers
• Additional Resources
• Q & A
Today’s Agenda
2Copyright 2015 Senturus, Inc. All Rights Reserved

Mark Madsen
President
Third Nature
Introduction: Today’s Presenters
Copyright 2015 Senturus, Inc. All Rights Reserved 3
John Peterson
Co-Founder and CEO
Senturus, Inc.

This slide deck is part of the “Rethinking the Data
Warehouse” recorded webinar. To view the FREE
recording of the entire presentation and download the
slide deck, go to:
www.senturus.com/resources/rethinking-the-data-
warehouse/
Senturus’ comprehensive library of recorded
webinars, demos, white papers, presentations and case
studies is available on our website:
www.senturus.com
Hear the Recording

Resource Library
Senturus’ whole purpose is to make you
successful with Business Analytics.
Thus, we offer a series of technology-
neutral webinars, training on specific
software, demonstrations, and no-holds-
barred reviews of new software releases.
We host dozens of live webinars every
year and we offer a comprehensive
library of recorded
webinars, demos, white
papers, presentations and case studies on
our website--a wealth of learning
resources. Most of our content is custom
created and constantly updated, so visit
us often to see what’s new in the
industry.
www.senturus.com/resources/

Who we are
SENTURUS INTRODUCTION

Technology Depth + Business Acumen
Senturus: Business Architects
for Business Analytics
C-Level
Business
Acumen
Technical/To
ol Expertise
Deep Data
Experience
Project
Management
Rigor
Business
Intelligence
Enterprise
Planning
Predictive
Analytics

A Few of Our 800+ Clients

Emerging Practices and Technologies to
Meet Today’s Requirements
RETHINKING THE DATA WAREHOUSE

slide deck, go to:
warehouse/
Senturus’ comprehensive library of recorded webinars,
demos, white papers, presentations and case studies is
available on our website:
www.senturus.com
Hear the Recording

11
Emerging Practices and
Architecture to Meet
Today’s Analytic
Requirements
May, 2015
Mark Madsen
Third Nature

CopyrightThird Nature, Inc.
As a technology moves from emerging to commodity the
nature of acquiring, using and managing it changes
Generate
options
Innovation
Novel practice
Maximize value
Maturation
Complicated
Constrain
choices
Adaptation
Good practice
Optimize
Standardize /
minimize choice
Acquisition
Best practice
Minimize costs
Saturation
Simple
Innovation
Complex
Agile & open
source* methods
6 Sigma & process
methods

The market is cyclical: databases in No-tation
1970: NoSQL = We have no SQL
1980: NoSQL = Know SQL
2000: NoSQL = No SQL?
2005: NoSQL = No SQL!
2010: NoSQL = Not only SQL
2015: NoSQL = No, SQL!
(R)DB(MS)

We are in a transitional phase in IT architecture
Then State of Practice Now, future
Architecture Timeshare Client/server Cloud
Data Core TXs All TXs, some
events, docs
All data
Rate of change Slow Rapid Continuous
Uses Few Many Everything
Latency Daily+++ < daily to
minutes
Immediate
Data platform Uniprocessor SMP, cluster Shared nothing

Today: repeating the experience of the prior eras
This is the turbulent
phase of the market as it
goes through rapid
development, then
product and service
changes.
The Internet combined with commodity computing is forcing a new
business and IT structural evolution, already underway.
Maturation SaturationInnovation
Big data
is here
Analytics
is here
BI/DW
is here

Think like an architect,
not like a consumer
No more “enterprise
standard” – now it’s all
about “what works”
The technology providers
are selling you what they
have, not what you need.
Follow the goals of the
business.
Translate the goals into
capabilities and match
those to the architecture
required.

The complaints about the data warehouse

Data
deficient
Takes
too long
Costs
too much
Function
deficient
IT root
causes
IT proximate
causes
What is said
in disputes
Lack of
agility
People & vendor
cost basis
Client-server
infrastructure
Lack of
“good enough”
competency
1980s-era
methods
Inappropriate
technology
Data hygiene
fetishes
Vendor lock
1990s-era
procurement
IT skills
deficit
Dysfunctional
OLTP portfolio

Data
deficient
Takes
too long
Costs
too much
Function
deficient
IT root
causes
IT proximate
causes
What is said
in disputes
How business
responds
IT FUD
responses
Lack of
agility
People & vendor
cost basis
Client-server
infrastructure
Lack of
“good enough”
competency
SaaS /
Cloud BI
Consultants
Self-service
BI
Shadow BI
Workarounds &
spreadmarts
Hidden costs
Loss of
control
Loss of
visibility
Loss of
knowledge
Security risks
Bad data risks
1980s-era
methods
Inappropriate
technology
Data hygiene
fetishes
Vendor lock
1990s-era
procurement
IT skills
deficit
Dysfunctional
OLTP portfolio

Data
deficient
Takes
too long
Costs
too much
Function
deficient
IT root
causes
IT proximate
causes
What is said
in disputes
How business
responds
IT FUD
responses
Lack of
agility
People & vendor
cost basis
Client-server
infrastructure
Lack of
“good enough”
competency
SaaS /
Cloud BI
Consultants
Self-service
BI
Shadow BI
Workarounds &
spreadmarts
Hidden costs
Loss of
control
Loss of
visibility
Loss of
knowledge
Security risks
Bad data risks
1980s-era
methods
Inappropriate
technology
Data hygiene
fetishes
Vendor lock
1990s-era
procurement
IT skills
deficit
Dysfunctional
OLTP portfolio
This is not a technology problem –
it is an architecture problem.

slide deck, go to:
warehouse/
www.senturus.com
Hear the Recording

Old market says: There’s nothing wrong with what
you have, just keep buying new products from us

The emerging big data market has an answer…

The data lake

The data lake after a little while

If “big” is your motivation for change, dig deeper
Source:Noumenal,Inc.

Architecture starts with a goal.
Answers: Why? What? Who? How?
Goal, capabilities, organization, implementation

Schema
In the DW world both data and processing are bounded
No consideration for feedback loops and change
Processing only
happens here
Carefully
controlled
SQL only
access
Nobodycreates
newinformation
Sources few and
well understood
Complex DI
is controlled
by IT
Schemas are few
and designed
Tools are authorized,
few in number and
kind
One way flow

The data warehouse vs business agility
All the data
Common, typed, tabular data
The bottleneck is you

Rate of change in the business exceeds that of the DW

In the big data world flow is unbounded and continuous
Feedback
loops allowed
End-of-analysis
dataset may be
start of a BI dataset
Continuous data
integration and delivery
Files are back as both
input and storage
Minimal
barrier of /
control on
collection
Areas of
provisioned
data
Any shape in,
rectangles out

slide deck, go to:
warehouse/
www.senturus.com
Hear the Recording

What’s the way out of the current problems?
architecture

Renovate the monolithic architecture

Principle: provide loose coupling between components

Principle: design for Isolation of unrelated change

There are really three workloads to consider, not two
1. Operational: OLTP systems
2. Analytic: (R)OLAP systems & Statistical
3. Processing: Computational systems
Unit of focus:
1. Transaction
2. Query
3. Computation
Different problems require different platforms

Deconstructing the data warehouse
There are three
things happening
in a DW:
▪ Data acquisition
▪ Data management
▪ Data delivery
Isolate them from
one another.
Data
Warehouse

Data integration: The data collection pattern
Take what people give you. Manage what is important.

Data collection
Collection can’t be limited by database scale and latency.
Immutability and persistence are required.
Incremental
Collect
Batch
One-time copy
Real time
Manage Integrate

slide deck, go to:
warehouse/
www.senturus.com
Hear the Recording

DATA ARCHITECTURE
We’re so focused on the light switch that we’re not
talking about the light

Decoupled Data Architecture
The core of the data warehouse isn’t the
database, it’s the data architecture that the
database and tools implement.
We need a data architecture that is not limiting:
▪ Deals with data and schema change easily
▪ Does not always require up front modeling
▪ Does not limit the format or structure of data
▪ Assumes a full range of data latencies, from
streaming to one-time bulk loads, both in and out,

Data architecture: phased or staged data
Multiple locations for use, differing quality levels

Integrate
Manage
Decouple the data architecture by phase
Use
This implies new data management and modeling approaches
Collect
Transactions Observations Declarations

Phases, not layers
Some tools require specific repositories or models.
Others can reach in to get what they need. Do not
enforce a single constraint.

It’s better to have visibility over everything than limited
visibility and control over a few things.
Control is an illusion, you never had it.

A visibility pattern
Give people a place for their ungoverned data.
Provide access to everything, in every phase.
You need a usage policy and a promotion policy.
Data
Warehouse

Agile methods without agile architectures fail
How you build is as important as what you build

Reinforcing relationships keep architectures from
changing, despite radical technology shifts
Note how only one third is tech
Architectural
Regime
MethodologyTechnology
Organization
Organization
defines where the
work is done and
the roles.
Technology
defines what
work can be done
in a given area. Methodology
defines how
work is done
and what that
work is.
Slide 50

How do you migrate to a new architecture?
“Any organization that designs a system … will
inevitably produce a design whose structure is a
copy of the organization's communication
structure.”
—M. Conway
Take advantage of conway’s law.

Move to continuous delivery

slide deck, go to:
warehouse/
www.senturus.com
Hear the Recording

Don’t follow the market
Some people can’t resist
getting the next new thing
because it’s new and new is
always better.
Many IT organizations are like
this, promoting a solution and
hunting for the problem that
matches it.
Better to ask “What is the
problem for which this
technology is the answer?”

We need “good enough” practices
We don’t need best practices, we need worst failures.

Conclusions
Big data is an opportunity to modernize, take
advantage of it.
You do need new tech, don’t delude yourself.
You still need most of the old tech, it works.
Architecture is key: deconstruct what’s wrong,
define the new, build toward it selectively.
Changing methods will change architecture, you
can’t build the new using the old methods.
Agile and continuous delivery go together.
People are more important than products (again).

“The future, according to some scientists, will be exactly like
the past, only far more expensive.” ~ John Sladek

CC Image Attributions
Thanks to the people who supplied the creative commons licensed images used in this presentation:
shady_puppy_sales.jpg - http://www.flickr.com/photos/brizzlebornandbred/5001120150
cuneiform_proto_3000bc.jpg - http://www.flickr.com/photos/takomabibelot/3124619443/
cuneiform_undo.jpg - http://www.flickr.com/photos/charlestilford/2552654321/
scroll_kerouac.jpg - http://www.flickr.com/photos/ari/93966538/
House on fire - http://flickr.com/photos/oldonliner/1485881035/
Manuscripts on shelf - http://flickr.com/photos/peterkaminski/1688635/
manuscript_illum.jpg - http://www.flickr.com/photos/diorama_sky/2975796332/
manuscript_page.jpg - http://www.flickr.com/photos/calliope/306564541/
subway dc metro - http://flickr.com/photos/musaeum/509899161/Circos, Hierarchical Edge
Bundles:Visualization of Adjacency Relations in Hierarchical Data, Danny Holten
text composition - http://flickr.com/photos/candiedwomanire/60224567/
twitter_network_bw.jpg - http://www.flickr.com/photos/dr/2048034334/
donuts_4_views.jpg - http://www.flickr.com/photos/le_hibou/76718773/
subway dc metro - http://flickr.com/photos/musaeum/509899161/

CC Image Attributions
Thanks to the people who supplied the creative commons licensed images used in this presentation:
cuneiform_undo.jpg - http://www.flickr.com/photos/charlestilford/2552654321/
cuneiform_proto_3000bc.jpg - http://www.flickr.com/photos/takomabibelot/3124619443/
scroll_kerouac.jpg - http://www.flickr.com/photos/ari/93966538/
firemen not noticing fire.jpg - http://flickr.com/photos/oldonliner/1485881035/
outdated gumshoe.jpg - http://flickr.com/photos/olivander/372385317/
manuscript_page.jpg - http://www.flickr.com/photos/calliope/306564541/
manuscript_illum.jpg - http://www.flickr.com/photos/diorama_sky/2975796332
well town hall.jpg - http://flickr.com/photos/tuinkabouter/1135560976/
pyramid_camel_rider.jpg - http://www.flickr.com/photos/khalid-almasoud/1528054134/
uniform_umbrellas.jpg - http://www.flickr.com/photos/mortimer/221051561/
open air market - http://flickr.com/photos/baboon/309793875/
train_to_sea.jpg - http://www.flickr.com/photos/innoxiuss/457069767/
wheat_field.jpg - http://www.flickr.com/photos/ecstaticist/1120119742/
Open air market - http://flickr.com/photos/baboon/309793875/
changing of the guard.jpg - http://flickr.com/photos/mambo1935/160739264/
Gare do Oriente Lisbon airport bridge.jpg - http://flickr.com/photos/higaara/228673603/
winding_road.jpg - http://www.flickr.com/photos/batt_57/4000701633/
Tokyo forum - http://flickr.com/photos/fukagawa/2004106475/
riot police line small - http://flickr.com/photos/73594239@N00/25719098/

About the Presenter
Mark Madsen is president of Third Nature, a
technology research and consulting firm
focused on business intelligence, data
integration and data management. Mark is
an award-winning author, architect and CTO
whose work has been featured in numerous
industry publications. Over the past ten years
Mark received awards for his work from the
American Productivity & Quality Center,
TDWI, and the Smithsonian Institute. He is an
international speaker, a contributor to
Forbes Online and on the O’Reilly Strata
program committee. For more information
or to contact Mark, follow @markmadsen on
Twitter or visit http://ThirdNature.net

About Third Nature
Third Nature is a research and consulting firm focused on new and emerging technology
and practices in analytics, business intelligence, information strategy and data
management. If your question is related to data, analytics, information strategy and
technology infrastructure then you‘re at the right place.
Our goal is to help organizations solve problems using data. We offer education, consulting
and research services to support business and IT organizations as well as technology
vendors.
We fill the gap between what the industry analyst firms cover and what IT needs. We
specialize in product and technology analysis, so we look at emerging technologies and
markets, evaluating technology and hw it is applied rather than vendor market positions.

BI Assessment
• Comprehensive Review of BI Stack Components: Server and Application
Layer, Data & Transformation Layer, and BI Tools Layer
• Deliverables include Grading and Roadmap for each of the Stack
Components
• Cost Starting at $9,995
Complimentary Consultation
• One-hour meeting with Senturus experts to discuss your challenges you
may have in your analytics environment and potential
solutions/recommendations
Contact info@senturus.com or 888.601.6010 ext. 85
Senturus BI Assessment And Consultation
63Copyright 2015 Senturus, Inc. All Rights Reserved.

www.senturus.com/events
Upcoming Events

*Custom, tailored training also available*
Cognos and Tableau Training Options

slide deck, go to:
warehouse/
www.senturus.com
Hear the Recording

Thank You!
www.senturus.com
888-601-6010
info@senturus.com
Copyright 2014 by Senturus, Inc.
This entire presentation is copyrighted and may not be
reused or distributed without the written consent of
Senturus, Inc.

Rethinking The Data Warehouse: Emerging Practices and Technologies to Meet Today’s Requirements

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (6)

Similar to Rethinking The Data Warehouse: Emerging Practices and Technologies to Meet Today’s Requirements

Similar to Rethinking The Data Warehouse: Emerging Practices and Technologies to Meet Today’s Requirements (20)

More from Senturus

More from Senturus (20)

Recently uploaded

Recently uploaded (20)

Rethinking The Data Warehouse: Emerging Practices and Technologies to Meet Today’s Requirements