A discussion as to what an ODS is and is not and where it fits within the enterprise

1. Guident
Guident
Why ODS? The Role Of The ODS In Today’s
BI World And How Oracle Technology Helps
C. Scyphers
Chief Architect
Guident Technologies
Guident || 198 Van Buren Street, Suite 120 Herndon, VA 20170 || Tel: Proprietary||of
Confidential and 703.326.0888 Web: www.guident.com || Email: info@guident.com
1
Guident

2. Topics Guident
 Guident Corporate Overview
 Overview Of RDBMS Implementations
 Where ODS Fits
 What An ODS Is – And Is Not
 The Classes Of ODS
 An Example
 Oracle Tools To Support An ODS
 Questions
 Conclusion
2

3. Who We Are… Guident
 BI Professional Services and Consulting Firm
 Washington, DC area headquarters
 Founded in 1996
 140+ professionals
 Top 20 Ranking in the 2006 & 2007 Washington Technology Fast 50
 Significant client base in both the Government and Commercial
sectors
 Certified by the SBA as an 8(a) business through July 2011
3

4. Business Intelligence Services Guident
 BI Core Technologies/Partnerships
 BI Strategy/Competency Centers, Data Warehousing, Data Quality, Enterprise
Performance Management, Analytics Solutions
 Over 150 implementations
 Oracle BI Partner Advisory Council member
 Focused BI Methodology
Come visit us at the
Guident Booth!
Raffling Off iPod Nano! 4

5. Topics Guident
 Guident Corporate Overview
 Overview Of RDBMS Implementations
 Where ODS Fits
 What An ODS Is – And Is Not
 The Classes Of ODS
 An Example
 Oracle Tools To Support An ODS
 Questions
 Conclusion
5

6. Introduction Of The Metaphor Guident
 Use a “real world” example.
A hypothetical company, called “Misa”
“Misa” will be a credit card company, much like
their competitors, Fastercard
In the interests of complete disclosure, I have not worked for any credit card
companies, nor do I have any insight or inside knowledge of their either
operational processes or their databases, other than the bill I get every month.
 Please don’t sue me.
6

7. Database Uses Guident
 Databases persist information in a logical fashion so as to facilitate efficient use of the
data stored.
 The efficient use may be rapid data entry (i.e. a Point Of Sale operation where the
speed of a customer’s check out is of paramount importances) or an analytical view of
the totality of the data.
 These goals are frequently referred to as OLTP and OLAP (OnLine Transactional
Processing and OnLine Analytical Processing, respectively).
 Both OLTP and OLAP operations require very different database structures, hardware
resources and administrative support.
 In practice, most databases tend to support both operations in varying combinations
of priority.
7

8. OLTP Databases Guident
 A very common (if not the most common) implementation of database technology is
with an OnLine Transactional Processing system.
 The primary focus of an OLTP database is the efficient processing of atomic
transactions to add, change and/or delete data.
 As a result, analyzing the data collected during the individual transactions receives
secondary importance.
 Attempting to report trends, metrics and other business intelligence related
evaluations can be unwieldy and slow.
In our example, Misa will use an OLTP database to track each individual
purchase by a given consumer using one of their credit products (a credit card,
for instance).
8

9. OLAP Databases Guident
 A relatively recent implementation of database technology is OnLine Analytical
Processing.
 Over the last two decades, the ability to analyze historical data in real time
interactions (as opposed to long running batch operations) has revolutionized the way
in which business evaluate their operations.
 A hallmark of OLAP databases is a radically modified data structure (when compared
to an OLTP database) optimized for efficient fetching of data in a fashion meaningful
to end users.
 Add, change and/or delete operations suffer as a result of this optimized approach.
 A direct result of this modified data structure is that
the database does not support alterations easily.
Data tends to be inserted only, with subsequence
“updates” to a data point taking precedence over
any prior incarnations.
In our example, our credit card company may
want to examine the purchase patterns of
their consumer base to identify and/or predict
future purchase patterns and possible
partnerships.
9

10. Topics Guident
 Guident Corporate Overview
 Overview Of RDBMS Implementations
 Where ODS Fits
 What An ODS Is – And Is Not
 The Classes Of ODS
 An Example
 Oracle Tools To Support An ODS
 Questions
 Conclusion
10

11. The Missing Link Guident
OLTP OLAP
 One or more OLTP databases,  One or more OLAP instances (possibly
containing the individual operations a data warehouse, but not
occurring on a daily basis within an necessarily), structured in such a way
organization. as to return relevant facts about
historical activities to an analyst as
quickly as possible.
 The business problem lies in analyzing current data at a higher level than individual
transactions.
With Misa, they have a strong business need to detect fraudulent transactions as
quickly as possible. They could attempt to use either the OLTP or the OLAP
systems.
 The analyst could comb through  The analyst could explore the historical
the OLTP databases, hunting for data within the OLAP environment
potentially fraudulent transactions.
 Depending on the freshness of the
 However, this is almost data within the OLAP instance, this
certainly going to be too will probably not be able to catch
resource intensive. the fraud before large amounts of
damage and/or losses are
incurred.
11

12. The Missing Link, part II Guident
 Additionally, the OLTP database(s) may not be the sole source of data needed for the
analysis.
 Other data sources may need to be considered:
 Demographic data such as age, race, sex, family size, et al.
 Economic data (income range, own vs. rent, credit history, etc).
 Geographical data (zip code, neighborhood information, etc.)
12

13. Loading The Data Guident
 One of the common ways of loading data into a database – particularly when loading
data for analytical purposes – is to using an ETL process.
 In this process, the data is first Extracted from a given source, Transformed into a
format more desirable in the target and then Loaded into the target database.
 A typical ETL process will bring several disparate data sources together into a
cohesive set of tables.
 These tables tend to be groups as
fact tables and dimensions in OLAP
environments.
 During the load process, the data may be
cleansed, semi-normalized into
dimensions (particularly if a snowflake
design is being used) and (oft times)
summarized and/or aggregated into roll
up values for analysis.
13

14. ODS - Bridging The Gap Guident
 An Operational Data Store (ODS) can be used to bridge the business needs gap
between OLTP systems and the OLAP environment.
OLTP OLAP
OLTP ODS OLAP
Rapid Inserts/Updates/ Rapid Inserts/Updates/ Comparatively Slower
Deletes Are Supported Deletes Are Supported Inserts (Due To ETL)
Live Data “Current” Data Historical Data
Live Perspective “Current” Perspective Historical Perspective
Optimized For Inserts/ Optimized For Reporting Optimized For Reporting
Updates/Deletes
14

15. Topics Guident
 Guident Corporate Overview
 Overview Of RDBMS Implementations
 Where ODS Fits
 What An ODS Is – And Is Not
 The Classes Of ODS
 An Example
 Oracle Tools To Support An ODS
 Questions
 Conclusion
15

16. The Definition Of An ODS Guident
 The canonical definition of an ODS is:
“a subject-oriented, integrated, volatile, current-valued, detailed-only collection of data in support of an
organization’s need for up-to-the-second, operational, integrated, collective information”
(Bill Immon)
 In reality, some summarization and/or aggregation tends to occur to support the direct
business needs at hand. Still, the intent of an ODS remains the atomic transactions.
 An ODS, much like a data warehouse or data mart, is organized around business
subjects like “Customer”, “Vendor”, and the like. An ODS is not typically focused on a
function or application like “A/P”, “A/R” or “Order Entry”.
 An ODS will bridge subjects and present an overview of a business area, rather than
a stovepipe view.
For example, any data relevant to the analysis and understanding of the
“Customer” business subject will be pulled into the ODS from whatever sources
are needed.
16

17. The Definition Of An ODS (continued) Guident
 An ODS will most likely reflect the “current” state of data for the subject – however the
word “current” is defined.
 For some groups, current may be the present accounting period. For others, it
may be a rolling seven day window.
 In these environments, the ODS will not contain anything beyond the operational
window, nor will it contain multiple versions (or snapshots) of the data.
 To maintain the current nature of the data, an ODS will have to be updated (from it’s
constituent data sources) on a sufficiently frequent cycle to ensure that the data in the
ODS remains “current”.
 An ODS will be sufficiently detailed to answer the specific business needs of the
subject.
 An ODS is classified into one of four categories based on both the rate data is loaded
into the system and the source of the data.
17

18. An ODS is Not…. Guident
 A common misconception of an ODS is that it must sit exactly between the OLTP
system and the OLAP environment. While is it possible (and sometimes optimal) to
use an ODS as a staging area for a subset of the data needed for an OLAP
environment, an ODS addresses very different business needs.
 An ODS typically will not be valuable at the department level of analysis. The value of
an ODS comes from leveraging data across the enterprise.
In the Misa analogy, a customer ODS would probably be used by marketing,
sales, customer support and billing, just to list four quick examples.
 An ODS is not the end-all, be-all for a specific subject matter.
 The BI value of an ODS comes from two main sources:
 The ability to promote rapid responses to
changes in the environment
 A quick way to orient analysts in the best
direction for a more comprehensive
investigation.
18

19. An ODS Is… Guident
 A way of getting specific answers to specific questions about the current state of the
world.
 An analogous way to think of the difference between an ODS and a data warehouse
is the difference between short term memory and long term memory.
 An ODS is short term memory – all about what has happened very recently
 A data warehouse is long term memory – what happened a decade ago
 An ODS is designed, deployed and tuned for fetch performance on a (comparatively)
small, focused set of data.
19

20. How To Use An ODS Guident
 An ODS excels about providing current data on a particular subject area.
 This naturally lends itself to being a data source for some dashboard
implementations.
 The ODS would have the most current information needed for a green/red light
implementation of metric analysis.
 A company could use an ODS as a data source for an up/down analysis of
servers, instances, services and other technical administrative details.
Misa might elect to use an ODS for real time fraud analysis or to alert key
personnel to an unexpected global surge in activity.
20

21. How To Use An ODS (continued) Guident
 Another valuable use for an ODS is in customer facing operations.
 An ODS can be used to provide phone support operators (sales, customer service,
technical support, etc.) with the most up-to-date information about a customer quickly
(usually faster than either a data warehouse or an OLTP system).
With Misa, a customer might call the call
center about her account.
The answering technician would then pull
up her current account, showing:
Most Recent Billing (And Payment ODS
Status)
Any Security Warnings
Any Active Disputes
 By going to the ODS, the response should be sub-second, if not instantaneous.
 If any additional data is needed (prior billings, prior disputes, etc.), those queries
could be serviced by the OLAP environment.
21

22. Topics Guident
 Guident Corporate Overview
 Overview Of RDBMS Implementations
 Where ODS Fits
 What An ODS Is – And Is Not
 The Classes Of ODS
 An Example
 Oracle Tools To Support An ODS
 Questions
 Conclusion
22

23. The Classes Of ODS – Class I Guident
 Transactions are loaded into the ODS from the data sources as soon as the source
transaction is committed.
 This is otherwise known as a real time ODS.
 In this model, any ETL processes must be very simple to maintain the constraints of
real time processing.
 In practice, this is very uncommon in production, mostly due to the prohibitive cost of
implementation.
23

24. The Classes Of ODS – Class II Guident
 Transactions are loaded into the ODS from the data sources within a very short
period of time.
 The period of time can be from a few minutes up to a few hours. A ceiling of ~6 hours
is typical.
 In this model, normal ETL processing is supported.
 Most ODS configurations tends to be a variation on this class.
24

25. The Classes Of ODS – Class III Guident
 Transactions are loaded into the ODS from the data sources on a daily schedule
 While it is certainly possible to populate the data on a schedule longer than a day, it is
also very uncommon to do that.
 In this model, normal ETL processing is supported.
 Variations on this approach are the second most common ODS implementation.
25

26. The Classes Of ODS – Class IV Guident
 Aggregate data are loaded into the ODS from an OLAP environment on a scheduled
basis.
 The primary difference between this approach and a data mart populated by
replicating a subset of data from a warehouse is:
 A data mart will have historical data within its confines
 An ODS will only have “current” data.
 In this model, normal ETL processing is supported, but usually not needed.
 This is rarely implemented in pure form; however, DWs can (and frequently do) serve
as data sources for any ODS.
Data
Warehouse
26

27. Topics Guident
 Guident Corporate Overview
 Overview Of RDBMS Implementations
 Where ODS Fits
 What An ODS Is – And Is Not
 The Classes Of ODS
 An Example
 Oracle Tools To Support An ODS
 Questions
 Conclusion
27

28. Detecting Fraud With An ODS Guident
 So, let’s solve a business problem. MISA wants to be able to detect fraudulent
behavior as soon as possible. How do you think we could use an ODS to do this?
 Organizing the data by
 Unique customer
 Integrating relevant data:
 Mailing address of the customer
 Location and Type of the stores
 Amount spent,
 Date and time of the transaction
 Declaring a rolling 14 day window
 A sufficient set of data should be on hand to support analysis.
28

29. Tips For Designing & Building An ODS Guident
 Have good requirements and a good data model
 Many – if not most – problems that occur in an ODS deployment can be directly
traced to an incomplete understanding and mapping of the business problem(s).
 Do not overpopulate the ODS with unnecessary data.
 Only include attributes directly relevant to the business problems the ODS will
solve.
 Adding extraneous attribute for ulterior purposes is very tempting, but will only
serve to lessen the value of the ODS in itself.
 Use an iterative approach. As with many OLAP
implementations, a quick ROI builds political capital
within the organization, allowing for future growth.
 As an ODS will most likely receive cleansed data
as part of it’s ETL process, it may be more efficient
to use the ODS as a data source for a data
warehouse or data mart.
 If so, avoid duplication of effort.
29

30. Where An ODS Fits In The Enterprise Guident
OLTP
ODS
Multiple ODS:
Customer for C/S
Customer for Sales
Customer for Marketing
Fraud for Security
OLAP
30

31. Topics Guident
 Guident Corporate Overview
 Overview Of RDBMS Implementations
 Where ODS Fits
 What An ODS Is – And Is Not
 The Classes Of ODS
 An Example
 Oracle Tools To Support An ODS
 Questions
 Conclusion
31

32. Oracle Tools For An ODS Guident
 Any number of Oracle tools can be used to assist with the design,
development, deployment and administration of an ODS.
 TimesTen is very good for Class I & II databases where speed is of
paramount importance.
 Oracle Warehouse Builder can quickly build an ETL process from a
host of disparate data sources.
 Fusion Middleware – in particular, Oracle Data Integrator – can
further extend the reach of the ODS into even more data sources.
 Enterprise Edition of the RDBMS can support virtually any ODS
requirement.
 Partitioning  Change Data Capture
 Scalability/VLDB  Parallelism – Whenever Possible!
 Heterogeneous Support
32

33. Topics Guident
 Guident Corporate Overview
 Overview Of RDBMS Implementations
 Where ODS Fits
 What An ODS Is – And Is Not
 The Classes Of ODS
 An Example
 Oracle Tools To Support An ODS
 Questions
 Conclusion
33

34. Q&A Guident
Any Questions?
34

35. Topics Guident
 Guident Corporate Overview
 Overview Of RDBMS Implementations
 Where ODS Fits
 What An ODS Is – And Is Not
 The Classes Of ODS
 An Example
 Oracle Tools To Support An ODS
 Questions
 Conclusion
35

36. Conclusions Guident
 An ODS acts as a current, bird’s-eye view of a specific business subject.
 An ODS typically pulls from multiple data sources to construct its environment.
 An ODS bridges the gap between OLTP and OLAP systems, providing a powerful tool
for real time analysis and reactions to an organization.
 Most ODS operations lag behind the live production OLTP data by some period of
time, usually measured in hours.
36

37. Contact Us Guident
To find out how Guident can assist you in aligning business and
technology to produce significant performance improvement, growth,
and return on investment, please contact:
C. Scyphers
Chief Architect
Office: 703.326.0888
E-mail: cscyphers@guident.com
Web: www.guident.com
37