UKOUG Tech 15 - ODI Lifecycle and Data Governance

info@rittmanmead.com www.rittmanmead.com @rittmanmead
Jerome Francoisse | UKOUG Tech 2015
Oracle Data Integrator –
Development Lifecycle & Data Governance
1

Jérôme Françoisse
• Consultant for Rittman Mead
- Oracle BI/DW Architect/Analyst/Developer
• ODI Trainer
• Providing ODI support on OTN Forums
• ODI 12c Beta Program Member
• Blogger at http://www.rittmanmead.com/blog/
• Email : jerome.francoisse@rittmanmead.com
• Twitter : @JeromeFr
2

About Rittman Mead
3
• World’s leading specialist partner for technical
excellence, solutions delivery and innovation in
Oracle Data Integration, Business Intelligence,
Analytics and Big Data
• Providing our customers targeted expertise; we are a
company that doesn’t try to do everything… only
what we excel at
• 70+ consultants worldwide including 1 Oracle ACE
Director and 2 Oracle ACEs
• Founded on the values of collaboration, learning,
integrity and getting things done
Optimizing your investment in Oracle Data Integration
• Comprehensive service portfolio designed to support
the full lifecycle of any analytics solution
• 4 UKOUG Partner of the Year Award 2015

User Engagement
4
Visual Redesign Business User Training
Ongoing SupportEngagement Toolkit
Average user adoption for BI
platforms is below 25%
Rittman Mead’s User Engagement Service can help

Successful ODI project
5
• Having good developers is great
• … but it’s not enough!
• We also need :
- Assurance that the code deployed in production is correct
- The right data
- Good data

Agenda
6
• Lifecycle Management
- Architecture
- Promotion
- Versioning
• Data Governance
- Oracle Enterprise Metadata Management
- Oracle Enterprise Data Quality

ODI 12c Key Components
8
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Development)
Execution
Execution Repository
(Production)
• ODI Studio (the main development IDE)
- Designer Navigator
- Operator Navigator
- Topology Navigator
- Security Navigator
• ODI Agents (executing mappings and packages)
• ODI Console (web-based interface)
• ODI Repositories (contain definitions of models,
mappings and other objects)

Physical and Logical Topology
9
DEV_SCOTT
SH
HR
SCOTT
HR
SCOTT
SH
HUM_RES
DEV_OP
TEST_OP
PROD_OP
Physical
SCOTT
HR
Contexts Logical
DEV
TEST
PROD
SH

Architecture
10
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Development)
Execution
(Production)

How many repositories do I need?
11
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Development)
Execution
(Production)
Execution
(Test)

12
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Development)
Execution
(Production)
Execution
(Test)
Security
Topology
Versioning
Master Repository

13
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Development)
Execution
(Production)
Execution
(Test)
Security
Topology
Versioning
Master Repository
Execution
(Pre-Production)
Security
Topology
Versioning
Master Repository

14
Models
Projects
Execution
Work Repository
(Development)
Execution
(Production)
Execution
(Test)
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Hotfix)
Execution
(Pre-Production)
Security
Topology
Versioning
Master Repository
Security
Topology
Versioning
Master Repository

15
Models
Projects
Execution
Work Repository
(Development)
Execution
(Production)
Execution
(Test)
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Hotfix)
Execution
(Pre-Production)
Security
Topology
Versioning
Master Repository
Security
Topology
Versioning
Master Repository
Execution
(Smoke Test)

Problems
16
• As a developer, how do I create a consistent set of objects for a release?
- For development environments
- For execution environments
• As a release manager, how do I easily promote code from one
environment to another?
• As a release manager, how do I easily keep track of these sets of
objects?
• As a release manager, how do I roll back to a previous release?

The Solution – Deployment Archive (DA)
• Archive file containing a set of consistent ODI objects
• Applied to Deployed Repositories (test/production)
• 3 types :
- Initial
- Patch
- Rollback
• Development or Execution
17

Example
18
Development
environment
QA environment
(Deployed repository)
Developer
Release Manager
Bug
Creates Initial
Release 1.0.0.0
Initialize
repository with
Release 1.0.0.0
Creates Patch
Release 1.1.0.0
to add a
features
Apply Patch
Release 1.1.0.0
and create
rollback DA
Apply Rollback
Release 1.1.0.0
Creates Patch
Release 1.1.0.1
to fix the
previous bug
Apply Patch
Release 1.1.0.1
and create
rollback DA

Developer / Administrator
19
• Create a deployment archive

20
• Create a deployment archive
from VCS label
• Best Practice :
Link documentation and DDL
statements to be applied

21
• Create a deployment
archive from selected
objects
• Best Practice :
Mention the bug number in
the description

22
• No need to think about development or
execution-only environment

Release Manager
23
• Apply Deployment Archives

Release Manager
24
• Initial DA
• Can choose from
development or
execution-only archive

Release Manager
25
• Patch DA
• Can choose from
development or
execution-only archive
• Best Practice :
Always create a
Rollback DA!

Release Manager
26
• See applied DAs –
Inventory
• See available DAs –
View Details

Release Manager
27
• Rollback DA

Automation
28
• Available through ODI SDK
• Scenarios :
- Import and be sure you always generate a rollback DA
- Import daily into a smoke test environment
- Import the same release in Hotfix and Production at the
same time

Other Problems
29
• How to rollback on the Development environment?
• How to restore delete objects in the Development
environment?

The Solution - VCS
30
• Integration with Subversion (SVN)
- Everything is done through ODI Studio
- Git support in the next release
- Tagging
- Branching / Merging

VCS - Features
31
• Add an object to VCS

VCS - Features
32
• Create a new version of an object

VCS - Features
33
• View and compare versions

VCS - Features
34
• Restore a version

VCS - Features
35
• Tags
- Snapshot of consistent set of versioned objects
- Ready for promotion – Deployment Archive
- … or for branching
- Created from
‣ Whole repository (Full Tag)
‣ Selected Objects and dependencies (Partial Tag)

VCS - Features
36
• Branches
- Created from a Tag
- Populate a new environment – Master and Work repository
- For a new feature or a new team
- Can be merged back in the trunk

VCS - Features
37
• Branches
Trunk
Branch 1
Branching
Release 1
101 2 3 5
4 6 7 9
Merging
11
Bug Fixes for Release 1
New features for Release 2
New features for Release 2New features for Release 1
8

Data Governance
38
• The right data
• The good data
People
Process Tools

Data Governance
39
• The right data
• The good data

Data Governance
40
• The right data
Which reports uses
my application data?
Application User Sysadmin
Which reports use
the mainframe data?
ETL Developer
What is the impact if I
change this
mapping?
DBA
Who is using this
database I need to
upgrade?
Data Steward
Where is this data
entered?
Who should I notify if
there is a problem?
Which data are
available?
Data Scientist BI Developer
Where does this data
come from?
Executive
How are sales
figures
calculated?

Oracle Enterprise Metadata Management
41
• Lineage
- From reports to source
• Impact Analysis
- From source to reports
• DB - ETL - BI Metadata
• Business Glossary
• Semantic Linking
• Supporting features
- Harvesting
- Automatic Stitching
- Versioning
- Collaboration - Annotations

42

44

45

46

47

48

Data Governance
49
• The right data
• The good data

info@rittmanmead.com www.rittmanmead.com @rittmanmead 50
• The good data
Oracle Enterprise Data Quality

Oracle Enterprise Data Quality - Profiling
51

Oracle Enterprise Data Quality - Cleansing/Matching
52

Oracle Enterprise Data Quality - Cleansing/Matching

Oracle Enterprise Data Quality - ODI Integration

Summary
• ODI - Deployment Archives
- Promote, Rollback and keep track of your code
- For development or execution-only environment
- Easy for Developers and Release Managers
• ODI - VCS
- Be able to get back to any version to object
- Branching for Team development
• OEMM
- Lineage, impact analysis and business glossary
• OEDQ
- Profile, Cleanse, Match, Monitor, Govern

Further Reading / Testing
56
• ODI Lifecycle
- Managing versions in SVN
https://blogs.oracle.com/dataintegration/entry/oracle_data_integrator_12_21
- Parallel Development using SVN Branches
https://blogs.oracle.com/dataintegration/entry/oracle_data_integrator_12_22
- ODI Getting Started VM 12.2.1
http://www.oracle.com/technetwork/middleware/data-integrator/odi-demo-2032565.html
• OEMM
- Oracle By Example
http://www.oracle.com/technetwork/middleware/oemm/learnmore/index.html
• OEDQ Getting Started 12.2.1
http://www.oracle.com/technetwork/middleware/oedq/downloads/edq-vm-download-2424092.html

Questions?
57

UKOUG Tech 15 - ODI Lifecycle and Data Governance

Recommended

Recommended

More Related Content

Similar to UKOUG Tech 15 - ODI Lifecycle and Data Governance

Similar to UKOUG Tech 15 - ODI Lifecycle and Data Governance (20)

Recently uploaded

Recently uploaded (20)

UKOUG Tech 15 - ODI Lifecycle and Data Governance

Editor's Notes