Building a Collaborative Data Architecture

EMBARCADERO TECHNOLOGIESEMBARCADERO TECHNOLOGIES
Build a Collaborative Data Architecture
Ron Huizenga
Senior Product Manager – ER/Studio

EMBARCADERO TECHNOLOGIES
Agenda
• What’s happening in the world of data?
– Data trends, data usage, data thirst
• Collaboration defined
– Enabling collaboration
• The need for architecture
• What is data architecture?
• Model based data architecture
– For solution development
– To mitigate organizational data landscape complexity
• Business driven data architecture
• Concluding remarks
2

Increasing volumes,
velocity, and variety of
Enterprise Data
30% - 50% year/year
growth
Decreasing % of
enterprise data which is
effectively utilized
5% of all Enterprise data
fully utilized
Increased risk from data
misunderstanding and
non-compliance
$600bn/annual cost for
data clean-up in U.S.
Enterprise Data Trends

Quenching the Thirst - Big Data?
• Volume
• Velocity
• Variety
• Veracity

Business Stakeholders’ Data Usage
5
Suspect that business stakeholders
INTERPRET DATA INCORRECTLY
Yes,
frequently
14%
Yes,
occasionally
67%
No, never
9%
I don’t know
10%
Suspect that business stakeholders make decisions
USING THE WRONG DATA?
Yes,
frequently
11%
Yes,
occasionally
64%
No, never
13%
I don’t know
12%

Data Model Usage & Understanding
6
13%
3%
16%
19%
31%
18%
0% 5% 10% 15% 20% 25% 30% 35%
We don’t use data models
Other
Our data team does most data
models but developers also build
them as needed
Our database administrators own
data modeling
Developers develop their own data
models
We have a data modeling team that
is responsible for data models
What is your organization’s approach to data modeling?
How well does your organization’s technology leadership team
understand the value of using data models?
Completely
understand
20%
Understand
somewhat
60%
Don’t
understand
17%
I don’t know
3%
87%

Collaboration
• Collaborate
– to work jointly with others or together especially in an
intellectual endeavor
– to cooperate with or willingly assist an enemy of one's country
and especially an occupying force
– to cooperate with an agency or instrumentality with which one
is not immediately connected
• Collaborative
– produced or conducted by two or more parties working
together
7

Communication is Critical
8

Communication – Full Definition
9
Technically correct
Functionally useless

Communication – Simple Definition
• Communication
– The act or process of using words, sounds, signs, or behaviors to
express or exchange information or to express your ideas,
thoughts, feelings, etc., to someone else
– A message that is given to someone : a letter, telephone call,
etc.
• Communications
– The ways of sending information to people by using technology
10

The need for architecture?
Evolution:
• 38 years of construction
• 147 builders
• No Blueprints
• No Planning
Result:
• 7 stories
• 65 doors to blank walls
• 13 staircases abandoned
• 24 skylights in floors
• 160 rooms, 950 doors
• 47 fireplaces, 17 chimneys
• Miles of hallways
• Secret passages in walls
• 10,000 window panes (all bathrooms are fitted with windows)
3

Data Architecture (as defined by DAMA)
• A master set of data models and design approaches identifying the strategic data requirements and the
components of data management solutions, usually at an enterprise level. Enterprise data architecture typically
consists of
– 1) an enterprise data model (contextual/subject area, conceptual or logical),
– 2) state transition diagrams depicting the lifecycle of major entities,
– 3) a robust information value chain analysis identifying data stakeholder roles, organizations, processes and applications, and
– 4) data integration architecture identifying how data will flow between applications and databases. The data integration
architecture may divide into
• database architecture
• master data management architecture
• data warehouse / business intelligence architecture
• meta data architecture.
• Some enterprises also include
– 5) lists of controlled domain values (code sets), and
– 6) the responsibility assignments of data stewards to subject areas, entities and code sets.
• The enterprise data architecture is an important part of the larger enterprise architecture that includes business,
process and technology architecture
12

Data Architecture
• As defined in Wikipedia:
– Data architecture is composed of models, policies, rules or
standards that govern which data is collected, and how it is
stored, arranged, integrated, and put to use in data
systems and in organizations.
13

ER/Studio Enterprise Team Edition
5

Key Skill Sets
• Data Design & Management
• ETL and Software Development
• Data Analysis / Stats
• Business Analysis & Discovery
Value Delivered
• Validation
• Integration
• Enrichment
• Usability
Value and the New Lifecycle
15
Discover
Document
(Model)
Integrate

Data Landscape Complexity
16
• Comprised of:
– Proliferation of disparate systems
– Mismatched departmental solutions
– Many database platforms
– Big data platforms
– ERP, SAAS
– Obsolete legacy systems
• Compounded by:
– Poor decommissioning strategy
– Point-to-point interfaces
– Data warehouse, data marts, ETL …
Data Archaeologist?

Discovery and Identification Through Models
• Identify candidate data sources
• Reverse engineer data sources into models
• Identify, name and define
• Classify through metadata
• Map “like” items across models
• Data lineage / chain of custody
• Repository
• Collaboration & publishing
17

Addressing Complexity through Models
• Multi-level sub-models: allow business decomposition
• Reverse engineering: wide variety of platforms including Big Data
• What and where?
– Naming standards
– Universal mappings
• Document and define
– Metadata extensions (attachments)
– Business glossaries
• Data in context: business processes
• Data lineage
• Repository, collaboration & publishing
18

Automated Naming Standards
19

ER/Studio: Universal Mappings
• Ability to link “like” or related objects
– Within same model file
– Across separate model files
• Entity/Table level
• Attribute/Column level
20

ER Studio: Attachment of Metadata extensions
21

ER/Studio: Data Dictionary
22

ER/Studio: Extended Notation for MongoDB
23

Clarify with Business Data Objects
24

Alternate Perspectives for Different Audiences
25

Data Lineage
26

ER/Studio Team Server: Enterprise Collaboration
27

ER/Studio Team Server – Model Explorer
28

The Need for Common Understanding
29

Business Glossary – Why?
• Maximize understanding of the core business concepts
and terminology of the organization
• Minimize misuse of data due to inaccurate understanding
of the business concepts and terms
• Improve alignment of the business organization with the
technology assets (and technology organization)
• Maximize the accuracy of the results to searches for
business concepts, and associated knowledge
30

Team Server: Glossary/Terms
31

Enhanced Communication: Glossary Integration
32

Addressing Governance
33
Data
Governance
Data
Architecture
Management
Data
Development
Database
Operations
Management
Data Security
Management
Reference &
Master Data
Management
Data
Warehousing
& Business
Intelligence
Management
Document &
Content
Management
Metadata
Management
Data Quality
Management

Collaborative, Business-Driven Data Architecture
• Improve visibility and collaboration with ER/Studio
• Enable more efficient and automated data modeling
• Share models and metadata across the organization
• Establish business glossaries with consistent terms
and definitions
• Build a solid foundation for compliance, data
governance, and master data management
34

Thank you!
• Learn more about the ER/Studio product family:
http://www.embarcadero.com/data-modeling
• Trial Downloads:
http://www.embarcadero.com/downloads
• To arrange a demo, please contact Embarcadero
Sales: sales@embarcadero.com, (888) 233-2224
35

Building a Collaborative Data Architecture

More Related Content

What's hot

Similar to Building a Collaborative Data Architecture

More from DATAVERSITY

Recently uploaded

Building a Collaborative Data Architecture