You can watch the replay for this webcast in the IDERA Resource Center: http://ow.ly/1Bzr50A58Tg Databases are often like the big bang, suddenly they just exist, right? Well, not really, someone had to do the due diligence and design them conceptually then come up with a physical model for implementation. If it’s a transaction database, then we're done and the application connects and, voila, data starts filling our database. But in other situations such as business intelligence, analytics, conversions, etc. we must move data from other systems into the database. In this session we are going to discuss these two key aspects: database design patterns and Extract, Transform and Load (ETL). We will talk about the role of data modeling and SQL Server Integration Services for data migration. About Stan: Stan Geiger is a Senior Product Manager at IDERA with over 25 years using Microsoft SQL Server. Stan has worked in various industries from fraud detection to healthcare. He has held several positions including database developer, DBA, and BI Architect, and has experience building Data Warehouse and ETL platforms, BI Analytics and OLTP systems.

1. © 2018 IDERA, Inc. All rights reserved.
DATABASES DON’T BUILD AND
POPULATE THEMSELVES
Stan Geiger, Sr. Manager of Product

2. © 2018 IDERA, Inc. All rights reserved.
AGENDA
▪ Data Modeling
• Benefits
• Conceptual, Logical and Physical Models
• Relational vs. NoSQL
▪ Design Patterns
• Datawarehouse/Analytics
• Late binding vs. Early binding
• Dimensional
• Transactional
▪ Deployment
▪ Data Loading

3. “
© 2018 IDERA, Inc. All rights reserved.
Cogito, ergo sum
I think, therefore I am

4. © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.© 2018 IDERA, Inc. All rights reserved.
DATA MODELING
A lot of databases are built like this: Then after a while they
end up like this:

5. © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.© 2018 IDERA, Inc. All rights reserved.
DATA MODELING
We want:
Benefits
• Higher quality applications
• Cost Reduction
• Faster time to market
• Organizational
understanding
• Increased performance
• Better documentation
• Fewer data errors
• Data Mining

6. © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.© 2018 IDERA, Inc. All rights reserved.
Data Modeling Concepts

7. © 2018 IDERA, Inc. All rights reserved.
DATA MODELING
Conceptual Models
• High level description of business information
• Includes main concepts and relationships

8. © 2018 IDERA, Inc. All rights reserved.
DATA MODELING
Logical Models
• Includes all entities and relationships among
them
• All attributes for each entity are specified
• The primary key for each entity is specified
• Foreign keys (keys identifying the relationship
between different entities) are specified.
• Normalization occurs at this level (for relational
models)

9. © 2018 IDERA, Inc. All rights reserved.
DATA MODELING
Physical Models
• Contains relationships between tables and addresses cardinality and
nullability of the relationships.
• Specific version of DBMS, location, data storage or technology to be used.
• Columns have exact datatypes, lengths assigned and default values.
• Primary and Foreign keys, views, indexes, access profiles, and
authorizations, etc. are defined.

10. © 2018 IDERA, Inc. All rights reserved.
DATA MODELING

11. © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.© 2018 IDERA, Inc. All rights reserved.
Database Concepts

12. © 2018 IDERA, Inc. All rights reserved.
DATA MODELING
Relational vs. NoSQL
• Data modeling often starts from the application-specific queries as opposed to
relational modeling
• NoSQL does not require predefined structure
• Design how to organize the data.
• Data is aggregated instead of normalized

13. © 2018 IDERA, Inc. All rights reserved.
DATA MODELING
NoSQL Modeling
• Denormalization
• Aggregates
• Soft Schemas

14. © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.© 2018 IDERA, Inc. All rights reserved.
Datawarehouse/Data Lake

15. © 2018 IDERA, Inc. All rights reserved.
DATA MODELING
Late Binding vs. Early Binding
• Data is typically bound to “business rules”
• Business rules are often bound in the model
• Implemented through foreign keys, constraints, triggers, etc.
• Late binding occurs when the business rules are “bound” at selection
time.
• Late binding is particularly effect in industries like healthcare where the
rules are constantly changing.

16. © 2018 IDERA, Inc. All rights reserved.
DATA MODELING
▪ Early Binding vs Late Binding
• NoSQL databases tend to use more late binding.
• The notion of data lakes are late binding.
• Once your definition is set in the data model, it is very difficult and time-
consuming to change it afterward.
• Late binding has speed and flexibility

17. © 2018 IDERA, Inc. All rights reserved.

18. © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.© 2018 IDERA, Inc. All rights reserved. 18© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.

19. © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.© 2018 IDERA, Inc. All rights reserved. 19© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.

20. © 2018 IDERA, Inc. All rights reserved.
DIMENSIONAL MODELING
▪ Based on the business processes
▪ Uses Facts and Dimensions
▪ Typically Star or Snowflake schema

21. © 2018 IDERA, Inc. All rights reserved.
DIMENSIONAL MODELING
▪ Choose the business process
▪ Declare the grain
▪ Identify the dimensions
▪ Identify the fact

22. © 2018 IDERA, Inc. All rights reserved.
OLTP MODELING
▪ Efficient for transaction processing
▪ Eliminates redundancy of data
▪ Makes transactions easy to save
▪ Querying is often comple.

23. © 2018 IDERA, Inc. All rights reserved.
DEPLOYMENT
▪ Built on the concept of forward engineering
▪ Takes logical model to a physical model
▪ Database/platform specific

24. © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.© 2018 IDERA, Inc. All rights reserved.
Loading Data

25. © 2018 IDERA, Inc. All rights reserved.
DATA LOADING
ETL vs ELT or Data Warehouse vs Data Lake

26. © 2018 IDERA, Inc. All rights reserved.
DATA LOADING
▪ When to use ETL
• Bulk data movement
• Comple rules and transformations are
needed.
• Combine large amounts of data from
multiple sources
▪ When to use ELT
• When ingestion speed is king.
• When more intel is better intel.
• When you know you will need to scale.

27. “
© 2018 IDERA, Inc. All rights reserved.
“If you do not know where you come from, then you don't know
where you are, and if you don't know where you are, then you don't
know where you're going. And if you don't know where you're
going, you're probably going wrong.”

28. © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.© 2018 IDERA, Inc. All rights reserved.
Lagniappe

29. © 2018 IDERA, Inc. All rights reserved.
DATA GOVERNANCE
▪ Increased consistency and confidence in decision making
▪ Decreased risk of regulatory fines
▪ Improves data security
▪ Maximizes the income generation potential of data
▪ Designates accountability for information quality
▪ Minimizes and eliminates re-work
▪ Optimizes staff effectiveness
▪ Establishes process performance baselines to enable
improvement efforts

30. © 2018 IDERA, Inc. All rights reserved.
DATA GOVERNANCE
Practice of Data Governance
1. Balanced, Lean Governance
2. Data Quality
3. Access to data
4. Data Literacy
5. Data Content
6. Analytic Prioritization
7. Master Data Management

31. © 2018 IDERA, Inc. All rights reserved.
DATA LINEAGE
Benefits of tracking
• Error Resolution
• Impact Analysis
• Regulatory Reporting

32. © 2018 IDERA, Inc. All rights reserved.
DATA MODELING TOOLS
▪ Database modeling – logical and physical
▪ Entity relationship diagramming
▪ Dimensional modeling
▪ Business Data Objects
▪ Automation with macros
▪ Model repository – version control and
collaboration
▪ Agile change management
▪ All major relational DBMS platforms
▪ Big data platforms

33. © 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.© 2018 IDERA, Inc. All rights reserved. 33© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential.
Demo