Database Design Patterns<br />Stephen Forte<br />
Speaker.Bio.ToString()<br />CTO and co-Founder of Corzen, Inc.<br />Sold to Wanted Technologies (TXV: WAN)<br />Internatio...
What Are Design Patterns?<br />What Are Design Patterns?<br />Design Patterns for Databases?<br />Data Model Patterns<br /...
What  Are Design Patterns?<br />Standard solutions to common problems in software design <br />Real-world, proven techniqu...
Where Did They Come From?<br />Roots in architecture field for following patterns in building buildings and cities<br />Ur...
Gang of Four<br />The Gang of Four (GoF) patterns are generally considered the foundation for all other patterns<br />Clas...
Why Use Design Patterns?<br />Can speed up the development process by providing tested, proven development paradigms <br /...
General Solution Templates<br />Design patterns provide general solutions<br />Documented in a format that doesn't require...
What Are Design Patterns?<br />Design Patterns for Databases?<br />Design Patterns for Databases?<br />Data Model Patterns...
Design Patterns for Databases?<br />Why not?<br />All the logic applies to database design as to software<br />They exist ...
Database Design Patterns<br />Data model patterns<br />Multi-User Highly Transactional design pattern<br />Slowly Changing...
What Are Design Patterns?<br />Design Patterns for Databases?<br />Data Model Patterns<br />Infrastructure Patterns<br />A...
Data Model Patterns	<br />Choosing the most efficient physical database design should be easy<br />Almost every database y...
Data Model Patterns<br />Multi-User Highly Transactional (OLTP)<br />Slowly Changing Dimensions (SCD)<br />Data Warehouse ...
Transactional Design Pattern<br />Any data model that collects data<br />Your typical OLTP<br />Even if “transactions” (BE...
Data Model Categories<br />Order Entry<br />People and Organizations<br />Accounting<br />Sports Scores <br />Time/Item T...
Slowly Changing Dimension Design Pattern<br />Anything that can be derived from an OLTP database for reporting or web view...
SCD Design Pattern<br />Optimized for reporting, Web access<br />Usually the #1 mistake is to try to merge this into one s...
SCD Databases<br />My former employer: Zagat<br />Raw data OLTP of about 4 million records<br />Published SCD of about 200...
demo<br />Building an SCD Database<br />
Data Warehouse Design Pattern<br />DSS data with pre-calculated rollups<br />Star schema<br />Simplest data warehouse sche...
Star Schema<br />Fact tables and dimension tables<br />Fact tables in star schema are mostly in third normal form, but dim...
Star Schema<br />
demo<br />Populating Fact Tables<br />
What Are Design Patterns?<br />Design Patterns for Databases?<br />Data Model Patterns<br />Infrastructure Patterns<br />A...
Infrastructure Patterns<br />Highly-available patterns<br />Data partitioning data patterns<br />Horizontal Partitioning<b...
Highly-Available Patterns<br />“% 9s” uptime: 99.999%<br />Translation<br />Less than 5 minutes of unscheduled downtime a ...
HA Patterns<br />PoF pattern avoids a single point of failure<br />Redundancy on all fronts-database, hardware<br />Log sh...
Data Partitioning Data Patterns<br />Solving the problem of data bloat<br />Advantages of moving less frequently accessed ...
Data Partitioning Data Patterns<br />Horizontal Partitioning<br />Vertical Partitioning<br />
Horizontal Partitioning<br />Table Partitioning <br />The process of splitting a table into pieces with the same structure...
Horizontal Partitioning<br />Good for archived data over a time series<br />Move to a separate device-can be a read only d...
Horizontal Partitioning<br />SQL Server 2005 Table and Index Partitioning <br />Keep data in one table<br />Groups of data...
DEMO<br />Horizontal Partitioning<br />
Data Partitioning Data Patterns<br />Horizontal Partitioning<br />Vertical Partitioning<br />
Vertical Partitioning<br />The process of splitting the table into different parts, with fewer columns, but the same amoun...
Vertical Partitioning<br />Great for larger data that is not often needed or changed<br />Will help with inserts, updates ...
DEMO<br />Vertical Partitioning<br />
Choosing the Correct Partitioning<br />Horizontal v Vertical Partitioning-When to use?<br />Horizontal good for archived d...
Summary<br />Design patterns are useful for database design<br />Designing a table schema follows one of  three patterns:<...
Q&A<br />
Thanks<br />Please fill out your evaluation form!Remember to put “Design Patterns” in the subject<br />stevef@orcsweb.com<...
Upcoming SlideShare
Loading in …5
×

Db design

1,294 views

Published on

I am not the author of this file but wanted to collect it for study purpose.

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,294
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Db design

  1. 1. Database Design Patterns<br />Stephen Forte<br />
  2. 2. Speaker.Bio.ToString()<br />CTO and co-Founder of Corzen, Inc.<br />Sold to Wanted Technologies (TXV: WAN)<br />International Conference Speaker for 10+ Years<br />RD, MVP and INETA Speaker <br />Wrote a few books <br />SQL Server 2005 Developers Guide (Microsoft Press)<br />Co-moderator & founder of NYC .NET Developers Group<br />http://www.nycdotnetdev.com<br />Former CTO of Zagat Survey <br />
  3. 3. What Are Design Patterns?<br />What Are Design Patterns?<br />Design Patterns for Databases?<br />Data Model Patterns<br />Infrastructure Patterns<br />Agenda<br />
  4. 4. What Are Design Patterns?<br />Standard solutions to common problems in software design <br />Real-world, proven techniques <br />About design and interaction of objects<br />Provide a communication platform concerning elegant, reusable solutions to commonly encountered programming challenges <br />
  5. 5. Where Did They Come From?<br />Roots in architecture field for following patterns in building buildings and cities<br />Urban planning<br />Christopher Alexander (1950s)<br />Started to develop for IT in the 1970s<br />Smalltalk contributed patterns<br />In 1987 Kent Beck and Ward Cunningham presented their “patterns” at the OOPSLA<br />Classic book: Steve Burbeck<br />Application Programming in Smalltalk-80 (1992)<br />
  6. 6. Gang of Four<br />The Gang of Four (GoF) patterns are generally considered the foundation for all other patterns<br />Classic Text: Design Patterns: Elements of Reusable Object-Oriented Software (1994)<br />Gamma<br />Helm<br />Vissides<br />Johnson<br />
  7. 7. Why Use Design Patterns?<br />Can speed up the development process by providing tested, proven development paradigms <br />Proven design elements<br />Used from other software engineering projects<br />Withstood scrutiny from the software development community<br />Mitigates a project’s risk<br />Makes a project more predictable<br />Allow developers to communicate using well-known and well understood names for software interactions <br />
  8. 8. General Solution Templates<br />Design patterns provide general solutions<br />Documented in a format that doesn't require specifics tied to a particular problem<br />Design patterns are only a template<br />Up to you to apply to a problem and implement<br />
  9. 9. What Are Design Patterns?<br />Design Patterns for Databases?<br />Design Patterns for Databases?<br />Data Model Patterns<br />Infrastructure Patterns<br />Agenda<br />
  10. 10. Design Patterns for Databases?<br />Why not?<br />All the logic applies to database design as to software<br />They exist but have not been documented as well as they have for software development<br />Not for C# code, but for actual databases<br />
  11. 11. Database Design Patterns<br />Data model patterns<br />Multi-User Highly Transactional design pattern<br />Slowly Changing Dimensions design pattern<br />Data Warehouse design pattern<br />Infrastructure patterns<br />Highly-available patterns<br />Point of Failure design pattern<br />Data partitioning patterns<br />Horizontal Partitioning<br />Vertical Partitioning<br />
  12. 12. What Are Design Patterns?<br />Design Patterns for Databases?<br />Data Model Patterns<br />Infrastructure Patterns<br />Agenda<br />Data Model Patterns<br />
  13. 13. Data Model Patterns <br />Choosing the most efficient physical database design should be easy<br />Almost every database you design should fall into one of three patterns<br />The most common mistake is mixing different models into the same database<br />
  14. 14. Data Model Patterns<br />Multi-User Highly Transactional (OLTP)<br />Slowly Changing Dimensions (SCD)<br />Data Warehouse (OLAP)<br />
  15. 15. Transactional Design Pattern<br />Any data model that collects data<br />Your typical OLTP<br />Even if “transactions” (BEGIN TRANS) are not used<br />Standard database normalization <br />First normal form<br />Second normal form<br />Third normal form<br />Can explore templates for data models based on the category of data you are collecting<br />
  16. 16. Data Model Categories<br />Order Entry<br />People and Organizations<br />Accounting<br />Sports Scores <br />Time/Item Tracking<br />Inventory<br />
  17. 17. Slowly Changing Dimension Design Pattern<br />Anything that can be derived from an OLTP database for reporting or web view<br />Biggest mistake is to run a query against loads of normalized “raw” data<br />Traditionally “flatter”<br />Some de-normalized aspects<br />Good implementation of XML <br />Highly indexed since the data is “published”<br />
  18. 18. SCD Design Pattern<br />Optimized for reporting, Web access<br />Usually the #1 mistake is to try to merge this into one system<br />ETL becomes very important in this design pattern<br />ETL<br />Published Database<br />OLTP<br />
  19. 19. SCD Databases<br />My former employer: Zagat<br />Raw data OLTP of about 4 million records<br />Published SCD of about 200K records, updated nightly as reviews came in<br />My current employer: Corzen<br />Raw data OLTP of about 800 million records<br />Published SCD data of about 12 million records, updated weekly as the time series dictates<br />Even a pure play e-commerce site like Amazon.com has a OLTP and SCD pattern:<br />OLTP: All ordering<br />SCD: Product catalog<br />
  20. 20. demo<br />Building an SCD Database<br />
  21. 21. Data Warehouse Design Pattern<br />DSS data with pre-calculated rollups<br />Star schema<br />Simplest data warehouse schema<br />Consisting of a single "fact table" with a compound primary key<br />One segment for each "dimension,“ with additional columns of additive, numeric facts<br />
  22. 22. Star Schema<br />Fact tables and dimension tables<br />Fact tables in star schema are mostly in third normal form, but dimensional tables are in de-normalized second normal form <br />If you want to normalize dimensional tables, they look like snowflakes (another pattern) <br />The same problems of relational databases arise!<br />Need complex queries! <br />Avoid the temptation to normalize the dimensional tables!!!<br />
  23. 23. Star Schema<br />
  24. 24. demo<br />Populating Fact Tables<br />
  25. 25. What Are Design Patterns?<br />Design Patterns for Databases?<br />Data Model Patterns<br />Infrastructure Patterns<br />Agenda<br />Infrastructure Patterns<br />
  26. 26. Infrastructure Patterns<br />Highly-available patterns<br />Data partitioning data patterns<br />Horizontal Partitioning<br />Vertical Partitioning<br />
  27. 27. Highly-Available Patterns<br />“% 9s” uptime: 99.999%<br />Translation<br />Less than 5 minutes of unscheduled downtime a year<br />
  28. 28. HA Patterns<br />PoF pattern avoids a single point of failure<br />Redundancy on all fronts-database, hardware<br />Log shipping<br />Microsoft SQL Server 2005 changes the equation<br />Always on Technologies available with SP1<br />Point of Failure Pattern<br />
  29. 29. Data Partitioning Data Patterns<br />Solving the problem of data bloat<br />Advantages of moving less frequently accessed data to different disks<br />Performance<br />Storage<br />
  30. 30. Data Partitioning Data Patterns<br />Horizontal Partitioning<br />Vertical Partitioning<br />
  31. 31. Horizontal Partitioning<br />Table Partitioning <br />The process of splitting a table into pieces with the same structure but fewer rows<br />Annual, Quarterly, Region<br />
  32. 32. Horizontal Partitioning<br />Good for archived data over a time series<br />Move to a separate device-can be a read only device<br />Will drive a developer crazy if you have to UNION it all back-but usually worth the tradeoff<br />
  33. 33. Horizontal Partitioning<br />SQL Server 2005 Table and Index Partitioning <br />Keep data in one table<br />Groups of data stored in other files<br />Only get a benefit if you have more than one disk<br />
  34. 34. DEMO<br />Horizontal Partitioning<br />
  35. 35. Data Partitioning Data Patterns<br />Horizontal Partitioning<br />Vertical Partitioning<br />
  36. 36. Vertical Partitioning<br />The process of splitting the table into different parts, with fewer columns, but the same amount of rows<br />Use a One-to-One relationship<br />
  37. 37. Vertical Partitioning<br />Great for larger data that is not often needed or changed<br />Will help with inserts, updates and deletes performance<br />Used by MySpace to break out audio, video, images, HTML, etc<br />http://www.baselinemag.com/article2/0,1540,2082921,00.asp<br />
  38. 38. DEMO<br />Vertical Partitioning<br />
  39. 39. Choosing the Correct Partitioning<br />Horizontal v Vertical Partitioning-When to use?<br />Horizontal good for archived data<br />Vertical good for inconsistent data that can be separated but still related to the primary key<br />You can use both-depending on your table <br />
  40. 40. Summary<br />Design patterns are useful for database design<br />Designing a table schema follows one of three patterns:<br />OLAP, SCD, OLTP<br />Infrastructure patterns help before and after your initial design<br />
  41. 41. Q&A<br />
  42. 42. Thanks<br />Please fill out your evaluation form!Remember to put “Design Patterns” in the subject<br />stevef@orcsweb.com<br />Stephenforte.net/owdasblog<br />

×