SQL Database Design For Developers at PhpTek 2025.pptx

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

SQL Database Design
For Developers
Scott Keck-Warren
php[tek] 2025
https://tinyurl.com/db-design-Cascadia-PHP
https://scot...

1.Never Worked With a DBA On a Project
2.Every Project Has At Least One SQL
Database

1.Never Worked With a DBA On a Project
2.Every Project Has At Least One SQL
Database

My (Early) Relationship to SQL
Database:
Less Than Ideal

How Did I Get Better?

Trial By Fire

Problems
• “Random” slowness
• Data inconsistencies
• Weird Bugs

SQL Database Design
For Developers

ScottKeckWarren

class ScottKeckWarren
{
}

use PsrHugHuggable; // PSR-8
class ScottKeckWarren implements Huggable
{
public function hug(Huggable $h)
{
// only if ask...

use PsrHugHuggable; // PSR-8
class ScottKeckWarren implements Huggable
{
use Partner;
public function hug(Huggable $h)
{
/...

use PsrHugHuggable; // PSR-8
class ScottKeckWarren implements Huggable
{
use Partner;
use BestDadEver;
public function hug...

use PsrHugHuggable; // PSR-8
class ScottKeckWarren implements Huggable
{
use Partner;
use BestDadEver;
public function hug...

use PsrHugHuggable; // PSR-8
class ScottKeckWarren implements Huggable
{
use Partner;
use BestDadEver;
use PhpDeveloper;
p...

use PsrHugHuggable; // PSR-8
class ScottKeckWarren implements Huggable
{
use Partner;
use BestDadEver;
use PhpDeveloper;
u...

scott.keck-warren.com

Scott’s Rules For Database
Design

Never Worked With a DBA On
a Project

Scott’s Rules For Database Design
1. Backup Your Database and Test It
2. Normalize Your Database For Data
Deduplication
3....

Backup Your Database and
Test It

“Backups are Unimportant” “Backups are the Most Important”
One “Lost” File

mysql> delete from users where id = 1;

mysql> delete from users where id > 1;

Backup Your Database and Test It
3 - 2 - 1 Backups
1. 3 copies
2. 2 different mediums
3. 1 “offsite”

Backup Your Database and Test It
Router
Router
Internet
Internet
Internet
Switch
Switch
Application
Backups

Backup Your Database and Test It
Router
Router
Internet
Internet
Internet
Switch
Switch
Application
Backups

Backup Your Database and Test It
Router
Router
Internet
Internet
Internet
Switch
Switch
Application
Backups

Backup Your Database and Test It
Router
Router
Internet
Internet
Internet
Switch
Switch
Application
Backups

Backup Your Database and Test It
Internet
Internet
Internet
Switch
Switch
Application
Backups

Backup Your Database and Test It
• At Least Monthly Test Restoring It
• Ideally as part of DR plan

How Should I Backup My Data?

Backup Your Database and Test It
• 4 Types of Backups
• Continuous Data Protection - Constantly backing up changes
• Full ...

Backup Your Database and Test It
• Continuous Data Protection
• Ideal method to backup data
• Database don’t generally sup...

Backup Your Database and Test It
• Always Full Backups
• Makes it really easy to do a restore
• When DB is small - perfect...

Grandfather-Father-Son
Backups

Grandfather-Father-Son Backups
• Grandfather - full offsite backup for long term storage
• Father - full onsite backup for...

Full Backups In MySQL
mysqldump —single-transaction > full_backup.sql

$Partial Backups In MySQL mysqldump —single-transaction —-where=“updated_at>={last_full}” > partial_backup.sql$

Partial Backups In MySQL
update users set x=“y” where id = 1
users
/var/log/mysql/mysql-bin.######

Partial Backups In MySQL
# in /etc/mysql/mysql.conf.d/mysqld.cnf
Log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days =...

Partial Backups In MySQL
mysqldump —single-transaction --flush-logs
--master-data=2 > full_backup.sql

Partial Backups In MySQL
mysqlbinlog /var/log/mysql/mysql-bin.000002 |
mysql -uroot -p mydb

Normalize Your Database For
Data Deduplication

Users Table

Users Table
• Email address
• Password
• Active state
• Hire Date
• Listing of previous passwords
• Office Name
• Office C...

Users Table
• Email address (string)
• Password (string)
• Active state (string)
• Hire Date (string)
• Listing of previou...

Users Table
email
passwor
d
activ
e
hire_date
previous_
password
office_name office_phone office_city office_zip
alice@exa...

Normalize Your Database For Data
Deduplication
“[T]he process of structuring a relational
database in accordance with a se...

Normalize Your Database For Data
Deduplication
• UNF: Unnormalized form
• 1NF: First normal form
• 2NF: Second normal form...

Normalize Your Database For Data
Deduplication
• UNF: Unnormalized form
• 1NF: First normal form
• 2NF: Second normal form...

Normalize Your Database For Data
Deduplication
• Boyce–Codd Normal Form:
• X should be a superkey for every
functional dep...

Unnormalized Form

Unnormalized Form
• A table doesn’t meet any of the conditions of normalization
• Essentially a spreadsheet
email
passwor
...

First Normal Form (1NF)

First Normal Form (1NF)
1. The table contains a unique identifier, also called the primary key, that
is used to identify t...

1NF - users
email password active hire_date
previous_p
assword
office_
name
office_phone office_city office_zip
alice@exa
...

1NF - users
• In MOST cases, a unique identifier should be:
• Auto-incrementing int -> good “fast” solution
• UUID -> slow...

1NF - users
id email password active hire_date
previous_
password
office_
name
office_phone
office_cit
y
office_zip
1 alic...

1NF - users
id email password active hire_date
previous_
password
office_
name
office_phone
office_cit
y
office_zip
1 alic...

1NF - user_password_histories

1NF - user_password_histories
id user_id password

1NF - user_password_histories
id user_id password
1 1 hash1
2 1 hash5
3 1 hash6
4 2 hash2
5 2 hash7
6 2 hash8
7 3 hash3
8 ...

1NF - users
id email password active hire_date
previous_
password
office_
name
office_phone
office_cit
y
office_zip
1 alic...

1NF - users
id email password active hire_date
office_
name
office_phone office_city office_zip
1 alice@exa
mple.com
hash1...

Second Normal Form (2NF)

Second Normal Form (2NF)
1. Is already in 1NF
2. All the non-key columns are dependent on the primary key of the
table

Second Normal Form (2NF)
id email password active hire_date
office_
name
office_phone office_city office_zip
1 alice@exa
m...

2nd - offices
id name phone city zip
1 Main Office
555-555-5555
Saginaw 48609
2 main office 5555555555 Saginaw 48609
3 Man...

2NF - users
id email password active hire_date
office_
name
office_phone office_city office_zip
1 alice@exa
mple.com
hash1...

2NF - users
id email password active hire_date
office_
name
office_phone
office_cit
y
office_zip office_id
1 alice@exa
mpl...

2NF - users
id email password active hire_date office_id
1 alice@example.c
om
hash1 1 1/1/2024 1
2 avery@example.c
om
NULL...

Third Normal Form (3NF)

Third Normal Form (3NF)
1. Is already in 2NF
2. It contains columns that are non-transitively dependent on the
primary key

3NF - offices
id name phone city zip
1 Main Office
555-555-5555
Saginaw 48609
2 main office 5555555555 Saginaw 48609
3 Man...

3NF - zips
id city
48609 Saginaw
48640 Midland
48642 Midland
48901 Lansing

3NF - zips
id city state
48609 Saginaw MI
48640 Midland MI
48642 Midland MI
48901 Lansing MI

Old Table Structure

New Table Structure

Use The Database Engine to
Keep Data Clean

Why?

Why?
$validated = $request->validate([
"email" => "required|unique:users|max:255",
]);

mysql> insert into users
(password)
values
(“just a password?");
Query OK, 1 row affected (0.01 sec)

mysql> insert into users
(password)
values
(“just a password?");
Query OK, 1 row affected (0.01 sec)

Garbage In -> Garbage Out

Garbage Data -> Bugs

To Prevent Bugs:
Make The Database
Work For Us

id email password active hire_date office_id
1 alice@example.c
om
hash1 1 1/1/2024 1
2 avery@example.c
om
NULL 1 8/11/2024...

Use Correct Column Types

Use Correct Column Types
id email password active hire_date office_id
1 alice@example.c
om
hash1 1 1/1/2024 1
2 avery@exam...

Use Correct Column Types
• Numeric: INT, TINYINT, BIGINT, FLOAT, REAL, etc.
• Date/Time: DATE, TIME, DATETIME, etc.
• Stri...

Use Correct Column Types
create table users (
id int primary key auto_increment,
email char(255),
password char(255),
acti...

Use Correct Column Types
create table users (
id int primary key auto_increment,
email char(255),
password char(255),
acti...

Use Correct Column Types
mysql> insert into users (hire_date) values ("tuesday");
ERROR 1292 (22007): Incorrect date value...

Use NOT NULL for Required
Fields

Use NOT NULL for Required Fields
mysql> insert into users
(password)
values
(“just a password?");
Query OK, 1 row affected...

Use NOT NULL for Required Fields
create table users (
id int primary key auto_increment,
email char(255),
password char(25...

Use NOT NULL for Required Fields
create table users (
id int primary key auto_increment,
email char(255) not null,
passwor...

Use NOT NULL for Required Fields
mysql>
insert into users
(password)
values
(“just a password?”);
ERROR 1364 (HY000): Fiel...

Use NOT NULL for Required Fields
mysql>
insert into users
(password)
values
(“just a password?”);
ERROR 1364 (HY000): Fiel...

Use NOT NULL for Required Fields
mysql>
insert into users
(email, password)
values
(“s@s”, "just a password?");
ERROR 1364...

Use UNIQUE for Unique Values

Use UNIQUE for Unique Values
mysql> insert into users (email) values ("scott@keck-warren.com");
Query OK, 1 row affected (...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active tinyint(1),
hire_date ...

Use UNIQUE for Unique Values
create table users (
id int primary key auto_increment,
email char(255),
password char(255),
...

Use UNIQUE for Unique Values
mysql> insert into users (email) values ("scott@keck-warren.com");
ERROR 1062 (23000): Duplic...

Use UNIQUE for Unique Values
create table users (
id int primary key auto_increment,
email char(255),
password char(255),
...

Use Foreign Keys For
References To Other Tables

Use Foreign Keys For References To Other Tables
id name phone city zip_id
1 Main Office
555-555-5555
Saginaw 48609
2 main ...

Use Foreign Keys For References To Other Tables
id name phone city zip_id
1 Main Office
555-555-5555
Saginaw 48609
2 main ...

Use Foreign Keys For References To Other Tables
id name phone city zip_id
1 Main Office
555-555-5555
Saginaw 48609
2 main ...

Use Foreign Keys For References To Other Tables
id name phone city zip_id
1 Main Office
555-555-5555
Saginaw 48609

Constraints - Foreign Keys

select count(*)
from users;
select name, email, offices.name
from users
inner join offices
on users.office_id = offices.id;

Orphaned Rows

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active tinyint(1),
hire_date ...

mysql> insert into users (email, office_id) values ("s@kw.com", 1000000);
Query OK, 1 row affected (0.03 sec)

mysql> insert into users (email, office_id) values ("s@kw.com", 1000000);
Query OK, 1 row affected (0.03 sec)

Foreign Key Constraints

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

Foreign Key Constraints
mysql> insert into users (email, office_id) values ("s@kw.com", 1000000);
ERROR 1452 (23000): Cann...

Foreign Key Constraints
mysql> insert into users (email, office_id) values ("s@kw.com", 1000000);
ERROR 1452 (23000): Cann...

Foreign Key Constraints
mysql> insert into users (email, office_id) values ("s@kw.com", 1);
Query OK, 1 row affected (0.01...

Foreign Key Constraints
mysql> delete from offices where id = 1;
ERROR 1451 (23000): Cannot delete or update a parent row:...

Foreign Key Constraints
mysql> delete from offices where id = 1;
ERROR 1451 (23000): Cannot delete or update a parent row:...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

Foreign Key Constraints
mysql> insert into user_password_histories (user_id, password) values (2, "test1");
Query OK, 1 ro...

Foreign Key Constraints
mysql> delete from users where id = 2;
Query OK, 1 row affected (0.01 sec)
mysql> select * from us...

Foreign Key Constraints
mysql> delete from users where id = 2;
Query OK, 1 row affected (0.01 sec)
mysql> select * from us...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

Downsides to Constraints

Performance

Use Triggers For Complex
Requirements

Use Triggers For Complex
Requirements
• Triggers add additional power to DB
• Operate based on create, update, or delete

Use Triggers For Complex
Requirements
id email password active hire_date office_id
1 alice@example.c
om
hash1 1 1/1/2024 1...

Use Triggers For Complex
Requirements
CREATE TRIGGER users_before_insert_valid_active
BEFORE INSERT ON users FOR EACH ROW
...

Use Triggers For Complex
Requirements
CREATE TRIGGER users_before_insert_validate_active
BEFORE INSERT ON users FOR EACH R...

Use Triggers For Complex
Requirements
CREATE TRIGGER users_before_insert_validate_active
BEFORE INSERT ON users FOR EACH R...

Use Triggers For Complex
Requirements
CREATE TRIGGER users_before_insert_validate_active
BEFORE INSERT ON users FOR EACH R...

Use Triggers For Complex
Requirements
CREATE TRIGGER users_before_insert_validate_active
BEFORE INSERT ON users FOR EACH R...

Use Triggers For Complex
Requirements
CREATE TRIGGER users_before_insert_validate_active
BEFORE INSERT ON users FOR EACH R...

Use Triggers For Complex
Requirements
CREATE TRIGGER users_before_update_validate_active
BEFORE UPDATE ON users FOR EACH R...

Use Triggers For Complex
Requirements
mysql> insert into users (active) values (2);
ERROR 1644 (45000): active must be 0 o...

Use Triggers For Complex
Requirements
mysql> insert into users (active) values (2);
ERROR 1644 (45000): active must be 0 o...

type = ?
Require fields 1,2,3
1
Require fields 2,5,11
2

Proactively Add Indexes to
Keep Queries Performant

Indexes in Databases

select *
from users
where hire_date = "2023-12-03"

Indexes in Databases

Indexes in Databases

Indexes in Databases
2023-12-03
Index: hire_date
2023-12-04
2023-12-05
2023-12-06

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

create table users (
id int primary key auto_increment,
email char(255),
password char(255),
active char(255),
hire_date d...

Start Out Simple

Start Out Simple
• We “automatically” create indexes for:
• All foreign key relationships
• Any column that’s searched

Start Out Simple
• Every month or so run database tools to find missing indexes
• set global log_queries_not_using_indexes...

Denormalize Your Database
For Speed

Denormalize Your Database For
Speed
“[D]enormalization is the process of trying
to improve the read performance of
a datab...

Denormalize Your Database For
Speed
Office Jan Feb March … Nov Dec
Charges (USD) 10 7 8 … 8 7
Credits (USD) 2 0 0 … 1 1
Co...

Denormalize Your Database For
Speed
Office Jan Feb March … Nov Dec
Charges (USD) 10 7 8 … 8 7
Credits (USD) 2 0 0 … 1 1
Co...

Denormalize Your Database For
Speed
• What we found:
• 10s of millions charges
• 100s of thousand credits
• 10s of thousan...

Methods of Denormalization
1. Pre-joining
2. Table Splitting
3. Adding derived and redundant columns
4. Mirrored tables

Pre-joining
create table charges (
id int primary key auto_increment,
amount not null int,
office_id not null int,
created...

Pre-joining
create table charges (
id int primary key auto_increment,
amount not null int,
office_id not null int,
office_...

Table Splitting
create table charges (
id int primary key auto_increment,
amount not null int,
office_id not null int,
cre...

Table Splitting
create table charges (
id int primary key auto_increment,
amount not null int,
office_id not null int,
des...

Adding Derived and Redundant
Columns
create table charges (
id int primary key auto_increment,
amount not null int,
office...

Adding Derived and Redundant
Columns
create table charges (
id int primary key auto_increment,
amount not null int,
office...

Mirrored Tables
Office Jan Feb March … Nov Dec
Charges (USD) 10 7 8 … 8 7
Credits (USD) 2 0 0 … 1 1
Costs (USD) 2 2 2 … 2 ...

Mirrored Tables
create table monthly_stats (
id int primary key auto_increment,
month_and_year varchar(255) not null,
char...

Mirrored Tables
Office Jan Feb March … Nov Dec
Charges (USD) 10 7 8 … 8 7
Credits (USD) 2 0 0 … 1 1
Costs (USD) 2 2 2 … 2 ...

Generating Denormalize Data
1. Delayed
1. Generate using cron job (every day or hour)
2. “Instantly”
1. Internal logic ins...

What You Need to Know

What You Need to Know
1. Backup Your Database and Test It
2. Normalize Your Database For Data
Deduplication
3. Use The Dat...

What You Need to Know
1. The table contains a unique identifier, also called the primary key, that
is used to identify the...

What You Need to Know
• Make the DB Work With You
• Correct Column Types
• NOT NULL for Required Fields
• UNIQUE for Uniqu...

What You Need to Know
• Use indexes on commonly searched columns
• Start simple

What You Need to Know
• Pre-joining
• Table Splitting
• Adding derived and redundant columns
• Mirrored tables

2924 West Road, East Lansing, Michigan 48823

Thank The Organizers

Thank The Speakers

Questions/Follow Me
• Questions?
• Please rate the talk
• https://tinyurl.com/db-design-tek-2024

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1049688 views
Storytelling For The Web: Integrate... by Chiara Aliotta 964375 views
Artificial Intelligence, Data and C... by OECD Directorate ... 895413 views
How to Leverage AI to Boost Employe... by SocialHRCamp 377996 views
2024 State of Marketing Report – by... by Marius Sescu 214368 views
Everything You Need To Know About C... by Expeed Software 232744 views

View on Slideshare

1 of 204 Ad

View on Slideshare

1 of 204 Ad