SlideShare a Scribd company logo
1 of 55
AVOIDING DATABASE MIGRATION
HELLby Edward Coleridge Smith
jedatron
edcs
WHAT IS A DATABASEMIGRATION?
on is an incremental and reversible change to
WHY USE DATABASEMIGRATIONS?
The Schema is Part of the Project
When you checkout a copy of the project, you get a copy
of the database schema too
WHY USE DATABASEMIGRATIONS?
No Need To Access Production Environments
Your database schema is part of your applications source
code
No need to log into your production database and grab
SQL dumps
WHY USE DATABASEMIGRATIONS?
The Database Schema is Versioned
The database schema is versioned with your application
source code
Migrations run in both directions so you can revert your
database schema back to any point in time
WHY USE DATABASEMIGRATIONS?
Dev & Production Environments Will Match
Since your database schema is described using code,
your database schema will be created in a repeatable and
predictable manner
WHY USE DATABASEMIGRATIONS?
Great For Testing!
Your test suite can create and destroy a database on the
fly and on demand
HOW DO I MAKE A MIGRATION?
Live Demo
<?php
use IlluminateDatabaseMigrationsMigration;
use IlluminateDatabaseSchemaBlueprint;
class CreateRolesTable extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
Schema::create('roles', function (Blueprint $table) {
});
}
WHAT DOES A MIGRATION LOOKLIKE?
<?php
use IlluminateDatabaseMigrationsMigration;
use IlluminateDatabaseSchemaBlueprint;
class CreateRolesTable extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
Schema::create('roles', function (Blueprint $table) {
$table->string('id');
$table->timestamps();
$table->softDeletes();
$table->primary('id');
});
}
WHAT DOES A MIGRATION LOOKLIKE?
WHAT IS A DATABASE SEED?
ed automatically inserts data into an applicati
WHY USE DATABASE SEEDS?
Makes Your Application Feel ‘Real’
Have you ever had to sign up to your own app during
development?
Have you ever had to insert a few rows into the database
for test runs?
WHY USE DATABASE SEEDS?
Production Data Stays In Production
Are you using a local copy of your production database for
development?
WHY USE DATABASE SEEDS?
Great For Testing!
You can quickly seed your database so your application is
ready for functional and acceptance tests
WHY USE DATABASE SEEDS?
Great For Inserting Application Settings
Does your application have settings like roles or scopes
which need to be in the database for it to work?
HOW DO I MAKE A SEED?
Live Demo
<?php
use IlluminateDatabaseSeeder;
class RolesTableSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
}
}
WHAT DOES A SEED LOOK LIKE?
<?php
use IlluminateDatabaseSeeder;
use IlluminateSupportFacadesDB;
use VascLabApiRole;
class RolesTableSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
DB::table('roles')->insert([
['id' => Role::ACCOUNT_ADMIN],
['id' => Role::REPORTS],
['id' => Role::REPORTS_READ],
['id' => Role::SUBSCRIPTION_ADMIN],
['id' => Role::MAIN_CONTACT]
]);
}
}
WHAT DOES A SEED LOOK LIKE?
<?php
use CarbonCarbon;
use IlluminateDatabaseSeeder;
use VascLabApiUser;
class UsersTableSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
User::create([
'salutation' => 'Mr',
'firstname' => 'Edward',
'lastname' => 'Coleridge Smith',
'email' => 'e.coleridgesmith@edcs.me',
'password' => bcrypt('password'),
'activated_at' => Carbon::now()
]);
}
}
WHAT DOES A SEED LOOK LIKE?
<?php
use IlluminateDatabaseSeeder;
use VascLabApiOrganisation;
class OrganisationsTableSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
factory(Organisation::class, 5)->create();
}
}
WHAT DOES A SEED LOOK LIKE?
WHAT DOES A SEED LOOK LIKE?
<?php
use IlluminateDatabaseSeeder;
class DevelopmentSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
$this->call(UsersTableSeeder::class);
$this->call(OrganisationsTableSeeder::class);
}
}
MODEL FACTORIES IN LARAVEL
<?php
/** @var $factory IlluminateDatabaseEloquentFactory */
use VascLabApiOrganisation;
$factory->define(Organisation::class, function (FakerGenerator $faker) {
$faker->addProvider(new FakerProvideren_GBAddress($faker));
return [
'name' => $faker->company,
'business_name' => $faker->company,
'address_1' => $faker->buildingNumber,
'address_2' => $faker->streetName,
'postal_town' => $faker->city,
'postcode' => $faker->postcode,
'country' => 'GB'
];
});
CATEGORISING YOUR SEEDS
Database Settings Seeds for Testing
Seeds for
Development or Demos
What Kinds of Data?
Entries which are required in
your database at all times
Entries which are required in
your database during testing
Faked data which can be use
by your developers or for
demos
How Much Data?
Data that’s required for your
application to work which
can’t live in config files
The bare minimum amount of
data that’s needed to test
your application
As much fake data as need
to make your application feel
‘real’
When Should
They Run?
Whenever you deploy and
update to your application
Before you run your tests
Manually run whenever you
need ‘fresh’ data
CATEGORISING YOUR SEEDS
Database Settings
Should contain entries which are required in your
database at all times
Should only seed data which is required for your
application to work and cannot live in a configuration file
Should be run whenever you deploy an update to your
application
CATEGORISING YOUR SEEDS
Seeds For Testing
Should contain a small sample of data required to run
tests against your application
Should be the bare minimum required for you application
to run
Should be run before you run your application test suite
CATEGORISING YOUR SEEDS
Seeds For Development and Demos
faked data which can be used for development and demo runs of y
uld contain as much fake data needed to make your application feel
be run on an ad-hoc basis whenever you need fresh data in your ap
DATABASE SEED STRATEGIES
Truncate Append
Remove all existing data from table
first
then re-seed
Check if row already exists in table
only add record if it doesn’t
DATABASE SEED STRATEGIES
Truncate
Remove all existing data from table
first
then re-seed
DATABASE SEED STRATEGIES
Append
Check if row already exists in table
only add record if it doesn’t
DATABASE SEED STRATEGIES
Truncate Append
Generally used for inserting data
used for development and testing
Generally used for settings that are
needed in both dev and production
WAIT, WHY NOT USEMIGRATIONS?
• A migration can only ever run once, unless
you refresh your database
• As your applications grow you’ll probably
need more settings
• More settings means more migrations
• Things get out of control fast!
Fixing This Was Not Fun!
WAIT, WHY NOT USEMIGRATIONS?
TRUNCATION STRATEGY SEEDS
DB::table($table)->truncate();
DB::statement('SET foreign_key_checks = 0');
DB::statement('SET foreign_key_checks = 1');
<?php
class DevelopmentSeeder extends Seeder
{
/**
* The tables which are affected by this seed.
*
* @var array
*/
protected $tables = [
'organisations',
'organisation_user',
'role_user',
'users',
'patients',
'reports',
'report_revisions',
'oauth_access_tokens',
'oauth_access_token_scope'
];
TRUNCATION STRATEGY SEEDS
TRUNCATION STRATEGY SEEDS
<?php
use IlluminateDatabaseSeeder;
use VascLabApiOrganisation;
class OrganisationsTableSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
factory(Organisation::class, 5)->create();
}
}
<?php
class DatabaseSeeder extends Seeder
{
/**
* The individual table seeders which should be run.
*
* @var array
*/
protected $seeds = [
RolesTableSeeder::class,
OauthGrantsTableSeeder::class,
OauthClientsTableSeeder::class,
OauthScopesTableSeeder::class
];
/**
* Run the database seeds.
*
* @return void
APPEND STRATEGY SEEDS
<?php
use CarbonCarbon;
use IlluminateDatabaseSeeder;
class OauthGrantsTableSeeder extends Seeder
{
/**
* The grants which are being created.
*
* @var array
*/
private $grants = [
'password',
'refresh_token',
'client_credentials'
];
/**
APPEND STRATEGY SEEDS
protected $tables = [];
/**
* The individual table seeders which should be run.
*
* @var array
*/
protected $seeds = [];
/**
* Ensures that the app is running in a development or test environment.
*
* @return void
*/
protected function shouldSeedsBeRun()
{
if (app()->environment() === 'production') {
$this->command->getOutput()->writeln(
"<error>This seeder cannot be run in production.</error>"
);
die;
}
}
/**
CODE DEFENSIVELY
PUTTING IT ALL TOGETHER
Live Demo
use CarbonCarbon;
use IlluminateDatabaseMigrationsMigration;
class InsertMissingSoloUnits extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
// Don't run this migration during testing
if (App::runningUnitTests()) {
return;
}
DB::unprepared(File::get(app_path() . '/../database/2015-07-16.sql'));
$pods = DB::table(‘__psl_part_numbers')
->join('pod_models', '__psl_part_numbers.model', '=', 'pod_models.name')
->join('pod_connectors', 'pod_models.id', '=', 'pod_connectors.model_id')
->get([
'__psl_part_numbers.psl_number AS ppid',
'__psl_part_numbers.sim_number AS sim_number',
PRO TIP, KEEP IT SIMPLE
THINGS ARE GOING WELL! WE NEED ANAPP…
• You’ve built an awesome product with Laravel
and business is going great!
• Your team has decided that it’s time to branch
out and build something with React Native
• You now need a ReST API to go alongside
your traditional web app
• Both products need to speak to the same
database…
COMPOSER TO THE RESCUE!
• We can use a Composer package to share
migrations and seeds between our projects
• Composer can access and install packages
from private Git repos
• Your original web app can install your
database as a main dependancy
• Your ReST API can install your database as a
dev dependancy
DATABASE PACKAGES
Web App ReST API
In charge of the production
database
Runs migrations and seeds in dev
and production environments
Only reads from production
database
Runs migrations and seeds in dev,
testing and staging environments
vagrant@homestead:~/Code/PP/database$ tree
.
|____composer.json
|____migrations
| |____2014_10_12_000000_create_users_table.php
| |____...
|____README.md
|____src
| |____Seeds
| | |____DatabaseSeeder.php
| | |____DevelopmentSeeder.php
| | |____Development
| | | |____UsersTableSeeder.php
| | | |____...
| | |____Production
| | | |____RolesTableSeeder.php
| | | |____...
STRUCTURING YOUR COMPOSERPACKAGE
SETTING UP THE COMPOSER PACKAGE
{
"name": "pod-point/database",
"description": "Migrations and seeds for the POD Point database",
"license": "proprietary",
"type": "library",
...
"autoload": {
"classmap": [
"migrations"
],
"psr-4": {
"PodPointDatabase": "src/"
}
}
}
{
"name": "pod-point/mis",
"description": "The POD Point MIS",
"license": "proprietary",
"type": "project",
"repositories": [
{
"type": "git",
"url": "git@github.com:pod-point/database.git"
}
],
"require": {
"pod-point/database": "^1.1",
...
},
...
}
INSTALLING THE COMPOSER PACKAGE
<?php
namespace PodPointInstallsWebAppProviders;
use IlluminateDatabaseEloquentFactory;
use IlluminateSupportServiceProvider;
class AppServiceProvider extends ServiceProvider
{
/**
* Bootstrap the application services.
*
* @return void
*/
public function boot()
{
$this->app->useDatabasePath(base_path('vendor/app/database/'));
}
/**
* Register the application services.
*
LARAVEL CONFIGURATION
INSTALLING THE COMPOSER PACKAGE
{
"name": "pod-point/api",
"description": "The POD Point API",
"license": "proprietary",
"type": "project",
"repositories": [
{
"type": "git",
"url": "git@github.com:pod-point/database.git"
}
],
...,
"require-dev": {
"pod-point/database": "^1.1",
...
}
}
<?php
namespace PodPointInstallsWebAppProviders;
use IlluminateDatabaseEloquentFactory;
use IlluminateSupportServiceProvider;
class AppServiceProvider extends ServiceProvider
{
/**
* Bootstrap the application services.
*
* @return void
*/
public function boot()
{
$this->app->useDatabasePath(base_path('vendor/app/database/'));
}
/**
* Register the application services.
*
LARAVEL CONFIGURATION
DATABASE PACKAGES
Web App ReST API
In charge of the production
database
Runs migrations and seeds in dev
and production environments
Only reads from production
database
Runs migrations and seeds in dev,
testing and staging environments
SUMMARY
Top Tips For Database Migrations
Only use migrations to manage your database schema
Avoid using raw SQL in your migrations at all costs
Keep your migrations as simple as possible
SUMMARY
Top Tips For Database Seeds
Remember that seeds can call other seeds, so it’s
possible to categorise them
You can use seeds in production to insert settings into the
database
Keep your seeds as simple as possible
SUMMARY
And Finally…
A set of well maintained migrations and seeds will save
you time in the long run
It’s possible to share your database files between projects
using composer
jedatron
edcs
to.edcs.me/db-migration-hell

More Related Content

What's hot

SharePoint 24x7x365 Architecting for High Availability, Fault Tolerance and D...
SharePoint 24x7x365 Architecting for High Availability, Fault Tolerance and D...SharePoint 24x7x365 Architecting for High Availability, Fault Tolerance and D...
SharePoint 24x7x365 Architecting for High Availability, Fault Tolerance and D...Eric Shupps
 
Acquia Drupal Certification
Acquia Drupal CertificationAcquia Drupal Certification
Acquia Drupal CertificationPhilip Norton
 
Hands-on-Lab: Adding Value to HBase with IBM InfoSphere BigInsights and BigSQL
Hands-on-Lab: Adding Value to HBase with IBM InfoSphere BigInsights and BigSQLHands-on-Lab: Adding Value to HBase with IBM InfoSphere BigInsights and BigSQL
Hands-on-Lab: Adding Value to HBase with IBM InfoSphere BigInsights and BigSQLPiotr Pruski
 
Creating other schema objects
Creating other schema objectsCreating other schema objects
Creating other schema objectsSyed Zaid Irshad
 
Skillwise-Spring framework 1
Skillwise-Spring framework 1Skillwise-Spring framework 1
Skillwise-Spring framework 1Skillwise Group
 
Sql Server 2008 Enhancements
Sql Server 2008 EnhancementsSql Server 2008 Enhancements
Sql Server 2008 Enhancementskobico10
 
Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…Aaron Shilo
 
DBA Brasil 1.0 - DBA Commands and Concepts That Every Developer Should Know
DBA Brasil 1.0 - DBA Commands and Concepts That Every Developer Should KnowDBA Brasil 1.0 - DBA Commands and Concepts That Every Developer Should Know
DBA Brasil 1.0 - DBA Commands and Concepts That Every Developer Should KnowAlex Zaballa
 
Oracle Database 12c - New Features for Developers and DBAs
Oracle Database 12c  - New Features for Developers and DBAsOracle Database 12c  - New Features for Developers and DBAs
Oracle Database 12c - New Features for Developers and DBAsAlex Zaballa
 
End-to-end Troubleshooting Checklist for Microsoft SQL Server
End-to-end Troubleshooting Checklist for Microsoft SQL ServerEnd-to-end Troubleshooting Checklist for Microsoft SQL Server
End-to-end Troubleshooting Checklist for Microsoft SQL ServerKevin Kline
 

What's hot (16)

SharePoint 24x7x365 Architecting for High Availability, Fault Tolerance and D...
SharePoint 24x7x365 Architecting for High Availability, Fault Tolerance and D...SharePoint 24x7x365 Architecting for High Availability, Fault Tolerance and D...
SharePoint 24x7x365 Architecting for High Availability, Fault Tolerance and D...
 
Acquia Drupal Certification
Acquia Drupal CertificationAcquia Drupal Certification
Acquia Drupal Certification
 
Oracle SQL Tuning
Oracle SQL TuningOracle SQL Tuning
Oracle SQL Tuning
 
Hands-on-Lab: Adding Value to HBase with IBM InfoSphere BigInsights and BigSQL
Hands-on-Lab: Adding Value to HBase with IBM InfoSphere BigInsights and BigSQLHands-on-Lab: Adding Value to HBase with IBM InfoSphere BigInsights and BigSQL
Hands-on-Lab: Adding Value to HBase with IBM InfoSphere BigInsights and BigSQL
 
Creating other schema objects
Creating other schema objectsCreating other schema objects
Creating other schema objects
 
Skillwise-Spring framework 1
Skillwise-Spring framework 1Skillwise-Spring framework 1
Skillwise-Spring framework 1
 
Developing for SP2013
Developing for SP2013Developing for SP2013
Developing for SP2013
 
Sql Server 2008 Enhancements
Sql Server 2008 EnhancementsSql Server 2008 Enhancements
Sql Server 2008 Enhancements
 
Apex und EBR
Apex und EBRApex und EBR
Apex und EBR
 
Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…
 
DBA Brasil 1.0 - DBA Commands and Concepts That Every Developer Should Know
DBA Brasil 1.0 - DBA Commands and Concepts That Every Developer Should KnowDBA Brasil 1.0 - DBA Commands and Concepts That Every Developer Should Know
DBA Brasil 1.0 - DBA Commands and Concepts That Every Developer Should Know
 
Oracle Database 12c - New Features for Developers and DBAs
Oracle Database 12c  - New Features for Developers and DBAsOracle Database 12c  - New Features for Developers and DBAs
Oracle Database 12c - New Features for Developers and DBAs
 
Spring batch
Spring batchSpring batch
Spring batch
 
ODTUG Webinar AWR Warehouse
ODTUG Webinar AWR WarehouseODTUG Webinar AWR Warehouse
ODTUG Webinar AWR Warehouse
 
JSP
JSPJSP
JSP
 
End-to-end Troubleshooting Checklist for Microsoft SQL Server
End-to-end Troubleshooting Checklist for Microsoft SQL ServerEnd-to-end Troubleshooting Checklist for Microsoft SQL Server
End-to-end Troubleshooting Checklist for Microsoft SQL Server
 

Similar to How To Avoid Database Migration Hell

Zend Framework 1.9 Setup & Using Zend_Tool
Zend Framework 1.9 Setup & Using Zend_ToolZend Framework 1.9 Setup & Using Zend_Tool
Zend Framework 1.9 Setup & Using Zend_ToolGordon Forsythe
 
Spring data jpa are used to develop spring applications
Spring data jpa are used to develop spring applicationsSpring data jpa are used to develop spring applications
Spring data jpa are used to develop spring applicationsmichaelaaron25322
 
Merging and Migrating: Data Portability from the Trenches
Merging and Migrating: Data Portability from the TrenchesMerging and Migrating: Data Portability from the Trenches
Merging and Migrating: Data Portability from the TrenchesAtlassian
 
PROGRAMMING IN JAVA -unit 5 -part I
PROGRAMMING IN JAVA -unit 5 -part IPROGRAMMING IN JAVA -unit 5 -part I
PROGRAMMING IN JAVA -unit 5 -part ISivaSankari36
 
Deploying opetus
Deploying opetusDeploying opetus
Deploying opetusGyani Feku
 
Jdbc in servlets
Jdbc in servletsJdbc in servlets
Jdbc in servletsNuha Noor
 
SOUG_Deployment__Automation_DB
SOUG_Deployment__Automation_DBSOUG_Deployment__Automation_DB
SOUG_Deployment__Automation_DBUniFabric
 
DevOps Drive-In: Automate Database Deployments in Your Continuous Delivery Pi...
DevOps Drive-In: Automate Database Deployments in Your Continuous Delivery Pi...DevOps Drive-In: Automate Database Deployments in Your Continuous Delivery Pi...
DevOps Drive-In: Automate Database Deployments in Your Continuous Delivery Pi...Serena Software
 
Handling Database Deployments
Handling Database DeploymentsHandling Database Deployments
Handling Database DeploymentsMike Willbanks
 
Part 7 packaging and deployment
Part 7 packaging and deploymentPart 7 packaging and deployment
Part 7 packaging and deploymenttechbed
 
E business suite r12.2 changes for database administrators
E business suite r12.2 changes for database administratorsE business suite r12.2 changes for database administrators
E business suite r12.2 changes for database administratorsSrinivasa Pavan Marti
 
E business suite r12.2 changes for database administrators
E business suite r12.2 changes for database administratorsE business suite r12.2 changes for database administrators
E business suite r12.2 changes for database administratorsSrinivasa Pavan Marti
 

Similar to How To Avoid Database Migration Hell (20)

Zend Framework 1.9 Setup & Using Zend_Tool
Zend Framework 1.9 Setup & Using Zend_ToolZend Framework 1.9 Setup & Using Zend_Tool
Zend Framework 1.9 Setup & Using Zend_Tool
 
Spring data jpa are used to develop spring applications
Spring data jpa are used to develop spring applicationsSpring data jpa are used to develop spring applications
Spring data jpa are used to develop spring applications
 
SSDT unleashed
SSDT unleashedSSDT unleashed
SSDT unleashed
 
Merging and Migrating: Data Portability from the Trenches
Merging and Migrating: Data Portability from the TrenchesMerging and Migrating: Data Portability from the Trenches
Merging and Migrating: Data Portability from the Trenches
 
WSS And Share Point For Developers
WSS And Share Point For DevelopersWSS And Share Point For Developers
WSS And Share Point For Developers
 
PROGRAMMING IN JAVA -unit 5 -part I
PROGRAMMING IN JAVA -unit 5 -part IPROGRAMMING IN JAVA -unit 5 -part I
PROGRAMMING IN JAVA -unit 5 -part I
 
Datamigration
DatamigrationDatamigration
Datamigration
 
Deploying opetus
Deploying opetusDeploying opetus
Deploying opetus
 
Pyramid patterns
Pyramid patternsPyramid patterns
Pyramid patterns
 
Jdbc in servlets
Jdbc in servletsJdbc in servlets
Jdbc in servlets
 
SOUG_Deployment__Automation_DB
SOUG_Deployment__Automation_DBSOUG_Deployment__Automation_DB
SOUG_Deployment__Automation_DB
 
DevOps Drive-In: Automate Database Deployments in Your Continuous Delivery Pi...
DevOps Drive-In: Automate Database Deployments in Your Continuous Delivery Pi...DevOps Drive-In: Automate Database Deployments in Your Continuous Delivery Pi...
DevOps Drive-In: Automate Database Deployments in Your Continuous Delivery Pi...
 
Jdbc
JdbcJdbc
Jdbc
 
Database CI/CD Pipeline
Database CI/CD PipelineDatabase CI/CD Pipeline
Database CI/CD Pipeline
 
Apache DeltaSpike: The CDI Toolbox
Apache DeltaSpike: The CDI ToolboxApache DeltaSpike: The CDI Toolbox
Apache DeltaSpike: The CDI Toolbox
 
Apache DeltaSpike the CDI toolbox
Apache DeltaSpike the CDI toolboxApache DeltaSpike the CDI toolbox
Apache DeltaSpike the CDI toolbox
 
Handling Database Deployments
Handling Database DeploymentsHandling Database Deployments
Handling Database Deployments
 
Part 7 packaging and deployment
Part 7 packaging and deploymentPart 7 packaging and deployment
Part 7 packaging and deployment
 
E business suite r12.2 changes for database administrators
E business suite r12.2 changes for database administratorsE business suite r12.2 changes for database administrators
E business suite r12.2 changes for database administrators
 
E business suite r12.2 changes for database administrators
E business suite r12.2 changes for database administratorsE business suite r12.2 changes for database administrators
E business suite r12.2 changes for database administrators
 

Recently uploaded

Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsUnveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsAlberto González Trastoy
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideChristina Lin
 
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...OnePlan Solutions
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...gurkirankumar98700
 
What is Binary Language? Computer Number Systems
What is Binary Language?  Computer Number SystemsWhat is Binary Language?  Computer Number Systems
What is Binary Language? Computer Number SystemsJheuzeDellosa
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantAxelRicardoTrocheRiq
 
Diamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with PrecisionDiamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with PrecisionSolGuruz
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...soniya singh
 
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackCloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackVICTOR MAESTRE RAMIREZ
 
DNT_Corporate presentation know about us
DNT_Corporate presentation know about usDNT_Corporate presentation know about us
DNT_Corporate presentation know about usDynamic Netsoft
 
Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language ModelsUnlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Modelsaagamshah0812
 
How To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.jsHow To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.jsAndolasoft Inc
 
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...harshavardhanraghave
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...MyIntelliSource, Inc.
 
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfThe Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfkalichargn70th171
 
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsJhone kinadey
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...stazi3110
 
How To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected WorkerHow To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected WorkerThousandEyes
 
Optimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVOptimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVshikhaohhpro
 

Recently uploaded (20)

Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsUnveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
 
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
 
What is Binary Language? Computer Number Systems
What is Binary Language?  Computer Number SystemsWhat is Binary Language?  Computer Number Systems
What is Binary Language? Computer Number Systems
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service Consultant
 
Diamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with PrecisionDiamond Application Development Crafting Solutions with Precision
Diamond Application Development Crafting Solutions with Precision
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
 
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackCloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStack
 
DNT_Corporate presentation know about us
DNT_Corporate presentation know about usDNT_Corporate presentation know about us
DNT_Corporate presentation know about us
 
Unlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language ModelsUnlocking the Future of AI Agents with Large Language Models
Unlocking the Future of AI Agents with Large Language Models
 
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS LiveVip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live
 
How To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.jsHow To Use Server-Side Rendering with Nuxt.js
How To Use Server-Side Rendering with Nuxt.js
 
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
 
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfThe Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
 
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial Goals
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
 
How To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected WorkerHow To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected Worker
 
Optimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVOptimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTV
 

How To Avoid Database Migration Hell

  • 1. AVOIDING DATABASE MIGRATION HELLby Edward Coleridge Smith
  • 3. WHAT IS A DATABASEMIGRATION? on is an incremental and reversible change to
  • 4. WHY USE DATABASEMIGRATIONS? The Schema is Part of the Project When you checkout a copy of the project, you get a copy of the database schema too
  • 5. WHY USE DATABASEMIGRATIONS? No Need To Access Production Environments Your database schema is part of your applications source code No need to log into your production database and grab SQL dumps
  • 6. WHY USE DATABASEMIGRATIONS? The Database Schema is Versioned The database schema is versioned with your application source code Migrations run in both directions so you can revert your database schema back to any point in time
  • 7. WHY USE DATABASEMIGRATIONS? Dev & Production Environments Will Match Since your database schema is described using code, your database schema will be created in a repeatable and predictable manner
  • 8. WHY USE DATABASEMIGRATIONS? Great For Testing! Your test suite can create and destroy a database on the fly and on demand
  • 9. HOW DO I MAKE A MIGRATION? Live Demo
  • 10. <?php use IlluminateDatabaseMigrationsMigration; use IlluminateDatabaseSchemaBlueprint; class CreateRolesTable extends Migration { /** * Run the migrations. * * @return void */ public function up() { Schema::create('roles', function (Blueprint $table) { }); } WHAT DOES A MIGRATION LOOKLIKE?
  • 11. <?php use IlluminateDatabaseMigrationsMigration; use IlluminateDatabaseSchemaBlueprint; class CreateRolesTable extends Migration { /** * Run the migrations. * * @return void */ public function up() { Schema::create('roles', function (Blueprint $table) { $table->string('id'); $table->timestamps(); $table->softDeletes(); $table->primary('id'); }); } WHAT DOES A MIGRATION LOOKLIKE?
  • 12. WHAT IS A DATABASE SEED? ed automatically inserts data into an applicati
  • 13. WHY USE DATABASE SEEDS? Makes Your Application Feel ‘Real’ Have you ever had to sign up to your own app during development? Have you ever had to insert a few rows into the database for test runs?
  • 14. WHY USE DATABASE SEEDS? Production Data Stays In Production Are you using a local copy of your production database for development?
  • 15. WHY USE DATABASE SEEDS? Great For Testing! You can quickly seed your database so your application is ready for functional and acceptance tests
  • 16. WHY USE DATABASE SEEDS? Great For Inserting Application Settings Does your application have settings like roles or scopes which need to be in the database for it to work?
  • 17. HOW DO I MAKE A SEED? Live Demo
  • 18. <?php use IlluminateDatabaseSeeder; class RolesTableSeeder extends Seeder { /** * Run the database seeds. * * @return void */ public function run() { } } WHAT DOES A SEED LOOK LIKE?
  • 19. <?php use IlluminateDatabaseSeeder; use IlluminateSupportFacadesDB; use VascLabApiRole; class RolesTableSeeder extends Seeder { /** * Run the database seeds. * * @return void */ public function run() { DB::table('roles')->insert([ ['id' => Role::ACCOUNT_ADMIN], ['id' => Role::REPORTS], ['id' => Role::REPORTS_READ], ['id' => Role::SUBSCRIPTION_ADMIN], ['id' => Role::MAIN_CONTACT] ]); } } WHAT DOES A SEED LOOK LIKE?
  • 20. <?php use CarbonCarbon; use IlluminateDatabaseSeeder; use VascLabApiUser; class UsersTableSeeder extends Seeder { /** * Run the database seeds. * * @return void */ public function run() { User::create([ 'salutation' => 'Mr', 'firstname' => 'Edward', 'lastname' => 'Coleridge Smith', 'email' => 'e.coleridgesmith@edcs.me', 'password' => bcrypt('password'), 'activated_at' => Carbon::now() ]); } } WHAT DOES A SEED LOOK LIKE?
  • 21. <?php use IlluminateDatabaseSeeder; use VascLabApiOrganisation; class OrganisationsTableSeeder extends Seeder { /** * Run the database seeds. * * @return void */ public function run() { factory(Organisation::class, 5)->create(); } } WHAT DOES A SEED LOOK LIKE?
  • 22. WHAT DOES A SEED LOOK LIKE? <?php use IlluminateDatabaseSeeder; class DevelopmentSeeder extends Seeder { /** * Run the database seeds. * * @return void */ public function run() { $this->call(UsersTableSeeder::class); $this->call(OrganisationsTableSeeder::class); } }
  • 23. MODEL FACTORIES IN LARAVEL <?php /** @var $factory IlluminateDatabaseEloquentFactory */ use VascLabApiOrganisation; $factory->define(Organisation::class, function (FakerGenerator $faker) { $faker->addProvider(new FakerProvideren_GBAddress($faker)); return [ 'name' => $faker->company, 'business_name' => $faker->company, 'address_1' => $faker->buildingNumber, 'address_2' => $faker->streetName, 'postal_town' => $faker->city, 'postcode' => $faker->postcode, 'country' => 'GB' ]; });
  • 24. CATEGORISING YOUR SEEDS Database Settings Seeds for Testing Seeds for Development or Demos What Kinds of Data? Entries which are required in your database at all times Entries which are required in your database during testing Faked data which can be use by your developers or for demos How Much Data? Data that’s required for your application to work which can’t live in config files The bare minimum amount of data that’s needed to test your application As much fake data as need to make your application feel ‘real’ When Should They Run? Whenever you deploy and update to your application Before you run your tests Manually run whenever you need ‘fresh’ data
  • 25. CATEGORISING YOUR SEEDS Database Settings Should contain entries which are required in your database at all times Should only seed data which is required for your application to work and cannot live in a configuration file Should be run whenever you deploy an update to your application
  • 26. CATEGORISING YOUR SEEDS Seeds For Testing Should contain a small sample of data required to run tests against your application Should be the bare minimum required for you application to run Should be run before you run your application test suite
  • 27. CATEGORISING YOUR SEEDS Seeds For Development and Demos faked data which can be used for development and demo runs of y uld contain as much fake data needed to make your application feel be run on an ad-hoc basis whenever you need fresh data in your ap
  • 28. DATABASE SEED STRATEGIES Truncate Append Remove all existing data from table first then re-seed Check if row already exists in table only add record if it doesn’t
  • 29. DATABASE SEED STRATEGIES Truncate Remove all existing data from table first then re-seed
  • 30. DATABASE SEED STRATEGIES Append Check if row already exists in table only add record if it doesn’t
  • 31. DATABASE SEED STRATEGIES Truncate Append Generally used for inserting data used for development and testing Generally used for settings that are needed in both dev and production
  • 32. WAIT, WHY NOT USEMIGRATIONS? • A migration can only ever run once, unless you refresh your database • As your applications grow you’ll probably need more settings • More settings means more migrations • Things get out of control fast!
  • 33. Fixing This Was Not Fun! WAIT, WHY NOT USEMIGRATIONS?
  • 34. TRUNCATION STRATEGY SEEDS DB::table($table)->truncate(); DB::statement('SET foreign_key_checks = 0'); DB::statement('SET foreign_key_checks = 1');
  • 35. <?php class DevelopmentSeeder extends Seeder { /** * The tables which are affected by this seed. * * @var array */ protected $tables = [ 'organisations', 'organisation_user', 'role_user', 'users', 'patients', 'reports', 'report_revisions', 'oauth_access_tokens', 'oauth_access_token_scope' ]; TRUNCATION STRATEGY SEEDS
  • 36. TRUNCATION STRATEGY SEEDS <?php use IlluminateDatabaseSeeder; use VascLabApiOrganisation; class OrganisationsTableSeeder extends Seeder { /** * Run the database seeds. * * @return void */ public function run() { factory(Organisation::class, 5)->create(); } }
  • 37. <?php class DatabaseSeeder extends Seeder { /** * The individual table seeders which should be run. * * @var array */ protected $seeds = [ RolesTableSeeder::class, OauthGrantsTableSeeder::class, OauthClientsTableSeeder::class, OauthScopesTableSeeder::class ]; /** * Run the database seeds. * * @return void APPEND STRATEGY SEEDS
  • 38. <?php use CarbonCarbon; use IlluminateDatabaseSeeder; class OauthGrantsTableSeeder extends Seeder { /** * The grants which are being created. * * @var array */ private $grants = [ 'password', 'refresh_token', 'client_credentials' ]; /** APPEND STRATEGY SEEDS
  • 39. protected $tables = []; /** * The individual table seeders which should be run. * * @var array */ protected $seeds = []; /** * Ensures that the app is running in a development or test environment. * * @return void */ protected function shouldSeedsBeRun() { if (app()->environment() === 'production') { $this->command->getOutput()->writeln( "<error>This seeder cannot be run in production.</error>" ); die; } } /** CODE DEFENSIVELY
  • 40. PUTTING IT ALL TOGETHER Live Demo
  • 41. use CarbonCarbon; use IlluminateDatabaseMigrationsMigration; class InsertMissingSoloUnits extends Migration { /** * Run the migrations. * * @return void */ public function up() { // Don't run this migration during testing if (App::runningUnitTests()) { return; } DB::unprepared(File::get(app_path() . '/../database/2015-07-16.sql')); $pods = DB::table(‘__psl_part_numbers') ->join('pod_models', '__psl_part_numbers.model', '=', 'pod_models.name') ->join('pod_connectors', 'pod_models.id', '=', 'pod_connectors.model_id') ->get([ '__psl_part_numbers.psl_number AS ppid', '__psl_part_numbers.sim_number AS sim_number', PRO TIP, KEEP IT SIMPLE
  • 42. THINGS ARE GOING WELL! WE NEED ANAPP… • You’ve built an awesome product with Laravel and business is going great! • Your team has decided that it’s time to branch out and build something with React Native • You now need a ReST API to go alongside your traditional web app • Both products need to speak to the same database…
  • 43. COMPOSER TO THE RESCUE! • We can use a Composer package to share migrations and seeds between our projects • Composer can access and install packages from private Git repos • Your original web app can install your database as a main dependancy • Your ReST API can install your database as a dev dependancy
  • 44. DATABASE PACKAGES Web App ReST API In charge of the production database Runs migrations and seeds in dev and production environments Only reads from production database Runs migrations and seeds in dev, testing and staging environments
  • 45. vagrant@homestead:~/Code/PP/database$ tree . |____composer.json |____migrations | |____2014_10_12_000000_create_users_table.php | |____... |____README.md |____src | |____Seeds | | |____DatabaseSeeder.php | | |____DevelopmentSeeder.php | | |____Development | | | |____UsersTableSeeder.php | | | |____... | | |____Production | | | |____RolesTableSeeder.php | | | |____... STRUCTURING YOUR COMPOSERPACKAGE
  • 46. SETTING UP THE COMPOSER PACKAGE { "name": "pod-point/database", "description": "Migrations and seeds for the POD Point database", "license": "proprietary", "type": "library", ... "autoload": { "classmap": [ "migrations" ], "psr-4": { "PodPointDatabase": "src/" } } }
  • 47. { "name": "pod-point/mis", "description": "The POD Point MIS", "license": "proprietary", "type": "project", "repositories": [ { "type": "git", "url": "git@github.com:pod-point/database.git" } ], "require": { "pod-point/database": "^1.1", ... }, ... } INSTALLING THE COMPOSER PACKAGE
  • 48. <?php namespace PodPointInstallsWebAppProviders; use IlluminateDatabaseEloquentFactory; use IlluminateSupportServiceProvider; class AppServiceProvider extends ServiceProvider { /** * Bootstrap the application services. * * @return void */ public function boot() { $this->app->useDatabasePath(base_path('vendor/app/database/')); } /** * Register the application services. * LARAVEL CONFIGURATION
  • 49. INSTALLING THE COMPOSER PACKAGE { "name": "pod-point/api", "description": "The POD Point API", "license": "proprietary", "type": "project", "repositories": [ { "type": "git", "url": "git@github.com:pod-point/database.git" } ], ..., "require-dev": { "pod-point/database": "^1.1", ... } }
  • 50. <?php namespace PodPointInstallsWebAppProviders; use IlluminateDatabaseEloquentFactory; use IlluminateSupportServiceProvider; class AppServiceProvider extends ServiceProvider { /** * Bootstrap the application services. * * @return void */ public function boot() { $this->app->useDatabasePath(base_path('vendor/app/database/')); } /** * Register the application services. * LARAVEL CONFIGURATION
  • 51. DATABASE PACKAGES Web App ReST API In charge of the production database Runs migrations and seeds in dev and production environments Only reads from production database Runs migrations and seeds in dev, testing and staging environments
  • 52. SUMMARY Top Tips For Database Migrations Only use migrations to manage your database schema Avoid using raw SQL in your migrations at all costs Keep your migrations as simple as possible
  • 53. SUMMARY Top Tips For Database Seeds Remember that seeds can call other seeds, so it’s possible to categorise them You can use seeds in production to insert settings into the database Keep your seeds as simple as possible
  • 54. SUMMARY And Finally… A set of well maintained migrations and seeds will save you time in the long run It’s possible to share your database files between projects using composer

Editor's Notes

  1. Turn of notifications on your laptop Close down stuff you don’t need Delete unused seeds and migrations from Laracon project Run `composer dumpautoload` Open Sequel Pro and delete all the tables in VascLab database Open iTerm tab and `cd Code/ECS/laracon` for migration and seed creation demos Zoom in 4 times Open iTerm tab and `cd Code/ECS/vasclab/api` to run example migration and seeds Zoom in 4 times
  2. You work for POD Point We make electric vehicle charging stations POD Point ship Laravel based web applications and ReST APIs and React Native mobile apps Personal interests are ReST APIs and Single Page Applications built using JavasScript We’re hiring! Crash course on migrations and seeds Different organisation techniques Time for questions at end
  3. Incremental change to schema Should be reversible Larval makes a distinction between migrations and seeds
  4. New devs will have copy of DB schema when project is checked out War story: Don’t have to constantly hassle your new colleagues on your first day for stuff
  5. No need to access production DB - this provides a safety net for your developers War story: Accidentally deleting production data
  6. Migrations can run in both directions War story: Installs tool foreign key broken migration could be rolled back
  7. Dev environment matches production War story: Setting incorrect data type when storing Facebook users IDs on David Bowie Is app
  8. Can use in-memory SQLite War story: Getting great test coverage on VascLab API using functional testing
  9. Explain the command - php artisan make migration Try to avoid copying and pasting migrations Filename and autoloading is fussy Go to console for demo
  10. Scaffolded file from Artisan command Extends Laravel’s base migration Two methods one for running the migration and one for reversing it Up method is called during a forward migration Scroll to next code block Down method is called during a rollback migration Always write code to reverse your migration
  11. The Schema class has methods for creating, updating and deleting tables The Blueprint class has methods for nearly everything you could want to do Timestamps method adds `created_at` and `updated_at` Soft Deletes adds `deleted_at` We’re manually creating an index in this migration too Scroll to next code block Down method drops table created by migration - opposite action to the up method Try and avoid using raw SQL in here
  12. Inserts data into database Usually used for testing Can be used for initial application setup though
  13. You can use seeds to create users and business data to use during development, testing and demos Works well with functional and acceptance tests as well as IRL run throughs
  14. Don’t do this! What happens if your laptop is stolen? Letting your dev team log into the production database all the time can be risky Seeds can be setup to make your application feel ‘real’ War story: POD Point MIS demo seeds - used to create real looking data, tailored to the customer without breaking data protection rules
  15. You can seed your database so it contains just the right amount of data to run tests against War story: VascLab API functional tests using Laravel test runner and a seeded database - hit 100% code coverage without too much effort
  16. You can use seeds to insert data like this into both dev and production databases Seeds can be run multiple times so it’s a better strategy than using a migration War story: All my OAuth projects use this method to insert scopes
  17. Using Artisan commands is a good habit to get into Could use IDE to create class which extends Laravel’s base seeder though Go to console for demo
  18. The Artisan command makes an empty class like this Has a run method in it Any code in the run method will executed when the seed is called
  19. You can use the query builder to insert data
  20. You could use models directly to create new records
  21. You could use model factories to create multiple new records
  22. A seed can also call other seeds using the call method and the classname of the seed This is great for grouping your seeds up into different categories
  23. Factories describe models in your application They live in the databases directory Can have as many factories as you need Can even have multiple factories for a single model if needed They’re a callback which return an array of data Data is used to create a new instance of a model Can use Faker to populate array These are great for testing Almost make database seeding in testing obsolete
  24. Tend to use three categories for my database seeds Settings, Testing and Development
  25. Database settings are things that your application needs to work Generally config items that need to be in the database rather than a config file OAuth Scopes User Roles Database settings should be seeded upon every deployment of your app Could integrate this as part of your build process War Story: use is in VascLab API to insert OAuth related settings which need to be in the database
  26. Test seeds are for creating a database to test your application with Users Products Could use model factories instead now Seeds for testing should be run before every test Good idea to run each test on a fresh database War Story: Use the in POD Point ordering tool and MIS to get the database ready for extensive functional testing
  27. Seeds for development or demos are used to run you application in browser Loads of users Loads of products and orders if we’re using the e-commerce example again Seeds for Development and Demoing should just be run manually Whenever you need a fresh version of your database - i.e. when you mess up and need to start over! War Story: Use this in POD Point MIS demo to prepare the database for sales meeting Can hone the seeds to make the data look ‘nice’ for your customers War Story: Use this in VascLab API and other projects so that I don’t need to carry around sensitive customer data with me
  28. Two strategies when seeding data in production - truncation or appending
  29. The truncation strategy simply truncates then re-seeds This is a super simple technique - like using a lump hammer on a small nail Probably need to disable foreign key checks during seeding and re-enable them afterwards Needs to be done with an SQL query Can cause problems if your using zero-downtime deployment techniques Could wrap your seeds in a transaction Generally not supported though - `TRUNCATE` cannot run in a transaction in MySQL as it causes an implicit commit
  30. The append strategy checks whether or not a row exists in the table already Bit more complex when it comes to deleting seeded data No need to disable foreign key checks Works well with zero-downtime deployments We only delete data that’s no longer needed
  31. Tend to use truncate for dev data It’s a quick and simple strategy Tend to use append for seeds that need to be run in production It works well with our zero downtime deployments
  32. Don’t use migrations to insert data into your database May seem like a great idea at first - a path to migration hell You may think you only need a few roles or scopes in your database Need to be able to adapt application Migration can only run once
  33. Here’s the old list of migrations for our management tool at POD Point We used migrations to insert data and as the business grew so did the data requirements The list just goes on and on and on! Getting the list of migrations down to a collection of schema only operations took many weeks to fix
  34. Laravel’s query builder is able to truncate a database table Truncating individual tables is a relatively simple process You probably have foreign keys on your tables so you’re going to need to disable these Not so good since you’ve got to use a raw SQL statement
  35. This is an example of a truncation seeder This is the development seeder class, so all it does is call other seeds Has an array of tables which need to be truncated Scroll to next block Has an array containing all the individual seeds we’d like to run Scroll to next block This method does the truncating Scroll to next block This method calls the truncate method and runs each individual seed
  36. This is an example of one of the individual seeds called in the previous example It’s super simple because it will always be run on a truncated table Just using a model factory to create some fake organisations to test an API out with
  37. This is an example of an append seeder This is the production seeder class, so all it does is call other seeds Similar to the truncation seeder, but only calls individual seeds No need to handle any truncation Has the same seeds property containing the names of the seeds to run Scroll to next block The run method now just calls each individual seed
  38. This is an example of one of the individual seeds called in the previous example It’s inserting the grant types needed for my API project This seed will be run in development and production We have an array of grants which need to be in the database at all times Scroll to next block The run method is a bit more complicated Needs to delete grants that should not be in the database and then only insert rows that don’t exist Uses where not in to delete Then inserts the diff of existing rows and the rows that we want in the DB
  39. Need to use force argument to seed in production It’s a good idea to make it impossible to run development seeds in a production environment Make sure your development seeders always make a call to a method like this before doing anything destructive
  40. Show empty database Run migrations, show schema has been created Run main seeder, show settings are in DB Run dev seeder, show all the faked data in DB
  41. This is an example of a bad migration from an old project at POD Point If your migrations or seeds start to look like this, you’re going to have a bad time Raw SQL in a migration Crazy selects and joins all over the place Hundreds of lines in a single method
  42. We have a Laravel web app which has direct access to the database via models The web app works brilliantly and we need a mobile app We could make the API endpoints a part of our web app Authentication is going to be different API has a different job to do it makes sense to setup two projects for this We’ve got an awesome set of migrations and seeds that our API now needs access to! War Story: Actual scenario which happened at POD Point We would make changes to MIS which changed the database schema Forgot to make sure these changes didn’t affect our API 😳
  43. Use Composer to get around this issue This will be a quick overview of how we can do this Complicated topic but worth knowing You can create a repo which contains your migrations and seeds, this can be private too We use GitHub ad POD Point and once you’ve setup an API key for Composer to use, it can access your private repositories
  44. Original web app maintains control over the database Runs all migrations and seeds against the database Second project installs the database package as a dev dependancy Only runs migrations and seeds during development and testing Must use semVer and testing to ensure both your applications can access database Complicated strategy Lots of moving parts
  45. This is a file tree showing the structure of a database package Truncated because there were loads of files! It’s quite similar to Laravel’s database directory We have a folder containing all our migrations Then we have a src directory which contains our seeds We put the seed classes in an src directory so we can autoload them using PSR-4
  46. Composer package has a file called composer.json Contains information about the package and dependancies Laravel users might be familiar with this Tell Composer that we’d like to autoload migrations using classmap Autoload seeds using PSR-4 Settings used are mostly lifted from Laravel’s configuration
  47. This is an example composer file for our main web app The database package is a main dependancy since we would like it to get installed in production Note the repositories namespace We don’t really want people accessing our database schema It’s a private GitHub repo which Composer can access No need to publish stuff like this on Packagist
  48. Tell Laravel where the database files are located We do this in the app service provider Can create another service provider for this though Make a call to use database path and specify the location of package Can manually specify path of migrations and seeds in Artisan command Bit annoying though, so setting up like this can be helpful
  49. This is an example composer file for our main new ReST API The database package is a dev dependancy since we don’t want it installed on production The database package will only get installed in environments like dev and testing Still using a private package since we don’t want to share our schema
  50. It’s a good idea to specify the database path in a service provider Stops you from having to tell Laravel where your seeds are!
  51. Summarise packages Web app can now manage the schema API project can use the schema during development and testing API can also create a schema in CI environments if needed
  52. Thanks for listening Can download slides from the internet Got time for a couple of questions