The document discusses best practices for managing database migrations and seeds in Laravel projects. It recommends:
- Using migrations only for database schema changes and keeping them simple and reversible.
- Categorizing seeds into different types for settings, testing data, and development data. Seeds can insert data and call other seeds.
- Sharing migrations and seeds between projects using a Composer package for the database files. This allows multiple projects to use the same database schema.
- Following strategies like truncation for development seeds and append-only for settings to prevent issues from duplicate data.
- Defensively coding seeds to prevent running in production and only insert required data.
Well maintained migrations and seeds
3. WHAT IS A DATABASEMIGRATION?
on is an incremental and reversible change to
4. WHY USE DATABASEMIGRATIONS?
The Schema is Part of the Project
When you checkout a copy of the project, you get a copy
of the database schema too
5. WHY USE DATABASEMIGRATIONS?
No Need To Access Production Environments
Your database schema is part of your applications source
code
No need to log into your production database and grab
SQL dumps
6. WHY USE DATABASEMIGRATIONS?
The Database Schema is Versioned
The database schema is versioned with your application
source code
Migrations run in both directions so you can revert your
database schema back to any point in time
7. WHY USE DATABASEMIGRATIONS?
Dev & Production Environments Will Match
Since your database schema is described using code,
your database schema will be created in a repeatable and
predictable manner
11. <?php
use IlluminateDatabaseMigrationsMigration;
use IlluminateDatabaseSchemaBlueprint;
class CreateRolesTable extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
Schema::create('roles', function (Blueprint $table) {
$table->string('id');
$table->timestamps();
$table->softDeletes();
$table->primary('id');
});
}
WHAT DOES A MIGRATION LOOKLIKE?
12. WHAT IS A DATABASE SEED?
ed automatically inserts data into an applicati
13. WHY USE DATABASE SEEDS?
Makes Your Application Feel ‘Real’
Have you ever had to sign up to your own app during
development?
Have you ever had to insert a few rows into the database
for test runs?
14. WHY USE DATABASE SEEDS?
Production Data Stays In Production
Are you using a local copy of your production database for
development?
15. WHY USE DATABASE SEEDS?
Great For Testing!
You can quickly seed your database so your application is
ready for functional and acceptance tests
16. WHY USE DATABASE SEEDS?
Great For Inserting Application Settings
Does your application have settings like roles or scopes
which need to be in the database for it to work?
19. <?php
use IlluminateDatabaseSeeder;
use IlluminateSupportFacadesDB;
use VascLabApiRole;
class RolesTableSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
DB::table('roles')->insert([
['id' => Role::ACCOUNT_ADMIN],
['id' => Role::REPORTS],
['id' => Role::REPORTS_READ],
['id' => Role::SUBSCRIPTION_ADMIN],
['id' => Role::MAIN_CONTACT]
]);
}
}
WHAT DOES A SEED LOOK LIKE?
20. <?php
use CarbonCarbon;
use IlluminateDatabaseSeeder;
use VascLabApiUser;
class UsersTableSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
User::create([
'salutation' => 'Mr',
'firstname' => 'Edward',
'lastname' => 'Coleridge Smith',
'email' => 'e.coleridgesmith@edcs.me',
'password' => bcrypt('password'),
'activated_at' => Carbon::now()
]);
}
}
WHAT DOES A SEED LOOK LIKE?
22. WHAT DOES A SEED LOOK LIKE?
<?php
use IlluminateDatabaseSeeder;
class DevelopmentSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
$this->call(UsersTableSeeder::class);
$this->call(OrganisationsTableSeeder::class);
}
}
24. CATEGORISING YOUR SEEDS
Database Settings Seeds for Testing
Seeds for
Development or Demos
What Kinds of Data?
Entries which are required in
your database at all times
Entries which are required in
your database during testing
Faked data which can be use
by your developers or for
demos
How Much Data?
Data that’s required for your
application to work which
can’t live in config files
The bare minimum amount of
data that’s needed to test
your application
As much fake data as need
to make your application feel
‘real’
When Should
They Run?
Whenever you deploy and
update to your application
Before you run your tests
Manually run whenever you
need ‘fresh’ data
25. CATEGORISING YOUR SEEDS
Database Settings
Should contain entries which are required in your
database at all times
Should only seed data which is required for your
application to work and cannot live in a configuration file
Should be run whenever you deploy an update to your
application
26. CATEGORISING YOUR SEEDS
Seeds For Testing
Should contain a small sample of data required to run
tests against your application
Should be the bare minimum required for you application
to run
Should be run before you run your application test suite
27. CATEGORISING YOUR SEEDS
Seeds For Development and Demos
faked data which can be used for development and demo runs of y
uld contain as much fake data needed to make your application feel
be run on an ad-hoc basis whenever you need fresh data in your ap
28. DATABASE SEED STRATEGIES
Truncate Append
Remove all existing data from table
first
then re-seed
Check if row already exists in table
only add record if it doesn’t
31. DATABASE SEED STRATEGIES
Truncate Append
Generally used for inserting data
used for development and testing
Generally used for settings that are
needed in both dev and production
32. WAIT, WHY NOT USEMIGRATIONS?
• A migration can only ever run once, unless
you refresh your database
• As your applications grow you’ll probably
need more settings
• More settings means more migrations
• Things get out of control fast!
35. <?php
class DevelopmentSeeder extends Seeder
{
/**
* The tables which are affected by this seed.
*
* @var array
*/
protected $tables = [
'organisations',
'organisation_user',
'role_user',
'users',
'patients',
'reports',
'report_revisions',
'oauth_access_tokens',
'oauth_access_token_scope'
];
TRUNCATION STRATEGY SEEDS
36. TRUNCATION STRATEGY SEEDS
<?php
use IlluminateDatabaseSeeder;
use VascLabApiOrganisation;
class OrganisationsTableSeeder extends Seeder
{
/**
* Run the database seeds.
*
* @return void
*/
public function run()
{
factory(Organisation::class, 5)->create();
}
}
37. <?php
class DatabaseSeeder extends Seeder
{
/**
* The individual table seeders which should be run.
*
* @var array
*/
protected $seeds = [
RolesTableSeeder::class,
OauthGrantsTableSeeder::class,
OauthClientsTableSeeder::class,
OauthScopesTableSeeder::class
];
/**
* Run the database seeds.
*
* @return void
APPEND STRATEGY SEEDS
38. <?php
use CarbonCarbon;
use IlluminateDatabaseSeeder;
class OauthGrantsTableSeeder extends Seeder
{
/**
* The grants which are being created.
*
* @var array
*/
private $grants = [
'password',
'refresh_token',
'client_credentials'
];
/**
APPEND STRATEGY SEEDS
39. protected $tables = [];
/**
* The individual table seeders which should be run.
*
* @var array
*/
protected $seeds = [];
/**
* Ensures that the app is running in a development or test environment.
*
* @return void
*/
protected function shouldSeedsBeRun()
{
if (app()->environment() === 'production') {
$this->command->getOutput()->writeln(
"<error>This seeder cannot be run in production.</error>"
);
die;
}
}
/**
CODE DEFENSIVELY
41. use CarbonCarbon;
use IlluminateDatabaseMigrationsMigration;
class InsertMissingSoloUnits extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
// Don't run this migration during testing
if (App::runningUnitTests()) {
return;
}
DB::unprepared(File::get(app_path() . '/../database/2015-07-16.sql'));
$pods = DB::table(‘__psl_part_numbers')
->join('pod_models', '__psl_part_numbers.model', '=', 'pod_models.name')
->join('pod_connectors', 'pod_models.id', '=', 'pod_connectors.model_id')
->get([
'__psl_part_numbers.psl_number AS ppid',
'__psl_part_numbers.sim_number AS sim_number',
PRO TIP, KEEP IT SIMPLE
42. THINGS ARE GOING WELL! WE NEED ANAPP…
• You’ve built an awesome product with Laravel
and business is going great!
• Your team has decided that it’s time to branch
out and build something with React Native
• You now need a ReST API to go alongside
your traditional web app
• Both products need to speak to the same
database…
43. COMPOSER TO THE RESCUE!
• We can use a Composer package to share
migrations and seeds between our projects
• Composer can access and install packages
from private Git repos
• Your original web app can install your
database as a main dependancy
• Your ReST API can install your database as a
dev dependancy
44. DATABASE PACKAGES
Web App ReST API
In charge of the production
database
Runs migrations and seeds in dev
and production environments
Only reads from production
database
Runs migrations and seeds in dev,
testing and staging environments
46. SETTING UP THE COMPOSER PACKAGE
{
"name": "pod-point/database",
"description": "Migrations and seeds for the POD Point database",
"license": "proprietary",
"type": "library",
...
"autoload": {
"classmap": [
"migrations"
],
"psr-4": {
"PodPointDatabase": "src/"
}
}
}
47. {
"name": "pod-point/mis",
"description": "The POD Point MIS",
"license": "proprietary",
"type": "project",
"repositories": [
{
"type": "git",
"url": "git@github.com:pod-point/database.git"
}
],
"require": {
"pod-point/database": "^1.1",
...
},
...
}
INSTALLING THE COMPOSER PACKAGE
48. <?php
namespace PodPointInstallsWebAppProviders;
use IlluminateDatabaseEloquentFactory;
use IlluminateSupportServiceProvider;
class AppServiceProvider extends ServiceProvider
{
/**
* Bootstrap the application services.
*
* @return void
*/
public function boot()
{
$this->app->useDatabasePath(base_path('vendor/app/database/'));
}
/**
* Register the application services.
*
LARAVEL CONFIGURATION
49. INSTALLING THE COMPOSER PACKAGE
{
"name": "pod-point/api",
"description": "The POD Point API",
"license": "proprietary",
"type": "project",
"repositories": [
{
"type": "git",
"url": "git@github.com:pod-point/database.git"
}
],
...,
"require-dev": {
"pod-point/database": "^1.1",
...
}
}
50. <?php
namespace PodPointInstallsWebAppProviders;
use IlluminateDatabaseEloquentFactory;
use IlluminateSupportServiceProvider;
class AppServiceProvider extends ServiceProvider
{
/**
* Bootstrap the application services.
*
* @return void
*/
public function boot()
{
$this->app->useDatabasePath(base_path('vendor/app/database/'));
}
/**
* Register the application services.
*
LARAVEL CONFIGURATION
51. DATABASE PACKAGES
Web App ReST API
In charge of the production
database
Runs migrations and seeds in dev
and production environments
Only reads from production
database
Runs migrations and seeds in dev,
testing and staging environments
52. SUMMARY
Top Tips For Database Migrations
Only use migrations to manage your database schema
Avoid using raw SQL in your migrations at all costs
Keep your migrations as simple as possible
53. SUMMARY
Top Tips For Database Seeds
Remember that seeds can call other seeds, so it’s
possible to categorise them
You can use seeds in production to insert settings into the
database
Keep your seeds as simple as possible
54. SUMMARY
And Finally…
A set of well maintained migrations and seeds will save
you time in the long run
It’s possible to share your database files between projects
using composer
Turn of notifications on your laptop
Close down stuff you don’t need
Delete unused seeds and migrations from Laracon project
Run `composer dumpautoload`
Open Sequel Pro and delete all the tables in VascLab database
Open iTerm tab and `cd Code/ECS/laracon` for migration and seed creation demos
Zoom in 4 times
Open iTerm tab and `cd Code/ECS/vasclab/api` to run example migration and seeds
Zoom in 4 times
You work for POD Point
We make electric vehicle charging stations
POD Point ship Laravel based web applications and ReST APIs and React Native mobile apps
Personal interests are ReST APIs and Single Page Applications built using JavasScript
We’re hiring!
Crash course on migrations and seeds
Different organisation techniques
Time for questions at end
Incremental change to schema
Should be reversible
Larval makes a distinction between migrations and seeds
New devs will have copy of DB schema when project is checked out
War story: Don’t have to constantly hassle your new colleagues on your first day for stuff
No need to access production DB - this provides a safety net for your developers
War story: Accidentally deleting production data
Migrations can run in both directions
War story: Installs tool foreign key broken migration could be rolled back
Dev environment matches production
War story: Setting incorrect data type when storing Facebook users IDs on David Bowie Is app
Can use in-memory SQLite
War story: Getting great test coverage on VascLab API using functional testing
Explain the command - php artisan make migration
Try to avoid copying and pasting migrations
Filename and autoloading is fussy
Go to console for demo
Scaffolded file from Artisan command
Extends Laravel’s base migration
Two methods one for running the migration and one for reversing it
Up method is called during a forward migration
Scroll to next code block
Down method is called during a rollback migration
Always write code to reverse your migration
The Schema class has methods for creating, updating and deleting tables
The Blueprint class has methods for nearly everything you could want to do
Timestamps method adds `created_at` and `updated_at`
Soft Deletes adds `deleted_at`
We’re manually creating an index in this migration too
Scroll to next code block
Down method drops table created by migration - opposite action to the up method
Try and avoid using raw SQL in here
Inserts data into database
Usually used for testing
Can be used for initial application setup though
You can use seeds to create users and business data to use during development, testing and demos
Works well with functional and acceptance tests as well as IRL run throughs
Don’t do this!
What happens if your laptop is stolen?
Letting your dev team log into the production database all the time can be risky
Seeds can be setup to make your application feel ‘real’
War story: POD Point MIS demo seeds - used to create real looking data, tailored to the customer without breaking data protection rules
You can seed your database so it contains just the right amount of data to run tests against
War story: VascLab API functional tests using Laravel test runner and a seeded database - hit 100% code coverage without too much effort
You can use seeds to insert data like this into both dev and production databases
Seeds can be run multiple times so it’s a better strategy than using a migration
War story: All my OAuth projects use this method to insert scopes
Using Artisan commands is a good habit to get into
Could use IDE to create class which extends Laravel’s base seeder though
Go to console for demo
The Artisan command makes an empty class like this
Has a run method in it
Any code in the run method will executed when the seed is called
You can use the query builder to insert data
You could use models directly to create new records
You could use model factories to create multiple new records
A seed can also call other seeds using the call method and the classname of the seed
This is great for grouping your seeds up into different categories
Factories describe models in your application
They live in the databases directory
Can have as many factories as you need
Can even have multiple factories for a single model if needed
They’re a callback which return an array of data
Data is used to create a new instance of a model
Can use Faker to populate array
These are great for testing
Almost make database seeding in testing obsolete
Tend to use three categories for my database seeds
Settings, Testing and Development
Database settings are things that your application needs to work
Generally config items that need to be in the database rather than a config file
OAuth Scopes
User Roles
Database settings should be seeded upon every deployment of your app
Could integrate this as part of your build process
War Story: use is in VascLab API to insert OAuth related settings which need to be in the database
Test seeds are for creating a database to test your application with
Users
Products
Could use model factories instead now
Seeds for testing should be run before every test
Good idea to run each test on a fresh database
War Story: Use the in POD Point ordering tool and MIS to get the database ready for extensive functional testing
Seeds for development or demos are used to run you application in browser
Loads of users
Loads of products and orders if we’re using the e-commerce example again
Seeds for Development and Demoing should just be run manually
Whenever you need a fresh version of your database - i.e. when you mess up and need to start over!
War Story: Use this in POD Point MIS demo to prepare the database for sales meeting
Can hone the seeds to make the data look ‘nice’ for your customers
War Story: Use this in VascLab API and other projects so that I don’t need to carry around sensitive customer data with me
Two strategies when seeding data in production - truncation or appending
The truncation strategy simply truncates then re-seeds
This is a super simple technique - like using a lump hammer on a small nail
Probably need to disable foreign key checks during seeding and re-enable them afterwards
Needs to be done with an SQL query
Can cause problems if your using zero-downtime deployment techniques
Could wrap your seeds in a transaction
Generally not supported though - `TRUNCATE` cannot run in a transaction in MySQL as it causes an implicit commit
The append strategy checks whether or not a row exists in the table already
Bit more complex when it comes to deleting seeded data
No need to disable foreign key checks
Works well with zero-downtime deployments
We only delete data that’s no longer needed
Tend to use truncate for dev data
It’s a quick and simple strategy
Tend to use append for seeds that need to be run in production
It works well with our zero downtime deployments
Don’t use migrations to insert data into your database
May seem like a great idea at first - a path to migration hell
You may think you only need a few roles or scopes in your database
Need to be able to adapt application
Migration can only run once
Here’s the old list of migrations for our management tool at POD Point
We used migrations to insert data and as the business grew so did the data requirements
The list just goes on and on and on!
Getting the list of migrations down to a collection of schema only operations took many weeks to fix
Laravel’s query builder is able to truncate a database table
Truncating individual tables is a relatively simple process
You probably have foreign keys on your tables so you’re going to need to disable these
Not so good since you’ve got to use a raw SQL statement
This is an example of a truncation seeder
This is the development seeder class, so all it does is call other seeds
Has an array of tables which need to be truncated
Scroll to next block
Has an array containing all the individual seeds we’d like to run
Scroll to next block
This method does the truncating
Scroll to next block
This method calls the truncate method and runs each individual seed
This is an example of one of the individual seeds called in the previous example
It’s super simple because it will always be run on a truncated table
Just using a model factory to create some fake organisations to test an API out with
This is an example of an append seeder
This is the production seeder class, so all it does is call other seeds
Similar to the truncation seeder, but only calls individual seeds
No need to handle any truncation
Has the same seeds property containing the names of the seeds to run
Scroll to next block
The run method now just calls each individual seed
This is an example of one of the individual seeds called in the previous example
It’s inserting the grant types needed for my API project
This seed will be run in development and production
We have an array of grants which need to be in the database at all times
Scroll to next block
The run method is a bit more complicated
Needs to delete grants that should not be in the database and then only insert rows that don’t exist
Uses where not in to delete
Then inserts the diff of existing rows and the rows that we want in the DB
Need to use force argument to seed in production
It’s a good idea to make it impossible to run development seeds in a production environment
Make sure your development seeders always make a call to a method like this before doing anything destructive
Show empty database
Run migrations, show schema has been created
Run main seeder, show settings are in DB
Run dev seeder, show all the faked data in DB
This is an example of a bad migration from an old project at POD Point
If your migrations or seeds start to look like this, you’re going to have a bad time
Raw SQL in a migration
Crazy selects and joins all over the place
Hundreds of lines in a single method
We have a Laravel web app which has direct access to the database via models
The web app works brilliantly and we need a mobile app
We could make the API endpoints a part of our web app
Authentication is going to be different
API has a different job to do it makes sense to setup two projects for this
We’ve got an awesome set of migrations and seeds that our API now needs access to!
War Story: Actual scenario which happened at POD Point
We would make changes to MIS which changed the database schema
Forgot to make sure these changes didn’t affect our API 😳
Use Composer to get around this issue
This will be a quick overview of how we can do this
Complicated topic but worth knowing
You can create a repo which contains your migrations and seeds, this can be private too
We use GitHub ad POD Point and once you’ve setup an API key for Composer to use, it can access your private repositories
Original web app maintains control over the database
Runs all migrations and seeds against the database
Second project installs the database package as a dev dependancy
Only runs migrations and seeds during development and testing
Must use semVer and testing to ensure both your applications can access database
Complicated strategy
Lots of moving parts
This is a file tree showing the structure of a database package
Truncated because there were loads of files!
It’s quite similar to Laravel’s database directory
We have a folder containing all our migrations
Then we have a src directory which contains our seeds
We put the seed classes in an src directory so we can autoload them using PSR-4
Composer package has a file called composer.json
Contains information about the package and dependancies
Laravel users might be familiar with this
Tell Composer that we’d like to autoload migrations using classmap
Autoload seeds using PSR-4
Settings used are mostly lifted from Laravel’s configuration
This is an example composer file for our main web app
The database package is a main dependancy since we would like it to get installed in production
Note the repositories namespace
We don’t really want people accessing our database schema
It’s a private GitHub repo which Composer can access
No need to publish stuff like this on Packagist
Tell Laravel where the database files are located
We do this in the app service provider
Can create another service provider for this though
Make a call to use database path and specify the location of package
Can manually specify path of migrations and seeds in Artisan command
Bit annoying though, so setting up like this can be helpful
This is an example composer file for our main new ReST API
The database package is a dev dependancy since we don’t want it installed on production
The database package will only get installed in environments like dev and testing
Still using a private package since we don’t want to share our schema
It’s a good idea to specify the database path in a service provider
Stops you from having to tell Laravel where your seeds are!
Summarise packages
Web app can now manage the schema
API project can use the schema during development and testing
API can also create a schema in CI environments if needed
Thanks for listening
Can download slides from the internet
Got time for a couple of questions