Continuous Integration, the minimum viable product
Upcoming SlideShare
Loading in...5
×
 

Continuous Integration, the minimum viable product

on

  • 29,812 views

What does it mean to 'do' Continuous Integration? It used to be enough to execute your unit tests in CI. But the bar is steadily raising for engineering practices. In the last decade we've seen ...

What does it mean to 'do' Continuous Integration? It used to be enough to execute your unit tests in CI. But the bar is steadily raising for engineering practices. In the last decade we've seen tremendous improvements inacceptance testing. JavaScript is now a platform in it's own right. Cloudcomputing is now vital. There's growing interest in deployment to prod.So Continuous Integration is under more pressure than ever. As the bar slowly raises for engineering practices, we ll present 2011's minimum viable feature set for Continuous Integration

Statistics

Views

Total Views
29,812
Views on SlideShare
4,148
Embed Views
25,664

Actions

Likes
6
Downloads
87
Comments
3

20 Embeds 25,664

http://seleniumhq.wordpress.com 24977
http://feeds.feedburner.com 583
http://translate.googleusercontent.com 32
http://deimos.worthanote.com 24
http://webcache.googleusercontent.com 20
http://twitter.com 6
http://tweetedtimes.com 5
http://www.b12345.somee.com 4
https://twitter.com 2
http://seleniumhq.wordpress.com} {1707154861|||pingback 1
http://seleniumhq.wordpress.com} {1707154861|||pingback 1
http://www.slideshare.net 1
http://www.newsblur.com 1
http://172.16.0.243:15871 1
http://10.113.32.26:15871 1
http://seleniumhq.wordpress.com HTTP 1
http://feedproxy.google.com 1
http://us-w1.rockmelt.com 1
http://translate.google.com 1
http://www.linkedin.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • @simpsonjulian Thanks
    Are you sure you want to
    Your message goes here
    Processing…
  • Hi there,

    No problem! I'm talking about database migration tools for changing the schema of your database. There's lots of those.

    For doing data migrations, that's really up to you.

    HTH,

    Julian
    Are you sure you want to
    Your message goes here
    Processing…
  • Thanks for your slides. I have a question when you say use data migrations but don't write your own tool. Not sure if I get it,. We have some old tool in excel now we want to migrate it to mysql. I was thinking of writing some kind of tool or script to migrate it. Tool in python or java so that we can track it on hudson if it fails in migration.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Explain that chris is not an imaginary friend\nDo the experience check\nExplain the vagueness in pitching\nask them to ask questions\nexplai\n \n\n\n
  • Explain Chris\nDo the experience check\nAsk for questions\nTake a deep fucking breath\n\n
  • This is the existing metaphor for CI\n“Toyoda’s automatic loom stopped whenever the thread of the warp was snapped, .. the loom could not produce defectives, because it had an automatic stopping device”\nBut CI is not about machines, it’s about peopl;e\n
  • \n
  • This is CM done wrong\n
  • Which leads to this\n
  • it should be this\n
  • The 1840’s called, they want your CI status back\n
  • 1. CI Principles\nBuild everything, whenever it changes\nSingle Source of Truth\nthe database schema counts as everything\nthe tests count as everything\nCheck in frequently\nWe all build together (ie Branching is Bad!)\nFEEDBACK!\n\n
  • 1. CI Principles\nBuild everything, whenever it changes\nSingle Source of Truth\nthe database schema counts as everything\nthe tests count as everything\nCheck in frequently\nWe all build together (ie Branching is Bad!)\nFEEDBACK!\n\n
  • 1. CI Principles\nBuild everything, whenever it changes\nSingle Source of Truth\nthe database schema counts as everything\nthe tests count as everything\nCheck in frequently\nWe all build together (ie Branching is Bad!)\nFEEDBACK!\n\n
  • 1. CI Principles\nBuild everything, whenever it changes\nSingle Source of Truth\nthe database schema counts as everything\nthe tests count as everything\nCheck in frequently\nWe all build together (ie Branching is Bad!)\nFEEDBACK!\n\n
  • 1. CI Principles\nBuild everything, whenever it changes\nSingle Source of Truth\nthe database schema counts as everything\nthe tests count as everything\nCheck in frequently\nWe all build together (ie Branching is Bad!)\nFEEDBACK!\n\n
  • 1. CI Principles\nBuild everything, whenever it changes\nSingle Source of Truth\nthe database schema counts as everything\nthe tests count as everything\nCheck in frequently\nWe all build together (ie Branching is Bad!)\nFEEDBACK!\n\n
  • \n
  • \n
  • AR migrations, dbdeploy, dbdeploy.net, tarantino, dbmigrate, et al\nDML and DDL must never meet\nReference data may depend on a schema\ntest it realistically - test against a real dataset if you can\n\n \n\n
  • AR migrations, dbdeploy, dbdeploy.net, tarantino, dbmigrate, et al\nDML and DDL must never meet\nReference data may depend on a schema\ntest it realistically - test against a real dataset if you can\n\n \n\n
  • AR migrations, dbdeploy, dbdeploy.net, tarantino, dbmigrate, et al\nDML and DDL must never meet\nReference data may depend on a schema\ntest it realistically - test against a real dataset if you can\n\n \n\n
  • AR migrations, dbdeploy, dbdeploy.net, tarantino, dbmigrate, et al\nDML and DDL must never meet\nReference data may depend on a schema\ntest it realistically - test against a real dataset if you can\n\n \n\n
  • \n
  • \n
  • Don’t use email\nIf you use IDE or desktop widgets\nYou need to communicate the state to all\n\n\n
  • Don’t use email\nIf you use IDE or desktop widgets\nYou need to communicate the state to all\n\n\n
  • Don’t use email\nIf you use IDE or desktop widgets\nYou need to communicate the state to all\n\n\n
  • \n
  • \n\nNext is scaling your tests (Jules can allude to his current client on this one...)\n\nScaling your CI infrastructure:\n\nAOL incident \n\nAfter that you need to make sure your software can scale in production (twitter...) you can \n\nThen comes the feedback loop of testing scaling, and making sure that your test environment reflects all the moving parts of production accurately enough without trying to match the scale...\n\n
  • \n\nNext is scaling your tests (Jules can allude to his current client on this one...)\n\nScaling your CI infrastructure:\n\nAOL incident \n\nAfter that you need to make sure your software can scale in production (twitter...) you can \n\nThen comes the feedback loop of testing scaling, and making sure that your test environment reflects all the moving parts of production accurately enough without trying to match the scale...\n\n
  • \n\nNext is scaling your tests (Jules can allude to his current client on this one...)\n\nScaling your CI infrastructure:\n\nAOL incident \n\nAfter that you need to make sure your software can scale in production (twitter...) you can \n\nThen comes the feedback loop of testing scaling, and making sure that your test environment reflects all the moving parts of production accurately enough without trying to match the scale...\n\n
  • Web application testing is a core driver of CI pain\n
  • Web application testing is a core driver of CI pain\n
  • Web application testing is a core driver of CI pain\n
  • \n
  • \n
  • Key message. Have this proportion of tests.\nYou might not use the same tools for each tier\nA tool for testing the DOM is not a general purpose testing tool\nYou can make your application easier to test by disabling GUI features (e.g. ads, painful ajax interactions)\nGUI tests ideally will take a slice of functionality rather than logging in, testing and logging out.\nMost CI builds are slow because of GUI tests.\nParallelizing GUI tests is dangerous unless you attempt to optimise first\nSome browsers are assloads slower than others\n\n\n\n\n\n\n
  • \n
  • \n
  • You’re not a bloody devop\nThese are the people who didn’t want to be left behind\nCollaboration is #1\nPimp our other talk\n
  • You’re not a bloody devop\nThese are the people who didn’t want to be left behind\nCollaboration is #1\nPimp our other talk\n
  • You’re not a bloody devop\nThese are the people who didn’t want to be left behind\nCollaboration is #1\nPimp our other talk\n
  • You’re not a bloody devop\nThese are the people who didn’t want to be left behind\nCollaboration is #1\nPimp our other talk\n
  • The ultimate scaling - production\n\nThere is a lot we can talk about here about actually getting code in to production. Simple stuff like making sure your deployment scripts are treated as the top level code it is and tested accordingly...\n
  • The ultimate scaling - production\n\nThere is a lot we can talk about here about actually getting code in to production. Simple stuff like making sure your deployment scripts are treated as the top level code it is and tested accordingly...\n
  • The ultimate scaling - production\n\nThere is a lot we can talk about here about actually getting code in to production. Simple stuff like making sure your deployment scripts are treated as the top level code it is and tested accordingly...\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Demo of the build pipeline plugin\nVery engineer-focused approach\nSetting up the CD system is easy, retro-fitting all the tests to your app: hard\nNo excuse not to deploy all the time to staging systems\n\n\n
  • Focus on driving down cycle time\nGreater emphasis on app releases\nUsing all of the pluming of CD\nYou’re only done when you’re in prod\nvenn diagram?\nsits on pillars of previous\n
  • \n
  • \n

Continuous Integration, the minimum viable product Continuous Integration, the minimum viable product Presentation Transcript

  • Continuous Integration The Minimum Viable Product Julian Simpson - @builddoctor Chris Read - @cread
  • http://www.flickr.com/photos/84889127@N00/5151115063/Where’s Chris?
  • http://www.flickr.com/photos/brook/50700239/Loom? Really?
  • http://www.flickr.com/photos/sarah_elizabeth_simpson/5923553033/
  • http://www.flickr.com/photos/krunkwerke/5304623530
  • http://www.flickr.com/photos/imlsdcc/5576596277/
  • http://www.flickr.com/photos/mainmanwalkin/3074193461/mainline release branches
  • http://www.flickr.com/photos/redarrow101/1338479041/
  • CI Principles
  • CI Principles• Build and test everything, when it changes
  • CI Principles• Build and test everything, when it changes• The database schema counts as everything
  • CI Principles• Build and test everything, when it changes• The database schema counts as everything• Tests count as everything
  • CI Principles• Build and test everything, when it changes• The database schema counts as everything• Tests count as everything• And JavaScript
  • CI Principles• Build and test everything, when it changes• The database schema counts as everything• Tests count as everything• And JavaScript• Check in often and early
  • CI Principles• Build and test everything, when it changes• The database schema counts as everything• Tests count as everything• And JavaScript• Check in often and early• Builds are feedback
  • Simple CI Demo
  • More build information
  • Databases
  • Databases• Use database migrations
  • Databases• Use database migrations• Don’t write your own tool
  • Databases• Use database migrations• Don’t write your own tool• Don’t mix DDL and DML
  • Databases• Use database migrations• Don’t write your own tool• Don’t mix DDL and DML• Data can cause migrations to fail in prod
  • 1 class CreateTheGems < ActiveRecord::Migration 2 def self.up 3 create_table :the_gems do |t| 4 t.string :name 5 t.timestamps 6 end 7 end 8 9 def self.down10 drop_table :the_gems11 end12 end
  • 1 class AddGemStatus < ActiveRecord::Migration2 def self.up3 add_column :the_gems, :status, :string, :default => "pending"## at least there are 3 values. pending, declined & approved4 end56 def self.down7 remove_column :the_gems, :status8 end9 end
  • Notifications
  • Notifications• Email is evil
  • Notifications• Email is evil• IDE or tray tools are OK
  • Notifications• Email is evil• IDE or tray tools are OK• Big Visible tools are awesome
  • XFD Demo
  • Scaling
  • Scaling• Solid State Disks are your friend
  • Scaling• Solid State Disks are your friend• Scaling tests is a different concern: http:// test-load-balancer.github.com/
  • Scaling• Solid State Disks are your friend• Scaling tests is a different concern: http:// test-load-balancer.github.com/• Don’t fake infrastructure to make it fast
  • Testing the Web
  • Testing the Web• You can make your application easier to test
  • Testing the Web• You can make your application easier to test• Don’t forget to test everything
  • Testing the Web• You can make your application easier to test• Don’t forget to test everything• Be damn careful when you parallelize tests
  • If you can’t test them, validate them
  • Though testing is better
  • GUI AcceptanceUnit/Integration
  • Sauce Labs Demo
  • DevOpshttp://www.flickr.com/photos/shaggypaul/193098324/
  • DevOps• most misunderstood meme of recent times http://www.flickr.com/photos/shaggypaul/193098324/
  • DevOps• most misunderstood meme of recent times• agile approach to systems admin http://www.flickr.com/photos/shaggypaul/193098324/
  • DevOps• most misunderstood meme of recent times• agile approach to systems admin• developer-sysadmin collaboration http://www.flickr.com/photos/shaggypaul/193098324/
  • DevOps• most misunderstood meme of recent times• agile approach to systems admin• developer-sysadmin collaboration• provides feedback to the business http://www.flickr.com/photos/shaggypaul/193098324/
  • Production
  • Production• Deployment scripts should be rehearsed in CI
  • Production• Deployment scripts should be rehearsed in CI• CI servers should be built the same way as prod
  • Production• Deployment scripts should be rehearsed in CI• CI servers should be built the same way as prod• Tools like Puppet and Chef help you do that
  • Puppet Demo
  • New server: 45 secondsknox:infrastructure jsimpson$rake puppet:remotebundle exec lib/instance.rbNode ready at 44.457065bootstrapping at 44.457127setting the ssh hostkey at 44.457141copying the code over at 44.828861about to make code dir at 44.828986About to rsync at 45.731777
  • Slowest part? My ISPRsync done at 327.996322bootstrapping rubygems at 327.997617updating system configExtracting templates from packages: 100%Successfully installed bundler-1.0.151 gem installedBuilding native extensions. This couldtake a while...Successfully installed json-1.5.31 gem installedbootstrapped rubygems at 366.733525bootstrapping at 366.733569bootstrapped at 366.733608bootstrapped at 373.198161
  • ContinuousDeployment
  • http://www.flickr.com/photos/rsgranne/179321532/Continuous Delivery ... satisfy the customer through early and continuous delivery of valuable software ... (ask Jez)
  • Further reading
  • Thank you Any Questions? @builddoctorjulian@build-doctor.com