Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Advanced Developer Testing

868 views

Published on

Developers have unique opportunities to influence software quality. We explore three advanced testing techniques that belong in every developers toolbox: (1) characterization, (2) approval, and (3) mutation testing.

Published in: Software
  • Be the first to comment

Advanced Developer Testing

  1. 1. Three Advanced
 DeveloperTestingTechniques Alistair McKinnell @amckinnell
  2. 2. How do we 
 change code safely?
  3. 3. Changing Code Existing Behaviour New Behaviour
  4. 4. Characterization Testing Approval Testing Code Coverage Mutation Testing
  5. 5. Gilded Rose
  6. 6. 1 class GildedRose 2 attr_reader :items 3 4 def initialize(items) 5 @items = items 6 end 7 8 def update_quality 9 @items.each do |item| 10 if item.name != 'Aged Brie' and 11 item.name != 'Backstage passes to a TAFKAL80ETC concert' 12 if item.quality > 0 13 if item.name != 'Sulfuras, Hand of Ragnaros' 14 item.quality = item.quality - 1 15 end 16 end 17 else 18 if item.quality < 50 19 item.quality = item.quality + 1 20 if item.name == 'Backstage passes to a TAFKAL80ETC concert' 21 if item.sell_in < 11 22 if item.quality < 50 23 item.quality = item.quality + 1 24 end 25 end 26 if item.sell_in < 6 27 if item.quality < 50 28 item.quality = item.quality + 1 29 end 30 end 31 end 32 end 33 end 34 if item.name != 'Sulfuras, Hand of Ragnaros' 35 item.sell_in = item.sell_in - 1 36 end 37 if item.sell_in < 0 38 if item.name != 'Aged Brie' 39 if item.name != 'Backstage passes to a TAFKAL80ETC concert' 40 if item.quality > 0 41 if item.name != 'Sulfuras, Hand of Ragnaros' 42 item.quality = item.quality - 1 43 end 44 end 45 else 46 item.quality = item.quality - item.quality 47 end 48 else 49 if item.quality < 50 50 item.quality = item.quality + 1 51 end 52 end 53 end 54 end 55 end 56 57 end !
  7. 7. “Conjured” items degrade 
 twice as fast as normal items. New Behaviour
  8. 8. Existing Behaviour
 (Refactored) New Behaviour
 (Conjured Item) Changing Code
  9. 9. Are we ready to 
 change code safely?
  10. 10. Nope Refactoring safely
 requires tests.
  11. 11. Characterization Testing Approval Testing Code Coverage Mutation Testing
  12. 12. describe GildedRose do it 'knows how to update quality for items' do items = [Item.new('Mail Armour', 10, 20)] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq(nil) end end
  13. 13. describe GildedRose do it 'knows how to update quality for items' do items = [Item.new('Mail Armour', 10, 20)] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq(nil) end end expected: nil got: "Mail Armour, 9, 19"
  14. 14. describe GildedRose do it 'knows how to update quality for items' do items = [Item.new('Mail Armour', 10, 20)] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq(nil) end end expected: nil got: "Mail Armour, 9, 19"
  15. 15. describe GildedRose do it 'knows how to update quality for items' do items = [Item.new('Mail Armour', 10, 20)] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19') end end
  16. 16. describe GildedRose do it 'knows how to update quality for items' do items = [Item.new(‘Mail Armour', 10, 20)] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19') subject.update_quality expect(subject.items[0].to_s).to eq(nil) end end
  17. 17. describe GildedRose do it 'knows how to update quality for items' do items = [Item.new('Mail Armour', 10, 20)] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19') subject.update_quality expect(subject.items[0].to_s).to eq(nil) end end expected: nil got: "Mail Armour, 8, 18"
  18. 18. describe GildedRose do it 'knows how to update quality for items' do items = [Item.new('Mail Armour', 10, 20)] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19') subject.update_quality expect(subject.items[0].to_s).to eq(nil) end end expected: nil got: "Mail Armour, 8, 18"
  19. 19. describe GildedRose do it 'knows how to update quality for items' do items = [Item.new('Mail Armour', 10, 20)] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19') subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 8, 18') end end
  20. 20. describe GildedRose do it 'knows how to update quality for items' do items = [ Item.new('Mail Armour', 10, 20), Item.new('Aged Brie', 4, 9) ] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19') subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 8, 18') end end
  21. 21. describe GildedRose do it 'knows how to update quality for items' do items = [ Item.new('Mail Armour', 10, 20), Item.new('Aged Brie', 4, 9) ] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19') subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 8, 18') end end
  22. 22. describe GildedRose do it 'knows how to update quality for items' do items = [ Item.new('Mail Armour', 10, 20), Item.new(‘Aged Brie', 4, 9) ] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19') expect(subject.items[1].to_s).to eq(nil) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 8, 18') expect(subject.items[1].to_s).to eq(nil) end end
  23. 23. describe GildedRose do it 'knows how to update quality for items' do items = [ Item.new('Mail Armour', 10, 20), Item.new(‘Aged Brie', 4, 9) ] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19') expect(subject.items[1].to_s).to eq(nil) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 8, 18') expect(subject.items[1].to_s).to eq(nil) end end
  24. 24. describe GildedRose do it 'knows how to update quality for items' do items = [ Item.new('Mail Armour', 10, 20), Item.new(‘Aged Brie', 4, 9) ] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19') expect(subject.items[1].to_s).to eq(‘Aged Brie, 3, 10') subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 8, 18') expect(subject.items[1].to_s).to eq(‘Aged Brie, 2, 11') end end
  25. 25. describe GildedRose do it 'knows how to update quality for items' do items = [ Item.new('Mail Armour', 10, 20), Item.new(‘Aged Brie', 4, 9) ] subject = GildedRose.new(items) subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19') expect(subject.items[1].to_s).to eq(‘Aged Brie, 3, 10') subject.update_quality expect(subject.items[0].to_s).to eq('Mail Armour, 8, 18') expect(subject.items[1].to_s).to eq(‘Aged Brie, 2, 11') end end Let’s organize our test
  26. 26. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  27. 27. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def items end def characterize(subject, days) end def expected end
  28. 28. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def items item_attributes = [ ['Mail Armour', 10, 20], ['Aged Brie', 4, 9] ] item_attributes.map { |args| Item.new(*args) } end
  29. 29. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def characterize(subject, days) characterization = [] (1..days).each do subject.update_quality subject.items.each { |item| characterization << item.to_s } end characterization end
  30. 30. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def expected [ 'Mail Armour, 9, 19', 'Aged Brie, 3, 10', 'Mail Armour, 8, 18', 'Aged Brie, 2, 11' ] end
  31. 31. Are we ready to 
 change code safely?
  32. 32. Nope Characterization
 is incomplete.
  33. 33. Characterization Testing Approval Testing Code Coverage Mutation Testing
  34. 34. 53%
  35. 35. if item.name == 'Backstage passes to a TAFKAL80ETC concert' if item.sell_in < 11 if item.quality < 50 item.quality = item.quality + 1 end end if item.sell_in < 6 if item.quality < 50 item.quality = item.quality + 1 end end end
  36. 36. if item.name == 'Backstage passes to a TAFKAL80ETC concert' if item.sell_in < 11 if item.quality < 50 item.quality = item.quality + 1 end end if item.sell_in < 6 if item.quality < 50 item.quality = item.quality + 1 end end end def items item_attributes = [ ['Mail Armour', 10, 20], ['Aged Brie', 4, 9] ] item_attributes.map { |args| Item.new(*args) } end
  37. 37. if item.name == 'Backstage passes to a TAFKAL80ETC concert' if item.sell_in < 11 if item.quality < 50 item.quality = item.quality + 1 end end if item.sell_in < 6 if item.quality < 50 item.quality = item.quality + 1 end end end def items item_attributes = [ ['Mail Armour', 10, 20], ['Aged Brie', 4, 9] ] item_attributes.map { |args| Item.new(*args) } end
  38. 38. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def items item_attributes = [ ['Mail Armour', 10, 20], ['Aged Brie', 4, 9], ['Backstage passes to a TAFKAL80ETC concert', 15, 17] ] item_attributes.map { |args| Item.new(*args) } end
  39. 39. def items item_attributes = [ ['Mail Armour', 10, 20], ['Aged Brie', 4, 9], ['Backstage passes to a TAFKAL80ETC concert', 15, 17] ] item_attributes.map { |args| Item.new(*args) } end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  40. 40. def expected [ 'Mail Armour, 9, 19', 'Aged Brie, 3, 10', 'Backstage passes to a TAFKAL80ETC concert, 14, 18', 'Mail Armour, 8, 18', 'Aged Brie, 2, 11', 'Backstage passes to a TAFKAL80ETC concert, 13, 19', ] end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  41. 41. def expected [ 'Mail Armour, 9, 19', 'Aged Brie, 3, 10', 'Backstage passes to a TAFKAL80ETC concert, 14, 18', 'Mail Armour, 8, 18', 'Aged Brie, 2, 11', 'Backstage passes to a TAFKAL80ETC concert, 13, 19', ] end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  42. 42. 60%
  43. 43. if item.name == 'Backstage passes to a TAFKAL80ETC concert' if item.sell_in < 11 if item.quality < 50 item.quality = item.quality + 1 end end if item.sell_in < 6 if item.quality < 50 item.quality = item.quality + 1 end end end
  44. 44. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end if item.name == 'Backstage passes to a TAFKAL80ETC concert' if item.sell_in < 11 if item.quality < 50 item.quality = item.quality + 1 end end if item.sell_in < 6 if item.quality < 50 item.quality = item.quality + 1 end end end
  45. 45. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end if item.name == 'Backstage passes to a TAFKAL80ETC concert' if item.sell_in < 11 if item.quality < 50 item.quality = item.quality + 1 end end if item.sell_in < 6 if item.quality < 50 item.quality = item.quality + 1 end end end
  46. 46. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 20) expect(characterization).to eq(expected) end end
  47. 47. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 20) expect(characterization).to eq(expected) end end This is what 
 we want to do
  48. 48. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 20) expect(characterization).to eq(expected) end end def expected [ 'Mail Armour, 9, 19', 'Aged Brie, 3, 10', 'Backstage passes to a TAFKAL80ETC concert, 14, 18', 'Mail Armour, 8, 18', 'Aged Brie, 2, 11', 'Backstage passes to a TAFKAL80ETC concert, 13, 19’, and many more expected results ] end
  49. 49. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def characterize(subject, days) characterization = [] (1..days).each do subject.update_quality characterization.concat(subject.items.map(&:to_s)) end Digest::SHA2.hexdigest(characterization.join) end
  50. 50. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def characterize(subject, days) characterization = [] (1..days).each do subject.update_quality characterization.concat(subject.items.map(&:to_s)) end Digest::SHA2.hexdigest(characterization.join) end
  51. 51. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def expected [ 'Mail Armour, 9, 19', 'Aged Brie, 3, 10', 'Backstage passes to a TAFKAL80ETC concert, 14, 18', 'Mail Armour, 8, 18', 'Aged Brie, 2, 11', 'Backstage passes to a TAFKAL80ETC concert, 13, 19’, ] end
  52. 52. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def expected nil end
  53. 53. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def expected nil end expected: nil got: “c869cb44cd36e1553d18 … d958277a43b1adc8e8e5”
  54. 54. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def expected nil end expected: nil got: “c869cb44cd36e1553d18 … d958277a43b1adc8e8e5”
  55. 55. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def expected ‘c869cb44cd36e1553d18 … d958277a43b1adc8e8e5’ end
  56. 56. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 20) expect(characterization).to eq(expected) end end def expected ‘c869cb44cd36e1553d18 … d958277a43b1adc8e8e5’ end
  57. 57. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 20) expect(characterization).to eq(expected) end end expected: “c869cb44cd36e1553d18 … d958277a43b1adc8e8e5” got: “fb61e7fdfffcba653fec … 9ae0512470a974b1561e" def expected ‘c869cb44cd36e1553d18 … d958277a43b1adc8e8e5’ end
  58. 58. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 20) expect(characterization).to eq(expected) end end def expected ‘fb61e7fdfffcba653fec … 9ae0512470a974b1561e’ end
  59. 59. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end def items item_attributes = [ ['Mail Armour', 10, 20], ['Mail Armour', 10, 1], ['Aged Brie', 4, 9], ['Aged Brie', 1, 49], ['Backstage passes to a TAFKAL80ETC concert', 15, 17], ['Backstage passes to a TAFKAL80ETC concert', 5, 49], ['Sulfuras, Hand of Ragnaros', -1, 80] ] item_attributes.map { |args| Item.new(*args) } end
  60. 60. 100%
  61. 61. Are we ready to 
 change code safely?
  62. 62. Nope Characterization
 is incomplete.
  63. 63. Characterization Testing Approval Testing Code Coverage Mutation Testing
  64. 64. class GildedRose attr_reader :items def initialize(items) @items = items end def update_quality @items.each do |item| … if item.name != 'Sulfuras, Hand of Ragnaros' item.sell_in = item.sell_in - 1 end … end end end
  65. 65. class GildedRose attr_reader :items def initialize(items) @items = items end def update_quality @items.each do |item| … # if item.name != 'Sulfuras, Hand of Ragnaros' item.sell_in = item.sell_in - 1 # end … end end end
  66. 66. class GildedRose attr_reader :items def initialize(items) @items = items end def update_quality @items.each do |item| … # if item.name != 'Sulfuras, Hand of Ragnaros' item.sell_in = item.sell_in - 1 # end … end end end Comment out 
 two lines
  67. 67. Test still passes? class GildedRose attr_reader :items def initialize(items) @items = items end def update_quality @items.each do |item| … # if item.name != 'Sulfuras, Hand of Ragnaros' item.sell_in = item.sell_in - 1 # end … end end end
  68. 68. def items item_attributes = [ ['Mail Armour', 10, 20], ['Aged Brie', 4, 9], ['Backstage passes to a TAFKAL80ETC concert', 15, 17] ] item_attributes.map { |args| Item.new(*args) } end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  69. 69. def items item_attributes = [ ['Mail Armour', 10, 20], ['Aged Brie', 4, 9], ['Backstage passes to a TAFKAL80ETC concert', 15, 17], ['Sulfuras, Hand of Ragnaros', -1, 80] ] item_attributes.map { |args| Item.new(*args) } end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  70. 70. def items item_attributes = [ ['Mail Armour', 10, 20], ['Aged Brie', 4, 9], ['Backstage passes to a TAFKAL80ETC concert', 15, 17], ['Sulfuras, Hand of Ragnaros', -1, 80] ] item_attributes.map { |args| Item.new(*args) } end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  71. 71. Are we ready to 
 change code safely?
  72. 72. Nope Characterization
 is incomplete.
  73. 73. github.com/mbj/mutant
  74. 74. Mutant configuration: Subjects: 2
 Mutations: 476 Kills: 454 Alive: 22 Coverage: 95.38% Expected: 100.00%
  75. 75. - if (item.quality > 0) + if (item.quality > 1) - if (item.quality < 50) + if (item.quality < 49) - if (item.quality < 50) + if (item.quality < 51) - if (item.quality < 50) + if true - if (item.quality < 50) + if 50 - if (item.name == "Backstage passes to a TAFKAL80ETC concert") + if item.name.eql?("Backstage passes to a TAFKAL80ETC concert")
  76. 76. def items item_attributes = [ ['Mail Armour', 10, 20], ['Aged Brie', 4, 9], ['Backstage passes to a TAFKAL80ETC concert', 15, 17], ['Sulfuras, Hand of Ragnaros', -1, 80] ] item_attributes.map { |args| Item.new(*args) } end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  77. 77. def items item_attributes = [ ['Mail Armour', 10, 20], ['Mail Armour', 10, 1], ['Aged Brie', 4, 9], ['Aged Brie', 1, 49], ['Backstage passes to a TAFKAL80ETC concert', 15, 17], ['Backstage passes to a TAFKAL80ETC concert', 5, 49], ['Sulfuras, Hand of Ragnaros', -1, 80] ] item_attributes.map { |args| Item.new(*args) } end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  78. 78. def items item_attributes = [ ['Mail Armour', 10, 20], ['Mail Armour', 10, 1], ['Aged Brie', 4, 9], ['Aged Brie', 1, 49], ['Backstage passes to a TAFKAL80ETC concert', 15, 17], ['Backstage passes to a TAFKAL80ETC concert', 5, 49], ['Sulfuras, Hand of Ragnaros', -1, 80] ] item_attributes.map { |args| Item.new(*args) } end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  79. 79. Subjects: 2 Mutations: 476 Kills: 454 Alive: 22 Coverage: 95.38% Expected: 100.00% Before After Subjects: 2 Mutations: 476 Kills: 474 Alive: 4 Coverage: 99.16% Expected: 100.00%
  80. 80. Are we ready to 
 change code safely?
  81. 81. Yep
  82. 82. New BehaviourExisting Behaviour Changing Code
  83. 83. Characterization Testing Approval Testing Code Coverage Mutation Testing
  84. 84. Refactoring
  85. 85. 1 class GildedRose 2 attr_reader :items 3 4 def initialize(items) 5 @items = items 6 end 7 8 def update_quality 9 @items.each { |item| update_item_quality(item) } 10 end 11 12 private 13 14 def update_item_quality(item) 15 return if item.name == 'Sulfuras, Hand of Ragnaros' 16 17 perform_inventory_rollover(item) 18 perform_inventory_expiration(item) 19 end 20 21 def perform_inventory_rollover(item) 22 item.sell_in -= 1 23 24 case item.name 25 when 'Aged Brie' 26 increase_quality(item) 27 when 'Backstage passes to a TAFKAL80ETC concert' 28 increase_quality(item) 29 increase_quality(item) if item.sell_in < 10 30 increase_quality(item) if item.sell_in < 5 31 else 32 decrease_quality(item) 33 end 34 end 35 36 def perform_inventory_expiration(item) 37 return unless expired?(item) 38 39 case item.name 40 when 'Aged Brie' 41 increase_quality(item) 42 when 'Backstage passes to a TAFKAL80ETC concert' 43 writeoff(item) 44 else 45 decrease_quality(item) 46 end 47 end 48 49 def expired?(item) 50 item.sell_in < 0 51 end 52 53 def decrease_quality(item) 54 item.quality -= 1 if 0 < item.quality 55 end 56 57 def increase_quality(item) 58 item.quality += 1 if item.quality < 50 59 end 60 61 def writeoff(item) 62 item.quality = 0 63 end 64 65 end
  86. 86. “Conjured” items degrade 
 twice as fast as normal items. New Behaviour
  87. 87. def perform_inventory_rollover(item) case item.name when 'Conjured Mana' decrease_quality(item) decrease_quality(item) end end def perform_inventory_expiration(item) case item.name when 'Conjured Mana' decrease_quality(item) decrease_quality(item) end end
  88. 88. def perform_inventory_rollover(item) case item.name when 'Conjured Mana' decrease_quality(item) decrease_quality(item) end end def perform_inventory_expiration(item) case item.name when 'Conjured Mana' decrease_quality(item) decrease_quality(item) end end
  89. 89. def items item_attributes = [ ['Mail Armour', 10, 20], ['Mail Armour', 10, 1], ['Aged Brie', 4, 9], ['Aged Brie', 1, 49], ['Backstage passes to a TAFKAL80ETC concert', 15, 17], ['Backstage passes to a TAFKAL80ETC concert', 5, 49], ['Sulfuras, Hand of Ragnaros', -1, 80], [‘Conjured Mana', 13, 50] ] item_attributes.map { |args| Item.new(*args) } end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  90. 90. def items item_attributes = [ ['Mail Armour', 10, 20], ['Mail Armour', 10, 1], ['Aged Brie', 4, 9], ['Aged Brie', 1, 49], ['Backstage passes to a TAFKAL80ETC concert', 15, 17], ['Backstage passes to a TAFKAL80ETC concert', 5, 49], ['Sulfuras, Hand of Ragnaros', -1, 80], [‘Conjured Mana', 13, 50] ] item_attributes.map { |args| Item.new(*args) } end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  91. 91. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 20) expect(characterization).to eq(expected) end end def expected ‘1768fa473f323772588a … 13eab4018717ea78ea0c’ end
  92. 92. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 20) expect(characterization).to eq(expected) end end expected: “1768fa473f323772588a … 13eab4018717ea78ea0c” got: “6cf1111d5865232381f1 … 9d00ff8e831395de5420" def expected ‘1768fa473f323772588a … 13eab4018717ea78ea0c’ end
  93. 93. New BehaviourExisting Behaviour Changing Code
  94. 94. Are we ready to 
 change code safely?
  95. 95. Nope Can’t distinguish existing and new behaviour.
  96. 96. Characterization Testing Approval Testing Code Coverage Mutation Testing
  97. 97. Whatever the code does. Existing Behaviour
  98. 98. “Conjured” items degrade
 twice as fast as normal items. New Behaviour
  99. 99. github.com/kytrinyx/approvals
  100. 100. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) verify { characterize(subject, 20) } end end def characterize(subject, days) characterization = [] (1..days).each_with_index do |day| subject.update_quality characterization << "Day #{day} of #{days}" subject.items.each { |item| characterization << " #{item.to_s}" } end characterization.join("n") end
  101. 101. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) verify { characterize(subject, 20) } end end def characterize(subject, days) characterization = [] (1..days).each_with_index do |day| subject.update_quality characterization << "Day #{day} of #{days}" subject.items.each { |item| characterization << " #{item.to_s}" } end characterization.join("n") end
  102. 102. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) verify { characterize(subject, 20) } end end def characterize(subject, days) characterization = [] (1..days).each_with_index do |day| subject.update_quality characterization << "Day #{day} of #{days}" subject.items.each { |item| characterization << " #{item.to_s}" } end characterization.join("n") end
  103. 103. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) verify { characterize(subject, 20) } end end
  104. 104. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) verify { characterize(subject, 20) } end end Approvals::ApprovalError: Approval Error: Approval file 
 “spec/fixtures/approvals/gildedrose/ 
 knows_how_to_update_quality_for_items.approved.txt"
 not found.
  105. 105. knows_how_to_update_quality_for_items.received.txt 1 Day 1 of 20 2 Mail Armour, 9, 19 3 Mail Armour, 9, 0 4 Aged Brie, 3, 10 5 Aged Brie, 0, 50 6 Backstage passes to a TAFKAL80ETC concert, 14, 18 7 Backstage passes to a TAFKAL80ETC concert, 4, 50 8 Sulfuras, Hand of Ragnaros, -1, 80 9 Day 2 of 20 10 Mail Armour, 8, 18 … 153 Day 20 of 20 154 Mail Armour, -10, 0 155 Mail Armour, -10, 0 156 Aged Brie, -16, 45 157 Aged Brie, -19, 50 158 Backstage passes to a TAFKAL80ETC concert, -5, 0 159 Backstage passes to a TAFKAL80ETC concert, -15, 0 160 Sulfuras, Hand of Ragnaros, -1, 80
  106. 106. knows_how_to_update_quality_for_items.approved.txt 1 Day 1 of 20 2 Mail Armour, 9, 19 3 Mail Armour, 9, 0 4 Aged Brie, 3, 10 5 Aged Brie, 0, 50 6 Backstage passes to a TAFKAL80ETC concert, 14, 18 7 Backstage passes to a TAFKAL80ETC concert, 4, 50 8 Sulfuras, Hand of Ragnaros, -1, 80 9 Day 2 of 20 10 Mail Armour, 8, 18 … 153 Day 20 of 20 154 Mail Armour, -10, 0 155 Mail Armour, -10, 0 156 Aged Brie, -16, 45 157 Aged Brie, -19, 50 158 Backstage passes to a TAFKAL80ETC concert, -5, 0 159 Backstage passes to a TAFKAL80ETC concert, -15, 0 160 Sulfuras, Hand of Ragnaros, -1, 80
  107. 107. knows_how_to_update_quality_for_items.approved.txt 1 Day 1 of 20 2 Mail Armour, 9, 19 3 Mail Armour, 9, 0 4 Aged Brie, 3, 10 5 Aged Brie, 0, 50 6 Backstage passes to a TAFKAL80ETC concert, 14, 18 7 Backstage passes to a TAFKAL80ETC concert, 4, 50 8 Sulfuras, Hand of Ragnaros, -1, 80 9 Day 2 of 20 10 Mail Armour, 8, 18 … 153 Day 20 of 20 154 Mail Armour, -10, 0 155 Mail Armour, -10, 0 156 Aged Brie, -16, 45 157 Aged Brie, -19, 50 158 Backstage passes to a TAFKAL80ETC concert, -5, 0 159 Backstage passes to a TAFKAL80ETC concert, -15, 0 160 Sulfuras, Hand of Ragnaros, -1, 80
  108. 108. New BehaviourExisting Behaviour Changing Code
  109. 109. def perform_inventory_rollover(item) case item.name when 'Conjured Mana' decrease_quality(item) decrease_quality(item) end end def perform_inventory_expiration(item) case item.name when 'Conjured Mana' decrease_quality(item) decrease_quality(item) end end
  110. 110. def perform_inventory_rollover(item case item.name when 'Conjured Mana' decrease_quality(item) decrease_quality(item) end end def perform_inventory_expiration(item) case item.name when 'Conjured Mana' decrease_quality(item) decrease_quality(item) end end
  111. 111. def items item_attributes = [ ['Mail Armour', 10, 20], ['Mail Armour', 10, 1], ['Aged Brie', 4, 9], ['Aged Brie', 1, 49], ['Backstage passes to a TAFKAL80ETC concert', 15, 17], ['Backstage passes to a TAFKAL80ETC concert', 5, 49], ['Sulfuras, Hand of Ragnaros', -1, 80], [‘Conjured Mana', 13, 50] ] item_attributes.map { |args| Item.new(*args) } end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  112. 112. def items item_attributes = [ ['Mail Armour', 10, 20], ['Mail Armour', 10, 1], ['Aged Brie', 4, 9], ['Aged Brie', 1, 49], ['Backstage passes to a TAFKAL80ETC concert', 15, 17], ['Backstage passes to a TAFKAL80ETC concert', 5, 49], ['Sulfuras, Hand of Ragnaros', -1, 80], [‘Conjured Mana', 13, 50] ] item_attributes.map { |args| Item.new(*args) } end describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) characterization = characterize(subject, 2) expect(characterization).to eq(expected) end end
  113. 113. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) verify { characterize(subject, 20) } end end
  114. 114. describe GildedRose do it 'knows how to update quality for items' do subject = GildedRose.new(items) verify { characterize(subject, 20) } end end Approvals::ApprovalError: Approval Error: Received file “spec/fixtures/approvals/gildedrose/ knows_how_to_update_quality_for_items.received.txt” does not match approved “spec/fixtures/approvals/gildedrose/ knows_how_to_update_quality_for_items.approved.txt".
  115. 115. knows_how_to_update_quality_for_items.received.txt 4 Aged Brie, 3, 10 5 Aged Brie, 0, 50 6 Backstage passes to a TAFKAL80ETC concert, 14, 18 7 Backstage passes to a TAFKAL80ETC concert, 4, 50 8 Sulfuras, Hand of Ragnaros, -1, 80 9 Conjured Mana, 12, 48 10 Day 2 of 20 11 Mail Armour, 8, 18 12 … knows_how_to_update_quality_for_items.approved.txt 4 Aged Brie, 3, 10 5 Aged Brie, 0, 50 6 Backstage passes to a TAFKAL80ETC concert, 14, 18 7 Backstage passes to a TAFKAL80ETC concert, 4, 50 8 Sulfuras, Hand of Ragnaros, -1, 80 9 Day 2 of 20 10 Mail Armour, 8, 18 11 …
  116. 116. knows_how_to_update_quality_for_items.received.txt 8 Sulfuras, Hand of Ragnaros, -1, 80 9 Conjured Mana, 12, 48 10 Day 2 of 20 11 Mail Armour, 8, 18 12 Mail Armour, 8, 0 13 Aged Brie, 2, 11 14 Aged Brie, -1, 50 15 Backstage passes to a TAFKAL80ETC concert, 13, 19 16 Backstage passes to a TAFKAL80ETC concert, 3, 50 17 Sulfuras, Hand of Ragnaros, -1, 80 18 Conjured Mana, 11, 46 19 Day 3 of 20 20 Mail Armour, 7, 17 21 Mail Armour, 7, 0 22 Aged Brie, 1, 12 23 Aged Brie, -2, 50 24 Backstage passes to a TAFKAL80ETC concert, 12, 20 25 Backstage passes to a TAFKAL80ETC concert, 2, 50 26 Sulfuras, Hand of Ragnaros, -1, 80 27 Conjured Mana, 10, 44 28 Day 4 of 20 29 Mail Armour, 6, 16 30 …
  117. 117. knows_how_to_update_quality_for_items.approved.txt 8 Sulfuras, Hand of Ragnaros, -1, 80 9 Conjured Mana, 12, 48 10 Day 2 of 20 11 Mail Armour, 8, 18 12 Mail Armour, 8, 0 13 Aged Brie, 2, 11 14 Aged Brie, -1, 50 15 Backstage passes to a TAFKAL80ETC concert, 13, 19 16 Backstage passes to a TAFKAL80ETC concert, 3, 50 17 Sulfuras, Hand of Ragnaros, -1, 80 18 Conjured Mana, 11, 46 19 Day 3 of 20 20 Mail Armour, 7, 17 21 Mail Armour, 7, 0 22 Aged Brie, 1, 12 23 Aged Brie, -2, 50 24 Backstage passes to a TAFKAL80ETC concert, 12, 20 25 Backstage passes to a TAFKAL80ETC concert, 2, 50 26 Sulfuras, Hand of Ragnaros, -1, 80 27 Conjured Mana, 10, 44 28 Day 4 of 20 29 Mail Armour, 6, 16 30 …
  118. 118. knows_how_to_update_quality_for_items.approved.txt 8 Sulfuras, Hand of Ragnaros, -1, 80 9 Conjured Mana, 12, 48 10 Day 2 of 20 11 Mail Armour, 8, 18 12 Mail Armour, 8, 0 13 Aged Brie, 2, 11 14 Aged Brie, -1, 50 15 Backstage passes to a TAFKAL80ETC concert, 13, 19 16 Backstage passes to a TAFKAL80ETC concert, 3, 50 17 Sulfuras, Hand of Ragnaros, -1, 80 18 Conjured Mana, 11, 46 19 Day 3 of 20 20 Mail Armour, 7, 17 21 Mail Armour, 7, 0 22 Aged Brie, 1, 12 23 Aged Brie, -2, 50 24 Backstage passes to a TAFKAL80ETC concert, 12, 20 25 Backstage passes to a TAFKAL80ETC concert, 2, 50 26 Sulfuras, Hand of Ragnaros, -1, 80 27 Conjured Mana, 10, 44 28 Day 4 of 20 29 Mail Armour, 6, 16 30 …
  119. 119. We win. We preserved
 existing behaviour and added new behaviour.
  120. 120. Characterization Testing Approval Testing Code Coverage Mutation Testing
  121. 121. Are we ready to 
 change code safely?
  122. 122. Changing Code Existing Behaviour New Behaviour
  123. 123. Changing Code New Behaviour
 (ApprovalTests) Existing Behaviour
 (CharacterizationTests)
  124. 124. Be amazing at
 changing code safely.
  125. 125. github.com/amckinnell/developer-testing
  126. 126. Recommended Reading

×