Presentation for Fog City Ruby based on https://blog.blockscore.com/how-to-write-better-code-using-mutation-testing/. Mutation testing can help you: * Improve your test coverage * Think about edge cases and catch more bugs before they break in production * Learn more about Ruby, the libraries you depend on, and your own codebase

1. How to Write Better Code with Mutation Testing
John Backus - @backus - john@blockscore.com

2. Talk Outline
Introduction

3. Talk Outline
Introduction
Improving coverage

4. Talk Outline
Introduction
Improving coverage
Learning about Ruby and the code you rely on

5. Line Coverage
Lines of Code Run by Tests Total Lines of Code÷

6. Mutation Coverage
How much of your code can I change
without breaking your tests?

7. Mutation Testing by Hand

8. 1 class Gluttons
2 def initialize(twitter_client)
3 @twitter = twitter_client
4 end
5
6 def recent
7 query = @twitter.search('"I really enjoy #pizza"')
8
9 query.first(2).map { |tweet| "@#{tweet.author}" }
10 end
11 end
1 RSpec.describe Gluttons do
2 it 'lists the two most recent gluttonous tweeters' do
3 tweets = [double(author: 'John'), double(author: 'Jane')]
4 gluttons = Gluttons.new(double(search: tweets))
5
6 expect(gluttons.recent).to eql(%w[@John @Jane])
7 end
8 end

26. 1 class Gluttons
2 def recent
3 - query = @twitter.search('"I really enjoy #pizza"')
4 + query = @twitter.search('"I really enjoy #hotdogs"')
1 example, 0 failures

27. 1 class Gluttons
2 def recent
3 - query = @twitter.search('"I really enjoy #pizza"')
4 + query = @twitter.search('')
1 example, 0 failures

28. 1 class Gluttons
2 def recent
3 - query = @twitter.search('"I really enjoy #pizza"')
4 + query = @twitter.search
1 example, 0 failures

29. 1 class Gluttons
2 def recent
3 - query.first(2).map { |tweet| "@#{tweet.author}" }
4 + query.first(1).map { |tweet| "@#{tweet.author}" }
Failure
expected: ["@John", "@Jane"]
got: ["@John"]

30. 1 - query.first(2).map { |tweet| "@#{tweet.author}" }
2 + query.first(3).map { |tweet| "@#{tweet.author}" }
3 end
1 example, 0 failures

31. Manual Mutation Testing
Tedious

32. Manual Mutation Testing
Hard to outsmart yourself

44. 1 def recent
2 - query = @twitter.search('"I really enjoy #pizza"')
3 + query = @twitter.search

45. 1 def recent
2 - query = @twitter.search('"I really enjoy #pizza"')
3 + query = @twitter.search(nil)

46. 1 - query.first(2).map { |tweet| "@#{tweet.author}" }
2 + query.last(2).map { |tweet| "@#{tweet.author}" }
3 end

47. 1 - query.first(2).map { |tweet| "@#{tweet.author}" }
2 + query.map { |tweet| "@#{tweet.author}" }
3 end

48. 1 it 'lists the two most recent gluttonous tweeters' do
2 tweets = [
3 double(author: 'John'),
4 double(author: 'Jane'),
5 double(author: 'Devon')
6 ]
7
8 client = double('Client')
9 gluttons = Gluttons.new(client)
10
11 allow(client)
12 .to receive(:search)
13 .with('"I really enjoy #pizza"')
14 .and_return(tweets)
15
16 expect(gluttons.recent).to eql(%w[@John @Jane])
17 end

49. Mutation Testing with mutant
Automated

50. Probably more clever than you
Mutation Testing with mutant

51. Example #1: Internal API

52. 1 it 'returns a user when given a valid id' do
2 expect(get(:show, id: 1)).to eq(id: 1, name: 'John')
3 end
4
5 it 'renders JSON error when given an invalid id' do
6 expect(get(:show, id: 0))
7 .to eq(error: "Could not find User with 'id'=0")
8 end
1 class UsersController < ApplicationController
2 def show
3 render json: User.find(params[:id].to_i)
4 rescue User::RecordNotFound => error
5 render json: { error: error.to_s }
6 end
7 end

53. 1 def show
2 - render json: User.find(params[:id].to_i)
3 + render json: User.find(Integer(params[:id]))
4 rescue User::RecordNotFound => error
5 render json: { error: error.to_s }
6 end

54. 1 def show
2 - render json: User.find(params[:id].to_i)
3 + render json: User.find(params.fetch(:id).to_i)
4 rescue User::RecordNotFound => error
5 render json: { error: error.to_s }
6 end

55. 1 def show
2 - render json: User.find(params[:id].to_i)
3 + render json: User.find(Integer(params.fetch(:id)))
4 rescue User::RecordNotFound => error
5 render json: { error: error.to_s }
6 end

56. 1 class UsersController < ApplicationController
2 def created_after
3 after = Date.parse(params[:after])
4 render json: User.recent(after)
5 end
6 end

57. 1 def created_after
2 - after = Date.parse(params[:after])
3 + after = Date.iso8601(params[:after])
4 render json: User.recent(after)
5 end

58. 1 def created_after
2 - after = Date.parse(params[:after])
3 + after = Date.iso8601(params[:after])
4 render json: User.recent(after)
5 end

59. 1 def created_after
2 - after = Date.parse(params[:after])
3 + after = Date.iso8601(params[:after])
4 render json: User.recent(after)
5 end
“2017-05-01"
"H29.05.01"
"Tue May 01 00:00:00 2017"
"Tue, 01 May 2017 00:00:00 +0000"
"Tue, 01 May 2017 00:00:00 GMT"
"May"
"I may be complete garbage"
“2017-05-01"
Date.parseDate.iso8601

60. Example #2: Hardening Regular Expressions

61. 1 usernames.select do |username|
2 username =~ /^(John|Alain).+$/
3 end

62. 1 usernames.select do |username|
2 - username =~ /^(John|Alain).+$/
3 + username =~ /A(John|Alain).+$/
4 end
1 usernames.select do |username|
2 - username =~ /^(John|Alain).+$/
3 + username =~ /^(John|Alain).+z/
4 end

63. 1 usernames.select do |username|
2 - username =~ /^(John|Alain).+$/
3 + username =~ /^(Alain).+$/
4 end
1 usernames.select do |username|
2 - username =~ /^(John|Alain).+$/
3 + username =~ /^(John).+$/
4 end

64. 1 usernames.select do |username|
2 - username =~ /^(John|Alain).+$/
3 + username =~ /^(?:John|Alain).+$/
4 end

65. 1 usernames.select do |username|
2 - username =~ /^(John|Alain).+$/
3 + username.match?(/^(John|Alain).+$/)
4 end

66. 1 usernames.select do |username|
2 - username =~ /^(John|Alain).+$/
3 + username.match?(/A(?:John|Alain).+z/)
4 end

67. Example #3: Learning about your HTTP Client

68. 1 def stars_for(repo)
2 url = "https://api.github.com/repos/#{repo}"
3 data = HTTParty.get(url).to_h
4
5 data['stargazers_count']
6 end

69. 1 def stars_for(repo)
2 url = "https://api.github.com/repos/#{repo}"
3 - data = HTTParty.get(url).to_h
4 + data = HTTParty.get(url)
5
6 data['stargazers_count']
7 end

70. Practicality

71. /^(John|Alain).+$/
/A(John|Alain).+$/
/^(John|Alain).+z/
/^(Alain).+$/
/^(John).+$/
/^(?:John|Alain).+$/

72. $ mutant --use rspec --since master

73. $ mutant
--use rspec
--since master
‘YourApp::User’

74. 1 module YourApp
2 class User < ActiveRecord::Base
3 # Dozens of includes, scopes, class methods, rails magic
4 # 100+ methods
5
6 def validate_email
7 # Simple method you're fixing
8 end
9 end
10 end

75. 1 RSpec.describe YourApp::User do
2 # 100s of tests and setup unrelated to your task
3
4 describe '#validate_email' do
5 # Half dozen tests you are focusing on
6 end
7 end

76. mutant - Your Secret Weapon

77. Thanks!
John Backus - @backus - john@blockscore.com