Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Android Mutation
Testing
Hazem Saleh
Mobile Architect @Nickelodeon
About
 More than twelve years of experience in Java and mobile solutions.
 Apache Open Source Committer.
 Author of fou...
Agenda
 Developers Life without Unit Testing.
 Unit Testing 101.
 Unit Testing in Android.
 Code Coverage and Jacoco.
...
Developers Life without Unit Testing
Complex integration between the system components.
Developers Life without Unit Testing
Unmanaged number of new/regression defects especially
when the system complexity incr...
Developers Life without Unit Testing
Low application quality.
Longer testing cycle.
test
fix
test
fix
test
fix
testfix
tes...
Unit Testing 101
A unit testing is a piece of code (usually a method) that
invokes another piece of code and later checks...
AutomatedRepeatable
Easy to run.
Fast
Easy to
understand
Incremental
Unit Testing 101
Unit Testing in ANDROID
 Generally, Unit tests in Android are implemented using:
– JUnit.
– Mockito (for mocking testable...
Unit Testing in ANDROID
Other challenge:
–Unit testing classes with huge dependency on
Android SDK is hard (such as activ...
Code Coverage and Jacoco
 Code Coverage represents the amount of source code
which will be executed when test cases run.
...
Code Coverage and Jacoco
 JaCoCo is a free code coverage library for Java.
 It includes the popular code coverage metric...
Code Coverage and Jacoco
 Fortunately, JaCoCo is fully Integrated with Gradle.
 JaCoCo works fine with Android projects....
Code Coverage and Jacoco
Demo
Sample URL:
https://github.com/hazems/Dagger-Sample
Challenges
 Traditional code
coverage only measure
the amount of executed
code.
 Traditional code
coverage does not dete...
Mutation testing 101
 Mutation testing is about seeding app source code with faults
(mutations).
 After seeding, unit te...
Mutation testing 101
 Mutations can be for example:
–Negate Conditionals Mutator (<= TO > and < TO >=)
–Increment Mutator...
What and Why PIT?
 One of the mutation testing tools for Java.
 It has advantages
–Fast to execute (unlike most of the J...
PIT Android Integration
 Gradle Plugin for PIT is available:
http://gradle-pitest-plugin.solidsoft.info/
https://github.c...
PIT Android Integration
 For using PIT in your Android apps, add PIT plugin to
your top-level build.gradle file as follow...
PIT Android Integration
 Configure PIT plugin basically as follows
pitest {
targetClasses = ['com.test.xyz.daggersample.p...
PIT Android Integration
 Check the mutation results by executing the following
command:
>./gradlew clean pitest
Mutation Testing Demo
Demo
Quiz
Q & A
Twitter: @hazems
Email: hazems@apache.org
Upcoming SlideShare
Loading in …5
×

[AnDevCon 2016] Mutation Testing for Android

3,561 views

Published on

Unit testing coverage is a great way to show us the amount of tested lines and branches of code, but is this really enough? The answer is "no" since unit testing coverage does not really fully measure the efficiency of the unit tests.

This is why there is a need for having techniques that show unit tests efficiency. Mutation testing is one of these powerful techniques. The main idea of mutation testing is to perform byte code modifications (mutations) to original Android app source code and then run app unit tests to check if they are strong enough to fail as a result of these mutations.

This session discusses mutation testing techniques, and demonstrates PIT as a powerful mutation testing tool for Android apps with demos.

Published in: Technology
  • Be the first to comment

[AnDevCon 2016] Mutation Testing for Android

  1. 1. Android Mutation Testing Hazem Saleh Mobile Architect @Nickelodeon
  2. 2. About  More than twelve years of experience in Java and mobile solutions.  Apache Open Source Committer.  Author of four technical books.  DeveloperWorks Contributing Author and DZone MVB.  Technical Speaker (JavaOne, ApacheCon, Geecon, JavaLand …etc).  JSR Experts Group Member.  An X-IBMer and Currently a Mobile Software Architect in Nickelodeon.
  3. 3. Agenda  Developers Life without Unit Testing.  Unit Testing 101.  Unit Testing in Android.  Code Coverage and Jacoco.  Code Coverage Demo.  Current challenges of traditional code coverage.  Mutation Testing 101.  What and Why PIT?  PIT Android Integration.  Mutation Testing Demo.  Quiz.  Q & A.
  4. 4. Developers Life without Unit Testing Complex integration between the system components.
  5. 5. Developers Life without Unit Testing Unmanaged number of new/regression defects especially when the system complexity increases.
  6. 6. Developers Life without Unit Testing Low application quality. Longer testing cycle. test fix test fix test fix testfix test fix test fix test fix
  7. 7. Unit Testing 101 A unit testing is a piece of code (usually a method) that invokes another piece of code and later checks the correctness of some assumptions. Unit testing helps in detecting BUGGY components in the early stages of the project. A test suite is a set of test cases, and a test case is a set of tests which verifies the system components.
  8. 8. AutomatedRepeatable Easy to run. Fast Easy to understand Incremental Unit Testing 101
  9. 9. Unit Testing in ANDROID  Generally, Unit tests in Android are implemented using: – JUnit. – Mockito (for mocking testable class dependencies).  However, there are some challenges: – Mocking Static methods. – Mocking final methods or classes. – Mocking private methods.  Thanks to PowerMock, the previous challenges are met.
  10. 10. Unit Testing in ANDROID Other challenge: –Unit testing classes with huge dependency on Android SDK is hard (such as activities / fragments). Thanks to Robolectric: –This challenge is also met: http://robolectric.org/ –Robolectric mocks all Android classes by Shadow classes. –Robolectric adds a great power to Android apps unit testing.
  11. 11. Code Coverage and Jacoco  Code Coverage represents the amount of source code which will be executed when test cases run.  In order to measure the amount of tested source code, there are popular coverage criteria: –Statement coverage –Function coverage –Branch coverage, which represents the amount of branches of each control structure (such as if) covered.
  12. 12. Code Coverage and Jacoco  JaCoCo is a free code coverage library for Java.  It includes the popular code coverage metrics: statement coverage, branch coverage, and method coverage.
  13. 13. Code Coverage and Jacoco  Fortunately, JaCoCo is fully Integrated with Gradle.  JaCoCo works fine with Android projects.  In order to configure JaCoCo with Gradle: –Enable code coverage for the build type(s) that you will be testing with. –Import JaCoCo in your gradle build file. –Configure JaCoCo with your Java sources.
  14. 14. Code Coverage and Jacoco Demo Sample URL: https://github.com/hazems/Dagger-Sample
  15. 15. Challenges  Traditional code coverage only measure the amount of executed code.  Traditional code coverage does not detect code faults.  Traditional code coverage does not show how strong the code is.
  16. 16. Mutation testing 101  Mutation testing is about seeding app source code with faults (mutations).  After seeding, unit tests then run.  If a unit test fails, then a mutation is killed (and means that your unit test is strong enough to face this mutation).  If a unit test succeeds, then a mutation is lived (and means that your unit test needs modification to be stronger).  In mutation testing, the quality of the test can be measured by the percentage of the killed mutations.
  17. 17. Mutation testing 101  Mutations can be for example: –Negate Conditionals Mutator (<= TO > and < TO >=) –Increment Mutator (i++ TO i--) –Invert Negatives Mutator (-I to I) –Math Mutator (+ to – and – to + and * to / and * to /) –Return Values Mutator (boolean  true to false) –Void Method Call Mutator (removing call).
  18. 18. What and Why PIT?  One of the mutation testing tools for Java.  It has advantages –Fast to execute (unlike most of the Java mutation testing tools). –Compatible with: •Ant •Maven •Gradle –Active. –Provides easy to read test reports.
  19. 19. PIT Android Integration  Gradle Plugin for PIT is available: http://gradle-pitest-plugin.solidsoft.info/ https://github.com/szpak/gradle-pitest-plugin  Gradle Plugin works perfect with Java projects.  However, for Android Gradle projects, you need to use this fork: https://github.com/koral--/gradle-pitest-plugin
  20. 20. PIT Android Integration  For using PIT in your Android apps, add PIT plugin to your top-level build.gradle file as follows buildscript { repositories { mavenCentral() } dependencies { classpath 'pl.droidsonroids.gradle:gradle- pitest-plugin:0.0.3' } }  Apply plugin: 'com.android.application’ apply plugin: 'pl.droidsonroids.pitest'
  21. 21. PIT Android Integration  Configure PIT plugin basically as follows pitest { targetClasses = ['com.test.xyz.daggersample.presenter.*'] /* Specify target classes to be mutated */ excludedClasses = ['**Factory*'] /* Exclude the unnecessary classes */ pitestVersion = "1.1.0" threads = 4 /* Specify number of threads to execute mutation testing */ outputFormats = ['XML', 'HTML'] /* Specify output format */ }
  22. 22. PIT Android Integration  Check the mutation results by executing the following command: >./gradlew clean pitest
  23. 23. Mutation Testing Demo Demo
  24. 24. Quiz
  25. 25. Q & A Twitter: @hazems Email: hazems@apache.org

×