Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Not sure if I have to check out this problem
Or just wait for the customer feedback
How to Narrow Down What to Test
@Zsolt...
@ZsoltFabok
or
#xp2013
Exercise 0: Setup
Download putty.exe if you are using windows:
http://<host>:8080/xp2013/putty.exe
Log in (user, userpass):
ssh user@<host> ...
Check the source directory in your browser:
http://<host>:8080/xp2013/<yourname>
“I get paid for code that works, not for tests, so my
philosophy is to test as little as possible to reach a given
level o...
I’d like to [re]start working on this
legacy application
My “where to start” list
#1 Determine which parts of
the code are really used
The good old Standish Group Study
7%
13%
16%19%
45%
Always
Often
Sometimes
Rarely
Never
The goal is to find those features which are
always or often used.
By studying coverage, access logs,
traces, web analytics...
Let’s have a look at the coverage
(using instrumented class files):
% cp jetty/cobertura.ser web.ser
% cp uploader/cobertur...
Example #1: overview
Example #2: execution
not even
executed
Example #3: number of execution
.NET wins
FileBasedMetadata (usage)
FileHelper (usage)
Exercise 1: Usage
Run the application:
$ run.sh “4+3”
Turn on data collection:
$ ant instrument
$ vi run.sh
Run the application a couple of ...
#2 Find out which parts of
the code change often
By checking how many times a file has
been committed into VCS:
% ./git_stat.sh
14, VerifierTask.java
13, index.jsp
11, FileB...
FileBasedMetadata (usage)
FileHelper (usage)
VerifierTask (changes)
index.jsp (changes)
FileBasedUserHome (changes)
Exercise 2: Have a beer (or
tea of coffee) :-)
#3 Determine which part of
the code changes data
Code review
“You have exactly 1 minute to explain to me
what that method does!”
FileBasedMetadata (usage)
FileHelper (usage, review)
VerifierTask (changes)
index.jsp (changes)
FileBasedUserHome (changes...
Exercise 3: Code Review
Check the code in the reports directory in your
browser:
http://<host>:8080/xp2013/<yourname>/.../target/
reports/src
#4 Determine where the code
is most likely going to fail
(e.g. with static code checkers)
PMD is not helpful at the moment,
but good to know about it
FileBasedMetadata (usage)
FileHelper (usage, review, bugs)
VerifierTask (changes)
index.jsp (changes)
FileBasedUserHome (c...
Let’s order our list and
we are done!
FileBasedMetadata (usage)
FileHelper (usage, review, bugs)
VerifierTask (changes)
index.jsp (changes)
FileBasedUserHome (c...
Exercise 4: Code Checkers
Check the reports directory in your browser:
http://<host>/xp2013/<yourname>/.../target/reports
Generate reports:
$ ant re...
Now we know where to start, and
now let’s talk about how to start.
Gaining 30% coverage in 2 minutes:
public class CheaterTest {
@Test
public void shouldIncreaseTheCoverage() {
HarvesterTas...
So, you start with an assertion:
public class FileHelperTest {
@Test
public void shouldReturnTheContentOfAFile() throws IO...
First test case is done:
public class FileHelperTest {
@Test
public void shouldReturnTheContentOfAFile() throws IOExceptio...
Now we have two test cases:
public class FileHelperTest {
@Test
public void shouldReturnTheContentOfAFile() throws IOExcep...
And we are done (assertion + coverage):
public class FileHelperTest {
private File input;
@Before
public void setUp() thro...
Exercise 5: Writing tests
Get test coverage (also runs tests):
$ ant cobertura
Run tests:
$ ant test
If you don’t want to write tests, you can find
e...
Well tested code
Well tested code everywhere
What about web applications?
(I’ll use a ruby on rails example, but the principles apply to other frameworks as well)
#2 Find out which parts of the code change
often (a.k.a VCS statistics)
#3 Determine which part of the code
changes data (...
A large variety of tools are available for:
#4 Determine where the code is most likely
going to fail (a.k.a static code ch...
% gem install rails_best_practices
% rails_best_practices -html .
Everything is nice and straightforward
until now, but the last remaining point
is tricky:
#1 Determine which parts of the ...
We can have coverage data in Ruby on Rails, too:
~/Temp/repos/sample_app % gem install simplecov
~/Temp/repos/sample_app %...
There is only one problem: the application
must be stopped in order to get the report,
which is not really efficient and us...
Fortunately, we can use a metric called
‘funnel’:
1. This is a huge lost in visitors,
would be good to know why.
2. Users visit this page more
often than the other page.
Slides: http://zsoltfabok.com/speaking/
Code: https://github.com/ZsoltFabok/
arithmetic.expression.evaluator/tree/xp2013/
Thank you very much for your attention!
http://zsoltfabok.com/ @ZsoltFabok
[xp2013] Narrow Down What to Test
[xp2013] Narrow Down What to Test
[xp2013] Narrow Down What to Test
[xp2013] Narrow Down What to Test
[xp2013] Narrow Down What to Test
[xp2013] Narrow Down What to Test
Upcoming SlideShare
Loading in …5
×

of

[xp2013] Narrow Down What to Test Slide 1 [xp2013] Narrow Down What to Test Slide 2 [xp2013] Narrow Down What to Test Slide 3 [xp2013] Narrow Down What to Test Slide 4 [xp2013] Narrow Down What to Test Slide 5 [xp2013] Narrow Down What to Test Slide 6 [xp2013] Narrow Down What to Test Slide 7 [xp2013] Narrow Down What to Test Slide 8 [xp2013] Narrow Down What to Test Slide 9 [xp2013] Narrow Down What to Test Slide 10 [xp2013] Narrow Down What to Test Slide 11 [xp2013] Narrow Down What to Test Slide 12 [xp2013] Narrow Down What to Test Slide 13 [xp2013] Narrow Down What to Test Slide 14 [xp2013] Narrow Down What to Test Slide 15 [xp2013] Narrow Down What to Test Slide 16 [xp2013] Narrow Down What to Test Slide 17 [xp2013] Narrow Down What to Test Slide 18 [xp2013] Narrow Down What to Test Slide 19 [xp2013] Narrow Down What to Test Slide 20 [xp2013] Narrow Down What to Test Slide 21 [xp2013] Narrow Down What to Test Slide 22 [xp2013] Narrow Down What to Test Slide 23 [xp2013] Narrow Down What to Test Slide 24 [xp2013] Narrow Down What to Test Slide 25 [xp2013] Narrow Down What to Test Slide 26 [xp2013] Narrow Down What to Test Slide 27 [xp2013] Narrow Down What to Test Slide 28 [xp2013] Narrow Down What to Test Slide 29 [xp2013] Narrow Down What to Test Slide 30 [xp2013] Narrow Down What to Test Slide 31 [xp2013] Narrow Down What to Test Slide 32 [xp2013] Narrow Down What to Test Slide 33 [xp2013] Narrow Down What to Test Slide 34 [xp2013] Narrow Down What to Test Slide 35 [xp2013] Narrow Down What to Test Slide 36 [xp2013] Narrow Down What to Test Slide 37 [xp2013] Narrow Down What to Test Slide 38 [xp2013] Narrow Down What to Test Slide 39 [xp2013] Narrow Down What to Test Slide 40 [xp2013] Narrow Down What to Test Slide 41 [xp2013] Narrow Down What to Test Slide 42 [xp2013] Narrow Down What to Test Slide 43 [xp2013] Narrow Down What to Test Slide 44 [xp2013] Narrow Down What to Test Slide 45 [xp2013] Narrow Down What to Test Slide 46 [xp2013] Narrow Down What to Test Slide 47 [xp2013] Narrow Down What to Test Slide 48 [xp2013] Narrow Down What to Test Slide 49 [xp2013] Narrow Down What to Test Slide 50 [xp2013] Narrow Down What to Test Slide 51 [xp2013] Narrow Down What to Test Slide 52 [xp2013] Narrow Down What to Test Slide 53 [xp2013] Narrow Down What to Test Slide 54 [xp2013] Narrow Down What to Test Slide 55 [xp2013] Narrow Down What to Test Slide 56 [xp2013] Narrow Down What to Test Slide 57 [xp2013] Narrow Down What to Test Slide 58 [xp2013] Narrow Down What to Test Slide 59 [xp2013] Narrow Down What to Test Slide 60
Upcoming SlideShare
Introduction to Software Development
Next
Download to read offline and view in fullscreen.

0 Likes

Share

Download to read offline

[xp2013] Narrow Down What to Test

Download to read offline

Narrow down what to test workshop material from XP2013

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

[xp2013] Narrow Down What to Test

  1. 1. Not sure if I have to check out this problem Or just wait for the customer feedback How to Narrow Down What to Test @ZsoltFabok http://zsoltfabok.com/ #xp2013 http://xp2013.org/ by Zsolt Fabok 2013-06-05
  2. 2. @ZsoltFabok or #xp2013
  3. 3. Exercise 0: Setup
  4. 4. Download putty.exe if you are using windows: http://<host>:8080/xp2013/putty.exe Log in (user, userpass): ssh user@<host> -p 3022 Create your environment: $ cd xp2013 $ mkdir <yourname> $ cp -r arithmetic.expression.evaluator <yourname> $ cd <yourname>/arithmetic.expression.evaluator $ ant init
  5. 5. Check the source directory in your browser: http://<host>:8080/xp2013/<yourname>
  6. 6. “I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it. I do tend to make sense of test errors, so I’m extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.” Kent Beck - September 10, 2010 quote: http://stackoverflow.com/questions/153234/how-deep-are-your-unit-tests/153565#153565
  7. 7. I’d like to [re]start working on this legacy application
  8. 8. My “where to start” list
  9. 9. #1 Determine which parts of the code are really used
  10. 10. The good old Standish Group Study 7% 13% 16%19% 45% Always Often Sometimes Rarely Never
  11. 11. The goal is to find those features which are always or often used. By studying coverage, access logs, traces, web analytics, heat maps, etc.
  12. 12. Let’s have a look at the coverage (using instrumented class files): % cp jetty/cobertura.ser web.ser % cp uploader/cobertura.ser ant.ser % ant usage_coverage usage_coverage: [cobertura-merge] Cobertura: Loaded information on 12 classes. [cobertura-merge] Cobertura: Loaded information on 11 classes. [cobertura-merge] Cobertura: Saved information on 16 classes. [cobertura-report] Cobertura: Loaded information on 16 classes. [cobertura-report] Report time: 600ms BUILD SUCCESSFUL Total time: 2 seconds
  13. 13. Example #1: overview
  14. 14. Example #2: execution not even executed
  15. 15. Example #3: number of execution .NET wins
  16. 16. FileBasedMetadata (usage) FileHelper (usage)
  17. 17. Exercise 1: Usage
  18. 18. Run the application: $ run.sh “4+3” Turn on data collection: $ ant instrument $ vi run.sh Run the application a couple of times: $ run.sh “4+3” Generate report: $ ant usage_report
  19. 19. #2 Find out which parts of the code change often
  20. 20. By checking how many times a file has been committed into VCS: % ./git_stat.sh 14, VerifierTask.java 13, index.jsp 11, FileBasedUserHome.java 11, FileBasedUser.java 11, FileBasedContentTracker.java 8, IntegrityCheckTask.java 7, MailSender.java
  21. 21. FileBasedMetadata (usage) FileHelper (usage) VerifierTask (changes) index.jsp (changes) FileBasedUserHome (changes)
  22. 22. Exercise 2: Have a beer (or tea of coffee) :-)
  23. 23. #3 Determine which part of the code changes data
  24. 24. Code review “You have exactly 1 minute to explain to me what that method does!”
  25. 25. FileBasedMetadata (usage) FileHelper (usage, review) VerifierTask (changes) index.jsp (changes) FileBasedUserHome (changes, review) FileBasedContentTracker (review)
  26. 26. Exercise 3: Code Review
  27. 27. Check the code in the reports directory in your browser: http://<host>:8080/xp2013/<yourname>/.../target/ reports/src
  28. 28. #4 Determine where the code is most likely going to fail (e.g. with static code checkers)
  29. 29. PMD is not helpful at the moment, but good to know about it
  30. 30. FileBasedMetadata (usage) FileHelper (usage, review, bugs) VerifierTask (changes) index.jsp (changes) FileBasedUserHome (changes, review) FileBasedContentTracker (review, bugs) HarversterTask (bugs) FileBasedContentTracker.fsck() (crap4j) FileBasedContentTracker.gc() (crap4j) VerifierTask.execute() (crap4j)
  31. 31. Let’s order our list and we are done!
  32. 32. FileBasedMetadata (usage) FileHelper (usage, review, bugs) VerifierTask (changes) index.jsp (changes) FileBasedUserHome (changes, review) FileBasedContentTracker (review, bugs) HarversterTask (bugs) FileBasedContentTracker.fsck() (crap4j) FileBasedContentTracker.gc() (crap4j) VerifierTask.execute() (crap4j)
  33. 33. Exercise 4: Code Checkers
  34. 34. Check the reports directory in your browser: http://<host>/xp2013/<yourname>/.../target/reports Generate reports: $ ant reports
  35. 35. Now we know where to start, and now let’s talk about how to start.
  36. 36. Gaining 30% coverage in 2 minutes: public class CheaterTest { @Test public void shouldIncreaseTheCoverage() { HarvesterTask harvester = new HarvesterTask(); Project project = new Project(); project.setBaseDir(new File(".")); harvester.setProject(project); harvester.setRepository("../repository"); harvester.setHistory("history"); harvester.setTemplate("templates"); harvester.execute(); } } Covered code != Tested code
  37. 37. So, you start with an assertion: public class FileHelperTest { @Test public void shouldReturnTheContentOfAFile() throws IOException { assertEquals("", FileHelper.getFileContent(null)); } } ➡ The ‘assertEquals’ makes sure that your test actually does something ➡ The ‘null’ parameter - along with the NullPointerException - will show you where to continue
  38. 38. First test case is done: public class FileHelperTest { @Test public void shouldReturnTheContentOfAFile() throws IOException { File input = File.createTempFile("foo", "bar"); assertEquals("", FileHelper.getFileContent(input)); } } ➡ Now the test is green, let’s continue with a more meaningful test case
  39. 39. Now we have two test cases: public class FileHelperTest { @Test public void shouldReturnTheContentOfAFile() throws IOException { File input = File.createTempFile("foo", "bar"); assertEquals("", FileHelper.getFileContent(input)); } @Test public void shouldReturnTheContentOfAFile() throws IOException { File input = File.createTempFile("foo", "bar"); new FileOutputStream(input).write("something".getBytes()); assertEquals("something", FileHelper.getFileContent(input)); } } ➡ Test method names remains the same until the body is filled properly
  40. 40. And we are done (assertion + coverage): public class FileHelperTest { private File input; @Before public void setUp() throws IOException { input = File.createTempFile("foo", "bar"); } @Test public void shouldReturnAnEmptyStringForAnEmptyFile() throws IOException { assertEquals("", FileHelper.getFileContent(input)); } @Test public void shouldReturnTheContentOfAFile() throws IOException { setInputFileContent("something"); assertEquals("something", FileHelper.getFileContent(input)); } private void setInputFileContent(String content) throws IOException { new FileOutputStream(input).write("something".getBytes()); } }
  41. 41. Exercise 5: Writing tests
  42. 42. Get test coverage (also runs tests): $ ant cobertura Run tests: $ ant test If you don’t want to write tests, you can find examples in this folder: samples
  43. 43. Well tested code Well tested code everywhere
  44. 44. What about web applications? (I’ll use a ruby on rails example, but the principles apply to other frameworks as well)
  45. 45. #2 Find out which parts of the code change often (a.k.a VCS statistics) #3 Determine which part of the code changes data (a.k.a code review) Points are just the same. and
  46. 46. A large variety of tools are available for: #4 Determine where the code is most likely going to fail (a.k.a static code checkers)
  47. 47. % gem install rails_best_practices % rails_best_practices -html .
  48. 48. Everything is nice and straightforward until now, but the last remaining point is tricky: #1 Determine which parts of the code are really used (a.k.a. coverage)
  49. 49. We can have coverage data in Ruby on Rails, too: ~/Temp/repos/sample_app % gem install simplecov ~/Temp/repos/sample_app % cat script/rails #!/usr/bin/env ruby require 'simplecov' SimpleCov.start do add_group "Models", "app/models" add_group "Controllers", "app/controllers" end APP_PATH = File.expand_path('../../config/application', __FILE__) # rest of the script/rails script
  50. 50. There is only one problem: the application must be stopped in order to get the report, which is not really efficient and user friendly.
  51. 51. Fortunately, we can use a metric called ‘funnel’:
  52. 52. 1. This is a huge lost in visitors, would be good to know why. 2. Users visit this page more often than the other page.
  53. 53. Slides: http://zsoltfabok.com/speaking/ Code: https://github.com/ZsoltFabok/ arithmetic.expression.evaluator/tree/xp2013/
  54. 54. Thank you very much for your attention! http://zsoltfabok.com/ @ZsoltFabok

Narrow down what to test workshop material from XP2013

Views

Total views

739

On Slideshare

0

From embeds

0

Number of embeds

70

Actions

Downloads

2

Shares

0

Comments

0

Likes

0

×