Quality comes free
! with Open Source Tools
Steven Mak
Sunday, 8 September, 13
Who am I?
•Agile, TDD Coaching, Ugly Code
Cleaning Dude
•I love coding - Java, C#, Javascript, C/
C++, PHP, Perl, and some...
What kind of tests in place?
3
Sunday, 8 September, 13
Why do you want these tests?
4
Sunday, 8 September, 13
Unit Test Example
5
• Arrange-Act-Assert / Given-When-Then
• One assertion per test / Test one behaviour per class
	 @Test...
Four-Phase Test Pattern
6
Test
Setup
Execute
Verify
Clean-up
Also called:Arrange-Act-Assert
Sunday, 8 September, 13
Four-Phase Test Pattern
6
Test
Setup
Execute
Verify
Clean-up
Might happen
partially in the test
fixture
Also called:Arrange...
Four-Phase Test Pattern
6
Test
Setup
Execute
Verify
Clean-up
Also called:Arrange-Act-Assert
Sunday, 8 September, 13
Four-Phase Test Pattern
6
Test
Setup
Execute
Verify
Clean-up
Avoid mixing up
the Execute and
theVerify
Also called:Arrange...
JUnit 4
7
import org.junit.Test;
import static org.junit.Assert.*;
public class PersonTest {
	 @Test
	 public void default...
JUnit 4
7
import org.junit.Test;
import static org.junit.Assert.*;
public class PersonTest {
	 @Test
	 public void default...
JUnit 4
7
import org.junit.Test;
import static org.junit.Assert.*;
public class PersonTest {
	 @Test
	 public void default...
JUnit 4
7
import org.junit.Test;
import static org.junit.Assert.*;
public class PersonTest {
	 @Test
	 public void default...
JUnit 4
7
import org.junit.Test;
import static org.junit.Assert.*;
public class PersonTest {
	 @Test
	 public void default...
JUnit 4
7
import org.junit.Test;
import static org.junit.Assert.*;
public class PersonTest {
	 @Test
	 public void default...
JUnit 4
7
import org.junit.Test;
import static org.junit.Assert.*;
public class PersonTest {
	 @Test
	 public void default...
Test Fixture
8
import org.junit.After;
import org.junit.Before;
public class PersonTest {
	 Person john = new Person();
	 ...
Test Fixture
8
import org.junit.After;
import org.junit.Before;
public class PersonTest {
	 Person john = new Person();
	 ...
Test Fixture
8
import org.junit.After;
import org.junit.Before;
public class PersonTest {
	 Person john = new Person();
	 ...
Test Fixture
8
import org.junit.After;
import org.junit.Before;
public class PersonTest {
	 Person john = new Person();
	 ...
Test Fixture
8
import org.junit.After;
import org.junit.Before;
public class PersonTest {
	 Person john = new Person();
	 ...
Test Fixture
8
import org.junit.After;
import org.junit.Before;
public class PersonTest {
	 Person john = new Person();
	 ...
JUnit lifecycle
@BeforeClass
For each @Test
Instantiate test class
@Before
Invoke the test
@After
@AfterClass
9
Sunday, 8 ...
FIRST
10
•Fast
•Isolates
•Repeatable
•Self-validating
•Timely
and more...
• Readable
• Try: One assertion per test
You sti...
Should be fast, very fast
11
Sunday, 8 September, 13
Test Driven Development
Why don’t we write code to make test pass?
•The one rule to rule them all
-Only ever write code to...
TDD Cycle
13
Test
Implement
Design
• Short
• Rhythmic
• Incremental
• Design-focused
• Disciplined
Sunday, 8 September, 13
Functional level
test as requirement, requirement as test
14
Sunday, 8 September, 13
Use Examples
15
With 3 judges giving
scores 4, 20, and 18,
the displayed score
should be 42.
When the first 2
judges have ...
Examples, Tests, and Spec
16
Examples Tests
Requirements
can become
elaborate
verify
Sunday, 8 September, 13
17
Discuss
in workshop
Develop
in concurrence
Deliver
for acceptance
I want to show scores for
the current pair.
OK. Can y...
18
The examples are translated into executable specification,
allowing a computer to run them against the product.
Discuss
...
19
Discuss
in workshop
Develop
in concurrence
Deliver
for acceptance
With 3 judges giving
scores 4, 20, and 18,
the displa...
19
Discuss
in workshop
Develop
in concurrence
Deliver
for acceptance
With 3 judges giving
scores 4, 20, and 18,
the displa...
It's all in the tables
20
Sunday, 8 September, 13
Test Tools
Robot Architecture
21
Test Data (Tables)
Robot Framework
Test Libraries
System Under Test
Test Library API
appl...
Test Cases are composed of
keyword-driven actions
22
!"#$%&'()*+%),'-./()0
Sunday, 8 September, 13
Test Cases are composed of
keyword-driven actions
22
!"#$%&'()*+%),'-./()0
this is the name of a test case
Sunday, 8 Septe...
Test Cases are composed of
keyword-driven actions
22
!"#$%&'()*+%),'-./()0
this is the name of a test case
these keywords ...
Test Cases are composed of
keyword-driven actions
22
!"#$%&'()*+%),'-./()0
this is the name of a test case
these keywords ...
2 types of keywords
23
Sunday, 8 September, 13
2 types of keywords
23
We can import keyword libraries for a test case
Sunday, 8 September, 13
2 types of keywords
23
We can import keyword libraries for a test case
...and libraries may be configured, too.
Sunday, 8 S...
2 types of keywords
23
We can import keyword libraries for a test case
...and libraries may be configured, too.
This keywor...
2 types of keywords
23
We can import keyword libraries for a test case
...and libraries may be configured, too.
This keywor...
24
Data-driven test cases
this is the name of a test case
these keywords form the test case
keywords receive arguments
Sun...
Variables
25
!"#$"%&'(
)#*+,-*++"./,&$.'0
Sunday, 8 September, 13
Technical
Activity
Workflow
Specification pyramid
26
RuleClarity
Stability
Specification
Users can
understand
Automation
Tech...
An Example
27
Sunday, 8 September, 13
RIDE Project
28
Sunday, 8 September, 13
Rule Based Test Case
29
Rule
Sunday, 8 September, 13
Workflow Definition
30
Workflow
Sunday, 8 September, 13
Technical Activity
31
Technical
Activity
Sunday, 8 September, 13
Rule Based Test Case
32
Rule
Sunday, 8 September, 13
Workflow Definition
33
Workflow
Sunday, 8 September, 13
Technical Activity
34
Technical
Activity
Sunday, 8 September, 13
35
import imaplib
class check_mail:
ROBOT_LIBRARY_SCOPE = 'GLOBAL'
__version__ = '0.1'
error_message = ""
def open_mail_bo...
36
public class MailLibrary {
	 public static final String ROBOT_LIBRARY_SCOPE = "GLOBAL";
	 public static final String RO...
Other choices
• Cucumber
• Fitnesse
• Robot Framework
37
Sunday, 8 September, 13
38
Sonar
Sunday, 8 September, 13
39
What does this mean?
Sunday, 8 September, 13
Longitudinal study
40http://almossawi.com/firefox/prose/
Anything happened
during some point in
time?
• Project deadline?
•...
Don’t forget your
version control system
41
http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&Object...
More visualization: JUnit project
42http://dkandalov.github.io/code-history-mining/junit.html
Files often commit together?...
Thank you for spending time with me this evening.
More feedback can be sent to:
43
Odd-e Hong Kong Ltd.
Steven Mak 麥天志
Agi...
Upcoming SlideShare
Loading in...5
×

Quality comes free with open source testing tools

487

Published on

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
487
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
14
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Quality comes free with open source testing tools

  1. 1. Quality comes free ! with Open Source Tools Steven Mak Sunday, 8 September, 13
  2. 2. Who am I? •Agile, TDD Coaching, Ugly Code Cleaning Dude •I love coding - Java, C#, Javascript, C/ C++, PHP, Perl, and some weird ones •I speak English, Cantonese, and Mandarin 2 Odd-e Pte. Ltd. Steven Mak 麥天志 Agile Coach Hong Kong Email: steven@odd-e.com Web: www.odd-e.com Twitter: stevenmak Sunday, 8 September, 13
  3. 3. What kind of tests in place? 3 Sunday, 8 September, 13
  4. 4. Why do you want these tests? 4 Sunday, 8 September, 13
  5. 5. Unit Test Example 5 • Arrange-Act-Assert / Given-When-Then • One assertion per test / Test one behaviour per class @Test public void makesReservationThroughReservationService() { // ARRANGE final Reservation reservation = new Reservation(); Reservations reservations = stubReservationsToReturn(reservation); HouseKeeping houseKeeping = stubHouseKeeping(); HotelRoom room = new HotelRoom(reservations, houseKeeping); // ACT Reservation reservationReturned = room.bookFor(new Customer()); // ASSERT assertSame(reservation, reservationReturned); } Sunday, 8 September, 13
  6. 6. Four-Phase Test Pattern 6 Test Setup Execute Verify Clean-up Also called:Arrange-Act-Assert Sunday, 8 September, 13
  7. 7. Four-Phase Test Pattern 6 Test Setup Execute Verify Clean-up Might happen partially in the test fixture Also called:Arrange-Act-Assert Sunday, 8 September, 13
  8. 8. Four-Phase Test Pattern 6 Test Setup Execute Verify Clean-up Also called:Arrange-Act-Assert Sunday, 8 September, 13
  9. 9. Four-Phase Test Pattern 6 Test Setup Execute Verify Clean-up Avoid mixing up the Execute and theVerify Also called:Arrange-Act-Assert Sunday, 8 September, 13
  10. 10. JUnit 4 7 import org.junit.Test; import static org.junit.Assert.*; public class PersonTest { @Test public void defaultNameOfThePersonIsJohn() throws Exception { Person john = new Person(); assertEquals("John", john.name()); } } Sunday, 8 September, 13
  11. 11. JUnit 4 7 import org.junit.Test; import static org.junit.Assert.*; public class PersonTest { @Test public void defaultNameOfThePersonIsJohn() throws Exception { Person john = new Person(); assertEquals("John", john.name()); } } Import JUnit package Sunday, 8 September, 13
  12. 12. JUnit 4 7 import org.junit.Test; import static org.junit.Assert.*; public class PersonTest { @Test public void defaultNameOfThePersonIsJohn() throws Exception { Person john = new Person(); assertEquals("John", john.name()); } } Import JUnit package Import assertion methods Sunday, 8 September, 13
  13. 13. JUnit 4 7 import org.junit.Test; import static org.junit.Assert.*; public class PersonTest { @Test public void defaultNameOfThePersonIsJohn() throws Exception { Person john = new Person(); assertEquals("John", john.name()); } } Import JUnit package Import assertion methods Test class containing tests Sunday, 8 September, 13
  14. 14. JUnit 4 7 import org.junit.Test; import static org.junit.Assert.*; public class PersonTest { @Test public void defaultNameOfThePersonIsJohn() throws Exception { Person john = new Person(); assertEquals("John", john.name()); } } Import JUnit package Import assertion methods Test class containing tests Annotation for test methods Sunday, 8 September, 13
  15. 15. JUnit 4 7 import org.junit.Test; import static org.junit.Assert.*; public class PersonTest { @Test public void defaultNameOfThePersonIsJohn() throws Exception { Person john = new Person(); assertEquals("John", john.name()); } } Import JUnit package Import assertion methods Test class containing tests Annotation for test methods Test method Sunday, 8 September, 13
  16. 16. JUnit 4 7 import org.junit.Test; import static org.junit.Assert.*; public class PersonTest { @Test public void defaultNameOfThePersonIsJohn() throws Exception { Person john = new Person(); assertEquals("John", john.name()); } } Import JUnit package Import assertion methods Test class containing tests Annotation for test methods Test method Assertion Sunday, 8 September, 13
  17. 17. Test Fixture 8 import org.junit.After; import org.junit.Before; public class PersonTest { Person john = new Person(); @Before public void initializePerson() { john.initialize(); } @After public void cleanUpPerson() { john.clean(); } @Test public void defaultNameOfThePersonIsJohn() throws Exception { assertEquals("John", john.name()); } } Sunday, 8 September, 13
  18. 18. Test Fixture 8 import org.junit.After; import org.junit.Before; public class PersonTest { Person john = new Person(); @Before public void initializePerson() { john.initialize(); } @After public void cleanUpPerson() { john.clean(); } @Test public void defaultNameOfThePersonIsJohn() throws Exception { assertEquals("John", john.name()); } } Import Before/ After annotations Sunday, 8 September, 13
  19. 19. Test Fixture 8 import org.junit.After; import org.junit.Before; public class PersonTest { Person john = new Person(); @Before public void initializePerson() { john.initialize(); } @After public void cleanUpPerson() { john.clean(); } @Test public void defaultNameOfThePersonIsJohn() throws Exception { assertEquals("John", john.name()); } } Import Before/ After annotations Run before each test Sunday, 8 September, 13
  20. 20. Test Fixture 8 import org.junit.After; import org.junit.Before; public class PersonTest { Person john = new Person(); @Before public void initializePerson() { john.initialize(); } @After public void cleanUpPerson() { john.clean(); } @Test public void defaultNameOfThePersonIsJohn() throws Exception { assertEquals("John", john.name()); } } Import Before/ After annotations Run before each test Run after each test Sunday, 8 September, 13
  21. 21. Test Fixture 8 import org.junit.After; import org.junit.Before; public class PersonTest { Person john = new Person(); @Before public void initializePerson() { john.initialize(); } @After public void cleanUpPerson() { john.clean(); } @Test public void defaultNameOfThePersonIsJohn() throws Exception { assertEquals("John", john.name()); } } Import Before/ After annotations Run before each test Data needed in each test. Run after each test Sunday, 8 September, 13
  22. 22. Test Fixture 8 import org.junit.After; import org.junit.Before; public class PersonTest { Person john = new Person(); @Before public void initializePerson() { john.initialize(); } @After public void cleanUpPerson() { john.clean(); } @Test public void defaultNameOfThePersonIsJohn() throws Exception { assertEquals("John", john.name()); } } Import Before/ After annotations Run before each test Data needed in each test. Run after each test The data and setup / teardown (before, after) is also sometimes called: test fixture Sunday, 8 September, 13
  23. 23. JUnit lifecycle @BeforeClass For each @Test Instantiate test class @Before Invoke the test @After @AfterClass 9 Sunday, 8 September, 13
  24. 24. FIRST 10 •Fast •Isolates •Repeatable •Self-validating •Timely and more... • Readable • Try: One assertion per test You still need higher-level tests! Sunday, 8 September, 13
  25. 25. Should be fast, very fast 11 Sunday, 8 September, 13
  26. 26. Test Driven Development Why don’t we write code to make test pass? •The one rule to rule them all -Only ever write code to fix a failing test •3 Steps: -Write a test (which fails “red”) -Code (to make test pass “green”) -Refactor (test still pass “green”) -Repeat forever 12 Sunday, 8 September, 13
  27. 27. TDD Cycle 13 Test Implement Design • Short • Rhythmic • Incremental • Design-focused • Disciplined Sunday, 8 September, 13
  28. 28. Functional level test as requirement, requirement as test 14 Sunday, 8 September, 13
  29. 29. Use Examples 15 With 3 judges giving scores 4, 20, and 18, the displayed score should be 42. When the first 2 judges have given their scores, e.g. 10 and 5, the intermediate score of 15 should be displayed already. No scores displayed as a dash (–), not zero. Maximum score from a judge is 20 points! Sunday, 8 September, 13
  30. 30. Examples, Tests, and Spec 16 Examples Tests Requirements can become elaborate verify Sunday, 8 September, 13
  31. 31. 17 Discuss in workshop Develop in concurrence Deliver for acceptance I want to show scores for the current pair. OK. Can you give me an example of these scores? So if the three judges gave scores 7, 11 and 2 then the displayed score should be 20. OK. And would those individual judges' scores be shown? Or just the total? Yes, individual scores. And they should be shown all the time, even before all judges have given their scores. Like if first two judges give a 7 and an 11 then we'll show 18 as an intermediate score? Exactly. With 3 judges giving scores 4, 20, and 18, the displayed score should be 42. When the first 2 judges have given their scores, e.g. 10 and 5, the intermediate score of 15 should be displayed already. No scores displayed as a dash (–), not zero. Maximum score from a judge is 20 points! As a competition organizer in order to show the judging to the audience on several monitors I want to have the scores for the current pair on a web page. Scoring As a competition organizer in order to show the judging to the audience on several monitors I want to have the scores for the current pair on a web page. Scoring As a competition organizer in order to show the judging to the audience on several monitors I want to have the scores for the current pair on a web page. Scoring As a competition organizer in order to show the judging to the audience on several monitors I want to have the scores for the current pair on a web page. Scoring Sunday, 8 September, 13
  32. 32. 18 The examples are translated into executable specification, allowing a computer to run them against the product. Discuss in workshop Develop in concurrence Deliver for acceptance Sunday, 8 September, 13
  33. 33. 19 Discuss in workshop Develop in concurrence Deliver for acceptance With 3 judges giving scores 4, 20, and 18, the displayed score should be 42. When the first 2 judges have given their scores, e.g. 10 and 5, the intermediate score of 15 should be displayed already. No scores displayed as a dash (–), not zero. Maximum score from a judge is 20 points! Sunday, 8 September, 13
  34. 34. 19 Discuss in workshop Develop in concurrence Deliver for acceptance With 3 judges giving scores 4, 20, and 18, the displayed score should be 42. When the first 2 judges have given their scores, e.g. 10 and 5, the intermediate score of 15 should be displayed already. No scores displayed as a dash (–), not zero. Maximum score from a judge is 20 points! Robot tests are written in tables so that computers can read them Sunday, 8 September, 13
  35. 35. It's all in the tables 20 Sunday, 8 September, 13
  36. 36. Test Tools Robot Architecture 21 Test Data (Tables) Robot Framework Test Libraries System Under Test Test Library API application interfaces Robot comes with a number of built-in test libraries and you can (should!) add your own. Test libraries can use any test tool necessary to interact with the system under test. Sunday, 8 September, 13
  37. 37. Test Cases are composed of keyword-driven actions 22 !"#$%&'()*+%),'-./()0 Sunday, 8 September, 13
  38. 38. Test Cases are composed of keyword-driven actions 22 !"#$%&'()*+%),'-./()0 this is the name of a test case Sunday, 8 September, 13
  39. 39. Test Cases are composed of keyword-driven actions 22 !"#$%&'()*+%),'-./()0 this is the name of a test case these keywords form the test case Sunday, 8 September, 13
  40. 40. Test Cases are composed of keyword-driven actions 22 !"#$%&'()*+%),'-./()0 this is the name of a test case these keywords form the test case keywords receive arguments Sunday, 8 September, 13
  41. 41. 2 types of keywords 23 Sunday, 8 September, 13
  42. 42. 2 types of keywords 23 We can import keyword libraries for a test case Sunday, 8 September, 13
  43. 43. 2 types of keywords 23 We can import keyword libraries for a test case ...and libraries may be configured, too. Sunday, 8 September, 13
  44. 44. 2 types of keywords 23 We can import keyword libraries for a test case ...and libraries may be configured, too. This keyword comes from the imported library. Sunday, 8 September, 13
  45. 45. 2 types of keywords 23 We can import keyword libraries for a test case ...and libraries may be configured, too. This keyword comes from the imported library. This is a user keyword, implemented in table format. (Think macros composed of other macros.) Sunday, 8 September, 13
  46. 46. 24 Data-driven test cases this is the name of a test case these keywords form the test case keywords receive arguments Sunday, 8 September, 13
  47. 47. Variables 25 !"#$"%&'( )#*+,-*++"./,&$.'0 Sunday, 8 September, 13
  48. 48. Technical Activity Workflow Specification pyramid 26 RuleClarity Stability Specification Users can understand Automation Technical Sunday, 8 September, 13
  49. 49. An Example 27 Sunday, 8 September, 13
  50. 50. RIDE Project 28 Sunday, 8 September, 13
  51. 51. Rule Based Test Case 29 Rule Sunday, 8 September, 13
  52. 52. Workflow Definition 30 Workflow Sunday, 8 September, 13
  53. 53. Technical Activity 31 Technical Activity Sunday, 8 September, 13
  54. 54. Rule Based Test Case 32 Rule Sunday, 8 September, 13
  55. 55. Workflow Definition 33 Workflow Sunday, 8 September, 13
  56. 56. Technical Activity 34 Technical Activity Sunday, 8 September, 13
  57. 57. 35 import imaplib class check_mail: ROBOT_LIBRARY_SCOPE = 'GLOBAL' __version__ = '0.1' error_message = "" def open_mail_box(self, server, user, pwd): try: self.mailbox = imaplib.IMAP4_SSL(server) self.mailbox.login(user, pwd) self.mailbox.select() except imaplib.IMAP4.error: self.error_message = 'Could not connect to the mailbox' def count_mail(self, *args): r, self.item = self.mailbox.search(None, *args) return len(self.item[0].split()) def get_mail_error_message(self): return self.error_message def close_mailbox(self): self.mailbox.close() self.mailbox.logout() Implement a Library Technical Activity Sunday, 8 September, 13
  58. 58. 36 public class MailLibrary { public static final String ROBOT_LIBRARY_SCOPE = "GLOBAL"; public static final String ROBOT_LIBRARY_VERSION = "1.0"; private Store store; private Folder folder; public MailLibrary() { } public void openMailbox(String host, String username, String password) throws MessagingException { Properties properties = new Properties(); properties.setProperty("mail.imap.socketFactory.class", "javax.net.ssl.SSLSocketFactory"); properties.setProperty("mail.imap.socketFactory.fallback", "false"); properties.setProperty("mail.imap.socketFactory.port", "993"); Session session = Session.getDefaultInstance(properties); store = session.getStore("imap"); store.connect(host, username, password); } public void checkMailbox() throws MessagingException { FlagTerm flagUnseen = new FlagTerm(new Flags(Flags.Flag.SEEN), false); SearchTerm searchTerm = new AndTerm(flagUnseen, new FromStringTerm("sender@mail.com")); folder = store.getFolder("INBOX"); folder.open(Folder.READ_ONLY); Message messages[] = folder.search(searchTerm); if (messages.length == 0) { throw new MessagingException("No mail found"); } } public void closeMailbox() throws MessagingException { if (folder != null) { folder.close(false); } if (store != null) { store.close(); } } } Technical Activity Sunday, 8 September, 13
  59. 59. Other choices • Cucumber • Fitnesse • Robot Framework 37 Sunday, 8 September, 13
  60. 60. 38 Sonar Sunday, 8 September, 13
  61. 61. 39 What does this mean? Sunday, 8 September, 13
  62. 62. Longitudinal study 40http://almossawi.com/firefox/prose/ Anything happened during some point in time? • Project deadline? • Firefighting? • Policy change? Sunday, 8 September, 13
  63. 63. Don’t forget your version control system 41 http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=16679 Sunday, 8 September, 13
  64. 64. More visualization: JUnit project 42http://dkandalov.github.io/code-history-mining/junit.html Files often commit together? Is there collective code ownership? Which part of the project have more attention? When commit happens? How frequent commit happens? Sunday, 8 September, 13
  65. 65. Thank you for spending time with me this evening. More feedback can be sent to: 43 Odd-e Hong Kong Ltd. Steven Mak 麥天志 Agile Coach Hong Kong Email: steven@odd-e.com Web: www.odd-e.com Twitter: stevenmak Sunday, 8 September, 13
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×