Presented by Robert Yates on Tuesday, March, 21 2017 at the Detroit Craftsman Guild user group meeting.

1. QA Best Practices
QA folk: how to interview, work with, and be them.
Lessons learned over years of coding, testing, and coding tests.
1

2. About Robert Yates
1. CS degree from UMBC
2. Certified Scrum Master
3. 9+ years testing
4. Started in .Net, moved to Java & OSS
5. Interested in everything computery
2

3. Contents
1. Fundamentals
2. Writing Issues
3. Test Automation
4. Miscellaneous
5. Resources/References
3

4. Tester
Subject Under Test
4

5. Fundamentals
5

6. System Architecture
1. Know it better than anyone except the architect
2. Know the points of failure
3. Determine which component a bug is in
4. First thing to do when coming to a new system
5. Full stack QA
6. Great interview question for QA
6

7. Think from all viewpoints
1. Developer
2. Customer
3. Product Owner
4. Designer (and a little customer again)
5. IT/NOC & Release Engineer
6. Jack of all trades, king of one
7

8. Embrace and understand your CI
1. Tests should run in CI
2. Build/Understand the jobs
3. It's just more automation!
4. Rube Goldberg machines are fun!
8

9. Don’t forget non-functional test types
1. Negative
2. Edge cases
3. Fault tolerance
4. Performance
5. Stress
6. Security
7. Usability 9

10. Just.. keep the negative tests limited to work
10

11. Make it easy to test across a variety of platforms
1. The most popular platforms
2. Then the most problematic platforms
3. If possible, run auto tests cross platform
4. Have platforms available to you easily and quickly
11

12. Review every commit you test
1. Find bugs before you build
2. Prevent future bugs by keeping quality high
3. Prevent FUD
4. See which parts changed
5. Standard reasons for code review
6. Have test code reviewed too!
12

13. There's a difference between assuring quality
and being obstructionist
1. Your view vs. others
2. Convey severity and risks → others’ decision
3. Minor issues can pass temporarily so progress continues
4. Don't let the perfect be the enemy of the good
5. Varies per target market
13

14. Writing Issues
14

15. If You See Something, Enter Something
1. Search for existing bugs first
2. Enter it even if it's only seen once
3. Seemingly unrelated bugs may turn out to be related
15

16. Bug fields - what to write/expect (pt. 1)
1. Summary/Description
2. Reproducibility
3. Impact on user
4. Workaround or temporary fix?
16

17. Bug fields - what to write/expect (pt. 2)
1. How to reproduce
a. Environment conditions
b. Steps
c. Current result
d. Expected result
2. System environment seen in
3. Platform 17

18. Relevant Attachments - Always
1. Screenshots
2. Videos
3. Logs
4. Do so even if a bug seems reliably reproducible
5. Lots of tools to do these things
18

19. Write as much detail as possible in issue comments
1. What you did, results
2. Put in the build version/commit
3. Can't say how many times that has helped me
19

20. Add auto tests for each bug when possible
1. Ensure it reproduces the bug
2. Helps understand the issue better
3. Builds confidence in your suite
4. Mention bug # in the comments for the test method
5. Conversely, put test name in the bug
20

21. Test
Automation
...or making robots do all
the work
21

22. Google’s test pyramid
Doesn’t always
apply, but a good
ideal to think
about
22

23. Decide your approach
Different resources and subject matter call for different
approaches
1. Unit testing only
2. Documentation as tests
3. External test project
23

24. Research the tools available
1. Lots of tools for everything imaginable
2. Look at strengths and weaknesses for your case
3. Each tool often has its own conventions
4. Ensure the tool can fit into your CI
5. If OSS, check for active development
24

25. Keep tests hermetic
1. Spin up new environments per test run
2. Isolate other environment variables if possible
3. Ensure one test doesn't affect another
4. If doing full end-to-end, use APIs to do setup
25

26. Keep test methods narrow
1. Don't test a dozen cases in one method
2. Faster to determine underlying cause
3. Early failures can mask other issues
26

27. Same code quality issues as all other code
1. Keep it clean
2. Keep it DRY
3. Others have gone into way more detail on this
4. Ensure it gets reviewed too
27

28. Useful patterns + code samples
1. Test behavior, not pixel perfect UI
2. Don’t estimate manual + auto separately
3. Share common test prep
4. Test matrices - lots of coverage and code reuse
5. Negative tests - a few different common ways to do
28

29. @Test
public void testFab() {
User user = UserFactory.createUser(api, orgID);
BacklinePage blpage = login(user, serverName);
switch (homePage) { // go to initial home page
case ONEONONE:
blpage = blpage.openNavDrawer().goToOneOnOne();
break;
case GROUP:
blpage = blpage.openNavDrawer().goToGroups();
break;
}
switch (fabPage) { // go to fab page
case USER:
blpage.goToUserSearch();
break;
case GROUP:
blpage.goToGroupSearch();
break;
}
}
@RunWith(Parameterized.class)
public class HomeScreenFloatingActionButtonTests
extends TestCase {
private HomePage homePage;
private FabPage fabPage;
public
HomeScreenFloatingActionButtonTests(HomePage homePage,
FabPage fabPage) {
this.homePage = homePage;
this.fabPage = fabPage;
}
@Parameters
public static Collection<Object[]> generateData()
{
ArrayList<Object[]> tests = new ArrayList<>();
for (HomePage homePage : HomePage.values()) {
for (FabPage fabPage : FabPage.values()) {
tests.add(new Object[]{homePage,
fabPage});
}
}
return tests;
}
JUnit Parameterized Test Example
29

30. JUnit Negative Test Examples - method expects
@Test(expected = TimeoutException.class)
public void loginWithBadPassword() {
User user = UserFactory.createUser(api, orgID);
// next line should throw exception
login(user.getEmail(), user.getPassword() + "z", user.getPin(), serverName);
}
30

31. JUnit Negative Test Examples - rule expects
@Rule public ExpectedException thrown = ExpectedException.none();
[...]
@Test
public void loginWithBadPin() {
[...]
LoginPage login = new LoginPage(driver);
PinPage pinPage = login.loginAs(email, password));
thrown.expect(AssertionError.class);
pinPage.enterPin("9999");
// end of method -- nothing beyond the line above will be executed
}
31

32. JUnit Negative Test Examples - try/catch
@Test
public void testOneOnOneChatWithInactiveUser() {
[...]
OneOnOneChatPage chat = oneOnOnePage.goToUserChat(userB.getLastName());
try {
chat.sendMessage("ShouldNotSend"); // should throw
NoSuchElementException
fail("User was able to send message to inactive user.");
} catch (NoSuchElementException e) {
// swallow expected exception
}
// more code can be executed after this
}
32

33. Miscellaneous
33

34. Write an app for the platform you’re testing
1. Understand general concepts + capabilities
2. Informs many parts of testing
3. Learn platform tools
34

35. Be Scrum Master
1. Gives perspective on all components
2. Makes sense to have overall pulse on the project
3. Discipline naturally translates to running meetings
4. Rotate Scrum Masters
35

36. ...Have OCD? ¯_(ツ)_/¯
1. Personal chicken and egg
2. Discipline and attention to detail
36

37. Interview questions for QA
1. Fav. project? Why? Architecture?
2. Personal coding projects?
3. Basic code challenge
4. Basic SQL challenge
5. Test scenarios for a feature of yours
37

38. Resources/References
38

39. GTAC - Google Test Automation Conference
1. At least be aware of, great resource for new
developments in the field
2. Find out about new tools and techniques
3. Streamed free!
4. Presentations on YouTube after the conference
39

40. Various Links!
1. Google Testing Blog
2. Agile Testing by Crispin and Gregory
3. OWASP Testing Guide
4. Security Now podcast
5. GTAC
40

41. Thank You and Q&A
Contact Info:
robert.k.yates@gmail.com
https://www.linkedin.com/in/robert-yates-5441b061
41