The document presents a test automation workshop led by Mike Duskis, focusing on integrating various test automation tools effectively. It includes case studies, challenges, and solutions related to automated testing for web and mobile applications primarily developed in Ruby on Rails. Key themes include problem decomposition, tool selection, and the design of a scalable test automation architecture.

1.
T15
Test
Automation
10/5/17
13:30
Integrate
Your
Test
Automation
Tools
for
More
Power
Presented
by:
Mike
Duskis
ProtectWise
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-­‐-­‐-­‐268-­‐-­‐-­‐8770
·∙·∙
904-­‐-­‐-­‐278-­‐-­‐-­‐0524
-­‐
info@techwell.com
-­‐
http://www.starwest.techwell.com/

2.
Mike
Duskis
ProtectWise
Mike
Duskis
got
his
test
automation
start
in
1997
using
Rational
Robot
on
a
multimedia
interleaving
product.
Through
the
years,
he
rotated
through
testing,
development,
and
test
management
roles
for
industries
ranging
from
higher
education
to
medical
devices
but
rarely
drifted
far
from
the
source
code.
He
currently
plays
test
automation
architect
at
ProtectWise,
an
enterprise
security
company
that
records,
analyzes,
and
visualizes
network
activity
on
demand.
The
service
lives
in
the
cloud,
but
the
office
overlooks
Union
Station
in
Denver,
Colorado.

3. Integrate
More Power
your test automation tools for
Anaheim, CA | October 5, 2017
Mike Duskis
Test Automation Architect
ProtectWise

4. INTRODUCTIONS

5. INTRODUCTIONS

6. INTRODUCTIONS

7. INTRODUCTIONS

8. INTRODUCTIONS

9. PRESENTATION MAP
Introductions
Case Study:
I need a tool
Decomposing
the Problem
A Basic Solution
Challenges and Adaptations

10. CASE STUDY: A REAL-WORLD QUESTION
We create websites and mobile apps in ruby/rails
for the most part, and we'd like to start utilizing a
tool for basic functional and regression testing,
but I'm not sure where to begin.
-- L.S., Senior QA Analyst

11. CASE STUDY: PROPOSED SOLUTIONS
Calaba.sh android & iOS for mobile apps… you can
wrap it up with cucumber to get the best BDD
experience, for web go to Selenium with Ruby
plugins.
-- S.M., QA Engineer

12. Been looking at a programme called Eggplant
recently.
-- H.D., Test Analyst
CASE STUDY: PROPOSED SOLUTIONS

13. Selenium tool with TestNG for Web applications
is good for Ruby… For mobile… Appium or
Eggplant
-- A.P., Senior Test/QA Consultant
CASE STUDY: PROPOSED SOLUTIONS

14. Robot framework is really good for web
automation
-- B.D., QA Manager
CASE STUDY: PROPOSED SOLUTIONS

15. I recommend either Watir or Selenium with Ruby
-- A.N., QA Manager
CASE STUDY: PROPOSED SOLUTIONS

16. Time Machine test tool supports URL API to help
facilitate automating temporal tests.
-- J.E., Sales Manager
CASE STUDY: PROPOSED SOLUTIONS

17. CASE STUDY: PROPOSED SOLUTIONS

18. CASE STUDY: PROPOSED SOLUTIONS

19. We create websites and mobile apps in
ruby/rails for the most part, and we'd like to
start utilizing a tool for basic functional and
regression testing, but I'm not sure where to
begin.
-- L.S., Senior QA Analyst
CASE STUDY: PROPOSED SOLUTIONS

20. DECOMPOSING THE PROBLEM
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool for basic
functional and
regression
testing

21. DECOMPOSING THE PROBLEM
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool for basic
functional and
regression
testing
Inputs: Systems
Under Test
(SUTs)

22. DECOMPOSING THE PROBLEM
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool for basic
functional and
regression
testing
Inputs: Systems
Under Test
(SUTs)
What is the
expected output of
a test process?

23. DECOMPOSING THE PROBLEM
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results
Inputs: Systems
Under Test
(SUTs)

24. DECOMPOSING THE PROBLEM
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results
Outputs: Reports
Inputs: Systems
Under Test
(SUTs)

25. DECOMPOSING THE PROBLEM
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results
Process: Exercise SUTs
Outputs: Reports
Inputs: Systems
Under Test
(SUTs)

26. A BASIC SOLUTION
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results
Tool that talks to
web interfaces

27. A BASIC SOLUTION
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results

28. A BASIC SOLUTION
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results
Tool that talks to
mobile apps

29. A BASIC SOLUTION
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results

30. A BASIC SOLUTION
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results
Tool that executes
functional tests

31. A BASIC SOLUTION
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results

32. A BASIC SOLUTION
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results
Tool that generates
reports

33. A BASIC SOLUTION
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results

34. A BASIC SOLUTION
We create
websites and
mobile apps in
ruby/rails for
the most part,
and we'd like to
start utilizing a
tool to execute
functional tests
and report their
results

35. A BASIC SOLUTION
simplest thing that could possibly work

36. CHALLENGE: DRIVER MECHANICS
What happens when a test talks to
Selenium or Appium directly?

37. CHALLENGE: DRIVER MECHANICS

38. ADAPTATION: SUT ABSTRACTION
Abstract behavior from mechanics
(Page Object Pattern)

39. ADAPTATION: SUT ABSTRACTION
page
fixtures
app
fixtures
Fixture = Page object +
everything else outside a
test's area of concern
(e.g. browser/device
discovery, authentication,
navigation, event
detection)

40. CHALLENGE: STARTING TESTS
page
fixtures
app
fixtures

41. ADAPTATION: MOVE TO C.I.
page
fixtures
app
fixtures

42. CHALLENGE: SCALABILITY
page
fixtures
app
fixtures

43. ADAPTATION: DISTRIBUTE BROWSERS
page
fixtures
app
fixtures
Distribute
browsers

44. ADAPTATION: DISTRIBUTE DEVICES
page
fixtures
app
fixtures
Distribute
browsers
Distributedevices

45. ADAPTATION: MANAGE TEST CODE
page
fixtures
app
fixtures
Distribute
browsers
Distributedevices

46. ADAPTATION: CONTAINERIZE
page
fixtures
app
fixtures
Distribute
browsers
Distributedevices
execution environment

47. CHALLENGE: EXTENSIBILITY
page
fixtures
app
fixtures
Distribute
browsers
Distributedevices
execution environment

48. ADAPTATION: PUBLISH EVENTS
page
fixtures
app
fixtures
Distribute
browsers
Distributedevices
event
system
reporter
reporter
execution environment

49. SIDEBAR: PUBLISH/SUBSCRIBE PATTERN
subscriber broker publisher

50. SIDEBAR: PUBLISH/SUBSCRIBE PATTERN
subscriber broker publisher
subscribe to topic
“foo”

51. SIDEBAR: PUBLISH/SUBSCRIBE PATTERN
subscriber broker publisher
subscribe to topic
“foo”
“foo”
(arbitrary payload)

52. SIDEBAR: PUBLISH/SUBSCRIBE PATTERN
subscriber broker publisher
subscribe to topic
“foo”
“foo”
(arbitrary payload)“foo”
(arbitrary payload)

53. SIDEBAR: PUBLISH/SUBSCRIBE PATTERN
reporter
event
system
test
subscribe to topic
“test started”
“test started”
test name = spam“test started”
test name = spam

54. ADAPTATION: PUBLISH EVENTS
page
fixtures
app
fixtures
Distribute
browsers
Distributedevices
event
system
reporter
reporter
execution environment

55. CHALLENGE: ARTIFACTS SCATTERED
page
fixtures
app
fixtures
Distribute
browsers
Distributedevices
event
system
reporter
reporter
execution environment

56. ADAPTATION: MANAGE ARTIFACTS
page
fixtures
app
fixtures
Distribute
browsers
Distributedevices
event
system
reporter
reporter
artifact
manager
execution environment

57. CHALLENGE: COMPLEX TEST SCRIPTS
page
fixtures
app
fixtures
Distribute
browsers
Distributedevices
event
system
reporter
reporter
artifact
manager
execution environment

58. ADAPTATION: BUILD SCAFFOLDS
test scaffold
page
fixtures
app
fixtures
Distribute
browsers
Distributedevices
event
system
reporter
reporter
artifact
manager
execution environment

59. SIDEBAR: BASIC SCAFFOLD PATTERN
Run setup
function
Run test
script and
capture
failure
Run
teardown
function

60. SIDEBAR: EXTENDED SCAFFOLD PATTERN
Publish
“test
started”
Provision
fixtures
Run setup
function
Run test
script and
capture
failure
Publish
“test
failed”
Run
teardown
function
Publish
“test
ended”

61. SIDEBAR: SAMPLE TEST CASES

62. SIDEBAR: TEST CASE SCAFFOLDING

63. ADAPTATION: BUILD SCAFFOLDS
test scaffold
page
fixtures
app
fixtures
Distribute
browsers
Distributedevices
event
system
reporter
reporter
artifact
manager
execution environment

64. RECAP: WHAT DID WE ACCOMPLISH?

65. RECAP: WHAT DID WE ACCOMPLISH?
1. We decomposed
the problem

66. RECAP: WHAT DID WE ACCOMPLISH?
1. We decomposed
the problem
2. We designed the
simplest solution that
could possibly work

67. RECAP: WHAT DID WE ACCOMPLISH?
1. We decomposed
the problem
2. We designed the
simplest solution that
could possibly work
3. We iterated in response
to challenges

68. RECAP: WHAT DID WE ACCOMPLISH?
4. We discovered common patterns

69. RECAP: WHAT DID WE ACCOMPLISH?
4. We discovered common patterns
Abstract the SUT using test fixtures

70. RECAP: WHAT DID WE ACCOMPLISH?
4. We discovered common patterns
Abstract the SUT using test fixtures
Choose a trigger (e.g. continuous integration)

71. RECAP: WHAT DID WE ACCOMPLISH?
4. We discovered common patterns
Abstract the SUT using test fixtures
Choose a trigger (e.g. continuous integration)
Distribute components using remote services

72. RECAP: WHAT DID WE ACCOMPLISH?
4. We discovered common patterns
Abstract the SUT using test fixtures
Choose a trigger (e.g. continuous integration)
Distribute components using remote services
Distribute test scripts using source control

73. RECAP: WHAT DID WE ACCOMPLISH?
4. We discovered common patterns
Abstract the SUT using test fixtures
Choose a trigger (e.g. continuous integration)
Distribute components using remote services
Distribute test scripts using source control
Distribute execution environment using containers

74. RECAP: WHAT DID WE ACCOMPLISH?
4. We discovered common patterns
Abstract the SUT using test fixtures
Choose a trigger (e.g. continuous integration)
Distribute components using remote services
Distribute test scripts using source control
Distribute execution environment using containers
Support arbitrary concerns by publishing events

75. RECAP: WHAT DID WE ACCOMPLISH?
4. We discovered common patterns
Abstract the SUT using test fixtures
Choose a trigger (e.g. continuous integration)
Distribute components using remote services
Distribute test scripts using source control
Distribute execution environment using containers
Support arbitrary concerns by publishing events
Manage artifacts centrally

76. RECAP: WHAT DID WE ACCOMPLISH?
4. We discovered common patterns
Abstract the SUT using test fixtures
Choose a trigger (e.g. continuous integration)
Distribute components using remote services
Distribute test scripts using source control
Distribute execution environment using containers
Support arbitrary concerns by publishing events
Manage artifacts centrally
Simplify scripts by building them on test scaffolds

77. mike@testbotwrangler.com

78. CREDITS
Orangeview Junior High School 2015
by Yvonne Tran
https://vimeo.com/119590100
Attribution-NonCommercial License
https://creativecommons.org/licenses/by-nc/3.0/
Commodore PET 4032
by Steve Gray
http://www.6502.org/users/sjgray/collections/commodore/c_4032.jpg
Attribution-NonCommmercial-ShareAlike 3.0 License
http://creativecommons.org/licenses/by-sa/3.0

79. CREDITS
Top of the world
by Tormod Ulsberg
https://www.flickr.com/photos/tormodspictures/15574265949/in/photostream/
Attribution-NonCommercial-NoDerivs 2.0 Generic License
https://creativecommons.org/licenses/by-nc-nd/2.0/
Quadro de Ferramentas
by Joaquim Pedro Oliveira Martin
https://www.flickr.com/photos/album-jua/5449853522/in/photostream/
Attribution-NonCommmercial-ShareAlike 2.0 Generic License
https://creativecommons.org/licenses/by-nc-sa/2.0/