Slides and link to audio from my talk + demo on how to "Automation across Platform, OS, Technologies with TaaS" at Agile India 2014, Bangalore on 1st March 2014

1. Automate
across
Pla.orm,
OS,
Technologies
with
TaaS
Anand
Bagmar
So#ware
Quality
Evangelist
Principal
Consultant,
@BagmarAnand

2. Organiza@on
Objec@ve
Money
/
Value
Time
Quality

3. Reality
–
Organiza@ons
are
spread
across
the
World

4. What
is
a
prac@ce
that
makes
Teams
successful?
Test
Automa=on

5. What
is
a
prac@ce
that
makes
Teams
unsuccessful?
Test
Automa=on

6. Where
is
Test
Automa@on
placed
in
the
Food
Chain
of
SDLC?
Is
Test
Automa@on
treated
as
1st
class
ci@zen
in
your
organiza@on?
• Value
– To
the
team?
– From
the
team?
• Quality
– Provide
feedback?
– Built
well?

7. Principles
&
Prac@ces
for
Building
a
Test
Automa@on
Framework

8. Code
Quality
Evolve
Extend
Design
PaDerns
Principles
for
building
an
Automa=on
Framework
Pairing
Refactor
Abstrac=on
Layers

9. No
Copy-­‐Paste
Con=nuous
Integra=on
(CI)
Screenshots,
Video
Recording
Configurable
Prac=ces
for
building
an
Automa=on
Framework
Logging
Tools
&
U=li=es
Test
Data

10. Audience
Survey
How
many
of
you
are
part
of
a
large
organiza@on?

11. Audience
Survey
…
large
organiza,on?
What
are
the
average
number
of
products
in
your
organiza@on’s
porVolio?

12. Audience
Survey
…
large
organiza,on?
…
number
of
products?
What
are
the
average
number
of
projects
across
all
these
products?

13. Audience
Survey
…
large
organiza,on?
…
number
of
products
and
projects?
What
is
the
typical
technology
stack
of
these
products?

14. Audience
Survey
…
large
organiza,on?
…
number
of
products
and
projects?
…
same
technology
stack?
How
many
of
you
are
part
of
merged
/
acquired
companies?
Do
you
have
the
same
technology
across
the
“new”
products?

15. Audience
Survey
…
large
organiza,on?
…
number
of
products
and
projects?
…
same
technology
stack?
…
merged
companies
–
same
technology
stack?
How
many
of
you
work
in
distributed
teams?

16. Audience
Survey
…
large
organiza,on?
…
number
of
products
and
projects?
…
same
technology
stack?
…
merged
companies
–
same
technology
stack?
…
distributed
teams?
Have
you
heard
of
a
Common
Test
Automa@on
framework?
Good?
Bad?
Easy?
Difficult?

18. Case
Study

19. Outlook
(Mac)
Outlook
(Windows)
Outlook
Web
Access
(OWA)
(Browser)
Outlook
Outlook
Sync
(Android)
…

20. How
will
you
Automate
the
Tes@ng
for
Outlook?

21. Outlook
for
Windows
Outlook
for
Mac
• White
/
QTP
• Automator
Outlook
Web
Access
(OWA)
• Selenium
/
Ruby
Outlook
Sync
for
Android
• Robo@um
/
Java

22. How
will
you
Automate
the
Integra@on
Test
….
For
a
user
who
has
installed
and
configured
Outlook
on
– Windows
OS
–
desktop
version,
and,
– Android
device
–
na@ve
mobile
app
– …
An
email
draTed
in
one
product
is
reflected
in
the
other
products

23. Create
&
Save
as
DraE
1
4
End-­‐2-­‐End
Integra-on
Tests
2
Verify
DraE
Verify
DraE
present
3
Orchestrator
Modify
DraE
(Test
Framework
for)
Outlook
on
Windows
(Test
Framework
for)
Outlook
Sync
on
Android

24. The
Problem
Can
you
use
the
same
technology
stack
for
automa=ng
the
tes=ng
of
each
variant
of
Outlook?

25. Outlook
for
Windows
Outlook
for
Mac
• White
/
QTP
• Automator
Outlook
Web
Access
(OWA)
• Selenium
/
Ruby
Outlook
Sync
for
Android
• Robo@um
/
Java

26. My
Original
Problem

27. The
Solu=on

28. What
is
TaaS?
“TaaS”
is
a
product
that
allows
you
to
validate
the
integra-on
across
a
variety
of
products
via
Test
Automa-on
• The
“correct”
way
• Pla.orm
/
OS
agnos=c
• Implementa=on
Language
Independent

29. What
is
TaaS?
Agnos-c
PlaVorm
/
OS
Implementa@on
Language

30. What
is
TaaS?
Contract
Decoupling
of
technology
barriers
Timeout

31. What
is
TaaS?
Simple
Parameters
Input
&
Output

32. What
is
TaaS?
Developed
in
Ruby
using
Sinatra

33. TaaS
is
Not
a
…
“Tool”
for
integra@on
different
products
-­‐ Lack
of
Security
-­‐ Probably
not
as
robust

34. TaaS
is
Not
a
…
Load
Tes@ng
Tool

35. How
does
TaaS
work?

36. Create
&
Save
as
DraE
1
End-­‐2-­‐End
Integra-on
TaaS
Tests
Client
Framework
Orchestrator
4
Verify
DraE
Test
Framework
TaaS
for
Server
Outlook
on
Windows
Service
Providers
2
Verify
DraE
present
3
Modify
DraE
Test
Framework
TaaS
for
Outlook
Server
Sync
on
Android

37. Demo
How
to
use
TaaS?
TaaS
Server
-­‐>
Ruby
project
TaaS
Service
provider
-­‐>
Cucumber-­‐jvm
Project
(Java)
Client
–
command
line
invoca@on

38. TaaS
Server
Implementa@on

39. What
is
a
Contract?
Specified
in
a
simple
yml
file

40. Contract
Defini@on

41. Contract
Defini@on

42. Implement
the
Contract
• Update
tests
to
-­‐
– Handle
input
parameters
• From
Environment
variables
– Execute
the
“test”

43. Implement
the
Contract

44. Parse
Input
Parameters

45. Return
the
Results
• Output
parameters
– As
console
output
between
special
markers
• Console
logs
• Excep@ons
(if
any)
• All
return
values
are
in
“json”
format

46. Collect
Output
Parameter
Values

47. Return
Output
Parameter
Values

48. Run
TaaS
Server
• Dedicated
machine
• Provide
IP
address
/
host
name
+
port
to
poten@al
clients

49. TaaS
Client
Implementa@on

50. Setup
TaaS
Client

51. Invoke
TaaS
Contract

52. TaaS
Technical
Summary

53. TaaS
Server
•
•
•
•
•
Specify
contract
details
Implement
contracts
Return
the
results
Run
TaaS
Server
(REST
service)
Serve
TaaS
Client
requests
– As
separate
processes

54. TaaS
Client
•
•
•
•
Collect
the
input
parameters
Invoke
the
contract
Consume
the
result
Con@nue
with
test
orchestra@on

55. Summary
of
Features
• Contract
– Decoupling
of
technology
barriers
– Timeout
• Passing
of
input
parameters
as
Environment
variables
• Result
as
json
– Output
parameters
– Console
logs
and
errors
– Excep@ons
• Developed
in
Ruby
using
Sinatra

56. Why
is
this
a
Good
Idea?
• Automate
the
last-­‐mile
• No
code
duplica@on
• Implementa@on
of
contract
lies
with
the
framework
tes@ng
that
product
– Evolves
with
product
changes
• Decoupling
of
technologies

57. Why
is
this
a
Good
Idea?
• Helps
in
Manual
Tes@ng
(setup
of
data)
• Anyone
can
use
it
• Each
product
is
tested
in
the
“best”
possible
way

58. How
can
you
use
TaaS?
•
•
•
•
Open
source
(apache2
license)
Available
on
github!
Available
on
rubygems.org
!
Refer
sample
project
(taas-­‐server)
to
get
started

59. Steps
to
Follow
• Nego@ate
contract
details
• Setup
&
configure
TaaS
Server
– Implement
contracts
– Return
the
results
– Run
TaaS
Server
(REST
service)
• Configure
TaaS
Client

60. Does
it
really
work?
>1.2K
downloads

61. Before
you
use
TaaS
…
Iden@fy
–
• Is
TaaS
the
right
thing
for
you?

62. How
can
You
help
TaaS?
•
•
•
•
•
Error
codes
for
the
service
Service
Discovery
Security
Create
a
Java
/
.Net
version
of
TaaS
Server
Asynchronous
requests

63. What
TaaS
did
for
me?
• Be
crea@ve
• Find
Innova@ve
solu@on
to
the
problem
• Another
open
source
contribu@on

64. Thank
you
abagmar@gmail.com
@BagmarAnand
about.me/anand.bagmar