Pub Testing #1
Vipin Jain
"End to end testing: key to application health checkup."
&
key to Application health checkup
Software complexity over time
Development getting Complex
So Many Types of Testing!
Why Do we need
End to End Testing
when we can test
each aspect of the
application?
Lessons learned
How to do EtoE testing?
 Ask Managers and Product Owners
 Their Answer: Build Testcases End to End and execute.
More testcases, better would be the testing.
Sure you want E2E testing?
DEVOPS CI and CD
MICROSERVICES
a
EtoE seems a very old way
Then How EtoE is
different and when
and how should we do
it?
Why We need it?
• Complex Modern
software
systems
• Multiple sub-
systems
interconnected
• If any subsystem
crashes then
whole system
will crash
How to plan?
Think from End User scenario and Think
Start to Finish
Study of end to end testing requirements and
describe all the systems and subsystems processes.
Test Environment setup and hardware/software
requirements
Description of roles and responsibilities for all the
systems
End to end requirements tracking and designing of test
cases
Input and output data for each system
Perform following activities
Things to care when designing
• A simple user
journey for buying
an item triggers
action in many
connected
systems.
• These are
independent, and
EtoE testing is
performed to
ensure that they
work together as
expected.
e.g.
• Can the cart hold
100 items?
• Can the delivery be
done on 29’Th Feb?
E-commerce application
How I arranged and managed
My Metrics for EtoE testing
Test Case preparation status
Test Case preparation progress against planned
Weekly Test Progress
week-wise details of percentage test completion-
Failed, not executed & executed against planned for
execution.
Defects Status & Details
Percentage of open & closed defects by week
Environment Availability
Total number of hours active against Total number of
hours scheduled per day for testing
More can harm !
• Avoid adding unnecessary tests
• Challenge every request to add
more tests to EtoE suite
• Executing more tests could be
counter productive.
Each added testcase adds time for
• data setup
• Users setup and access grants
• coordinating execution with
everyone
• monitoring all the messages and
ensuring that everything works
Automation approach
• Critical bugs come not from individual pieces
malfunctioning but rather from multiple pieces not
working correctly together, which is where EtoE testing is
required
• Building automation scripts for these is DIFFICULT but
worth trying for, as it involves script all testcases for
complete product rather than running Unit tests
automation.
• Off course, it is time consuming and require full
knowledge of the system.
Automation approach
Thank You !
Email:
Vipin.jain@metacube.com
Linkedin:
in.linkedin.com/in/vipinqalead/
Twitter:
vipin_QA

End to end testing - strategies

  • 1.
    Pub Testing #1 VipinJain "End to end testing: key to application health checkup." &
  • 2.
    key to Applicationhealth checkup
  • 3.
  • 4.
  • 6.
    So Many Typesof Testing!
  • 7.
    Why Do weneed End to End Testing when we can test each aspect of the application?
  • 8.
  • 9.
    How to doEtoE testing?  Ask Managers and Product Owners  Their Answer: Build Testcases End to End and execute. More testcases, better would be the testing.
  • 10.
    Sure you wantE2E testing? DEVOPS CI and CD MICROSERVICES a
  • 11.
    EtoE seems avery old way
  • 12.
    Then How EtoEis different and when and how should we do it?
  • 13.
    Why We needit? • Complex Modern software systems • Multiple sub- systems interconnected • If any subsystem crashes then whole system will crash
  • 14.
    How to plan? Thinkfrom End User scenario and Think Start to Finish
  • 15.
    Study of endto end testing requirements and describe all the systems and subsystems processes. Test Environment setup and hardware/software requirements Description of roles and responsibilities for all the systems End to end requirements tracking and designing of test cases Input and output data for each system Perform following activities
  • 16.
    Things to carewhen designing
  • 17.
    • A simpleuser journey for buying an item triggers action in many connected systems. • These are independent, and EtoE testing is performed to ensure that they work together as expected. e.g.
  • 18.
    • Can thecart hold 100 items? • Can the delivery be done on 29’Th Feb?
  • 19.
  • 20.
    How I arrangedand managed
  • 21.
    My Metrics forEtoE testing Test Case preparation status Test Case preparation progress against planned Weekly Test Progress week-wise details of percentage test completion- Failed, not executed & executed against planned for execution. Defects Status & Details Percentage of open & closed defects by week Environment Availability Total number of hours active against Total number of hours scheduled per day for testing
  • 23.
    More can harm! • Avoid adding unnecessary tests • Challenge every request to add more tests to EtoE suite • Executing more tests could be counter productive. Each added testcase adds time for • data setup • Users setup and access grants • coordinating execution with everyone • monitoring all the messages and ensuring that everything works
  • 24.
    Automation approach • Criticalbugs come not from individual pieces malfunctioning but rather from multiple pieces not working correctly together, which is where EtoE testing is required • Building automation scripts for these is DIFFICULT but worth trying for, as it involves script all testcases for complete product rather than running Unit tests automation. • Off course, it is time consuming and require full knowledge of the system.
  • 25.
  • 26.