*
Val Lines
* Senior project manager responsible for the quality

assurance practice at Motorists Mutual Insurance Company

* Over 25 years of IT experience, primarily in insurance and
banking on mainframe systems

* IT experience as a developer, tester, project manager,

resource manager and now quality assurance practitioner

* Contact Val
* Val.Lines@motoristsgroup.com
* 614.225.8626
* www.linkedin.com/in/vallines

*
* Primarily mainframe applications for our
production systems

* Low personnel turnover
* High reliance on subject matter experts for
each application

* QA position created in 2012
* No standardized testing practices in place
* Transformation to agile project methodology
* Agile team members work on multiple projects and
production support

* QA functions are spread throughout the team & multiple
locations
* QA practices are based on tester experience

* Test automation framework is nonexistent

* Batch test cycles
* Agile seminars, conferences, etc. all focus on testing with
tools that aren’t readily available for the mainframe

* Test data management in shared regions

*
* Metrics
* Rethinking mainframe development process into user stories
that fit in a single, 2-week sprint
* Usually not deployable at the end of one sprint
* End-to-end system testing
* Testing of date driven processes
* Production deployment activities are time consuming
* Rally vs. ALM/Quality Center
* Rally does a great job with user stories, sprints, etc.
* ALM/Quality center does a great job with traceability,
automated testing integration with QTP

*
*
* Business and IT management are committed to agile
* Management exception is needed to run a waterfall
project

* Agile coach conducts scrum training for all agile
team members (includes product owner)

* Product owners are very involved in the daily scrum
and project activities

* Agile community of practice
* Plan – Do – Check – Adjust

*
*
*
*
*
* Focus on basic regression scripts to build an automated
testing script library

* Can also be used to expedite data entry testing
* Establishing procedures so Operations can execute

* Established an agile project to give the team time and
priority to work on the scripts

* Started an automation community of practice that
meets monthly to discuss what we are learning

* Members are all using QTP for scripting mainframe, object
oriented, vendor package applications

*
* Scheduler to run the test batch cycle
* Controlled by one parameter card for all integrated systems
* All test jobs follow the same naming conventions
* Created a user ID that runs the jobs that does not have
access to production files

* Training on how to run a batch test cycle is easier

* Job output is stored the same way our production job output
is stored

* PDF generator to create pdf output from mainframe jobs
* Challenge test planners to be as efficient as possible with the
cycles that are needed

*
*
* Five test regions to be kept up to date
* Created a series of jobs that copy executable code from
the production region to the test regions

* Developers can update a parameter to prevent overlaying
work in progress

* No source code is overlaid

* Automated the schedule to execute this process weekly,
after hours

* Projects are scheduled into specific regions
* Test data management is a very manual process

*
*
*
*
* Challenge team to break down stories
* In some cases the stories have been moved from one

sprint to the next or broken down into part 1 and part 2
when it became evident it was too big
* Better approach is to break out what can be done on a
story in a sprint and make multiple stories

* Acceptance criteria may involve having another developer
review the interim deliverables

* Grooming involves all involved parties to get a better
sizing estimate
* Pair programming
* Plan – Do – Check – Adjust

*
*
* Front load as much testing as possible in each
sprint

* Acceptance testing for stories is thorough

* Each test cycle is a user story and moves

through any date driven processes that are
required

* With good user story acceptance testing

already done, the end-to-end testing moves a
little faster before the release

*
*
*
*
* ALM/QC is fully baked for test management
* Traceability
* Integration with QTP
* Metrics

* Rally is fully baked for user story management
and improving on the testing functions

* Traceability isn’t intuitive
* No integration with QTP
* Testing metrics

*
Contact Info:
Val.Lines@motoristsgroup.com
614.225.8626
www.linkedin.com/in/vallines

Val lines - Agile Testing in a Legacy World

  • 1.
  • 2.
    * Senior projectmanager responsible for the quality assurance practice at Motorists Mutual Insurance Company * Over 25 years of IT experience, primarily in insurance and banking on mainframe systems * IT experience as a developer, tester, project manager, resource manager and now quality assurance practitioner * Contact Val * Val.Lines@motoristsgroup.com * 614.225.8626 * www.linkedin.com/in/vallines *
  • 4.
    * Primarily mainframeapplications for our production systems * Low personnel turnover * High reliance on subject matter experts for each application * QA position created in 2012 * No standardized testing practices in place
  • 7.
    * Transformation toagile project methodology * Agile team members work on multiple projects and production support * QA functions are spread throughout the team & multiple locations * QA practices are based on tester experience * Test automation framework is nonexistent * Batch test cycles * Agile seminars, conferences, etc. all focus on testing with tools that aren’t readily available for the mainframe * Test data management in shared regions *
  • 8.
    * Metrics * Rethinkingmainframe development process into user stories that fit in a single, 2-week sprint * Usually not deployable at the end of one sprint * End-to-end system testing * Testing of date driven processes * Production deployment activities are time consuming * Rally vs. ALM/Quality Center * Rally does a great job with user stories, sprints, etc. * ALM/Quality center does a great job with traceability, automated testing integration with QTP *
  • 10.
  • 11.
    * Business andIT management are committed to agile * Management exception is needed to run a waterfall project * Agile coach conducts scrum training for all agile team members (includes product owner) * Product owners are very involved in the daily scrum and project activities * Agile community of practice * Plan – Do – Check – Adjust *
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
    * Focus onbasic regression scripts to build an automated testing script library * Can also be used to expedite data entry testing * Establishing procedures so Operations can execute * Established an agile project to give the team time and priority to work on the scripts * Started an automation community of practice that meets monthly to discuss what we are learning * Members are all using QTP for scripting mainframe, object oriented, vendor package applications *
  • 17.
    * Scheduler torun the test batch cycle * Controlled by one parameter card for all integrated systems * All test jobs follow the same naming conventions * Created a user ID that runs the jobs that does not have access to production files * Training on how to run a batch test cycle is easier * Job output is stored the same way our production job output is stored * PDF generator to create pdf output from mainframe jobs * Challenge test planners to be as efficient as possible with the cycles that are needed *
  • 18.
  • 19.
    * Five testregions to be kept up to date * Created a series of jobs that copy executable code from the production region to the test regions * Developers can update a parameter to prevent overlaying work in progress * No source code is overlaid * Automated the schedule to execute this process weekly, after hours * Projects are scheduled into specific regions * Test data management is a very manual process *
  • 20.
  • 21.
  • 22.
  • 23.
    * Challenge teamto break down stories * In some cases the stories have been moved from one sprint to the next or broken down into part 1 and part 2 when it became evident it was too big * Better approach is to break out what can be done on a story in a sprint and make multiple stories * Acceptance criteria may involve having another developer review the interim deliverables * Grooming involves all involved parties to get a better sizing estimate * Pair programming * Plan – Do – Check – Adjust *
  • 24.
  • 25.
    * Front loadas much testing as possible in each sprint * Acceptance testing for stories is thorough * Each test cycle is a user story and moves through any date driven processes that are required * With good user story acceptance testing already done, the end-to-end testing moves a little faster before the release *
  • 26.
  • 27.
  • 28.
  • 29.
    * ALM/QC isfully baked for test management * Traceability * Integration with QTP * Metrics * Rally is fully baked for user story management and improving on the testing functions * Traceability isn’t intuitive * No integration with QTP * Testing metrics *
  • 30.