The document discusses how a great quality assurance (QA) team can make a disproportionate contribution to project success. It outlines the attributes that define a great QA team, including having domain expertise, working independently and creatively to determine what needs to be done, using tools intelligently, and providing feedback that motivates other teams like developers to improve quality. A great QA team adds value by delivering high quality testing that finds defects early and gives stakeholders clear visibility into a project's quality level and progress.
Agile * Agile Principles * Agile Practices * Pair Programming * Extreme Programming * SOLID design principles * SDLC * Software Development
After working 10 years in multiple major "from-scratch" development projects, I finally got a chance to work in a truly Agile development project. Here is my take on how to make Agile work for your project.
Reduce Development Cost with Test Driven Developmentsthicks14
A collaboration between NetServ Applications and Celtic Testing Experts on Test Driven Development and Design. This presentation demonstrates how an organization can reduce development cost by implementing TDD.
Agile * Agile Principles * Agile Practices * Pair Programming * Extreme Programming * SOLID design principles * SDLC * Software Development
After working 10 years in multiple major "from-scratch" development projects, I finally got a chance to work in a truly Agile development project. Here is my take on how to make Agile work for your project.
Reduce Development Cost with Test Driven Developmentsthicks14
A collaboration between NetServ Applications and Celtic Testing Experts on Test Driven Development and Design. This presentation demonstrates how an organization can reduce development cost by implementing TDD.
Bert Jagers - Preserving Our Future Through Customer SatisfactionTEST Huddle
EuroSTAR Software Testing Conference 2008 presentation on Preserving Our Future Through Customer Satisfaction by Bert Jagers. See more at conferences.eurostarsoftwaretesting.com/past-presentations/
Pragmatic Not Dogmatic TDD Agile2012 by Joseph Yoder and Rebecca Wirfs-BrockJoseph Yoder
This presentation challenges the "norm" for TDD. Testing should be an integral part of your daily programming practice. But you don’t always need to derive your code via many test-code-revise-retest cycles to be test-driven. Some find it more natural to outline a related set of tests first, and use those test scenarios to guide them as they write code. Once they’ve completed a “good enough” implementation that supports the test scenarios, they then write those tests and incrementally fix any bugs as they go. As long as you don’t write hundreds of lines of code without any testing, there isn’t a single best way to be Test Driven. There’s a lot to becoming proficient at TDD. Developing automated test suites, refactoring and reworking tests to eliminate duplication, and testing for exceptional conditions, are just a few. Additionally, acceptance tests, smoke tests, integration, performance and load tests support incremental development as well. If all this testing sounds like too much work, well…let’s be practical. Testing shouldn’t be done just for testing’s sake. Instead, the tests you write should give you leverage to confidently change and evolve your code base and validate the requirements of the system. That’s why it is important to know what to test, what not to test, and when to stop testing.
I am a agile tester, because...(masterclass at the Barcelona Test Academy)Derk-Jan de Grood
These are the slides of the masterclass I gave 24-1-2018 at the Barcelona Test Academy. Using a assessment we started a discussion about what it takes to be an agile tester.
Description: Testing has transformed into Agile Testing. Testing has become a responsibilty of the whole development team. Many testers wonder what their role is now that everyone is testing. Some people say that Testing is WHAT we do and Agile is HOW we do it. In order to contribute effectively to quality solutions, agile testers need to combine the WHAT and HOW in their daily approach.
Time for a deep-dive. What defines the agile tester and how do we add value. In this workshop participant will fill ins an self-assessment based on the 12 characteristics of the agile tester. We’ll share the group results and create a snapshot of where we stand. Next, we will discuss each of the characteristics mean to us, how do they make agile testing work, help to boost the agility of the test process and how we can embed quality in the agile development process. In groups we will share examples from the trenches to go along with each of characteristics. What do we do to make it work, and what challenges do we encounter? Participants will help each other and be able to benchmark their own ideas.
This session aligns with the needs that I recognize with many colleagues. They are good testers, work successfully in an agile environment, but want to get better in explaining why they are a good tester. How does their attitude and approach contribute to valuable software solutions? Join this session if you want to improve yourself and want to get practical tips from the real world; If you want to learn what makes your testing agile and how is your agile mindset translated into a valuable testing approach. Participants can use the self-assessment to identify blind spots in their skills. Each of the participants will leave the room with a good understanding of where he/she stands: “I am an agile tester because…”
Michael Bolton - Two Futures of Software TestingTEST Huddle
EuroSTAR Software Testing Conference 2008 presentation on Two Futures of Software Testing by Michael Bolton. See more at conferences.eurostarsoftwaretesting.com/past-presentations/
To be most effective, test managers must develop and use metrics to help direct the testing effort and make informed recommendations about the software’s release readiness and associated risks. Because one important testing activity is to “measure” the quality of the software, test managers must measure the results of both the development and testing processes. Collecting, analyzing, and using metrics is complicated because many developers and testers are concerned that the metrics will be used against them. Join Rick Craig as he addresses common metrics—measures of product quality, defect removal efficiency, defect density, defect arrival rate, and testing status. Learn the guidelines for developing a test measurement program, rules of thumb for collecting data, and ways to avoid “metrics dysfunction.” Rick identifies several metrics paradigms and discusses the pros and cons of each. Delegates are urged to bring their metrics problems and issues for use as discussion points.
Jelle Calsbeek - Stay Agile with Model Based Testing revisedTEST Huddle
EuroSTAR Software Testing Conference 2009 presentation on Evolution of New Feature Verification in 3G Networks by Michael Monaghan. See more at conferences.eurostarsoftwaretesting.com/past-presentations/
EuroSTAR Software Testing Conference 2009 presentation on Spend Wisely, Test Well by John fodeh. See more at conferences.eurostarsoftwaretesting.com/past-presentations/
Real case studies of QA management in big teams (60-100 people). How to setup robust QA processes and approaches in them. Main impediments and problems, how to solve them. SAFe.
Bert Jagers - Preserving Our Future Through Customer SatisfactionTEST Huddle
EuroSTAR Software Testing Conference 2008 presentation on Preserving Our Future Through Customer Satisfaction by Bert Jagers. See more at conferences.eurostarsoftwaretesting.com/past-presentations/
Pragmatic Not Dogmatic TDD Agile2012 by Joseph Yoder and Rebecca Wirfs-BrockJoseph Yoder
This presentation challenges the "norm" for TDD. Testing should be an integral part of your daily programming practice. But you don’t always need to derive your code via many test-code-revise-retest cycles to be test-driven. Some find it more natural to outline a related set of tests first, and use those test scenarios to guide them as they write code. Once they’ve completed a “good enough” implementation that supports the test scenarios, they then write those tests and incrementally fix any bugs as they go. As long as you don’t write hundreds of lines of code without any testing, there isn’t a single best way to be Test Driven. There’s a lot to becoming proficient at TDD. Developing automated test suites, refactoring and reworking tests to eliminate duplication, and testing for exceptional conditions, are just a few. Additionally, acceptance tests, smoke tests, integration, performance and load tests support incremental development as well. If all this testing sounds like too much work, well…let’s be practical. Testing shouldn’t be done just for testing’s sake. Instead, the tests you write should give you leverage to confidently change and evolve your code base and validate the requirements of the system. That’s why it is important to know what to test, what not to test, and when to stop testing.
I am a agile tester, because...(masterclass at the Barcelona Test Academy)Derk-Jan de Grood
These are the slides of the masterclass I gave 24-1-2018 at the Barcelona Test Academy. Using a assessment we started a discussion about what it takes to be an agile tester.
Description: Testing has transformed into Agile Testing. Testing has become a responsibilty of the whole development team. Many testers wonder what their role is now that everyone is testing. Some people say that Testing is WHAT we do and Agile is HOW we do it. In order to contribute effectively to quality solutions, agile testers need to combine the WHAT and HOW in their daily approach.
Time for a deep-dive. What defines the agile tester and how do we add value. In this workshop participant will fill ins an self-assessment based on the 12 characteristics of the agile tester. We’ll share the group results and create a snapshot of where we stand. Next, we will discuss each of the characteristics mean to us, how do they make agile testing work, help to boost the agility of the test process and how we can embed quality in the agile development process. In groups we will share examples from the trenches to go along with each of characteristics. What do we do to make it work, and what challenges do we encounter? Participants will help each other and be able to benchmark their own ideas.
This session aligns with the needs that I recognize with many colleagues. They are good testers, work successfully in an agile environment, but want to get better in explaining why they are a good tester. How does their attitude and approach contribute to valuable software solutions? Join this session if you want to improve yourself and want to get practical tips from the real world; If you want to learn what makes your testing agile and how is your agile mindset translated into a valuable testing approach. Participants can use the self-assessment to identify blind spots in their skills. Each of the participants will leave the room with a good understanding of where he/she stands: “I am an agile tester because…”
Michael Bolton - Two Futures of Software TestingTEST Huddle
EuroSTAR Software Testing Conference 2008 presentation on Two Futures of Software Testing by Michael Bolton. See more at conferences.eurostarsoftwaretesting.com/past-presentations/
To be most effective, test managers must develop and use metrics to help direct the testing effort and make informed recommendations about the software’s release readiness and associated risks. Because one important testing activity is to “measure” the quality of the software, test managers must measure the results of both the development and testing processes. Collecting, analyzing, and using metrics is complicated because many developers and testers are concerned that the metrics will be used against them. Join Rick Craig as he addresses common metrics—measures of product quality, defect removal efficiency, defect density, defect arrival rate, and testing status. Learn the guidelines for developing a test measurement program, rules of thumb for collecting data, and ways to avoid “metrics dysfunction.” Rick identifies several metrics paradigms and discusses the pros and cons of each. Delegates are urged to bring their metrics problems and issues for use as discussion points.
Jelle Calsbeek - Stay Agile with Model Based Testing revisedTEST Huddle
EuroSTAR Software Testing Conference 2009 presentation on Evolution of New Feature Verification in 3G Networks by Michael Monaghan. See more at conferences.eurostarsoftwaretesting.com/past-presentations/
EuroSTAR Software Testing Conference 2009 presentation on Spend Wisely, Test Well by John fodeh. See more at conferences.eurostarsoftwaretesting.com/past-presentations/
Real case studies of QA management in big teams (60-100 people). How to setup robust QA processes and approaches in them. Main impediments and problems, how to solve them. SAFe.
Manual testing course in Chennai We are best Manual testing course center in Chennai
with 100 JOB Placements and we are listed in Top 10 Manual Testing training
institute in Chennai To know more about Manual testing course in Chennai Enroll Today
Htop Phone Number: 8489907817
Vist our
website: https://www.htopsolutions.com/manual-testing-training-in-chennai
Instagram: https://www.instagram.com/htop_solutions/
Linkedin: https://in.linkedin.com/company/htop-solutions
For more enquiry : https://forms.gle/LbXwyxcd9P96jbRB8
I recently did a talk on Agile and the importance of people over role/process obsessions. I didnt use this in the end, as I decided to adopt a more natural and example-based approach. Feel free to use however - the web is built on plagiarism after all ;)
parikshalabs.com provides the advance level services in software field like Software Testing Tools, Web Application Testing and also mobile Apps testing.
Agile development poses several challenges to effectively testing software. Many myths have become "common wisdom" about how testing is much more difficult, even impossible, in an agile environment. Aricent's software testing experts look at 7 of these myths, and based on their years of experience debunk them.
Foundations of Software Testing Lecture 4Iosif Itkin
This lecture is a part of the online course on Software Testing for Complex Intelligent Systems and Autonomous Vehicles. The course lectures provide the theoretical basics of testing autonomous systems based on artificial intelligence.
The fourth lecture of the course entitled Foundations of Software Testing reviews the ‘absence-of-errors fallacy’ and other principles of software testing, as well as the types and levels of software testing. The lecture also provides a fuller picture of the understanding of test objectives and methodologies by different schools of thought within the software testing domain.
QA Financial Forum London 2021 - Automation in Software Testing. Humans and C...Iosif Itkin
Speaker: Iosif Itkin, co-CEO & co-founder, Exactpro Systems
9th November 2021
Hilton Canary Wharf
Exactpro is an independent software testing business focused on mission-critical financial market infrastructures, primarily exchanges and clearing houses. In his presentation, Iosif will give a brief overview of research on the concept of model-based testing and the principal challenges of its application while testing complex distributed systems. He will also outline the broader context of interaction between humans and complex computer models.
Exactpro FinTech Webinar - Global Exchanges Test OraclesIosif Itkin
Global Exchanges series webinar to discuss Test Oracles. A test oracle is a mechanism for determining whether a test has passed or failed. The use of oracles involves comparing the output(s) of the system under test for a given test-case input, to the output(s) that the oracle determines the product should have. We will review various types of test oracles using examples from Exactpro’s Global Exchanges division projects and protocol-based interactions in trading systems.
Exactpro FinTech Webinar - Global Exchanges FIX ProtocolIosif Itkin
Exactpro’s Global Exchanges Division training session on FIX Trading Protocol.
The essence of the FIX protocol and its place in the overall structure of network protocols, FIX message attributes and the internal data types of the protocol.
Operational Resilience in Financial Market InfrastructuresIosif Itkin
A4Q World Congress 13-16 April 2021
Iosif Itkin
Exactpro provides independent software testing services for mission critical technology that underpins global financial markets – exchanges and clearing houses. Half of the top 20 global exchange groups on all continents around the globe rely on processes, platforms and people from Exactpro to improve their quality and reliability. The company has spent the last 11 years studying operational resilience in this crucial sector. The presentation will outline the key principles for software testing of the systems that process hundreds of millions of orders per day with roundtrip latencies below one hundred microseconds.
20 Simple Questions from Exactpro for Your Enjoyment This Holiday SeasonIosif Itkin
Warmest wishes for a happy holiday season and a wonderful New Year!
We look forward to our continued collaboration in 2020. Thank you for your support.
QA-Financial Forum 2019 in New York
13 November
Iosif Itkin, CEO and co-founder
Elena Treshcheva, Business Development Manager and Researcher
An October 2019 survey by BoE and FCA found that ML in financial organizations has already passed an initial development phase, and the usage of live ML applications is about to dramatically increase over the next three years. Artificial Intelligence systems are used in market surveillance, they are providing intellectual analysis of news feeds, and they are an important part of the conversational agents facing users and helping them with their business needs from identity verification to trading and portfolio management. How to ensure that an AI-powered system is up to its task? And what would that mean from the software testing perspective?
EXTENT 2019: Exactpro Quality Assurance for Financial Market InfrastructuresIosif Itkin
On Complex Software Systems Testing — Alexey Zverev, co-CEO & co-founder, Exactpro
Software Testing and Machine Learning
Mind the Gap. Applying Process Mining
Learning from Failure is not just for Humans
Dancing with Whales. Adaptive Log Classification System
On Traceability and the Illusion of Control
Building Partnerships
Demystifying DLT Testing One Network at a Time
Get the MOST from FIX
Georgia on My Mind
Build Software to Test Software — Iosif Itkin, co-CEO & co-founder, Exactpro
ClearTH Test Automation Framework: Case Study in IRS & CDS Swaps Lifecycle Mo...Iosif Itkin
Synchronize Europe
18th June 2019
Iosif Itkin, co-CEO and co-founder, Exactpro
Using the ISDA CDM Swaps application, simultaneously execute multiple end-to-end scenarios for DAML applications in capital markets - validate with actual contract data on ledger.
EXTENT Talks 2019 Tbilisi: Failover and Recovery Test Automation - Ivan ShamraiIosif Itkin
Ivan Shamray, Senior NFT Analyst, Exactpro
20 April 2019 EXTENT Talks, Tbilisi, Georgia
Tbilisi QA Community
EXTENT Talks is a meeting place for IT specialists working in various industries and seeking professional growth, practitioners from IT firms, as well as Quality Assurance enthusiasts of all backgrounds interested in actively participating in local IT events.
EXTENT Talks QA Community Tbilisi 20 April 2019 - Conference OpenIosif Itkin
EXTENT Talks is a meeting place for IT specialists working in various industries and seeking professional growth, practitioners from IT firms, as well as Quality Assurance enthusiasts of all backgrounds interested in actively participating in local IT events. The first EXTENT Talks were held in Tbilisi on 22 February 2019, initiating the creation a QA Community in Tbilisi and laying a foundation for an international platform for exchanging experience and knowledge in the field of software testing, development and IT. The program of the inaugural event included presentations on ISTQB, Software Testing, and Agile methodology from senior specialists. The next EXTENT Talks in Tbilisi will take place on 20 April 2019.
User-Assisted Log Analysis for Quality Control of Distributed Fintech Applica...Iosif Itkin
The First IEEE International Conference On Artificial Intelligence Testing (2019 IEEE AITest)
Iosif Itkin, Anna Gromova, Anton Sitnikov, Elena Treshcheva, Rostislav Yavorskiy, Evgenii Tsymbalov, Andrey Novikov and Kirill Rudakov
1 Exactpro, UK, Georgia, USA, Russia
2 Skolkovo Institute of Science and Technology, Russia
3 Higher School of Economics, Russia
Speakers: Iosif Itkin, CEO and Co-Founder and Elena Treshcheva, Business Development Manager and Researcher - Exactpro
Exactpro provides software testing services for mission-critical technology that underpins global financial markets. Exactpro clients are regulated by FCA, Bank of England and their counterparts from other countries. During this session, Elena and Iosif will talk about end-to-end software testing for post-trade systems in financial market infrastructures. What are the key challenges in quality assurance at this scale? What kind of cognitive biases affect SDLC? How precise is the knowledge about the systems under test? What constitutes good test evidence? How to deal with complexity in regulated environments?
Behaviour Driven Development: Oltre i limiti del possibileIosif Itkin
The QA Financial Forum: Milan 2019
23 January at the Excelsior Hotel Gallia.
Anna-Maria Lukina, Exactpro Business Development Director
The QA Financial Forum: Milan is one of the leading fintech conferences in Italy. The event focuses on the latest achievements in software risk management and automation of software testing. The predominant theme of the Milan event will be Quality Assurance for the entire Software Development Life Cycle (SDLC).
The topics under discussion will feature:
- Technologies for Automation & AI
- DevOps & CI/CD
- Value Stream Management
- Test Data Management
- Regulatory Compliance
- App Security & DevSecOps
- Testing and quality assurance of Blockchain platforms
The official language of the event is Italian.
On 17th January 2018 Exactpro successfully completed a management buyout from London Stock Exchange Group (LSEG), signed a new multi-year master services agreement with LSEG, and opened its head office in London.
What else has happened in 2018?
I wanted to take the opportunity to reflect on what has been an unusual year for Exactpro.
Integration front to back - Mr. Custodian tear down that wall
The scope of the application level has been continuous extended over the years, albeit with a focus on the area of pre-trade and trade.
Recently, there has been an increased interest to move further into the area of post-trade which is predominantly driven by the ISO 20022 standard. Is there really a need for new FIX messages in areas such as payments and
what are the integration problems needing a resolution?
Panellists
- Iosif Itkin, CEO, Exactpro
- Jim Northey, Co-Chair Global Technical Committee, Americas Region, FIX Trading Community, Chair Elect, ISO TC68 Financial
Services Technical Committee, and Consultant and Industry Standards Liaison, Itiviti
- Barry Young, Director, Aladdin Product Manager, BlackRock
BDD. The Outer Limits. Iosif Itkin at Youcon (in Russian)Iosif Itkin
Exactpro is supporting the 3rd annual IT-conference YouCon to take place on 14th October in Saratov, Russia. Over 900 programmers, systems engineers and architects, software QA engineers, and marketing specialists will gather to discuss the latest trends in programming technology. It is the largest IT industry event in Saratov.
Iosif Itkin, CEO of Exactpro, part of London Stock Exchange Group, will deliver a "BDD. The Outer Limits" presentation named after Iosif's favorite Sci-Fi series.
The topics to be covered are:
Behavior Driven Development concepts
Applying BDD in trading and clearing systems
Specification by Example and using production data
Combining Model-based testing and BDD
The Outer Limits
There will be an opportunity to ask questions, share thoughts and expertise in BDD, or just chat with a representative at the Exactpro stand at any time during the event.
Don't miss out, stop by and ask how you can get your Exactpro souvenir :)
We look forward to meeting you there!
#Exactpro #Youconsaratov
BDD. The Outer Limits. Iosif Itkin at Youcon (in Russian)
How a Great QA Team can make a Disproportionate Contribution to Project Success
1. ‘How a Great QA Team can make
a Disproportionate Contribution to Project
Success’
Paper Presented by Mark Ryland
2. Presentation Contents
1. Introduction
2. Project Lifecycle & Team Constituents
3. Team Dynamics
4. Typical QA
5. Great QA
3. Mark Ryland’s Credentials
• Close on 20 years large project delivery experience in Wholesale Financial Services
• Banks, exchanges and brokers
• Variety of systems types
• Projects from $10M to $150M
• All of these projects required QA testing
• Involved in building 2 offshore testing centres
• Schooled in QA testing best practise
4. Example Scope From Building
QA Centre
‘Location QA’ - ‘end to end’ testing with location
specific downstream systems
On-shore user acceptance testing ( executed by user
representatives)
Test execution ( system, regression, new functionality)
Off-shore
Maintenance of regression test cases
Test automation development
l oo Tt net si s no C
e T det a mt u A
Tt net si s no C
na Ml e W
o
na Ml e W
no t uce x E
l
l
i
5. Project Team v Lifecycle
Conception Initiation Development Test Implement
Project Sponsor
Project Manager
Business Analyst
Architect/Designer
Development Team
QA Team
Production Team
8. ‘Typical/Normal’ QA Experience
1. Doesn’t fully understand project objectives
2. Lacks detailed domain knowledge
3. Consumes lots of management & BA time
4. Is a bottleneck - test execution cycles take too long
5. Not flexible in dealing with unplanned change
6. Test coverage and quality is not good enough
7. QA team measure their success on how many test cases were executed (as
opposed to the value of them)
9. Attributes of a Great QA Team
1. The basics
2. Domain experts
3. Works out for itself what needs to be done
4. High calibre, motivated staff, stable team
5. Intelligent use of tools to speed up test execution/improve
6. Only raise genuine defects
7. A peer to other parts of the project team
10. Attributes of a Great QA Team
Cont….
8. Energy and diligence
9. Flexible orientation of test effort
10. Responsive to changes in project priorities
11. Disciplined organisation of the test environment
12. Creative
13. Independent & Objective
Adds Up to Making a ‘Real’ Contribution to Quality
& Hence Project Success
11. Measuring QA Team
Effectiveness
1. High
Quality Staff
9. Bespoke
Automation 2. Domain
Tools Available Knowledge
8. Detailed
Management of 3. Level of
Test Environment Energy
Enthusiasm &
Proactivity
7. Test Case
Quality 4. Planning &
Delivering Ability
6. Test 5. Creativity &
Process Flexibility in Problem
Effectiveness Solving
12. Independence & Objectivity
1. Understanding of Requirement
End
Business Manager Business Analyst Designer Devel –oper
Users
2. Delivery of Solution
3A. Independent
Measurement by
QA Team
3B. Independent Measurement by High Quality QA Team
13. Team Dynamics When the QA
Team is Excellent
1. Developers try harder to get it right in the first place
2. Developers get rapid/timely feedback on what’s wrong
3. Developers are work harder to solve defects to keep the project on track
4. Project sponsors get clear visibility on quality level and progress
5. Project designers and analysts are have clear problem statements to work from
6. Have (justified) confidence in the system – that will feed through to the users
7. Fewer problems are going to found in live & hence the project will cost less
Talk Synopsis ‘ How a great QA Team can make a Disproportionate Contribution to Project Success’ Intro Explore project team dynamics – how the different parts of the team work together – focussing on the contribution that QA can make Have in mind financial services electronic trading platforms
Describe sections
Close on 20 years large project delivery experience in Wholesale Financial Services Companies include: UBS, Credit Suisse, Bank of America Merrill Lynch (BAML), HSBC, Swiss Stock Exchange, London Stock Exchange Group, Newedge Systems Include: Sell side order and execution management, market access, position keeping & risk, matching engines, messaging infrastructures, post trade Projects from $10M to $150M All of these projects required QA testing Along the way was involved in building 2 offshore testing centres from scratch one at BAML (though Tata CS) the other at HSBC (Captive) Schooled in QA testing best practise
All large projects that are realised are the result of a team effort Never seen a successful project that did not rely on stong contributions from several parts of the project team Challenging projects call out something extra and it’s usual to see some individuals make outstanding contributions to the whole Observing and being part of that is one of the most rewarding aspects of large project work = buzz, camaraderie, satisfaction of doing an excellent job
Proposition for the Talk Thesis is that certain parts of the project can provide an exceptional contribution that raises the game of the others Look for image of a race with a pace maker The pace maker carries everyone else forward, others need to make an effort to keep up otherwise they can fall behind In the analogy the paced could come from any part of the project team You might expect it to come from the sponsor or the development manager of project manager or maybe the architect or lead analyst – in fact in a well performing team its likely to come from more that one place, but my thesis is that one of the places it can come from if the QA team Almost no-one would think it can come from there
QA Team Place within the Overall Team The general norm in my experience in that the QA team doesn’t make an outstanding contribution to success Typically the QA team is lagging;- doesn’t fully understand the project objectives, lacks detailed understanding, is a bottleneck in that test execution cycles take too long and become the critical path on the project, is not flexible enough to deal with unplanned changes, consumes lots of management time needing to be ‘spoon fed’ and there is a nagging feeling that despite best efforts the test coverage and quality is not good enough, QA team measure their own success on how many test cases were executed (as opposed to the value of them) - often the most frustrating aspect of project delivery This is ‘normal’ QA Results from; inexperience, poor domain knowledge low calibre staff, poor staff retention, industry standard test tools, ‘T&M’ business model, buying based on day rate
It Doesn’t Have to Be Like That Recent experience much better The basics: Clear planning, risk based testing, well organised test case library, test execution management tool, defect reporting database, test data management Domain experts already e.g. FIX, exchanges characteristics of products, attributes of similar preceding platforms Works out for itself what needs to be done & proposed that to the other members of the project team High calibre, motivated staff, stable team Intelligent use of tools to speed up test execution/improve coverage & has invested in a bank of specialist tools for the specific scenario Only raise genuine defects and communicate those clearly Be a peer to other parts of the project team
Apply energy and diligence Flexible orientation of test effort to different parts of the platform as needed Responsive to changes in project priorities – either externally driven or as a result of problems being found Disciplined organisation of the test environment Creative – comes up with suggestions and solutions to move the project forward – these may be outside of the domain where one would normally expect the QA team to operate Independence & Objectivity
Independence and Objectivity Subject to human interpretation & relationships - someone may say something is fine or not because they like or are influenced by the presentation Some project team members may ‘sell’ the results of the work, be arrogant or fail to understand what is really needed Benefits in independence of QA Gives a ‘Measurement’ on status of delivery which is quite distinct from how anybody might ‘spin’ the progress A great QA team can bring insight beyond the brief and specs they have been given and pick up things that have been missed by others The most useful feedback is objective, but that’s not easy; Challenges of objectivity Really need to understand the detail of how is supposed to work Defects articulated clearly with necessary supporting evidence Communicate based on facts not supposition Reserve judgement Consistency All of the above leads to earning respect
Developers are going to try harder to get it right in the first place Developers are going to get rapid/timely feedback on what’s wrong Developers are going to work harder to solve defects to keep the project on track Project sponsors are going to have clear visibility or quality level and progress Project designers and analysts are going to have clear problem statements to work from When the system goes live the project team is going to have (justified) confidence in the system – that will feed through to the users Fewer problems are going to found in live (where they are much more expensive to fix) & hence the project will cost less
Conclusion A great QA Team can make a Disproportionate Contribution to Project Success