- Mike Smith argues that software testing models often rely too heavily on test cases, which may not provide the best measures for control and risk management.
- An effective measurement framework separates objectives from initiatives and uses a complex model of relationships rather than a simple hierarchy. This provides better traceability and the ability to cope with change.
- Lessons can be learned across different domains of measurement and testing. An ideal testing model would incorporate concepts from performance management systems like the balanced scorecard to link testing to business outcomes.
- Many factors influence what level of measures and targets are suitable for a given situation, but the most important thing is that the model supports analysis and decision making to maintain control.
This document discusses acceptance testing, which is formal testing conducted by end users to determine if a system meets requirements and business processes before it is accepted. The document outlines what acceptance testing is, different types including user acceptance testing and operational acceptance testing, common application areas, how it fits into software development lifecycles, challenges, and guidelines for success. It also briefly discusses outsourcing acceptance testing.
Agile Implementation Challenges – Testing and more…SQALab
This document discusses the challenges of implementing agile methodologies like Scrum, particularly as they relate to testing. It notes that companies are increasingly adopting agile approaches, which require teams to work differently compared to traditional methods. Some challenges mentioned include adapting to new roles, processes, and culture; ensuring discipline and quality with fast iterations; learning new skills like programming; and managing high expectations from executives. The summary at the end emphasizes that overcoming these challenges requires teams to adopt, learn, and adapt to change.
This document outlines the process of building a new software testing team from scratch within a short timeframe. Key steps included defining team roles and hiring internal resources, setting up infrastructure and providing extensive training on testing processes, tools, and the software architecture. Training occurred both through classroom sessions and shadowing existing testing teams. The team focused on writing test cases, reviewing each other's work, and learning through hands-on testing and feedback. After 5 weeks of preparation, the new team was able to successfully test and go live with the new software on time and on budget, though quality could still be improved. Management support and extensive training of new testers were essential to the team's success.
14 lessons for successful testing outsourcingSQALab
This document provides 14 lessons for successfully outsourcing software testing based on real examples from Europe and Asia. It begins with an introduction to outsourcing testing and the differences between nearshoring and offshoring. The main lessons include deciding why to outsource, communication, training staff, logistics, measuring quality and processes, and automating where possible. The overall message is that outsourcing requires investment, control, and treating offshore resources well in order to be successful.
The document discusses a new model for testing that focuses on exploration of knowledge sources to build test models that inform testing. It outlines three patterns of software development (structured, agile, continuous) and argues testing involves exploring knowledge sources and building test models, with all testing being exploratory in nature. A new test process is proposed involving exploration support tools that capture testing plans and activity in real-time. The roles of developers and testers may become blurred in the future under this new model.
This document discusses effective testing in Agile environments. It introduces the Agile Testing Quadrants model which divides testing into four categories: technology-facing tests that support the team (unit and integration testing), business-facing tests that support the team (acceptance and functional testing), business-facing tests that critique the product (usability and exploratory testing), and technology-facing tests that critique the product (performance and security testing). It provides examples of tools that can be used for each testing quadrant and recommends practicing test automation, exploratory testing, and only documenting what is useful.
The document provides guidance for managing a team of junior testers. It discusses challenges such as lack of skills and experience in junior testers. It recommends setting clear expectations, providing frequent communication and feedback, ensuring knowledge sharing, and protecting the team to help them succeed. Patience and structure are important, as is repeating key messages, to help junior testers learn and improve. The goal is for the team to work cooperatively toward a common objective.
The document discusses 10 signs that an organization's software testing may not be enough. These include having excessive production bugs, bugs found during user acceptance testing, growing bug counts over test cycles, not investing in testing compared to competitors, lacking clear criteria for what constitutes "enough" testing, testers advising against releasing software, weak prevention efforts like code reviews, lack of developer unit testing, frequently reduced testing periods causing deadline problems, and high tester turnover. The document advocates treating testing as risk management, increasing test reuse and automation, and addresses common challenges and questions around software testing.
This document discusses acceptance testing, which is formal testing conducted by end users to determine if a system meets requirements and business processes before it is accepted. The document outlines what acceptance testing is, different types including user acceptance testing and operational acceptance testing, common application areas, how it fits into software development lifecycles, challenges, and guidelines for success. It also briefly discusses outsourcing acceptance testing.
Agile Implementation Challenges – Testing and more…SQALab
This document discusses the challenges of implementing agile methodologies like Scrum, particularly as they relate to testing. It notes that companies are increasingly adopting agile approaches, which require teams to work differently compared to traditional methods. Some challenges mentioned include adapting to new roles, processes, and culture; ensuring discipline and quality with fast iterations; learning new skills like programming; and managing high expectations from executives. The summary at the end emphasizes that overcoming these challenges requires teams to adopt, learn, and adapt to change.
This document outlines the process of building a new software testing team from scratch within a short timeframe. Key steps included defining team roles and hiring internal resources, setting up infrastructure and providing extensive training on testing processes, tools, and the software architecture. Training occurred both through classroom sessions and shadowing existing testing teams. The team focused on writing test cases, reviewing each other's work, and learning through hands-on testing and feedback. After 5 weeks of preparation, the new team was able to successfully test and go live with the new software on time and on budget, though quality could still be improved. Management support and extensive training of new testers were essential to the team's success.
14 lessons for successful testing outsourcingSQALab
This document provides 14 lessons for successfully outsourcing software testing based on real examples from Europe and Asia. It begins with an introduction to outsourcing testing and the differences between nearshoring and offshoring. The main lessons include deciding why to outsource, communication, training staff, logistics, measuring quality and processes, and automating where possible. The overall message is that outsourcing requires investment, control, and treating offshore resources well in order to be successful.
The document discusses a new model for testing that focuses on exploration of knowledge sources to build test models that inform testing. It outlines three patterns of software development (structured, agile, continuous) and argues testing involves exploring knowledge sources and building test models, with all testing being exploratory in nature. A new test process is proposed involving exploration support tools that capture testing plans and activity in real-time. The roles of developers and testers may become blurred in the future under this new model.
This document discusses effective testing in Agile environments. It introduces the Agile Testing Quadrants model which divides testing into four categories: technology-facing tests that support the team (unit and integration testing), business-facing tests that support the team (acceptance and functional testing), business-facing tests that critique the product (usability and exploratory testing), and technology-facing tests that critique the product (performance and security testing). It provides examples of tools that can be used for each testing quadrant and recommends practicing test automation, exploratory testing, and only documenting what is useful.
The document provides guidance for managing a team of junior testers. It discusses challenges such as lack of skills and experience in junior testers. It recommends setting clear expectations, providing frequent communication and feedback, ensuring knowledge sharing, and protecting the team to help them succeed. Patience and structure are important, as is repeating key messages, to help junior testers learn and improve. The goal is for the team to work cooperatively toward a common objective.
The document discusses 10 signs that an organization's software testing may not be enough. These include having excessive production bugs, bugs found during user acceptance testing, growing bug counts over test cycles, not investing in testing compared to competitors, lacking clear criteria for what constitutes "enough" testing, testers advising against releasing software, weak prevention efforts like code reviews, lack of developer unit testing, frequently reduced testing periods causing deadline problems, and high tester turnover. The document advocates treating testing as risk management, increasing test reuse and automation, and addresses common challenges and questions around software testing.
This document discusses why checklists are better than test cases for documentation in quality assurance. It argues that test cases become overcrowded and focus too much on documentation rather than core functions. Checklists are more time-saving and easy to update. An example compares a test case to a checklist for login/registration flows. The author's company Hipo uses a test pad and robot framework integrated with checklists to share with clients and team members.
New Model Testing: A New Test Process and ToolTEST Huddle
Paul Gerrard presented a new test process and tool called Cervaya that combines elements of structured and exploratory testing. The process involves testers surveying features using Cervaya to iteratively build system models and test plans. This shifts testing earlier in the development process. Cervaya logs tester activity, supports real-time collaboration, and could generate documentation. The goal is to make testing more aligned with agile and continuous delivery approaches. Gerrard invited collaboration on further developing Cervaya.
Growing a Company Test Community: Roles and Paths for TestersTEST Huddle
Over the past three years, our company’s test team has grown from three lonesome testers to a community of nine – with more planned. Since we don’t see testers as “click monkeys”, but as valuable and integrated project members who bring a specific skill set to the table, it’s important for us to choose testers well and to train them in various areas so that they can contribute, grow and see their own career path within testing.
To structure to our internal tester training program, we have been developing role descriptions, education paths and career options for our testers, which I’d like to share with you in this webinar.
View webinar - https://huddle.eurostarsoftwaretesting.com/resource/webinar/growing-company-test-community-roles-paths-testers/
I believe that our existing models of testing are not fit for purpose – they are inconsistent, controversial, partial, proprietary and stuck in the past. They are not going to support us in the rapidly emerging technologies and approaches. The certification schemes that should represent the interests and integrity of our profession don’t, and we are left with schemes that are popular, but have low value, lower esteem and attract harsh criticism. My goal in proposing the New Model is to stimulate new thinking in this area.
eurostarconferences.com
testhuddle.com
This document provides an overview of approaches to achieving zero defects in software development. It discusses how adopting a "zero defects" attitude can reduce defects by 50% overnight. It emphasizes defect prevention through proper root cause analysis and design reviews over testing. Iterative design, implementation, and review cycles are recommended to find and address issues early. The goal of quality assurance is to help developers produce fewer defects by focusing on prevention rather than finding defects after the fact. The ultimate goal is to deliver working software to customers with "no questions and no issues."
'The Real Agile Testing Quadrants' with Michael BoltonTEST Huddle
EuroSTAR Conferences, with the support of ISA Software Skillnet, Irish Software Innovation Network and SoftTest, were delighted to bring you a half-day software testing masterclass with Michael Bolton
In this session, Michael Bolton (who has extensive experience as a tester, as a programmer, and as a project manager) explained the role of skilled software testers, and why you might not want to think of testing as "quality assurance".
He present ideas about the relationship between management and testers, and about the service that testers really provide: making quality assurance possible by lighting the way of the project. For those of you who who attended this event, we really hope it was of use to you in your testing careers.
www.eurostarconferences.com
This document discusses testing approaches on agile projects. It states that testing on agile projects focuses on finding and preventing problems through techniques like feedback and risk assessment. Testing is integrated into the development process and strips away extras like separate test documents, scripts, and tools. The key is for testing to adapt to changes in risk, skills, architecture and timescales as the process changes, and to involve everyone by making testing a part of development.
The document provides an overview of agile testing principles and practices. It discusses that agile testing involves the entire cross-functional team working together to test software iteratively. Key aspects of agile testing covered include continuous feedback, delivering value to customers, enabling face-to-face communication, and keeping testing simple. The document also outlines typical testing activities in an agile project such as test planning, driving development, facilitating communication, and completing testing tasks within each sprint.
This document discusses the need for leadership in the testing community to drive innovation and change. It provides examples of challenges facing testers at different companies and how they are addressing them through approaches like shifting testing left into development, adopting agile practices, and using analytics. It argues that testing is no longer just an end phase but must be integrated into continuous delivery. For change to happen, testers will need to embrace new approaches, challenge old ways of thinking, and stand up as leaders to define the future of testing.
A Common Sense Guide to Agile Development and Testing that might just change your Agile approach forever.
Answering the 9 most common questions asked about Agile Testing:
- What is Agile Testing?
- Do we still need testers in Agile?
- What is an Agile Tester?
- What does a Software Tester Actually Do?
- Should we automate our testing?
- What tools should we use for our Agile Testing?
- How Much Should we Automate?
- How can we automate and still finish the sprint?
- How can we finish all our testing in the sprint?
A high quality download of the 9 points as a free "Print out and Keep" Poster is available at http://eviltester.com/agile
dbg Agile Testing Presentation, demonstrating the use of Test Charters, Exploratory Testing, Session Based Testing and Testing Tours. With thanks to James Bach, Lisa Crispin, Janet Gregory and James Whittaker
Scrum_BLR 11th meet up 13 dec-2014 - SDET - They Way to go for Testers - Jaya...Scrum Bangalore
The document discusses the evolving role of software testers and the benefits of testers becoming software development engineers in test (SDETs). It argues that testers should expand their skills from just testing to also include activities like programming, test automation, and code reviews. This will allow testers to find defects earlier and help reduce test cycle times. It outlines a path for testers to transition to SDETs by improving their technical skills and provides examples of how SDETs have contributed value in various organizations.
Ho Chi Minh City Software Testing Conference January 2015
Software Testing in the Agile World
Website: www.hcmc-stc.org
Author: Nhat Do, Vu Duong
Context-Driven Testing (CDT) rejects the notion of generalized “best practices” that apply to all projects, and instead accepts that different practices work best under different circumstances. The third principle of the seven defined in CDT states that people are the most important part of any project’s context. Less of a focus on processes and tools, with more emphasis on people and their collaboration empowers testers with the freedom to make choices about how best to do their job without following a restrictive plan.
In joining the game of workshop and some theory sharing in slides, you will a better understanding of Context-Driven Testing practices, principles and its benefits as well as know how is a nice Marriage of Agile and Context-Driven Testing.
You want to integrate skilled testing and development work. But how do you accomplish this without developers accidentally subverting the testing process or testers becoming an obstruction? Efficient, deep testing requires “critical distance” from the development process, commitment and planning to build a testable product, dedication to uncovering the truth, responsiveness among team members, and often a skill set that developers alone—or testers alone—do not ordinarily possess. James Bach presents a model—a redesign of the famous Agile Testing Quadrants that distinguished between business vs. technical facing tests and supporting vs. critiquing―that frames these dynamics and helps teams think through the nature of development and testing roles and how they might blend, conflict, or support each other on an Agile project. James includes a brief discussion of the original Agile Testing Quadrants model, which the presenters believe has created much confusion about the role of testing in Agile.
The document discusses Daniel Ricardo de Amorim and his role as an Agile QA Consultant. It explores how agile testers work as part of development teams rather than as separate roles. It sorts QA profiles into three dimensions: business, technical, and DevOps. The business dimension involves acceptance testing and BDD. The technical dimension covers programming skills, TDD, test automation, and security and performance testing. The DevOps dimension focuses on continuous delivery and automating repeatable tasks. All QAs keep the team focused on delivery, product quality, test ownership, and wear many hats to infect the team with quality practices.
Practical Test Strategy Using HeuristicsTEST Huddle
Key Takeaways
- See what makes a good test strategy
- Learn how to make a thorough test strategy
- Identify what is the ‘Heuristic Test Strategy Model’ is
- Develop a solid test strategy that fits fast
- Discover how diversification can help you to create a test strategy
1. Perficient is a leading IT consulting firm with over $250 million in annual revenues, 1,400+ consultants, and locations across North America and China.
2. The presenter, Mary Jiang, is a test lead at Perficient China with over 8 years of experience in software development, testing, and Agile methodologies.
3. Traditional software development often fails to meet budget, timeline, and customer expectations. Agile methods emphasize frequent delivery of working software, customer collaboration, and responding to change.
Tafline Murnane - The Carrot or The Whip-What Motivates Testers? - EuroSTAR 2010TEST Huddle
EuroSTAR Software Testing Conference 2010 presentation on The Carrot or The Whip-What Motivates Testers? by Tafline Murnane. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/
This document discusses career paths for testing engineers. It begins by describing a typical interview where the candidate's lack of technical skills is apparent. It then discusses a tester's concerns about their career progression and perceptions of testers. The document outlines stories of individuals who grew their careers in testing over long periods of time, taking on roles such as test automation engineer and testing director. It provides advice on making good career choices by gaining experience in one's current role and waiting for opportunities, rather than changing roles frequently or due to money alone. Specific career paths are suggested such as build master, release engineer, and testing management. The document emphasizes that experience over time strengthens one's position and makes them competitive for career growth.
This document discusses why checklists are better than test cases for documentation in quality assurance. It argues that test cases become overcrowded and focus too much on documentation rather than core functions. Checklists are more time-saving and easy to update. An example compares a test case to a checklist for login/registration flows. The author's company Hipo uses a test pad and robot framework integrated with checklists to share with clients and team members.
New Model Testing: A New Test Process and ToolTEST Huddle
Paul Gerrard presented a new test process and tool called Cervaya that combines elements of structured and exploratory testing. The process involves testers surveying features using Cervaya to iteratively build system models and test plans. This shifts testing earlier in the development process. Cervaya logs tester activity, supports real-time collaboration, and could generate documentation. The goal is to make testing more aligned with agile and continuous delivery approaches. Gerrard invited collaboration on further developing Cervaya.
Growing a Company Test Community: Roles and Paths for TestersTEST Huddle
Over the past three years, our company’s test team has grown from three lonesome testers to a community of nine – with more planned. Since we don’t see testers as “click monkeys”, but as valuable and integrated project members who bring a specific skill set to the table, it’s important for us to choose testers well and to train them in various areas so that they can contribute, grow and see their own career path within testing.
To structure to our internal tester training program, we have been developing role descriptions, education paths and career options for our testers, which I’d like to share with you in this webinar.
View webinar - https://huddle.eurostarsoftwaretesting.com/resource/webinar/growing-company-test-community-roles-paths-testers/
I believe that our existing models of testing are not fit for purpose – they are inconsistent, controversial, partial, proprietary and stuck in the past. They are not going to support us in the rapidly emerging technologies and approaches. The certification schemes that should represent the interests and integrity of our profession don’t, and we are left with schemes that are popular, but have low value, lower esteem and attract harsh criticism. My goal in proposing the New Model is to stimulate new thinking in this area.
eurostarconferences.com
testhuddle.com
This document provides an overview of approaches to achieving zero defects in software development. It discusses how adopting a "zero defects" attitude can reduce defects by 50% overnight. It emphasizes defect prevention through proper root cause analysis and design reviews over testing. Iterative design, implementation, and review cycles are recommended to find and address issues early. The goal of quality assurance is to help developers produce fewer defects by focusing on prevention rather than finding defects after the fact. The ultimate goal is to deliver working software to customers with "no questions and no issues."
'The Real Agile Testing Quadrants' with Michael BoltonTEST Huddle
EuroSTAR Conferences, with the support of ISA Software Skillnet, Irish Software Innovation Network and SoftTest, were delighted to bring you a half-day software testing masterclass with Michael Bolton
In this session, Michael Bolton (who has extensive experience as a tester, as a programmer, and as a project manager) explained the role of skilled software testers, and why you might not want to think of testing as "quality assurance".
He present ideas about the relationship between management and testers, and about the service that testers really provide: making quality assurance possible by lighting the way of the project. For those of you who who attended this event, we really hope it was of use to you in your testing careers.
www.eurostarconferences.com
This document discusses testing approaches on agile projects. It states that testing on agile projects focuses on finding and preventing problems through techniques like feedback and risk assessment. Testing is integrated into the development process and strips away extras like separate test documents, scripts, and tools. The key is for testing to adapt to changes in risk, skills, architecture and timescales as the process changes, and to involve everyone by making testing a part of development.
The document provides an overview of agile testing principles and practices. It discusses that agile testing involves the entire cross-functional team working together to test software iteratively. Key aspects of agile testing covered include continuous feedback, delivering value to customers, enabling face-to-face communication, and keeping testing simple. The document also outlines typical testing activities in an agile project such as test planning, driving development, facilitating communication, and completing testing tasks within each sprint.
This document discusses the need for leadership in the testing community to drive innovation and change. It provides examples of challenges facing testers at different companies and how they are addressing them through approaches like shifting testing left into development, adopting agile practices, and using analytics. It argues that testing is no longer just an end phase but must be integrated into continuous delivery. For change to happen, testers will need to embrace new approaches, challenge old ways of thinking, and stand up as leaders to define the future of testing.
A Common Sense Guide to Agile Development and Testing that might just change your Agile approach forever.
Answering the 9 most common questions asked about Agile Testing:
- What is Agile Testing?
- Do we still need testers in Agile?
- What is an Agile Tester?
- What does a Software Tester Actually Do?
- Should we automate our testing?
- What tools should we use for our Agile Testing?
- How Much Should we Automate?
- How can we automate and still finish the sprint?
- How can we finish all our testing in the sprint?
A high quality download of the 9 points as a free "Print out and Keep" Poster is available at http://eviltester.com/agile
dbg Agile Testing Presentation, demonstrating the use of Test Charters, Exploratory Testing, Session Based Testing and Testing Tours. With thanks to James Bach, Lisa Crispin, Janet Gregory and James Whittaker
Scrum_BLR 11th meet up 13 dec-2014 - SDET - They Way to go for Testers - Jaya...Scrum Bangalore
The document discusses the evolving role of software testers and the benefits of testers becoming software development engineers in test (SDETs). It argues that testers should expand their skills from just testing to also include activities like programming, test automation, and code reviews. This will allow testers to find defects earlier and help reduce test cycle times. It outlines a path for testers to transition to SDETs by improving their technical skills and provides examples of how SDETs have contributed value in various organizations.
Ho Chi Minh City Software Testing Conference January 2015
Software Testing in the Agile World
Website: www.hcmc-stc.org
Author: Nhat Do, Vu Duong
Context-Driven Testing (CDT) rejects the notion of generalized “best practices” that apply to all projects, and instead accepts that different practices work best under different circumstances. The third principle of the seven defined in CDT states that people are the most important part of any project’s context. Less of a focus on processes and tools, with more emphasis on people and their collaboration empowers testers with the freedom to make choices about how best to do their job without following a restrictive plan.
In joining the game of workshop and some theory sharing in slides, you will a better understanding of Context-Driven Testing practices, principles and its benefits as well as know how is a nice Marriage of Agile and Context-Driven Testing.
You want to integrate skilled testing and development work. But how do you accomplish this without developers accidentally subverting the testing process or testers becoming an obstruction? Efficient, deep testing requires “critical distance” from the development process, commitment and planning to build a testable product, dedication to uncovering the truth, responsiveness among team members, and often a skill set that developers alone—or testers alone—do not ordinarily possess. James Bach presents a model—a redesign of the famous Agile Testing Quadrants that distinguished between business vs. technical facing tests and supporting vs. critiquing―that frames these dynamics and helps teams think through the nature of development and testing roles and how they might blend, conflict, or support each other on an Agile project. James includes a brief discussion of the original Agile Testing Quadrants model, which the presenters believe has created much confusion about the role of testing in Agile.
The document discusses Daniel Ricardo de Amorim and his role as an Agile QA Consultant. It explores how agile testers work as part of development teams rather than as separate roles. It sorts QA profiles into three dimensions: business, technical, and DevOps. The business dimension involves acceptance testing and BDD. The technical dimension covers programming skills, TDD, test automation, and security and performance testing. The DevOps dimension focuses on continuous delivery and automating repeatable tasks. All QAs keep the team focused on delivery, product quality, test ownership, and wear many hats to infect the team with quality practices.
Practical Test Strategy Using HeuristicsTEST Huddle
Key Takeaways
- See what makes a good test strategy
- Learn how to make a thorough test strategy
- Identify what is the ‘Heuristic Test Strategy Model’ is
- Develop a solid test strategy that fits fast
- Discover how diversification can help you to create a test strategy
1. Perficient is a leading IT consulting firm with over $250 million in annual revenues, 1,400+ consultants, and locations across North America and China.
2. The presenter, Mary Jiang, is a test lead at Perficient China with over 8 years of experience in software development, testing, and Agile methodologies.
3. Traditional software development often fails to meet budget, timeline, and customer expectations. Agile methods emphasize frequent delivery of working software, customer collaboration, and responding to change.
Tafline Murnane - The Carrot or The Whip-What Motivates Testers? - EuroSTAR 2010TEST Huddle
EuroSTAR Software Testing Conference 2010 presentation on The Carrot or The Whip-What Motivates Testers? by Tafline Murnane. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/
This document discusses career paths for testing engineers. It begins by describing a typical interview where the candidate's lack of technical skills is apparent. It then discusses a tester's concerns about their career progression and perceptions of testers. The document outlines stories of individuals who grew their careers in testing over long periods of time, taking on roles such as test automation engineer and testing director. It provides advice on making good career choices by gaining experience in one's current role and waiting for opportunities, rather than changing roles frequently or due to money alone. Specific career paths are suggested such as build master, release engineer, and testing management. The document emphasizes that experience over time strengthens one's position and makes them competitive for career growth.
Software Quality Metrics for Testers - StarWest 2013XBOSoft
Presentation by Phil Lew at StarWest 2013.
When implementing software quality metrics, we need to first understand the purpose of the metrics and who will be using them. Will the metric be used to measure people or the process, to illustrate the level of quality in software products, or to drive toward a specific objective? QA managers typically want to deliver productivity metrics to management but management may want to see metrics that describe customer or user satisfaction. Philip Lew believes that software quality metrics without actionable objectives toward increasing customer satisfaction are a waste of time. Learn how to connect each metric with potential actions based on evaluating the metric. Metrics for the sake of information may be helpful but often just end up in spreadsheets of interest to no one. Take home methods to identify metrics that support actionable objectives. Once the metrics and their objectives have been established, learn how to define and use metrics for real improvement.
The document discusses evaluating training programs through various methods. It outlines the goals of evaluation, such as assessing progress, evaluating curriculum and staff, and justifying expenditures. Common myths about evaluation are debunked, such as the ideas that results cannot be measured or that evaluation will lead to criticism. Effective evaluation requires collecting data at different stages using instruments like questionnaires, observations, interviews and performance reviews. The document provides guidance on developing these instruments and conducting evaluations to improve training programs.
This document outlines an instructor-led presentation on software quality metrics. It introduces the instructor and his relevant experience. It then provides information from attendee feedback on previous sessions, including both positive and negative comments. The presentation agenda is then outlined, with topics like how measurement can help organizations understand, evaluate, and improve their processes. Group exercises are included to discuss defining software quality and examples of metrics used in real life. The presentation also covers best practices for using metrics, different models of software quality, and measuring quality within agile development.
Training needs analysis, skills auditing, training evaluation, calculating training ROI and strategic learning and development best practice principles and processes
The world has changed dramatically since LEAN and Six Sigma were popularized in the early 1990′s. Globalization, product proliferation, information technology, intense competition, and an activist regulatory environment have contributed to a rapid rise in complexity. As a result, many companies are finding that LEAN and Six Sigma aren’t delivering the results they expected. In this presentation, delivered by Chris Seifert at APICS 2013, we discuss a new approach that a select few companies are utilizing to achieve Operational Excellence in the face of complexity.
The document discusses various topics related to software engineering concepts including software development methodologies, requirements analysis, SWOT analysis, marketing mix (4Ps), software architecture, software testing, and test case preparation. It provides descriptions and comparisons of different software development methodologies like waterfall model, agile model, RAD model, spiral model, and scrum model. It also discusses software requirement analysis process, questions to consider for SWOT analysis, principles of software architecture around cohesion and coupling, and key fields to include in a test case.
Training needs analysis, skills auditing and training roi presentation 31 aug...Charles Cotter, PhD
This document discusses training needs analysis, skills auditing, and training return on investment. It provides an overview of the training process and cycle, including training needs analysis, skills auditing, workplace skills plans, and evaluating training return on investment. It describes a 6-step process for conducting a training needs analysis involving situational analysis, envisioning desired outcomes, identifying data collection methods, collecting data, sharing findings, and developing an implementation plan. Best practices for skills auditing are outlined, including using job analyses and developing performance standards. The skills auditing process involves determining skills requirements, auditing actual skills, and identifying development needs.
Process Management by Jan Mohammed.pptxJanMohammed3
This is a very generic presentation on Process Management concepts and its design + implementation. Use it to understand what process management entails and to coach your teams / customers.
What ISO Management Systems can learn from Balanced Scorecard?PECB
Balanced Scorecard is a Strategy Management System developed by Professors Kaplan and Norton. It is probably the most comprehensive system/tool in the modern world. It allows an organization balance its Strategy across 4 perspectives (Financial, Customer, Internal Process and Learning and Growth Perspectives). It further lets an organization break down each of these 4 perspectives based on 4 criteria which are Objectives, Measures, Target and Initiatives. There is a lot that ISO Implementers and Auditors need to learn from a Balanced Scorecard that will help in better delivering ISO engagements. This webinar will take a critical look at what is Balanced Scorecard and what ISO Consultants need to know to about it.
Main points covered:
• What is a Balance Scorecard?
• How Balance Scorecard allows organization to balance its Strategy across 4 perspectives (Financial, Customer, Internal Process and Learning and Growth Perspectives)
• How an organization breaks down each 4 perspective based on 4 criteria (Objectives, Measures, Target and Initiatives)
Presenter:
This webinar was presented by Orlando Olumide Odejide, who is the Chief Trainer for Training Heights Limited. Orlando is an experienced Enterprise Architect and Programme Director working on various technology solutions including SharePoint, SQL Server, Oracle, SAP, Odoo and Qlikview Technologies for clients in the Financial Services, Government and Manufacturing Sectors.
Link of the recorded session published on YouTube: https://youtu.be/XPPj9XhXl0s
The document discusses metrics for agile teams. It explains that metrics are important for several reasons, such as providing transparency, enabling business decisions, and changing day-to-day behavior. Good metrics should be vital, measure results not just outputs, track trends, be easy to collect, amplify learning, reinforce desired behavior, and optimize the whole. The document provides examples of common agile metrics like velocity, story completion, bugs, and technical debt. It emphasizes that metrics should be simple, visible to the team, and not threaten people.
Has your organization ever considered replacing a tester that did not write, for example, 15 test cases per day? Is the testing team blamed if defect leakage is greater than 5% into production? What drives decisions like these? The common thread in these examples is “Test Metrics”
Test Metrics... Everyone has an opinion about them. Some believe they are the most valuable way to communicate the results of testing. Some think that they are useless, misleading, and damaging to the communication of test results. Some believe that without measurement you are not managing the effort. And some believe that bad metrics are worse than no metrics at all.
Where does your organization fit in the metrics and measurement debates? Is your team aligned? Do you agree with the team? Do you use a reporting process for test results? Are you forced to report on metrics you don't believe are valuable? Do you have dozens of metrics that you are reporting periodically that no one looks at, and when they do look at them, there is room for misinterpretation?
In this session, Mike Lyles and Jay Philips will challenge the audience to discuss the topic of metrics and measurement, review multiple viewpoints on the topic, and address many of the questions that organizations have today around metrics and measurement.
Takeaways:
- Top metrics that are misused or misunderstood in most every organization.
- Metrics that you should you get rid of ASAP!
- Best and Worst metrics - based on opinions of the speakers & audience.
- Metrics that everyone should use – and how they compare to your organization’s metrics.
- Tools and processes that can help your organization better measure your testing.
** Presentation given at STPCon Spring 2014
Safeabilty: Analyzing the Relationship between Safety and Reliability PlantEngineering
-What if we treated maintenance and reliability improvement like safety?
-How Would that change our focus? our tactics?
-What techniques could we apply directly from the other areas?
-How are the two the same and how are they different?
This chapter discusses methods for measuring employee performance, including measuring results and behaviors. It outlines developing accountabilities, objectives, and performance standards to measure results. This involves identifying key tasks, determining objectives that are specific and time-bound, and setting standards related to quality and quantity. To measure behaviors, the chapter recommends identifying competencies and indicators, and choosing a measurement system such as comparative or absolute ratings. Examples of developing these elements are provided.
Quality improvement (QI) aims to continuously improve processes and outcomes by measuring performance, identifying areas for improvement, testing changes, and implementing successful changes. It focuses on systems rather than individuals. The Model for Improvement and the PDSA (Plan-Do-Study-Act) cycle are common frameworks for testing changes. Quality assurance focuses on conforming to standards through inspection, while QI is proactive and aims to prevent errors by improving systems and processes. Regular use of the PDSA cycle allows for incremental improvement through repeated small tests of change.
Software Quality Metrics Do's and Don'ts - XBOSoft-QAI WebinarXBOSoft
This slide presentation is from the QAI -Quest sponsored webinar with #XBOSoft where Philip Lew covered the Do's and Don'ts of #Software #Quality and Software #Testing Metrics.
This webinar discussed some of the most common mistakes in using software quality metrics and measurements. The primary take away is to learn from the mistakes of others, particularly where to use and not use #metrics to measure your testing and QA efforts. The last thing you want is to measure the wrong thing and create unwanted behavior. With knowledge of what not to do, we’ll then dive into how to develop a measurements and metrics framework that aligns with the organization’s business objectives. This means taking on a manager’s viewpoint so that your metrics don’t just measure testing progress, but also measure product quality and how it impacts an organization’s bottom line. As part of the webinar, we’ll discuss a variety of metrics that can be used to track work effort with results and enable you to plan and forecast your testing needs.
This document discusses best practices and common mistakes in implementing software quality metrics programs. It emphasizes the importance of understanding why metrics are being collected, measuring the right things in the proper context, and ensuring metrics are useful to stakeholders and help answer important questions. Common mistakes discussed include measuring the wrong things, forgetting context, collecting metrics sporadically, and failing to determine what constitutes "good" or "bad" metric values. The document provides examples of useful metrics and encourages linking metrics to goals, questions, and evaluation.
This document discusses using outcomes-based measurements for quality assurance. It defines outcomes as specific changes in attitudes, behaviors, knowledge, skills, status, or functioning expected to result from program activities. Choosing meaningful outcomes is emphasized as the first step, even if they are difficult to measure. Sample evaluation questions, a timeline for implementing an outcomes-based system, and how to measure success are provided. Common myths about outcomes evaluation are addressed, emphasizing that evaluation involves ongoing, practical processes rather than a complex science.
2015 ISACA NACACS - Audit as Controls FactoryNathan Anderson
The presentation provides an overview of data analytics concepts and tools that can be used for internal auditing. It discusses how audit analytics can help challenge traditional audit views and provide additional services while maintaining independence. Examples are given of how analytics can be used for monitoring controls, enhancing audits, and ad-hoc analysis of risks. Key lessons focus on ensuring diversity in analytic teams and being prepared to replace personnel. The presentation emphasizes using a toolbox approach to tools and affordably sourcing analytic talent from interns with the needed skills. Maintaining independence is discussed in the context of facilitating rather than directly implementing risk responses or managing risk.
The document summarizes a webinar on assessing compliance programs. It discusses why organizations conduct periodic assessments of their compliance programs, including regulator expectations, stakeholder expectations, and identifying risks and gaps. It also covers preparing for an assessment, including establishing goals and scope, collecting data through document reviews, surveys, interviews and focus groups. Finally, it discusses analyzing the data, reporting findings and recommendations, and generating an action plan to address recommendations. The overall purpose is to evaluate program effectiveness and identify areas for improvement.
Similar to Are you in control of Testing, or does Testing control you? (20)
This document discusses continuous performance testing (CPT) and introduces the Jagger CPT solution. It provides an overview of why performance testing is important, outlines the principles and goals of CPT, and describes the key parts of the Jagger CPT platform including load generation, metrics collection, test data management, and environment management. It also provides an example customer success story where Jagger was used for continuous performance testing of a large ecommerce site.
Мощь переполняет с JDI 2.0 - новая эра UI автоматизацииSQALab
This document provides an overview of the JDI (Java UI test automation framework). It discusses features of JDI including being UI element oriented, providing common UI elements and solutions to common problems. It provides examples of how to write tests using JDI annotations and page object pattern. The document also summarizes benefits of JDI such as reducing test code, improving test clarity, reuse across projects. Finally it outlines new features planned for JDI 2.0 including layout verification, page object generator, integration with Selenium and expanding JDI to other languages like Python.
The document discusses testing of geolocation systems. It provides an overview of geolocation, including definitions and importance. It then outlines the speaker's experience and work testing GIS systems. The rest of the document details approaches to testing geolocation, including simulating calls, checking responses and databases, and verifying accuracy. It also discusses common data formats, projections, tools like PostGIS and QGIS, and potential bugs to watch for like coordinate jumbling. The conclusion emphasizes starting simple, practicing to improve, and for tests to grow with knowledge as geolocation is important for future IT.
A workshop hosted by the South African Journal of Science aimed at postgraduate students and early career researchers with little or no experience in writing and publishing journal articles.
हिंदी वर्णमाला पीपीटी, hindi alphabet PPT presentation, hindi varnamala PPT, Hindi Varnamala pdf, हिंदी स्वर, हिंदी व्यंजन, sikhiye hindi varnmala, dr. mulla adam ali, hindi language and literature, hindi alphabet with drawing, hindi alphabet pdf, hindi varnamala for childrens, hindi language, hindi varnamala practice for kids, https://www.drmullaadamali.com
A review of the growth of the Israel Genealogy Research Association Database Collection for the last 12 months. Our collection is now passed the 3 million mark and still growing. See which archives have contributed the most. See the different types of records we have, and which years have had records added. You can also see what we have for the future.
How to Add Chatter in the odoo 17 ERP ModuleCeline George
In Odoo, the chatter is like a chat tool that helps you work together on records. You can leave notes and track things, making it easier to talk with your team and partners. Inside chatter, all communication history, activity, and changes will be displayed.
How to Build a Module in Odoo 17 Using the Scaffold MethodCeline George
Odoo provides an option for creating a module by using a single line command. By using this command the user can make a whole structure of a module. It is very easy for a beginner to make a module. There is no need to make each file manually. This slide will show how to create a module using the scaffold method.
This presentation was provided by Steph Pollock of The American Psychological Association’s Journals Program, and Damita Snow, of The American Society of Civil Engineers (ASCE), for the initial session of NISO's 2024 Training Series "DEIA in the Scholarly Landscape." Session One: 'Setting Expectations: a DEIA Primer,' was held June 6, 2024.
Main Java[All of the Base Concepts}.docxadhitya5119
This is part 1 of my Java Learning Journey. This Contains Custom methods, classes, constructors, packages, multithreading , try- catch block, finally block and more.
Walmart Business+ and Spark Good for Nonprofits.pdfTechSoup
"Learn about all the ways Walmart supports nonprofit organizations.
You will hear from Liz Willett, the Head of Nonprofits, and hear about what Walmart is doing to help nonprofits, including Walmart Business and Spark Good. Walmart Business+ is a new offer for nonprofits that offers discounts and also streamlines nonprofits order and expense tracking, saving time and money.
The webinar may also give some examples on how nonprofits can best leverage Walmart Business+.
The event will cover the following::
Walmart Business + (https://business.walmart.com/plus) is a new shopping experience for nonprofits, schools, and local business customers that connects an exclusive online shopping experience to stores. Benefits include free delivery and shipping, a 'Spend Analytics” feature, special discounts, deals and tax-exempt shopping.
Special TechSoup offer for a free 180 days membership, and up to $150 in discounts on eligible orders.
Spark Good (walmart.com/sparkgood) is a charitable platform that enables nonprofits to receive donations directly from customers and associates.
Answers about how you can do more with Walmart!"
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...PECB
Denis is a dynamic and results-driven Chief Information Officer (CIO) with a distinguished career spanning information systems analysis and technical project management. With a proven track record of spearheading the design and delivery of cutting-edge Information Management solutions, he has consistently elevated business operations, streamlined reporting functions, and maximized process efficiency.
Certified as an ISO/IEC 27001: Information Security Management Systems (ISMS) Lead Implementer, Data Protection Officer, and Cyber Risks Analyst, Denis brings a heightened focus on data security, privacy, and cyber resilience to every endeavor.
His expertise extends across a diverse spectrum of reporting, database, and web development applications, underpinned by an exceptional grasp of data storage and virtualization technologies. His proficiency in application testing, database administration, and data cleansing ensures seamless execution of complex projects.
What sets Denis apart is his comprehensive understanding of Business and Systems Analysis technologies, honed through involvement in all phases of the Software Development Lifecycle (SDLC). From meticulous requirements gathering to precise analysis, innovative design, rigorous development, thorough testing, and successful implementation, he has consistently delivered exceptional results.
Throughout his career, he has taken on multifaceted roles, from leading technical project management teams to owning solutions that drive operational excellence. His conscientious and proactive approach is unwavering, whether he is working independently or collaboratively within a team. His ability to connect with colleagues on a personal level underscores his commitment to fostering a harmonious and productive workplace environment.
Date: May 29, 2024
Tags: Information Security, ISO/IEC 27001, ISO/IEC 42001, Artificial Intelligence, GDPR
-------------------------------------------------------------------------------
Find out more about ISO training and certification services
Training: ISO/IEC 27001 Information Security Management System - EN | PECB
ISO/IEC 42001 Artificial Intelligence Management System - EN | PECB
General Data Protection Regulation (GDPR) - Training Courses - EN | PECB
Webinars: https://pecb.com/webinars
Article: https://pecb.com/article
-------------------------------------------------------------------------------
For more information about PECB:
Website: https://pecb.com/
LinkedIn: https://www.linkedin.com/company/pecb/
Facebook: https://www.facebook.com/PECBInternational/
Slideshare: http://www.slideshare.net/PECBCERTIFICATION
বাংলাদেশের অর্থনৈতিক সমীক্ষা ২০২৪ [Bangladesh Economic Review 2024 Bangla.pdf] কম্পিউটার , ট্যাব ও স্মার্ট ফোন ভার্সন সহ সম্পূর্ণ বাংলা ই-বুক বা pdf বই " সুচিপত্র ...বুকমার্ক মেনু 🔖 ও হাইপার লিংক মেনু 📝👆 যুক্ত ..
আমাদের সবার জন্য খুব খুব গুরুত্বপূর্ণ একটি বই ..বিসিএস, ব্যাংক, ইউনিভার্সিটি ভর্তি ও যে কোন প্রতিযোগিতা মূলক পরীক্ষার জন্য এর খুব ইম্পরট্যান্ট একটি বিষয় ...তাছাড়া বাংলাদেশের সাম্প্রতিক যে কোন ডাটা বা তথ্য এই বইতে পাবেন ...
তাই একজন নাগরিক হিসাবে এই তথ্য গুলো আপনার জানা প্রয়োজন ...।
বিসিএস ও ব্যাংক এর লিখিত পরীক্ষা ...+এছাড়া মাধ্যমিক ও উচ্চমাধ্যমিকের স্টুডেন্টদের জন্য অনেক কাজে আসবে ...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...Dr. Vinod Kumar Kanvaria
Exploiting Artificial Intelligence for Empowering Researchers and Faculty,
International FDP on Fundamentals of Research in Social Sciences
at Integral University, Lucknow, 06.06.2024
By Dr. Vinod Kumar Kanvaria
5. Trying to stay relevant
• Tool Author – Test Process Management (T-Plan)
• Businessman/Entrepreneur/Tester/Innovator
• UK Representative to ISTQB
• Chairman ISTQB Advanced Level Syllabus Working Group
• Founder member of CAT (Certified Agile Tester) SIG
• Practical Assessments (R&D Project)
• Web TV & Digital Marketing in sport
• Professional sports governance (in my spare time!)
• Writing a book!
6. Presentation Background
• Lack of control over projects/initiatives in IT, Business &
Sport
• Common faults over this broad perspective
• This presentation focuses on software testing as a form
of measurement
• But it uses analogies and common characteristics across
other perspectives
• I will use some examples and my experiences to help
highlight key points
8. My theme for a book
“Effective Measurement and Testing Delivers
Control of Anything”
9. Measurement & Testing
My Axioms
• Testing is more than just Testing!
• Software Testing can learn from other forms of
Measurement & Testing
• Other forms of Measurement & Testing can learn
from Software Testing
• Agile Development is helping the case for
Measurement & Testing
• Software Testing took a wrong turn when it
became obsessed with Test Cases!
10. Another Testing Axiom
•The answer to the ultimate question of Life, The Universe
and Everything?
Answer: “42”
(Hitchhikers Guide to the Galaxy – Douglas Adams)
•The answer to any question about Software Testing?
Answer: “It Depends!”
(Dale Perry – SQE Inc. USA)
12. Control – Some definitions
•The power to influence or direct people's behaviour or the
course of events. (lead)
•Determine the behaviour or supervise the running of. (lag
& lead)
13. Control vs Management
What’s the difference?
Corporate Governance Definition
1.Strategic Direction of the Business
1.Effective Monitoring and Control of the
Management of the Business
2.Accountable to the Shareholders of the Business
15. Being in Control – Can you answer
these questions?
•What are my measures of success?
•How will I know when I’ve met them?
•How sure will I be that I’m going to meet them at
any point in time?
•How do I know where I am against them at any
point in time?
•Can I manage change effectively?
•For an IT project, how useful are Test Cases to
help me keep in control?
16. Successful Outcomes?
• Reach destination port
• Get there on schedule
• All passengers & crew are safe
• Passengers have not been (too) seasick!
• Within budget (economical with fuel)
• Obstacles avoided!
17. How will I know when I’ve met the
successful outcomes?
• What are the key measures & targets?
• Speed?
• Direction?
• Position?
• Quality Attributes?
18. Keeping the ship under control!
Key Drivers
Initiatives taken based these
key goal & performance drivers
22. What is Business Performance
Measurement & Management?
• ‘Translating Strategy into Action’
– Kaplan & Norton
• Use of Business Balanced Scorecard
– Different perspectives
• Drives behaviour
• Measures outcomes
• Links actions to strategy
• Leads to predictable outcomes
23. Dangers of Performance Measurement &
Management?
Measures that are badly constructed can
destroy an organisation’s performance… effort
may be misdirected into unproductive actions
and a great deal of time and effort can be
wasted.
‘Balanced Scorecard in a week – Mike Bourne, Pippa Bourne’
24. 5 Key principles
1. Generic application
2. Objectives, Measures & Targets, Initiatives
3. Separate “What” & “How”
4. Cascading Scorecards
– One person’s ‘How’ is another person’s ‘What’
– Measures & Targets become objectives for next
person
1. Lead & Lag Indicators
– Goal Indicators (reactive)
– Performance Indicators (predicative)
26. Scorecards linked by
cascading ‘whats’ & ‘hows’
HOWWHAT
HOWWHAT
HOWWHAT
• Could be
1:many or
many:many
• Simplest way
is 1:1
HOW
HOWWHAT
HOWWHAT
HOWWHAT
WHAT
27. Summary of Outcome Driven Measurement
Framework/Model
• Separate “What” & “How”
• Complex Relationships in Model
• Information Traceability matrix Top to
Bottom
• Control of Complex Organisations &
Systems
• Good fit with Agile Development
Practices!
28. Now consider Software Testing Model
• What are the Measures & Targets that support
the Software Testing Process?
• Test Cases?
• Test Objectives?
• Test Requirements?
• Test Conditions?
• Testers focus on Test Cases, but are they the
best Measures/Targets to drive behaviour?
• Are they the best to manage risk and change?
30. Test(ing) & Software Testing
•The means by which the presence, quality, or genuineness of anything
is determined; a means of trial (lag)
•Software testing is a process of executing a program or application with
the intent of finding the software bugs. (lag) It can also be stated as the
process of validating and verifying that a software program or application
or product meets the business and technical requirements that guided it's
design and development. (lag & lead)
•Testing is a risk reduction process (lead)
31. The problem with Test Cases
Test case
A set of input values, execution preconditions,
expected results and execution postconditions,
developed for a particular objective or test
condition, such as to exercise a particular
program path or to verify compliance with a
specific requirement. [After IEEE 610]
32. The problem with Test Cases
• How big?
• When can they be defined?
• They may help me Test, but do they help me
Measure?
• Whose Measures are they?
• Do they separate the What from the How?
• History of Test Cases (‘Bottom-up’)
• What else fails if my Test Case fails - traceability?
• Tool problems – wrong model
• What if none defined?
34. 1479 test cases: but are they “useful”?
Now, let’s
start with a
classification
tree
1479
test cases, so
it must be good,
right?
Test
specification
process
Documentation
to agree
coverage
38. Test Model described by ‘methods’ people at large pan-
European financial institution.
Test
Requirements
Requirements
WHY
Test Design
Specifications
Test Case
Specifications
WHAT WHAT HOW
• “From Requirements to Test Case Specification”
in test management tool
39. A complex model to support software testing
May or may not
Need complex
model:
IT DEPENDS!
40. Guidelines for a better model
• Need to separate What & How
• Model needs Complex Relationships – many:many
• Test Conditions can be Considered targets for
success, organised by Measurement Attributes
– Functional, Behavioural
• Can and should be defined early
• Ability to cope with change
• Know where you are at any point in time
• Delivers Control of Complex Systems
• Good fit with Agile Development Practices!
• Business Outcome Driven Development
41. What factors affect the level of measures
and targets in a measurement framework?
• Risk
• Size & Complexity
• Dev Method
• Test Level
• Manageability & Maintainability
• Tool usage
• Ability to cope with change
• Quality and level of artefacts
• Regulatory requirements…....
42. What factors affect the level of measures
and targets in a measurement framework?
IT DEPENDS!!
Setting good measures and targets is
difficult - it requires strong analytical skills
People often neglect the –ve measures
and focus on the +ve
43. What can the world of Measurement learn
from Software Testing?
• Software Testing has been developed as a
discipline for many years
• Very detailed processes and techniques
compared with other forms of Measurement &
Testing
• Software Test Techniques
– Support Analysis and Design Activities
• Airport Example
44. Other Experiences
• Making a new Financial
Exchange Live
• An Office Move
• ISTQB Syllabus Development
• Practical Assessments
• Governance in Sport
• Mindset change and silos!
46. WHAT’S MISSING?
Testing Model
Control
Tests and Test
Cases
Test Requirements/
Test Conditions
Measures & Targets
Separated from Tests
Many:Many Reqs:Tests
Objectives/
Requirements
Initiatives
47. WHAT’S MISSING?
New Financial Exchange
New Market
Tests and Test
Cases
Regulator Approval
SPECIFICATION
Measurement
Framework
Retrospective
Test Conditions in
Complex Model
49. WHAT’S MISSING?
Sports Club Constitution Review
Need to Change
Old Subject Matter
Experience
VISION / STRATEGY
CONSTITUTION/
RULES
Terms of
Reference
REQUIREMENTS
50. My ‘Axioms’
“Test Driven Everything”
“Business Outcome Driven Development”
“Effective Measurement and Testing Delivers
Control of Anything”
“Top Down, meets Bottom Up”
51. References
• Testing Entities Paper – John Kent
• IEEE Paper – ‘Keystone Testing Entity’ Mike
Smith & Neil Thompson
• Value Flow Scorecards – Mike Smith & Neil
Thompson
• Balanced Scorecard in a week – Mike & Pippa
Bourne
• My book on Measurement & Testing – when?!
52. What can you take away from this?
• If you are building a pure hierarchical model to
manage testing, think again
• Can I optimise my model to give better control
without it becoming unmanageable?
• Am I relying too much on my testing tools – do I
need something else?
• Can I use my analytical skills to engage with the
business better?
• I encourage you to think about these concepts,
even if you think you are in control…...
53. If you think you’ve found a good move,
look for a better one!
54. My Measures of Success for this talk?!
1.Standing Ovation?
2.Loud Applause?
3.Polite Applause?
4.Silence?
5.I won’t be let into Russia again?!!
Editor's Notes
Is the dog wagging the tail (in control) or is the tail wagging the dog! (not in control)!! Very Successful marketing campaign 15 years ago – had a call from someone who opened it while testing on a Sunday!
INTRODUCTION SLIDES
Important to set a context up front, with more detail to follow. As well as a technology background, from programming to project management, I have been involved in various independent businesses and sport since 1984. Observation and age is allowing me to conduct a retrospective – with the aim of producing a book.
My kids would have you believe I was around in the Jurassic Period – maybe in terms of Technology I have!
Trying to stay relevant, but getting the balance right! (And note relevance to getting the balance right later regarding measurement frameworks!)
Still trying to come up with new ideas!
Play some music! (and sing a little)
Presenting to IT and Engineering undergraduates in India in February (Best question asked for a long time about being in IT for 40 years!)
Presenting at the ANZTB Conference in Canberra, Australia
Involved in cricket – this was few years ago on a school tour to Barbados. I’m an elected board member at Essex County Cricket Club (one of 18 professional teams in England)
Trying to stay relevant!
ADD AUTO TRANSITION – BETTER GRAPHIC REPRESENTATION
Semi-retiring? from Executive Business Management
Still many roles and interesting projects as an independent
Tool Author – Test Process Management (T-Plan)
Businessman/Entrepreneur/Tester/Innovator
Semi-retiring? from Executive Business Management
Still many roles and interesting projects as an independent
UK Representative to ISTQB
Chairman ISTQB Advanced Level Syllabus Working Group
Founder member of CAT (Certified Agile Tester) SIG
Practical Assessments (R&D Project)
Web TV & Digital Marketing in sport
Professional sports administration (in my spare time!)
Writing a book!
Background
Warning – Conceptual, Work in Progress!
Method Agnostic
This is a short high level overview
Have a wide perspective from Business to Technical
Strong interest in Innovation and R&D (tax credits!)
Have spoken and written papers over many years:
Software Testing and Test Process Management
Governance & Compliance
Business Performance Measurement & Management
Concerns over effectiveness of software testing as a measurement process/tool
Have found comparing principles of Business Performance Measurement and Software Testing very useful to highlight concerns
The subject of my book – Measurement & Testing!
Background
Warning – Conceptual, Work in Progress!
Method Agnostic
This is a short high level overview
Have a wide perspective from Business to Technical
Strong interest in Innovation and R&D (tax credits!)
Have spoken and written papers over many years:
Software Testing and Test Process Management
Governance & Compliance
Business Performance Measurement & Management
Concerns over effectiveness of software testing as a measurement process/tool
Have found comparing principles of Business Performance Measurement and Software Testing very useful to highlight concerns
The subject of my book – Measurement & Testing!
Important to look at what I mean by these words, since they can have several generic and contextual definitions
The Hitchhikers Guide to the Galaxy – Douglas Adams. In the radio series and the first novel, a group of hyper-intelligent pan-dimensional beings demand to learn the Answer to the Ultimate Question of Life, The Universe, and Everything from the supercomputer, Deep Thought, specially built for this purpose. It takes Deep Thought 7½ million years to compute and check the answer, which turns out to be 42.
Thanks to Dale Perry (SQE – Software Quality Engineering, USA) for this observation many years ago!
Important to look at what I mean by these words, since they can have several generic and contextual definitions.
May seem basic, but I’m going to explore these and help define them in the my context to help the audience understand the points I am trying to make – my axioms.
Control
The power to influence or direct people's behaviour or the course of events. (lead)
Determine the behaviour or supervise the running of. (lag & lead)
Measure(ment)
Ascertain the size, amount, or degree of (something) by using an instrument or device marked in standard units (lag)
Assess the importance, effect, or value of (something) (lag)
A plan or course of action taken to achieve a particular purpose. (lead)
Testing (and Software Testing)
The means by which the presence, quality, or genuineness of anything is determined; a means of trial (lag)
Software testing is a process of executing a program or application with the intent of finding the software bugs. (lag) It can also be stated as the process of validating and verifying that a software program or application or product meets the business and technical requirements that guided it's design and development. (lag)
Testing is a risk reduction process (lead)
There are many more, but these two help make the points.
Note the terms ‘lag’ and ‘lead’. Do people understand these terms? Further development of these later.
Many people find difficulty in distinguishing between “Control” & “Management”, but there is an important distinction. People who control things, do not necessarily manage things. This is important because we are considering the definition of “Control” and not “Management” in this presentation, although there is a clear link between them (but not just that the person in control is the “Super Manager”!. It’s useful to consider the meaning of “Governance” when analysing the meaning of “Control” and useful to consider a definition of Corporate Governance.
Careful with any analogy. Not saying that controlling a ship is the same as controlling an IT project, but can help to visualise something that is often difficult to portray using a complex IT system.
Remember the questions to ask
What are my measures of success?
How will I know when I’ve met them?
How sure will I be that I’m going to meet them at any point in time?
How do I know where I am against them at any point in time?
Can I manage change effectively?
What does being in control mean?
My set of questions that link up the Control, Measure and Test – Jumping ahead a little bit here in mentioning this in relation to Test Cases!
Ask some questions about control and the audience experience in Projects and Testing (and role profile of audience).
Key visual to demonstrate that there are potential complex relationships between the measures and targets in a measurement framework that delivers effective control.
We are now going to consider some principles of Business Performance Measurement & Management. This has been a key part of my analysis over the years to compare with IT projects and software testing as a measurement process.
Why? – What does success look like? What are my desirable outcomes?
What? – How will I know when I reached my desirable outcomes? (and monitored them and coped with changes along the way)
How? – What actions do I need to take in line with the measures and targets?
How, driven by the what, becomes the next level Why, which in turns leads to other subordinate whats – a ‘What’/’How’ framework
Reference Mike Smith and Neil Thompson’s paper for full detail – ‘Keystone Testing Entity’.
Test Requirements now equivalent to Test Conditions in ISTQB Glossary.
I prefer to emphasise the risk reduction process - one that will find bugs during that process. Note the historic emphasis on the purpose of software testing being to find bugs. This emphasis was in the first ISEB/ISTQB Foundation Syllabus released in 1996 and highlights the fact that the syllabus was based on a Unit Test Standard, not from a Business/System Testing basis.
From the ISTQB Glossary. Far too many terms and subjectivity. What if no Test Cases? Use of ‘Test’ – defined as a set of one or more test cases
Do you use a proprietary tool to manage tests? Does it help you control? Do you find you have to build your own control system because it doesn’t deliver what you need? That is a good sign!! Rather than be a slave to the tool, you are thinking what you really need to get in control.
This approach led to huge volumes of low and no value tests based on drilling down to a very low level at requirements and assuming all tests must fit into a hierarchy within the requirements hierarchy.
THIS PROJECT BECAME OUT OF CONTROL, COSTING TENS OF MILLIONS AND WAS EVENTUALLY SCRAPPED!
Can be extended and consolidated for a range of other measures for projects outside of software testing – ref Scorecards and Performance Measurement & Management. Need not be IT Depends. Explain factors affecting level and model required.
You may be doing this already, or you may say its not applicable to you. IT Depends! I’ve seen a lot of projects out of control and helped them with this approach. The failed project referred to earlier threw out this model!!
It is a very difficult activity, requiring string analytical skills
Ref MoneyBall re sports mindsets!
Case study produced
Case study produced
Mindset change – silos – even change to openness in command & control systems in safety critical world
Because I like using analogies , here is another one that I think is appropriate. Mindset change is often the hardest!
Let’s now look at Measurement and some simple definitions