Project Management for QA manager
Liang Gao (liangg@gmail.com)
Different Roles in a Modern Product Development
• Project/Program manager.
• Product Marketing people
• Development/QA manager. (people manager)
• Technical lead
• Team member (engineers)
QA Manager’s Role
• Managing team (people management)
• Interact with other function team in the product development
• Guardian of the product quality
• Make sure the team’s work is on time with Quality
QA Manager QA Team
Development
Manager
Documentation
Team
Product
Manager
SE/Support
Pre-Project Planning
• Understand the working scope and requirement
• Logistics
• Resource and Schedule.
• Define clear deliverables and acceptance criteria
• Communication with peers and team members
Understand the working
scope• Talk to Program manager
• What is the over all release schedule.
• How long is the coding, how long is the testing.
• Talk to product manager and marketing folks.
• Why we need this product/features.
• Impact on the revenu
• Talk to Development manager
• Understand the implementation in high level
• Define the code hand over criteria
Logistics
• Any product related trainings your team need and when?
• Any technology related trainings your team need and when?
• Equipments needed, budget and where to get them
• Device under test
• PCs
• Routers/switches
• Software
Resource and Schedule.
• Need to justify the resource and schedule that you need
• Need to have a productivity estimation of the team’s capability
on
• Number of test cases develop per day per engineer
• Number of test cases manual execution per day per engineer
• Number of test scripts develop per day per engineer
• A Week by Week detail schedule would be good
Sample Manual
Execution Report
Sample Regression
Report
Resource and Schedule.
• Leave buffer for the emergency and last minute change.
• Have your priority list of the projects you are doing right now.
• Communicate your priority list with your peers and upper
management
Deliverables and Acceptance
Criteria• Test case and test plan
• Manual testing
• Automated regression testing
• Automation script development
Test Case & Test Plan
Development
• Deliverables:
• Test case high level design (test purpose)
• Detail test case & test plan documentation.
• Acceptance Criteria
• On Schedule
• High level design reviewed with development team and
marketing people, passed development team and marketing
people’s internal review process.
• Test case & test plan reviewed with development team and
marketing people, passed development team and marketing
people's internal review process.
Test Case Manual Execution
• Deliverables
• Test case execution log
• Defects
• Acceptance Criteria
• On Schedule
• Case execution log reviewed and approved by development team
and marketing people.
• All test cases are either in a state of “PASS” or in a state of “FAIL”
with a open defect ID.
• No invalid defects filed by Sigma
• All defects are valid & reproducible.
• Defects response time from Sigma no later than 12 hours.
Test Case Automated
Regression
• Deliverable
• Automated Regression Report.
• Regression Defects
• Acceptance Criteria
• On Schedule.
• All scripts results are either in a state of “PASS” or in a state of
“FAIL” with an open defect ID.
• Results are in consistent with baseline result.
• All bugs reported are valid regression bugs after a clean baseline
establishment.
• Defects response time from Sigma no later than 12 hours.
Script Development –
Deliverables.
• Deliverables
• Script template
• Scripts
• Execution logs (results are either in a state of “PASS” or in a state
of “FAIL” with an open defect ID)
• Script code structure documentation
• Script integration guideline (to integrate in team’s testing
environment)
Script Development –
Acceptance Criteria
• Acceptance Criteria
• On schedule
• Each script passed on team predefined
• Hardware product list
• Software (platform) version list.
Log will be provided. (can fail but Sigma will provide an open bug
id)
• Scripts are reviewed and approved by team internal review
process
• Scripts can be executed with same results in team’s testing
environment as in Sigma testing environment
• 30 days post-project script support.
Communication and Sale
• Propose your test project plans to the upper management and
your peer.
• Justify your claim on the cost/budget
• Equipments
• Resources.
• Justify your schedules
• Communicate with the team on the project background, scope
and requirement
Standard Template and Standard Test Case
• Standard template of test plan/test cases.
• Standard template of scripts
• Standard test case library that every one can share
• CLI test cases
• GUI test cases.
• Negative values library
Test Plan Template Sample Scripts
Effective Review Meetings
• At lest 2 sets of reviews on the deliverables
• Internal review
• External review
• Invite
• Coder, development manager
• Marketing, product managers
• Sales Engineers
• Assign roles of “Recorder”, “Reviewer”, “Moderator”, “Reader”
in the review meeting
• Make sure all feedbacks are recorded, have a follow up plan
and have been updated on the deliverables
Action Item List
• Weekly reviews on the action item and its state
• New
• Pending
• In progress
• Completed (can be removed from the list)
• Make sure once a AI is on the list, it will be there for ever if not
“completed”, and track the status every week
• Always list last week's objectives and this
week'saccomplishments against them
Quality Checklist
Bug Quality
Checklist
Script Quality
Checklist
Tset Case Quality
Checklist
Review with Team Member’s
Work• In the tester's cube, not yours
• Show me your best work from last week
• Show me your most interesting bugs
• Show me your most interesting test cases
• What have you been testing? Why did you do it that way? Have
you thought about this?
Daily report for the bug trend and bug state
• Have a dedicated team member to send out
• Number of bugs filed today and by who
• Total number of bugs filed by each of the team members till
today
• Bugs that need team members to response today
Daily report on the team member’s productivity
• Number of the test cases manually or auto executed today by
each of the team member
• Number of the scripts developed today by each of the team
member
• Number of the test cases developed today by each of the team
member
Report and Post Mortem
Analysis• Generate your testing report to the upper management
• Defects: very important, no defects, no talk.
• Participate the release meeting to sign off or not
• Post Mortem analysis with the team
Generate your testing report
• Make sure your report is accurate and clean
• Failed test case must have an open bug ID
• High level summary for your boss:
• Total number of test cases developed and executed
• Total number of defects
• Total number of test cases.
• Bug trend – Bug open rate, close rate during the project period
Participate the release
meeting• Make a decision if you will sign off or not on the release
• Prepare your bug report to justify your claim
• Remember to release or not is a business decision, not a
testing decision
• Make sure you provide the accurate risk analysis to the upper
management
Post Mortem Analysis
• A round table survey and talk with in your team
• What you think we do good in this project
• What you think we do bad on this project
• If there is one thing you wish we could do better ,what would
that be?
• Read the post mortem report, it is a very good way to
understand your team, and do better in the next project.
Version control
• Version 1.0, February 2008, Liang Gao
Project management for qa manager

Project management for qa manager

  • 1.
    Project Management forQA manager Liang Gao (liangg@gmail.com)
  • 2.
    Different Roles ina Modern Product Development • Project/Program manager. • Product Marketing people • Development/QA manager. (people manager) • Technical lead • Team member (engineers)
  • 3.
    QA Manager’s Role •Managing team (people management) • Interact with other function team in the product development • Guardian of the product quality • Make sure the team’s work is on time with Quality QA Manager QA Team Development Manager Documentation Team Product Manager SE/Support
  • 4.
    Pre-Project Planning • Understandthe working scope and requirement • Logistics • Resource and Schedule. • Define clear deliverables and acceptance criteria • Communication with peers and team members
  • 5.
    Understand the working scope•Talk to Program manager • What is the over all release schedule. • How long is the coding, how long is the testing. • Talk to product manager and marketing folks. • Why we need this product/features. • Impact on the revenu • Talk to Development manager • Understand the implementation in high level • Define the code hand over criteria
  • 6.
    Logistics • Any productrelated trainings your team need and when? • Any technology related trainings your team need and when? • Equipments needed, budget and where to get them • Device under test • PCs • Routers/switches • Software
  • 7.
    Resource and Schedule. •Need to justify the resource and schedule that you need • Need to have a productivity estimation of the team’s capability on • Number of test cases develop per day per engineer • Number of test cases manual execution per day per engineer • Number of test scripts develop per day per engineer • A Week by Week detail schedule would be good Sample Manual Execution Report Sample Regression Report
  • 8.
    Resource and Schedule. •Leave buffer for the emergency and last minute change. • Have your priority list of the projects you are doing right now. • Communicate your priority list with your peers and upper management
  • 9.
    Deliverables and Acceptance Criteria•Test case and test plan • Manual testing • Automated regression testing • Automation script development
  • 10.
    Test Case &Test Plan Development • Deliverables: • Test case high level design (test purpose) • Detail test case & test plan documentation. • Acceptance Criteria • On Schedule • High level design reviewed with development team and marketing people, passed development team and marketing people’s internal review process. • Test case & test plan reviewed with development team and marketing people, passed development team and marketing people's internal review process.
  • 11.
    Test Case ManualExecution • Deliverables • Test case execution log • Defects • Acceptance Criteria • On Schedule • Case execution log reviewed and approved by development team and marketing people. • All test cases are either in a state of “PASS” or in a state of “FAIL” with a open defect ID. • No invalid defects filed by Sigma • All defects are valid & reproducible. • Defects response time from Sigma no later than 12 hours.
  • 12.
    Test Case Automated Regression •Deliverable • Automated Regression Report. • Regression Defects • Acceptance Criteria • On Schedule. • All scripts results are either in a state of “PASS” or in a state of “FAIL” with an open defect ID. • Results are in consistent with baseline result. • All bugs reported are valid regression bugs after a clean baseline establishment. • Defects response time from Sigma no later than 12 hours.
  • 13.
    Script Development – Deliverables. •Deliverables • Script template • Scripts • Execution logs (results are either in a state of “PASS” or in a state of “FAIL” with an open defect ID) • Script code structure documentation • Script integration guideline (to integrate in team’s testing environment)
  • 14.
    Script Development – AcceptanceCriteria • Acceptance Criteria • On schedule • Each script passed on team predefined • Hardware product list • Software (platform) version list. Log will be provided. (can fail but Sigma will provide an open bug id) • Scripts are reviewed and approved by team internal review process • Scripts can be executed with same results in team’s testing environment as in Sigma testing environment • 30 days post-project script support.
  • 15.
    Communication and Sale •Propose your test project plans to the upper management and your peer. • Justify your claim on the cost/budget • Equipments • Resources. • Justify your schedules • Communicate with the team on the project background, scope and requirement
  • 16.
    Standard Template andStandard Test Case • Standard template of test plan/test cases. • Standard template of scripts • Standard test case library that every one can share • CLI test cases • GUI test cases. • Negative values library Test Plan Template Sample Scripts
  • 17.
    Effective Review Meetings •At lest 2 sets of reviews on the deliverables • Internal review • External review • Invite • Coder, development manager • Marketing, product managers • Sales Engineers • Assign roles of “Recorder”, “Reviewer”, “Moderator”, “Reader” in the review meeting • Make sure all feedbacks are recorded, have a follow up plan and have been updated on the deliverables
  • 18.
    Action Item List •Weekly reviews on the action item and its state • New • Pending • In progress • Completed (can be removed from the list) • Make sure once a AI is on the list, it will be there for ever if not “completed”, and track the status every week • Always list last week's objectives and this week'saccomplishments against them
  • 19.
    Quality Checklist Bug Quality Checklist ScriptQuality Checklist Tset Case Quality Checklist
  • 20.
    Review with TeamMember’s Work• In the tester's cube, not yours • Show me your best work from last week • Show me your most interesting bugs • Show me your most interesting test cases • What have you been testing? Why did you do it that way? Have you thought about this?
  • 21.
    Daily report forthe bug trend and bug state • Have a dedicated team member to send out • Number of bugs filed today and by who • Total number of bugs filed by each of the team members till today • Bugs that need team members to response today
  • 22.
    Daily report onthe team member’s productivity • Number of the test cases manually or auto executed today by each of the team member • Number of the scripts developed today by each of the team member • Number of the test cases developed today by each of the team member
  • 23.
    Report and PostMortem Analysis• Generate your testing report to the upper management • Defects: very important, no defects, no talk. • Participate the release meeting to sign off or not • Post Mortem analysis with the team
  • 24.
    Generate your testingreport • Make sure your report is accurate and clean • Failed test case must have an open bug ID • High level summary for your boss: • Total number of test cases developed and executed • Total number of defects • Total number of test cases. • Bug trend – Bug open rate, close rate during the project period
  • 25.
    Participate the release meeting•Make a decision if you will sign off or not on the release • Prepare your bug report to justify your claim • Remember to release or not is a business decision, not a testing decision • Make sure you provide the accurate risk analysis to the upper management
  • 26.
    Post Mortem Analysis •A round table survey and talk with in your team • What you think we do good in this project • What you think we do bad on this project • If there is one thing you wish we could do better ,what would that be? • Read the post mortem report, it is a very good way to understand your team, and do better in the next project.
  • 27.
    Version control • Version1.0, February 2008, Liang Gao

Editor's Notes

  • #4 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #5 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #6 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #7 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #8 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #9 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #10 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #16 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #17 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #18 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #19 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #20 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #21 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #22 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #23 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #24 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #25 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #26 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。
  • #27 到底什么是系统测试,和解决方案测试。 我们可以给出一个定义,但那样子就太理论化了,所以我更愿意给出一个例子来。 看一看别的做的比较好的公司是怎么做的。