Download as PDF, PPTX
Uploaded byVerhaert Masters in Innovation
Innovation day 2013 2.4 frederik mortier (verhaert) - test management
This document provides guidance on organizing test management during a feasibility study. It recommends taking an efficient test approach that begins with quick and dirty testing to get early results with minimal resources. This is followed by more detailed test planning and testing based on identified risks. The goal is to reduce project risks through a phased testing approach while balancing budget, planning and performance. Testing conclusions should identify any issues found to inform next steps. An efficient test approach aims to maximize learning while controlling costs and timing.
More Related Content
Similar to Innovation day 2013 2.4 frederik mortier (verhaert) - test management
![ICF-related data: the new frontier of individualised, predictive healthcare ]...](https://cdn.slidesharecdn.com/ss_thumbnails/dtv8w8utqk2ngpvjhh1o-signature-ed09eebe028b42c0dd5fe83c2d913fb7e847b0bbdbaeeb79f594140f431a041d-poli-170903164202-thumbnail.jpg?width=640&height=640&fit=bounds)
More from Verhaert Masters in Innovation
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Innovation day 2013 2.4 frederik mortier (verhaert) - test management
- 1. 2 Test management: Howto organize your research and test campaign during the feasibility study CONFIDENTIAL Frederik Mortier Consultant applied physics & systems frederik.mortier@verhaert.com THEME 2: RISK MANAGEMENT IN INNOVATION
- 2. 3 Frederik Mortier Consultant AppliedPhysics & Systems 2012- now Researcher @ Ghent University 2011-2012 MSc. in Engineering – Electro-Mechanical 2009-2011 MSc. in Industrial Sciences – Electro-Mechanical 2005-2009 frederik.mortier@verhaert.com
- 3. 4 “A feasibility studyis not only meant to confirm a good idea…”
- 4. 5 “…, it couldalso detect a bad idea!”
- 5.
- 6. 7 Outline Context withinNPD Problems & Risks Efficient Test Approach Conclusion
- 7.
- 8. 9 Concept Design & Development Production Visualization TestingSimulation Review Quick&Dirty Detailed Conclusion Utility Allowability Desirability Feasibility Usability When What How Context within NPD
- 9. 10 Cut Tangible • Visualize •Simulate • Test • Review • Roadshow Risk focus • Criticalities • Added value • 360° Early • Rapid prototyping • First time right Options Context within NPD Risk reduction methods
- 10. 11 Risk Management Risk identification Risk analysis Riskevaluation Technology management Development Tools & methods Verification Risk mitigation Requirement mgmt Risk assessment Risk treatment Risk monitoring Validation Context within NPD
- 11. 12 Why Feasibility Study? Investigatefeasibility of the idea in the application •Confirm / deny added value •Determine required costs By determining: Feasibility Study Strengths vs. Weaknesses Opportunities vs. Threats Needed resources Context within NPD Risk mitigation
- 12. 13 Context within NPD AddedValue – What do we want to find out? Feasibility • Is the core concept feasible ? Proof-of technology • Does my core technology work in the lab ? Field demon- strator • Will my design function in the application?
- 13. 14 Context within NPD AddedValue – Validation vs. Verification Needs Requirements Design Test setup Implementation Documentation Validation Check if developed product fulfills its purpose (intended use) Are we building the right product? Field test : full system test in real life Demonstrator in application Test in production if the operator can measure a hole Ø and can detect faults. Validation = application owner (customer) Verification Check if developed product is compliant with specifications Are we building the product right? Lab test / Analysis / Simulation Breadboards / Proof-of-concept in lab Test on a sample plate if we can reach e.g accuracy, w.r.t. reference technology. Verification = development partner (engineer)
- 14. 15 Context within NPD CostRequired Why wait for the risk to come true ? More freedom to adjust in early project phases Early detection of risk allows for measures at lower cost and with less impact on schedule Normal Time Learning curve System specification Conceptual design Detailed design Tooling / purchase Integration Test Reversed Fail fast, fail cheap!
- 15. 16 Context within NPD WhyTesting? Different aspects of technical part of feasibility study: •Better guarantee on final results •Simulations often too complex/time consuming •Sometimes fastest method •New technologies need testing (no previous experience) Feasibility testing is done to learn! Avoid unnecessary testing Example: blister packaging & NASA Visualization Testing Simulation Review
- 16.
- 17. 18 Context within NPD TestManagement Managing of test activities: • Preparing and managing test plan • Preparation of tests • Choice of test approach + testing • Interpretation of test results • Documentation / reporting results Iterative process during feasibility study
- 18.
- 19. 20 Problems & Risks Risksduring feasibility phase Risks dealt with are the same as in the overall risk management • Budget • Planning • Performance Perfor- mance Planning Risk manage ment Budget Keep 3 axes in balance !
- 20. 21 Budget risks Uncontrolled budgets Timing •Long test times • Documentation Test setups / tools • Renting vs. buying • Too advanced setups too soon • Rework due to unforeseen events Perfor- mance Planning Risk manage ment Budget Problems & Risks
- 21. 22 Planning risks Planning Long durationof test campaign • Test conclusions too late • High criticality items too late Tests vs. Resources • Equipment availability • Personnel availability Lead times • Equipment/tools • Software • Test stands Perfor- mance Planning Risk manage ment Budget Problems & Risks
- 22. 23 Performance risks Performance Wrong result •Wrong measurement method • Wrong measurement tool • Inaccurate results • Wrong test order Wrong basic requirement • Wrong test criteria “risk based methods in NPD” Perfor- mance Planning Risk manage ment Budget Problems & Risks
- 23.
- 24. 25 Outline Objective of testing Evaluateperformance / behavior to remove / reduce project risk by testing on “representative hardware” Ways of testing • Proof of concept bread boarding • Works like real application demonstrator • Looks/works/tastes like real product prototype Efficient test approach Test Management
- 25. 26 Quick & Dirty Testing TestPlan Development Detailed Testing Conclusion Efficient test approach Test flow
- 26. 27 Quick & Dirty Testing TestPlan Development Detailed Testing Conclusion Efficient test approach Test flow
- 27. 28 Efficient test approach Quick& Dirty What? • Basic testing of hypothesis • Basic materials/tools/budgets • Wide window • Unexpected phenomena When? • Prior to full test plan development • Sometimes during FFE Why? • Upstream testing reversed learning curve! • Variables and operating windows • Significance of these variables risk based testing • Reduced cost no expensive tools / instrumentation • Reduced timing short tests, short tooling lead times Increased test output with reduced risk
- 28. 29 Efficient test approach Quick& Dirty Examples Basic testing:
- 29. 30 Efficient test approach Quick& Dirty So why don’t they? • Corporate culture administration during testing • Availability tools & instrumentation • Downstream test approach • Used of testing known things or by known methods • Afraid to get hands dirty..? Hot potato issue! • Foresee time & budget for Quick & Dirty testing • Ensure basic equipment is at hand Low threshold for testing should be encouraged
- 30. 31 Quick & Dirty Testing TestPlan Development Detailed Testing Conclusion Efficient test approach Test flow
- 31. 32 Efficient test approach Testplan development 1. Gathering inputs • Desk search • Quick & dirty test results • Specification document • Risk assessment • .. 2. Define test actions List items according to function / module / .. 3. Define acceptance criteria Based on gathered information 4. Assign priorities to each test based upon risk/criticality Risk based testing Assign risks well thought
- 32. 33 Efficient test approach Testplan development 5. Define test methods based on: • Variables • Measurement method • Automatic vs. manual 6. Define setup & tools • Measurement method • Cost (rent vs. hire) • Location (EMC, Clima-chamber) 7. Estimate timing • Lead time setup & tools • Estimated testing time Test plan will give priority / budget / timing overview!
- 33. 34 Quick & Dirty Testing TestPlan Development Detailed Testing Conclusion Efficient test approach Test flow
- 34. 35 Efficient test approach DetailedTesting • Instrumentation Calibration & sanity check! Setting up instruments correctly • Test stand Designed to be versatile & adaptable adaptation on the fly for faster results • Testing Test order based on priorities (high to low) Reassess priorities/tasks as you go Eyes open for undefined phenomena & assess their influence test plan iteration
- 35. 36 Efficient test approach DetailedTesting • Documentation On the fly most efficient, less hassle for test engineer use templates! Take pictures store them for later use, small effort Need for computer at test stand + internet
- 36. 37 Efficient test approach Breadboardexamples More advanced testing • Use of data-acquisition • Use of rapid prototyping • Use of automation devices
- 37. 38 Quick & Dirty Testing TestPlan Development Detailed Testing Conclusion Efficient test approach Test flow
- 38. 39 Efficient test approach Testconclusion Test action acceptance • Fill in test plan acceptance fail/pass • Comment on possible fails • Identify show stoppers as soon as possible Feasibility Study Strengths vs. Weaknesses Opportunities vs. Threats Needed resources
- 39.
- 40. 41 Efficient test approach Conclusion Reducethe barrier for Quick & Dirty testing Lower threshold to start testing + getting a feel with the matter Define test window + parameters for more detailed & accurate testing Plan your testing based on risks start with highest priorities/risks first Reassess test plan priorities as you go Test plan helps you control: • Priorities / risks • Cost tooling/setup • Timings Perfor- mance Planning Risk managem ent Budget Keep 3 axes in balance ! Maximize output, reverse the learning curve Reduce overall risk Risk based testing
- 41.
- 42. 43 VERHAERT MASTERS ININNOVATION® Headquarters Hogenakkerhoekstraat 21 9150 Kruibeke (B) tel +32 (0)3 250 19 00 fax +32 (0)3 254 10 08 ezine@verhaert.com More at www.verhaert.com VERHAERT MASTERS IN INNOVATION® Netherlands ESIC European Space Innovation Centre Kapteynstraat 1 2201 BB Noordwijk (NL) Tel: +31 (0)618 12 19 19 derk.schneemann@verhaert.com More at www.verhaert.com MASTERS IN INNOVATION® is a platform set up by VERHAERT to train, stimulate and incubate you as an innovator. We provide an extensive training program with different tracks and covering critical areas of new products and business innovation. Furthermore we manage the VERHAERT venturing program and organize our Innovation Day, an annual conference on best practices and insights on new products & business innovation.
Editor's Notes
- #6 feasibility study is niet enkel ter bevestiging Zekere risico’s aan verbonden (past binnen kader van risk management op inno day) Risico’s bespreken en daarna een efficiënte aanpak voorstellen
- #8 feasibility study is niet enkel ter bevestiging Situering binnen NPD Zekere risico’s aan verbonden (past binnen kader van risk management op inno day) Risico’s bespreken en daarna een efficiënte aanpak voorstellen
- #10 Situering van de presentatie Conceptniveau presentatie handelt over innovatieprojecten(kader inno day / verhaert) focus ligt op technische haalbaarheid Technische haalbaarheid nagaan dmv testen (uiteraard niet enigste mogelijkheid)
- #11 link naar andere presentaties en algemene risk mgmt aanpak duidelijk maken
- #12 feasibility is not only looking at the feasibility (monitoring), but also about making the concept feasible (treatment) Snapshot uit de presentatie ‘risk training’
- #14 Innovatief project technical feasibility Uitleg geven rondom HQMS (aerotecs project) Last item usability Field demonstrator user centered
- #15 validation demonstrator in application (feasibility & usability & desirability check) verification BB or PoC (feasibility check) Over specification will have its influence throughout the entire project! Example: Fusionics (wrong specification leads to expensive instrument) Airbus 5µm accuracy possible, however, customer does not have a need for such a precision
- #16 Early stage cost reduction by changing concept has significant influence in overall project costs Reversing the learning curve by learning as much as fast as possible overall advantage!
- #17 You test what you really want to know Simulations are possible for partially determined cases with many variable changes Human interpretation also for testing, but without translation from theoretical case to practical Blister package: Determine in feasibility study Bullnett the break force of the blister NASA already has very detailed information regarding human capabilities!
- #22 renting vs. buying: FLIR camera for 1 day, gekalibreerde meettoestellen
- #24 wrong basic requirement: Fusionics (te zwaar gespec’t) Hot cold treatment: Temperature target 2 °C very demanding and not usefull for treatment New application based on ice packs => good and bad characteristics copied Airbus (té goed en daardoor te omslachtig) Borehole measurement 5 µm target very demanding , set by quality department and not usefull for end user (production)
- #26 Proof of concept => breadboarding Works like real => application demonstrator looks/works/tastes like real => product prototype
- #27 Test plan schrijven + uitvoeren => Iteratief proces test plan: Banksys: - loopt het mis indien… ? - wanneer loopt het mis? andere insteek
- #28 Test plan schrijven + uitvoeren => Iteratief proces test plan: Banksys: - loopt het mis indien… ? - wanneer loopt het mis? andere insteek
- #29 quick desk search parameters/eigenschappen/moeilijkheden leren kennen
- #31 low threshold nog steeds hoog in de praktijk: - corporate culture (veel administratie) - beschikbaarheid instrumentatie/tools - often downstream test approach uitstellen testen (enkel voor validatie/verificatie) - niet durven handen vuil te maken? Foto’s opstelling beneden: -
- #32 Test plan schrijven + uitvoeren => Iteratief proces test plan: Banksys: - loopt het mis indien… ? - wanneer loopt het mis? andere insteek
- #35 Test plan schrijven + uitvoeren => Iteratief proces test plan: Banksys: - loopt het mis indien… ? - wanneer loopt het mis? andere insteek
- #38 Link to sentence in beginning bad idea/concept show example.
- #39 Test plan schrijven + uitvoeren => Iteratief proces test plan: Banksys: - loopt het mis indien… ? - wanneer loopt het mis? andere insteek
- #42 Zo snel mogelijk leren door Q&D met daarop gevolgd een test plan op basis van priorities/risks zal het totale risico van een feasibility study beperken.