Validating Medical Device Concepts with the Wizard of Oz
•
6 likes•1,552 views
The Wizard of Oz is an inspiration for interaction design within mobile phone UX communities. Medical device developers should also embrace wizard of oz methodologies. In this presentation I discuss how, by validating concepts early using interaction realistic prototypes, medical device developer have the opportunity to: generate innovation through interactions, optimise device usability and minimise use and development risk. As Medtec moves beyond devices, beyond treatment and beyond the hospital we are moving into an era of people centered, real time, connected and outcome driven wellness solutions. I argue that Medtec should move beyond the research and development of technology alone and into the research of interaction, to become, wizards of objects. #wizardofobjects
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (17)
Similar to Validating Medical Device Concepts with the Wizard of Oz
Similar to Validating Medical Device Concepts with the Wizard of Oz (20)
Recently uploaded
Recently uploaded (20)
Validating Medical Device Concepts with the Wizard of Oz
- 1. CONCEPT VALIDATION How, why and when to make and test interaction realistic prototypes Lucy Sheldon 3 June 2014 Medtec 2014 - Concept Validation © 2014 Cambridge Design Partnership LLP
- 2. © 2014 Cambridge Design Partnership LLP 2 THE WIZARD OF OZ AN UNLIKELY INSPIRATION FOR USABILITY AND INTERACTION DESIGN 3 June 2014 Medtec 2014 - Concept Validation
- 3. © 2014 Cambridge Design Partnership LLP 3 INTERFACES CREATING AN ILLUSION 3 June 2014 Medtec 2014 - Concept Validation Image attributed to: http://www.fda.gov/medicaldevices/deviceregulationandguidance/guid ancedocuments/ucm259748.htm
- 4. © 2014 Cambridge Design Partnership LLP 4 INTERFACES CREATING AN ILLUSION 3 June 2014 Medtec 2014 - Concept Validation ILLUSION
- 5. © 2014 Cambridge Design Partnership LLP 5 WHAT…… IS AN INTERACTION REALISTIC PROTOTYPE? 3 June 2014 Medtec 2014 - Concept Validation To Dorothy the wizard was real and so were all her experiences
- 6. © 2014 Cambridge Design Partnership LLP WHAT WHY HOW WHO WHEN 3 June 2014 Medtec 2014 - Concept Validation
- 7. © 2014 Cambridge Design Partnership LLP 7 WHAT…… IS VALIDATION? 3 June 2014 Medtec 2014 - Concept Validation * EU MDD ** USA FDA
- 8. © 2014 Cambridge Design Partnership LLP 8 3 June 2014 Medtec 2014 - Concept Validation A ONE OFF EVENT? Photograph attributed to: Billy Hicks: CC BY-SA 4.0 http://commons.wikimedia.org/wiki/File:Firework_photomontage.jpgc
- 9. © 2014 Cambridge Design Partnership LLP 9 Described Communicated Experienced More compact and more ergonomic than competitor devices WHAT…… IS A CONCEPT? 3 June 2014 Medtec 2014 - Concept Validation A CB
- 10. © 2014 Cambridge Design Partnership LLP WHAT WHY HOW WHO WHEN 3 June 2014 Medtec 2014 - Concept Validation
- 11. © 2014 Cambridge Design Partnership LLP 11 Define Design Develop V&V Launch WHY? GET CREATIVE Opportunity for innovation Effort of change Risk Aversion to risk 3 June 2014 Medtec 2014 - Concept Validation
- 12. © 2014 Cambridge Design Partnership LLP 12 WHY? CREATE FACTS AND EVIDENCE 3 June 2014 Medtec 2014 - Concept Validation Photograph shared with kind permission of CarolineLD. http://carolineld.blogspot.co.uk/2008/09/facts-not-opinions.html
- 13. © 2014 Cambridge Design Partnership LLP 13 WHY? ALLOW PEOPLE TO INTERACT WITH, AND CONTRIBUTE TO, YOUR VISION 3 June 2014 Medtec 2014 - Concept Validation Photograph attributed to: FutureShopMoncton CC BY-SA 3.0 http://en.wikipedia.org/wiki/Mapleton_Shopping_Area#mediaviewer/File:FutureShopMoncton.JP
- 14. © 2014 Cambridge Design Partnership LLP 14 WHY? CREATE A SHARED VISION 3 June 2014 Medtec 2014 - Concept Validation Image attributed to: http://wallpaperswide.com/
- 15. © 2014 Cambridge Design Partnership LLP 15 WHY? RELEASE THE TRUE VALUE IN THE USABILITY TEAM HF Carrot Because it creates better devices 3 June 2014 Medtec 2014 - Concept Validation HF Stick Because it satisfies regulations
- 16. © 2014 Cambridge Design Partnership LLP WHAT WHY HOW WHO WHEN 3 June 2014 Medtec 2014 - Concept Validation
- 17. © 2014 Cambridge Design Partnership LLP 17 HOW…… REMEMBER THE WIZARD 3 June 2014 Medtec 2014 - Concept Validation
- 18. © 2014 Cambridge Design Partnership LLP 18 HOW…… REMEMBER THE WIZARD 3 June 2014 Medtec 2014 - Concept Validation Who is your audience?
- 19. © 2014 Cambridge Design Partnership LLP 19 HOW…… REMEMBER THE WIZARD 3 June 2014 Medtec 2014 - Concept Validation What question(s) are you trying to answer?
- 20. © 2014 Cambridge Design Partnership LLP 20 HOW…… REMEMBER THE WIZARD 3 June 2014 Medtec 2014 - Concept Validation What is the most cost effective way to answer the question?
- 21. © 2014 Cambridge Design Partnership LLP 21 HOW….. EXAMPLE _ VITAL SIGNS MONITOR 3 June 2014 Medtec 2014 - Concept Validation Will continuous vital signs monitoring of breathing and heart reduce the time spent manually checking vital signs and increase the accuracy of triage?
- 22. © 2014 Cambridge Design Partnership LLP 22 HOW…. EXAMPLE _ VITAL SIGNS MONITOR 3 June 2014 Medtec 2014 - Concept Validation Multi casualty training exercise Paper and a pencil
- 23. © 2014 Cambridge Design Partnership LLP 23 Described Communicated Experienced HOW…… CHOOSE THE APPROPRIATE MEDIUM? 3 June 2014 Medtec 2014 - Concept Validation
- 24. © 2014 Cambridge Design Partnership LLP 24 Described Communicated Experienced HOW DESCRIPTIONS © 2014 Cambridge Design Partnership LLP 3 June 2014 Medtec 2014 - Concept Validation
- 25. © 2014 Cambridge Design Partnership LLP 25 HOW SKETCHES Described Communicated Experienced © 2014 Cambridge Design Partnership LLP
- 26. © 2014 Cambridge Design Partnership LLP 26 HOW STORYBOARDS 3 June 2014 Medtec 2014 - Concept Validation “Now I can concentrate on patient….. JONES “ AYW-0005-D-A Patient GODFREY‘s IPM alarm has sounded ….! “I think I have stabilised patient….. GODFREY “ “I think I’ll fit an IPM monitor to check….. “ GODFREY‘s breathing has increased ….! Described Communicated Experienced © 2014 Cambridge Design Partnership LLP
- 27. © 2014 Cambridge Design Partnership LLP 27 RR 20 HOW VIDEOS AND ANIMATIONS 3 June 2014 Medtec 2014 - Concept Validation RR 24 RR 28 RR 28 RR 28 RR 28 RR 28 RR 30 RR 30 RR 30 RR 33 RR 33 HR 120 RR 33 Described Communicated Experienced © 2014 Cambridge Design Partnership LLP
- 28. © 2014 Cambridge Design Partnership LLP 28 HOW LOW TECH MODELS 3 June 2014 Medtec 2014 - Concept Validation Described Communicated Experienced © 2014 Cambridge Design Partnership LLP
- 29. © 2014 Cambridge Design Partnership LLP 29 HOW HANDLING MODELS 3 June 2014 Medtec 2014 - Concept Validation Described Communicated Experienced © 2014 Cambridge Design Partnership LLP
- 30. © 2014 Cambridge Design Partnership LLP 30 HOW….. EXAMPLE _ INTENSIVE TREATMENT UNIT HUMIDIFIER FOR VENTILATION 3 June 2014 Medtec 2014 - Concept Validation
- 31. © 2014 Cambridge Design Partnership LLP 31 HOW SAMPLE INSTRUCTIONS 3 June 2014 Medtec 2014 - Concept Validation
- 32. © 2014 Cambridge Design Partnership LLP 32 HOW NON FUCTIONAL HANDLING MODEL 3 June 2014 Medtec 2014 - Concept Validation
- 33. © 2014 Cambridge Design Partnership LLP 33 HOW UNDERSTAND PEOPLE People are constrained by their knowledge of the world “as it is” 3 June 2014 Medtec 2014 - Concept Validation © 2014 Cambridge Design Partnership LLP http://commons.wikimedia.org/wiki/File:Prozac_pills.jpg
- 34. © 2014 Cambridge Design Partnership LLP 34 HOW 3 June 2014 Medtec 2014 - Concept Validation UNDERSTAND PEOPLE People tell you what they think you want to hear © 2014 Cambridge Design Partnership LLP
- 35. © 2014 Cambridge Design Partnership LLP 35 HOW 3 June 2014 Medtec 2014 - Concept Validation UNDERSTAND PEOPLE People feel that they are being tested, not your ideas © 2014 Cambridge Design Partnership LLP
- 36. © 2014 Cambridge Design Partnership LLP 36 HOW 3 June 2014 Medtec 2014 - Concept Validation UNDERSTAND PEOPLE People give criticism in context of the fidelity of the prototype or experience looks like the real thing? acts like the real thing? N Y N Y © 2014 Cambridge Design Partnership LLP
- 37. © 2014 Cambridge Design Partnership LLP WHAT WHY HOW WHO WHEN 3 June 2014 Medtec 2014 - Concept Validation © 2014 Cambridge Design Partnership LLP
- 38. © 2014 Cambridge Design Partnership LLP 38 WHO VALIDATES INTERACTION? 3 June 2014 Medtec 2014 - Concept Validation Described Communicated Experienced Marketing “How people choose” Usability “How people use” © 2014 Cambridge Design Partnership LLP
- 39. © 2014 Cambridge Design Partnership LLP WHAT WHY HOW WHO WHEN 3 June 2014 Medtec 2014 - Concept Validation © 2014 Cambridge Design Partnership LLP
- 40. © 2014 Cambridge Design Partnership LLP 40 WHEN NEW INTENDED USE? 3 June 2014 Medtec 2014 - Concept Validation © 2014 Cambridge Design Partnership LLP
- 41. © 2014 Cambridge Design Partnership LLP 41 Define Design Develop V&V Launch WHEN IS THIS ENOUGH? Usability and user experience Project Risk Described Communicated Experienced Summative Study Formative Study Research & Voice of Customer 3 June 2014 Medtec 2014 - Concept Validation © 2014 Cambridge Design Partnership LLP
- 42. © 2014 Cambridge Design Partnership LLP 42 Define Design Develop V&V Launch CONTINUALLY VALIDATE AND CO-CREATE Described Communicated Experienced Summative Study Formative Study Usability and user experience Project Risk Research & Voice of Customer 3 June 2014 Medtec 2014 - Concept Validation © 2014 Cambridge Design Partnership LLP
- 43. © 2014 Cambridge Design Partnership LLP 43 Define Design Develop V&V Launch CONTINUALLY VALIDATE AND CO-CREATE Described Communicated Experienced Usability and user experience Project Risk 3 June 2014 Medtec 2014 - Concept Validation © 2014 Cambridge Design Partnership LLP 8
- 44. © 2014 Cambridge Design Partnership LLP WHAT WHY HOW WHO WHEN AND FINALLY. . . . 3 June 2014 Medtec 2014 - Concept Validation © 2014 Cambridge Design Partnership LLP
- 45. © 2014 Cambridge Design Partnership LLP 45 AS MEDTEC MOVES….. BEYOND DEVICES 3 June 2014 Medtec 2014 - Concept Validation
- 46. © 2014 Cambridge Design Partnership LLP 46 Tom Varco AS MEDTEC MOVES….. BEYOND TREATMENT 3 June 2014 Medtec 2014 - Concept Validation Photograph attributed to: Tom Varco and licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.
- 47. © 2014 Cambridge Design Partnership LLP 47 AS MEDTEC MOVES….. BEYOND HOSPITALS AND LABS 3 June 2014 Medtec 2014 - Concept Validation Photograph attributed to: Tagisimon CC BY-SA 3.0.
- 48. © 2014 Cambridge Design Partnership LLP 48 BECOME A WIZARD. . . . OF NEW INTERACTIONS 3 June 2014 Medtec 2014 - Concept Validation UK OFFICE Church Road, Toft, Cambridge, CB23 2RF, UK Tel: +44 (0) 1223 264428 US OFFICE 228 Hamilton Avenue, 3rd Floor, Palo Alto, 94301, USA Tel: +41 (650) 798 5134 hello@cambridge-design.co.uk www.cambridge-design.co.uk THANK YOU Lucy Sheldon LAS@Cambridge-design.co.uk wizardofobjects Research and develop interactions, not just technology Validate concepts using low cost interaction prototypes Learn fast and learn cheaply Innovate & minimise risk © 2014 Cambridge Design Partnership LLP
Editor's Notes
- Who know…… The name of the experiment comes from The Wonderful Wizard of Oz story (1900), an unlikely inspiration in interaction design, is a story in which an old man hides behind a curtain and create the illusion that he is a powerful wizard. Until the point at which the wizard is revealved, all of Dorothy’s (the main character)s reactions were valid psychologically, anthropologically, and sociologically. To her the wizard was real ad so were all her experiences. John F. (“Jeff”) Kelley coined the phrases “Wizard of OZ” and “OZ Paradigm” for this purpose circa 1980 to describe the method he developed during his dissertation work at Johns Hopkins University. (His dissertation advisor was the late professor Alphonse Chapanis, the “Godfather of Human Factors and Engineering Psychology”.) Design methodology wherein an experimenter (the “wizard”), in a laboratory setting, simulates the behavior of a theoretical intelligent computer application (often by going into another room and intercepting all communications between participant and system). Sometimes this is done with the participant’s a-priori knowledge and sometimes it is a low-level deceit employed to manage the participant’s expectations and encourage natural behaviors. http://commons.wikimedia.org/wiki/File:The_Wonderful_Wizard_of_Oz_-_W.W._Denslow_cover.jpg
- It’s not just wizards who can make illusions, medical device manufacturers can too…… ….. By making models of interactions and experiences with a devices, to allow would be users to experience what a new device might be like to use ….. And this can be done, without actually bothering to design or make one! It’s the creation of an illusion and a testing procedure which allow you to discover how users interact with a product that doesn’t exist yet
- It’s not just wizards who can make illusions, medical device manufacturers can too…… ….. By making models of interactions and experiences with a devices, to allow would be users to experience what a new device might be like to use ….. And this can be done, without actually bothering to design or make one! It’s the creation of an illusion and a testing procedure which allow you to discover how users interact with a product that doesn’t exist yet
- By validating concepts using interaction realistic prototypes to fool your customers, dorothy! Does not need to…… work like the real thing look like the real thing Act like th real thing so that users and stakeholders can experience your visions(s) and so you can discover if your concept………. http://commons.wikimedia.org/wiki/File:The_Wizard_of_Oz_Judy_Garland_Terry_1939.jpg
- So, what the mobile and software industry has called “wizard of oz testing” I’ve rebranded for the medical device industry as “concept validation”
- Perhaps it shouldn’t just be left to the end
- And be done at other stages of development too? I believe we should validate concepts iteratively as they evolve so we can Prove that it is the right thing to do, kill it early, or learn enough to turn it into the right thing to do” http://commons.wikimedia.org/wiki/File:Firework_photomontage.jpg
- It can be described. Communicated – Feasible visions in the context of Constraints (technical/pipeline/roadmap, clinical, societal, market etc) .Or experienced - through an interaction prototype to discover whether the reality (the interaction) enables the value to be extracted And at each transition there is the opportunity for the value of a concept to be lost in translation or not be realised. But there any many costs effective ways of validating concepts ……
- Designers are not users Marketing departments are not user Engineers are not users Even human factors specialists are also not users In order to create facts about interaction, not opinions, based on what users want and need not what development teams think they might need – Validate concepts In a recent project CDP worked with a surgical device manufacturer to put non functional handling models in front of surgeons early in the design process. Eliminating requirements which were based on the assumptions of the development team and adding new, more important, actual user interface requirements were identified . This was done in parallel with technical R&D work prior to formal design control so didn’t add to the timeline of the project. Create traceable evidence for user interface requirements and customer requirements focusing the development resources.
- People can’t ask for what they don’t know is possible Peoples imaginations (and hence their desires, are bound by their experience), they accept inadequacies and deficiencies in their environment as normal Users are not designers, but by allowing them to interact and contribute to your ideas within the context of a well designed research programme they can contribute to your vision. http://en.wikipedia.org/wiki/Mapleton_Shopping_Area#mediaviewer/File:FutureShopMoncton.JPG
- Clearer device visions saving time and money through predictable and timely development programmes Arm the team with all they need to get the device designed and built in the right way: Clearly defined requirements Collective vision
- Oh, and one final reason to validate concepts early…….. The stick has driven usability in many organisations under the watchful eye and even direction of the compliance team Opportunity to get out more than regulatory compliance alone from usability processes and methodologies including; Delightful user experiences improve brand loyalty User centred devices and services improve outcomes and empower patients and carers Reduction training and support costs and these opportunities can only be realised if usability teams (and design teams) are involved in concept definition and creation through dedicated R&D of interactions.
- So how do you go about validating concepts…
- Until the point at which the wizard is revealved, all of Dorothy’s (the main character)s reactions were valid psychologically, anthropologically, and sociologically. To her the wizard was real ad so were all her experiences. The wizard did a good job of creating an illusion and in order for you to do so too you need to understand…..
- Fragmented decision-making in many healthcare markets makes it extremely difficult for companies to understand the requirements of all key stakeholders. To be selected for use, a device might have to be approved by a national or regional authority, selected by a healthcare provider, specified by a particular clinical team, and then chosen by doctors, often in consultation with patients. Finally, it may be the patient’s , nurses if clinicans own reactions to the device that define its success in use. Each of these stakeholders will have an incomplete picture of product attributes: payors might not understand the importance of usability in patient compliance, while a physician may be unaware of the ongoing cost of supporting a product in the field. As a result, the incentives to purchase in many medical device markets may be fundamentally different from the benefits ultimately enjoyed by end users. Depending on what your application is, you will have to decide which stakeholder take priority in validating ideas and interactions and sometimes it will be more than one and not just users. While some stakeholders don’t interact with a device directly , by letting them interact with your prototypes you will learn which features may help sway the decisions of the gatekeepers to the sale of your devices.
- Does this solve real problems can people use it to solve those problems‘ Is value perceived Is timing right which ideas are strong and which are weak are there any barriers we hadn’t identified,
- But, the key creative leap in concept validation is the design of……… Remembering that the fidelity of the experience is NOT reflected by the fidelity of the prototype Use anything to conjure up the experience, faking devices and scenarios is far cheaper than putting a device in the market
- We wanted to know
- And the most cost effective method to answer this question is through a standard multi casualty first response training exercise. And a piece of card where vital signs and trends can be updated to a script and the effect on the time spend monitoring vital signs and triaging accuracy can be evaluated You arrive at the scene of a road traffic accident. You are the first person on the scene with any medical skills. On initial assessment you notice there are two cars involved. One vehicle has caught fire with the other vehicle ten metres away with significant damage to the drivers side. All the passengers have moved from the scene. The injured people present you with signs and symptoms. Work as normal using either only smart triage tags or smart triage tags and an IPM to each patient:
- A key component in validating concepts is choosing an appropriate medium
- They can be solution agnostic or describe some of the solution. Either way, ultimately the solution is envisaged by the user or stakeholder as they try and answer the question. They may be tainted by their inability to envisage how this could be achieved. Or the solution they imagine, may be technically over, or under optimistic and tainted by their experiences with other devices and products. How much can you trust feedback to this stimuli?
- Validate concepts as a realistic vision in the context of constraints as you see them (technical, workflow, societal, experience) This can give you verbal feedback on reality of a communicated device vision and might help you decide whether it is worth investing in a technology further….
- Validating workflow in the context of a set of use scenario Allows feedback on reality of scenario and enables you to explore further scenarios
- To increase the fidelity of a visual experience, you could use videas and animations to detail important elements of the use process. Always remember that the fidelity of the experience matters, not the fidelity of the prototype. Here’s an example of how we communicated an interface concept of the patient vital signs monitor.
- These mediums of description and visual communication have their place, but don’t allow hand on interaction…… Which risks the Lots of opinions, which may be unfounded or irrelevant when a real product is in their hands in ralmuse scenario. Use a low techn interaction model so the participants who are experiencing a solution using more than sight and responding in real time to a device interface. The fidelity of the experience matters, not the fidelity of the prototype
- Validating whether the reality (the interaction) enables the value to be extracted safely (in a realistic scenario). Can the value of the technology and all it’s features be extracted through the form of the device and it’s interface?
- We wanted to know
- Next we took sample instructions to nurses
- Along with non functional handling models. One of the funniest insights we had during this study was when the nurse can it, looked at our device and tried to sit on it. Without putting the device in context with real users we would never had thought that our device displayed the visual appearance of a comfy seat for tired nurse legs and realised that either a design requirement of strength or visual messaging communicating discomfort would be needed to ensure device robustness.
- The final how is around understanding the problems with people. People are constrained by their knowledge of the world “as it is” and some may not be more or less able than others to understand designs that change the status quo and see the future clearly. Think about how the interaction might be different to existing experiences (with device or other products) design concept validation studies accordingly http://commons.wikimedia.org/wiki/File:Danpatch1.jpg In a recent test of a connected device we were careful to take feedback in context of the users experience with smartphone technology and progressiveness with new ideas.
- Hunt for honest opinions. One technique I use to do this is to ask participants to put themselves in the shoes of other users and stakeholder to get a more honest view. For example, how do you think less experience surgeons might feel about this technology. What kind of mistakes or assumptions might they make, what about your assistants. http://commons.wikimedia.org/wiki/File:P%C5%82atek_ucha.jpg Asking people to speak for others gives them an outlet for being more honest of recalling what use of a device would be like when they were more naive with a technology.
- With interaction prototypes there is always the risk that people can’t work out how to use the device and then feel like they are being testing. Make sure people know they are testing the device and not them And also look for non verbal communication relating to stress at trying to achieve a task. http://commons.wikimedia.org/wiki/File:Richard_Huish_College_Exam_Hall.jpg
- The key is asking why and probing in order to gain the insights to a level that a development decision can be made from it. For example, when testing a medical device concept I’ve had feedback from nurses before where they said. I wouldn’t use it. Why not. It doesn’t look sterile. If it did look sterile and was packaged, approved and labelled as sterile then how would you feel? Yes, I’d use it! I don’t think the nurses will like it. Why not? It’s too big to move around. How small would it need to be to make it appropriate for nurses to move around do you think? The size of a laptop would be good because it could easily fit in a storage cupboard when not in use. Can you show me the cupboard>? Yes. Great.
- At the start of the development process marketing are concerned with “how people choose products” Once a design is in development usability professional will get concerned with validating whether people can use a device safely and efficaciously In between there is the opportunity to validate interaction, iteratively, prior to design freeze optimise usability and minimise risk associated with user interfaces And it’s a low cost exercise, low/no tech prototypes access to small sample sizes representative users and stakeholders (5-8 from each grouo)
- Reduce risks and optimise value and outcomes through usability and user experience IT PAYS FOR ITSELF
- Services and solutions to improve outcomes or drive costs down
- Prevention and remote monitoring to improve outcomes more efficiently through real-time management of health conditions http://commons.wikimedia.org/wiki/File:Prozac_pills.jpg
- Enabling patients to manage their conditions at home and in the communityb without frequent, costly interventions Have a heart, have a brain and have the courage to…. http://en.wikipedia.org/wiki/University_College_Hospital#mediaviewer/File:University_College_Hospital_-_New_Building_-_London_-_020504.jpg