Your SlideShare is downloading. ×
0
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
BC MK-803 product use testing 3-15-12
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

BC MK-803 product use testing 3-15-12

478

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
478
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Steve Herbst SMDeveloper of Topical Storm software-a web-based outlining program that offerssuggestions to overcome writer’s blockThis presentation for MK-803:My product development story and what Iknow about Product Use Testing © 2012 Steve Herbst Confidential and Proprietary
  • 2. Cartoon by Barbara Smaller, The New Yorker 4/26/10© 2012 Steve Herbst Confidential and Proprietary
  • 3. Names don’t work that well compared to product.Google – we worry about the name … will the worldaccept … or Yahoo. Or Lotus. How silly. Lotus. Apple.And yet it’s the product. The product really, reallymatters. -Dan Bricklin, co-founder of VisiCalc -Interview May 7, 2004 © 2012 Steve Herbst Confidential and Proprietary
  • 4. My product development history 1998 2010 2011 Groundwork: Stay on target: Validation: NEXTReasoning software Education market Field test at STEPS runs on a PC goes crazy for Douglas School collaboration tools In Acton MA 2006 Inspiration: 2010 2011 Daughter writes Reality check: Enlightenment: crappy essay Customers Software is expect perfection Show biz © 2012 Steve Herbst Confidential and Proprietary
  • 5. My product development history 1998 2010 2011 Groundwork: Stay on target: Validation: NEXTReasoning software Education market Field test at STEPS runs on a PC goes crazy for Douglas School collaboration tools In Acton MA 2006 Inspiration: 2010 2011 Daughter writes Reality check: Enlightenment: crappy essay Customers Software is expect perfection Show biz © 2012 Steve Herbst Confidential and Proprietary
  • 6. Daughter’s 6th grade essay topic: Warm air causes hurricanes Using a Google search, she finds:Hurricanes start when warm, moist air from the ocean surface begins to rise rapidly, where itencounters cooler air that causes the warm water vapor to condense and to form storm clouds anddrops of rain. The condensation also releases latent heat, which warms the cool air above, causing itto rise and make way for more warm humid air from the ocean below.As this cycle continues, more warm moist air is drawn into the developing storm and more heat istransferred from the surface of the ocean to the atmosphere. This continuing heat exchange creates awind pattern that spirals around a relatively calm center, or eye, like water swirling down a drain.Done! © 2012 2011 Steve Herbst Confidential and Proprietary
  • 7. How doesWarm air warm aircauses Rule causehurricanes hurricanes ? © 2012 Steve Herbst Confidential and Proprietary
  • 8. Warm air causes hurricanesNoticing the pattern “A causes B”, the computer asks: How does warm air cause hurricanes? Evidence that warm air causes hurricanes What causes warm air? What do hurricanes cause? What else causes hurricanes? What else does warm air cause? Can warm air be used to predict hurricanes? Can preventing warm air prevent hurricanes? Do different kinds of warm air cause different kinds of hurricanes?And so on. © 2012 Steve Herbst Confidential and Proprietary
  • 9. Warm air causes hurricanesNoticing the pattern “A causes B”, the computer asks: How does warm air cause hurricanes? Evidence that warm air causes hurricanes What causes warm air? What do hurricanes cause? What else causes hurricanes? What else does warm air cause? Can warm air be used to predict hurricanes? Can preventing warm air prevent hurricanes? Do different kinds of warm air cause different kinds of hurricanes?And so on. © 2012 Steve Herbst Confidential and Proprietary
  • 10. Warm air causes hurricanesNoticing the pattern “A causes B”, the computer asks: How does warm air cause hurricanes? Evidence that warm air causes hurricanes What causes warm air? What do hurricanes cause? What else causes hurricanes? What else does warm air cause? Can warm air be used to predict hurricanes? Can preventing warm air prevent hurricanes? Do different kinds of warm air cause different kinds of hurricanes?And so on. © 2012 Steve Herbst Confidential and Proprietary
  • 11. Warm air causes hurricanesNoticing the pattern “A causes B”, the computer asks: How does warm air cause hurricanes? Evidence that warm air causes hurricanes What causes warm air? What do hurricanes cause? What else causes hurricanes? What else does warm air cause? Can warm air be used to predict hurricanes? Can preventing warm air prevent hurricanes? Do different kinds of warm air cause different kinds of hurricanes?And so on. © 2012 Steve Herbst Confidential and Proprietary
  • 12. Too much television causes ADHDNoticing the pattern “A causes B”, the computer asks: How does too much television cause ADHD? Evidence that too much television causes ADHD What causes too much television? What does ADHD cause? What else causes ADHD? What else does too much television cause? Can too much television be used to predict ADHD? Can preventing too much television prevent ADHD? Do different kinds of too much television cause different kids of ADHD?And so on. © 2012 Steve Herbst Confidential and Proprietary
  • 13. Too much television causes ADHDNoticing the pattern “A causes B”, the computer asks: How does too much television cause ADHD? Evidence that too much television causes ADHD What causes too much television? What does ADHD cause? What else causes ADHD? What else does too much television cause? Can too much television be used to predict ADHD? Can preventing too much television prevent ADHD? Do different kinds of too much television cause different kids of ADHD?And so on. © 2012 Steve Herbst Confidential and Proprietary
  • 14. Too much television causes ADHDNoticing the pattern “A causes B”, the computer asks: How does too much television cause ADHD? Evidence that too much television causes ADHD What causes too much television? What does ADHD cause? What else causes ADHD? What else does too much television cause? Can too much television be used to predict ADHD? Can preventing too much television prevent ADHD? Do different kinds of too much television cause different kids of ADHD?And so on. © 2012 Steve Herbst Confidential and Proprietary
  • 15. Product: An outliner that generates questions: We should focus on market share What evidence do you have that market share is important? What other factors are also important? Are there any unspoken assumptions behind this statement? What steps can we take to focus on market share? What is the purpose of market share? Who knows the most about market share? Who might disagree with this statement? How can we research this recommendation further? © 2012 Steve Herbst Confidential and Proprietary
  • 16. My product development history 1998 2010 2011 Groundwork: Stay on target: Validation: NEXTReasoning software Education market Field test at STEPS runs on a PC goes crazy for Douglas School collaboration tools In Acton MA 2006 Inspiration: 2010 2011 Daughter writes Reality check: Enlightenment: crappy essay Customers Software is expect perfection Show biz © 2012 Steve Herbst Confidential and Proprietary
  • 17. 2010November Learning (Boston)Conference: Building Learning Communities © 2012 Steve Herbst Confidential and Proprietary
  • 18. 2010High energy, very exciting. All these free tools: © 2012 Steve Herbst Confidential and Proprietary
  • 19. 2010Lesson learned: Hold steady to the original vision. Don’t rush to change the product in response to trends.Positive outcome: clarity Plenty of existing tools for sharing content. But there may be value in helping people create good written content to share. © 2012 Steve Herbst Confidential and Proprietary
  • 20. My product development history 1998 2010 2011 Groundwork: Stay on target: Validation: NEXTReasoning software Education market Field test at STEPS runs on a PC goes crazy for Douglas School collaboration tools In Acton MA 2006 Inspiration: 2010 2011 Daughter writes Reality check: Enlightenment: crappy essay Customers Software is expect perfection Show biz © 2012 Steve Herbst Confidential and Proprietary
  • 21. Reality check 2010Showed a civics demo to a history professor friend. Me: Isn’t this technology super cool? My program generates all these ideas with just a few powerful rules! John: This is bad grammar. And this question is vague. Teachers won’t accept it. © 2012 Steve Herbst Confidential and Proprietary
  • 22. Reality check 2010Lesson learned: Nobody cares how the software works, they want magic.Positive outcome: user focus Spare us the fancy “Artificial Intelligence” technology. Make it simple; hard-code the answers. Don’t mention “rules”. Concentrate on what people want from the product. © 2012 Steve Herbst Confidential and Proprietary
  • 23. My product development history 1998 2010 2011 Groundwork: Stay on target: Validation: NEXTReasoning software Education market Field test at STEPS runs on a PC goes crazy for Douglas School collaboration tools In Acton MA 2006 Inspiration: 2010 2011 Daughter writes Reality check: Enlightenment: crappy essay Customers Software is expect perfection Show biz © 2012 Steve Herbst Confidential and Proprietary
  • 24. Douglas School Acton MA May 2011 © 2012 Steve Herbst Confidential and Proprietary
  • 25. May 2011 Survey comments from 6th grade writing students: It helps writers that I would use this Yes I think it is a very have writers block program again successful website. and are unable to because it comes Could be a great tool for organize there ideas up with ideas that i kids writing essays. while trying to write would have not there essay or story. come up with.i really do enjoy filling inall of the checklists things I would definetly use a I would use this weband answering the program like this site agen becausequestions. i really think again because it really never befor have ithat topical storm is a makes you think about been able to make avery productive and fun what you are writing first draft of an essayway to write an essay. about. in a single day before. Thank you! © 2012 Steve Herbst Confidential and Proprietary
  • 26. May 2011Lessons learned: Sixth graders are awfully perceptive. They don’t want happy cartoon characters. They love the scoreboard. Make it Smartboard ready, touch-based. Simplify it beyond belief. Getting people to use the product is priceless. © 2012 Steve Herbst Confidential and Proprietary
  • 27. My product development history 1998 2010 2011 Groundwork: Stay on target: Validation: NEXTReasoning software Education market Field test at STEPS runs on a PC goes crazy for Douglas School collaboration tools In Acton MA 2006 Inspiration: 2010 2011 Daughter writes Reality check: Enlightenment: crappy essay Customers Software is expect perfection Show biz © 2012 Steve Herbst Confidential and Proprietary
  • 28. Software as Show Business (SASB) “I wouldn’t buy this – even for 5 dollars. But…” © 2012 Steve Herbst Confidential and Proprietary
  • 29. December 2011Early morning realization: Nobody wants this. © 2012 Steve Herbst Confidential and Proprietary
  • 30. “Enlightened persons who know the True Self work for the welfare of the world, unattached, ever helping to point humanity toward dharma (right action, living a truth-based life).” - THE BHAGAVAD GITA, 200 BCE – 200 AD© 2012 Steve Herbst Confidential and Proprietary
  • 31. Product Use Testing also called Usability Testinginvolves a task, for example:• make something with a tool• drive a vehicle• fill out a form• chop carrots• sleep in a tent © 2012 Steve Herbst Confidential and Proprietary
  • 32. Things to find out:• How quickly people do the task• How accurately• How long they stick with it• How they feel about the experience afterwards • Would they buy? Recommend?• How much they remember later © 2012 Steve Herbst Confidential and Proprietary
  • 33. MethodSimple:• Watch people use your product on a realistic task• See when they get confused or lose interest• Ask them for suggestions afterwards• Use this priceless information to improve the product• Testing is useful at every stage of development © 2012 Steve Herbst Confidential and Proprietary
  • 34. Conventional view of testing TEST HERE FIX © 2012 Steve Herbst Confidential and Proprietary
  • 35. Reality of testingSMALLTESTS… © 2012 Steve Herbst Confidential and Proprietary
  • 36. Test early & often• Test a mockup • For tangible goods: model, prototype • For software, focus test a “wireframe”: © 2012 Steve Herbst Confidential and Proprietary
  • 37. Testing in large organizations• Professional usability testers• Many test subjects• Fancy equipment – eye tracking, voice stress• Statistical analysis of results• Feh © 2012 Steve Herbst Confidential and Proprietary
  • 38. thinkeyetracking.comNielsen Norman Group University of Minnesota © 2012 Steve Herbst Confidential and Proprietary
  • 39. Simpler non-fanatic approach• A few test subjects• Realistic task• Listen carefully, take notes• Record the session if possible• Think, fix See: Steve Krug, Rocket Surgery Made Easy © 2012 Steve Herbst Confidential and Proprietary
  • 40. Step by step1. Define the goal of this test2. Choose a realistic task3. Select participants4. Prepare participants – expectation, motivation5. Pre-questionnaire? Understand the person6. Run the test7. Ask for reactions, suggestions8. Report results to the participants © 2012 Steve Herbst Confidential and Proprietary
  • 41. 1. Define the goal of this testFind out how to improve the product:  See where people get stuck  Gauge emotional reactions  Discover the most & least attractive features  Compare several products or concepts (“Use each of these machines to make espresso”) © 2012 Steve Herbst Confidential and Proprietary
  • 42. 1. Define the goal of this testNote:  A Product Use Test tests the product, not the participant  Task should not be overly difficult  Everything the user does is right; the product is wrong © 2012 Steve Herbst Confidential and Proprietary
  • 43. 2. Choose a realistic taskRealistic: Something that people need to do and that your product needs to facilitateExamples of tasks for a website:  Create a document  Make a purchase  Find the answer to a question  Sign up (how enticing is the lead-up?) © 2012 Steve Herbst Confidential and Proprietary
  • 44. 3. Select participantsWho:  People It takes only five users to uncover 80 percent of high-level usability problems. -Jakob Nielsen © 2012 Steve Herbst Confidential and Proprietary
  • 45. Why do the test subjects know better than us?They know very little about our product (just like all the people who will use it someday)They’re detached They’re focused on their own needs and couldn’t care less how much more work we have to do.They’re in a hurry (just like all the people …) © 2012 Steve Herbst Confidential and Proprietary
  • 46. 4. Prepare the participantsSet expectations, give motivation:  Credibility: Who is conducting this test?  How long will the test take?  If you will be recording, tell them  Guarantee confidentiality  Promise a reward: cash, iTunes card, free product or subscription  Ask them to sign a consent form  Let them view the test results later! © 2012 Steve Herbst Confidential and Proprietary
  • 47. 5. Questionnaire beforehandSample questions:  What’s your profession?  What types of tasks do you do?  (How) do you currently use this type of product?  What difficulties or needs do you have?  What other similar products do you know?  Blue sky: What product would you love to have? © 2012 Steve Herbst Confidential and Proprietary
  • 48. 6. Run the testWhile performing the task(s):  Ask them to think aloud:  What they are about to do  How they plan to do it  What step they are trying now  What they think a feature does  What surprises them  What questions they have  Why they want to kill you © 2012 Steve Herbst Confidential and Proprietary
  • 49. 7. Ask for reactions, suggestionsPost-interview or questionnaire:  Summarize this experience  What was difficult?  What was confusing?  What would you change?  You know, stuff like that © 2012 Steve Herbst Confidential and Proprietary
  • 50. 8. Report results to participantsThis is one of the strongest incentives to participate  Get contact info or post on a website  Report what most people liked & hated  Report any interesting ideas that came up © 2012 Steve Herbst Confidential and Proprietary
  • 51. Today’s Product Use Test is different• Everybody all at once• We can’t do think-aloud (unfortunately)• The program will save your work and comments, and I will look at them later• We’ll discuss when done• I will post a summary of the results © 2012 Steve Herbst Confidential and Proprietary
  • 52. Today’s Product Use Test1. Log in with your user name & password4. Pretend you want to write a college application essay5. Start by knowing nothing about the product3. See how far you get (that’s the goal of this test)4. Click this button to enter your comments:5. Whole class discussion at 9PMResults will be posted at: www.topicalstorm.com/BC/results This PowerPoint at: www.topicalstorm.com/BC/slides My email: steve.herbst@comcast.net © 2012 Steve Herbst Confidential and Proprietary

×