More Related Content
Viewers also liked
Viewers also liked (12)
Similar to Will schroeder racing with the clock 5 25 11
Similar to Will schroeder racing with the clock 5 25 11 (20)
Recently uploaded
Recently uploaded (20)
Will schroeder racing with the clock 5 25 11
- 1. Streamlining the Design Processby RACING WITH THE CLOCK Will Schroeder MathWorks
- 2. Your therapist will tell you … that the most important part of the hour is the last five minutes
- 3. What can we accomplish in two hours?
- 4. Choose Teams 4: Good size for a rapid design team
- 5. Set The Task The Use Case is the test task
- 6. Design Design time: 5minutes
- 7. Show and Tell Show and Tell 10minutes
- 8. Fix and Repeat 2X Keep strict time
- 9. At the break: Teams formed Design task introduced and explained Functional prototypes of 3 approaches Two design reviews for each design Hour #1
- 10. Prepare for Test Prep for test 2X
- 11. Test Test 2X Rotate and Test
- 12. Test 2X Rotate and Test
- 13. Test 2X Rotate and Test
- 14. Debrief 2X Debrief
- 15. In the second hour: Prototypes polished for testing 3 usability tests – for each design Users gave comparative feedback Developers, observing, absorbed all this Hour #2
- 16. Feedback was good ... Participants: “Having artifacts [designs and supporting materials] to go back to was very valuable.” “Seeing designs is better than simply recording them in a spec.” “I like including users in collaborative design.” “No time to over-think solutions.” “… the opportunity to view the ideas of other groups and borrow ‘guilt free’ was a good thing.” Developers: “… we had done quite a bit of design work on this feature” “… pleasantly surprised with the designs.” “… validated our design (there were many similarities), came up with new ideas, and exposed a design issue ….” “… created and explored more design ideas in minutes than we had come up with in a month.”
- 17. Have you heard the one about the talking dog?
- 18. One short year later ...
- 19. Still perfect team size 2X
- 20. Nature abhors a vacuum People LOVE their own ideas So they add unnecessary requirements The facilitator describes a problem Each team makes its own list of specific instances of the problem Then they chose one to seed their design Later some teams chose to cover more cases Giving pre-work failed Stern speeches failed Handing out lists failed 2X Starting a design task is very difficult
- 21. 2X Design always works out fine – 5 minutes is plenty
- 22. Show and tell enhanced the designs 2X
- 23. Not getting the full benefit Could go on longer (under control) Participants give notes – even draw ideas Facilitator prompts the explanation, orchestrates the design critique ... and keeps up the tempo 2X
- 25. Tests work fine with minimal or no preparation2X Wastes time, dilutes effort
- 26. Rotated users gave astounding feedback 2X
- 27. No scouting, no show and tell, not enough revision Participants did not see all tests ... And they did not take notes ... So there was not enough feedback And there was no “fixing” between tests One user testing all designs is about the useful limit Multiple rounds of testing requires more than two hours 2X
- 28. Make recording part of the process - List of pains - Keep and number sketches Take notes in show-and-tell Facilitate debriefing 2X No formal process
- 29. IN A NUTSHELL PIECES Players chose goals Blitz design sessions Show and tell Parallel testing Iteration No breaks GLUE Facilitation Time Pressure Structured activities For timing For feedback Levels of design
- 30. Getting the right design and the design right, MaryamTohidi, William Buxton, Ronald Baecker, Abigail Sellen- April 2006CHI '06: Proceedings of the SIGCHI conference on Human Factors in computing systems Questions? Will.Schroeder@MATHWORKS.COM
Editor's Notes
- Good afternoon - I’m Will Schroeder of the MathWorksI’m going to talk about combining brainstorming, rapid prototyping, and usability testing into a process that takes a small feature from design exploration all the way to where it is ready for wireframes Why bring all of these techniques together? Information transfer between processes is inefficient and wasteful Stopping and starting consumes time and energy better used for design Scheduling constraints make it difficult to march a team through three separate activities together – or even to keep a team together for that longWhen I put it that way, I’m hoping you will ask: “Why not put them all in a room, lock the door, and not let them out till they are done?”Well ... Why not?It’s pretty clear that two things need to be added if there is any hope of making this idea work:Structure. The participants need to know at every moment what to do – and what to do next. And reminded of this as frequently as necessaryA timekeeper to keep the process moving.Which means that this idea for streamlining the design process hasn’t a prayer without a facilitatorBut why are we racing with the clock?Two reasons: (besides, of course “Time is Money”)First, managers tend to measure value by time spent. For any new process to make headway against what seems to work fine already, it not only has to be better, it has to be quicker.Second, deadlines and imposed time management have beneficial effects all in themselvesAs Samuel Johnson said, "Depend upon it, sir, when a man knows he is to be hanged in a fortnight, it concentrates his mind wonderfully." So let’s talk about time compression a moment(SLIDE)
- The idea here is that the patient spends 45 out of 50 minutes fidgeting, fussing, denying, evading, and going off subject, and finally “gets it” in the last five minutesSurely some of that 45 minutes could be eliminated, but we don’t know how muchSo how much could we compress the design process?(SLIDE)
- I presented this scheme here a year ago, and we have been working with it and improving it since.I’m going to run through the original scheme quickly to give you the basic ideasThen I'll go back and tell you about what we have learned since.
- I chose teams, of course – we haven’t time for democracy.Teams of 4 worked well –so we stuck with that.We spread developers, quality engineers and usability specialists across the teamsI brought a lot of different prototyping materials, not knowing which ones people would want,Hoping that the variety would be inspiring – but they mostly sketched on 5” x 8” index cards
- We gave them two tasks (use cases) to guide their design, and screen shots of the existing GUI to work inHere’s an example of the scale of project we began with - for reference.Imagine a widget added to the Outlook GUI whose purpose was to control the announcement of arriving emails. It could be turned off and on, filter messages by type (rule), and control the noise (threat) level of what was announced.There would be a level-setting task and a filtering task, and the widget would need to find a place in the copies of the Outlook GUI we handed out
- Each team prototyped a concept– IN FIVE MINUTESThis is the part that ALWAYS works - it has always worked from the beginning.When they get ten minutes, they waste the first fiveAnd teams always produce something that bakes well in Show and TellThis is the core of the process – the remaining steps serve to firm up the concepts and match better them to the task goals
- Show and tell was the big surprise (for me, anyway) in the first sessionsTELL WHAT THEY DOIt’s amazing to watch people DISCOVERING how their design works (or doesn’t work) as they describe it to othersThis phase always produces lots of great feedbackShow and Tell requires structuring to ensure that All ideas are heard The session doesn’t lose momentumStructuring was one comment per person, picking the next person to talk (if necessary) and bringing show and tell to a close at the right time.
- Then they take what they have learned back into design, Fixing was enthusiastic: Holes in the design were patched Ideas were “borrowed”– and the first hour ended on a high note with a second show and tellWith each team eager to explain how they had improved their design, or resolved problems
- Why did we break?For two reasons:1) this was (at first) a kind of skunk works project – we did it at lunchtime so as not to appear frivolous with company resources.2) I was also concerned as to how long such a group could perform at top level, driven constantly by a facilitator.
- We began the second hour by preparing prototypes for testingThe clever ones among you will note that the teams are actually repeating work that they did – or should have done – and the end of the first hour – but they were also working out how they would test
- We ran three tests in parallel, with just three users who rotated through the designs
- If you’ve never heard the kind of user feedback you get after testing three designs aimed at the same tasks, you have a treat coming.I have always had my reservations about the think-aloud process in a single-user, single-design test, but when users compare different designs as they test them in sequence, the quality and value of the feedback reaches a whole new level.
- I got this idea from a paper given at CHI Montreal by MaryamTohidi – a student of Bill Buxton’s.I had been itching to try it ever since, and have learned that this is just about the only way to bring it offI have never been able to organize/schedule conventional testing this tightly.
- This left us with five minutes to debrief – which was not enough. Participants had to report design feedback after the sessionAlthough we were very excited at first about how much we were getting ...We realized that we were also leaving a lot on the table
- And so that was it – real time about 100 minutes from beginning to endSo how did it work?
- (Take a moment, let them read, take a drink of water)So this all sounds pretty good, doesn’t it?Time for a thought experiment
- Well, suppose you meet up – for the first time - with a dog that talks Not only does this dog talk, he talks really fast – like an auctioneerAlthough you have seen plenty of fast-talking people, because you have never seen a fast-talking dog beforeYou are probably blown away by this suave canineIt may be quite a while before you ask yourself whether what the dog said made sense or notThe process worked – it did not crash and burn – but was that beginner’s luck?How good was it REALLY?(SLIDE)
- For the past year, we have been paying close attention to what the dog is saying..We have done three successful real-time projects (not lunch-time) and We are wiser. I’m going to go through the process again step by step and:Talk about problems we encountered (and mostly circumvented or made go away)Improvements and new ideas which worked betterAnd a few ideas we haven’t tried yet.
- Four still seems to be the ideal team size. Smaller teams work, but are a bit lean on ideas and don’t revise as eagerly.Larger teams almost invariably leave one person sitting on the sidelines.There are “natural” roles for a team doing this kind of work, if the task is set properlySometimes people just fall into these roles (time pressure works wonders)Other times the facilitator must nudge them
- Getting everyone started from the same place is hardWhat’s supposed to happen in this (brief) time is that the team agrees to and focuses on a design objective which they all understand equally well, and which they proceed to solve together.(CLICK)What actually happens all too often is that instead of solving the set design problem, people will try to drive the design by imposing additional requirements. Does this sound familiar? Don’t think because the time is short that this kind of thing can’t happen.(CLICK)This is a tough problem. I tried a number of things that didn’t work. Everything top-down failed. Against my will I was forced to institute democracy.In the time I had originally given myself to speak, I had the team describe problem/solution scenarios using the widget, such as “”Working on an important letter, screen out everything that’s not vital.” or “Messages from sales are not getting through.”
- Design is fascinatingly effective again. If you can get them all to start from the same place, people do this REALLY WELL. There is a bounty of good ideas, just as in brainstorming. Now we must make the most we can from these ideas
- Show and tell is a natural way to develop conceptswhen the ideas are explained to someone who is not one of the originators.The implicit part of the design (the team’s assumptions) becomes explicit
- Show and tell brings immediate refinement to concepts with the requirement that they be explained to a third party.Things implicit in the design become explicit.(CLICK)It works immensely better when the facilitator orchestrates it. Set examples with positive comments, encourage people to give critiques in sequence, one idea at a time, prompt individuals to fill empty spaces, move things along(CLICK)It’s hard to forecast when the last really great idea is going to surfaceParticipants don’t digest fully what is going on (the group is now too big)(SLIDE)
- Breaks really don’t work(CLICK)Also, “Preparing for test” is just a waste of time. (CLICK) Testing works fine with little or no preparation.
- Interactive testing fleshes out the next level of detail – the details of interactionComparative testing generates a tremendous amount of valuable information about the details of the three designs(SLIDE)
- And that’s another problemWe are producing feedback a lot faster than we can consume it(CLICK)We tried to do too much – we wasted effort and missed too muchThe data we produced did not get ploughed back into the design work “while the iron was hot”(CLICK)To make full use of iterative parallel testing we need more time(Two hours should just about do it)
- A relaxed debriefing session afterwards might be pleasant, but recording data from memory is not efficient, and can be misleading.We have begun to build more debriefing into the process, and sequence the process to support better note-taking.