Embrace Uncertainty: Building a UX Process for an Agile World
Report
Tom IllmenseeFollow
Sep. 29, 2012•0 likes•1,792 views
Embrace Uncertainty: Building a UX Process for an Agile World
Sep. 29, 2012•0 likes•1,792 views
Download to read offline
Report
Technology
This talk was presented at Innovate Virginia on October 12, 2012 in Richmond, VA. A slightly shorter version was presented at The Web and Beyond 2012, Amsterdam. This set of slides replaced the shorter version on October 15, 2012. Handouts (which include references and a poster) are available by request.
Tom IllmenseeFollow
Recommended
T systems - agile workshop 0. sessionPéter Lukács
Design process week 1Julia Powles
Project Management Success: The 7 Pitfalls Every Project Should AvoidSPECengineering
Agile scrum-retrospectiveAgile Project Management
1st skill build assessment presentationsheffieldEnterprise
Design Studio Methodology: A quick why and howDaniel Naumann
More Related Content
Similar to Embrace Uncertainty: Building a UX Process for an Agile World
Build a Recipe for Better UX Process with Fresh Lean IngredientsTom Illmensee
BEST Practices - Testing & Optimization | Bredan RendanCaleb Whitmore
Designing the Unknown - IA Konferenz Berlin 2013Bogdan Stanciu
Research and Design for the UX Team of OneUIEpreviews
User Experience Design: 5 Techniques for Creating Better Websites and Applica...nForm User Experience
Agile Project Management Workshop.pptxAbhishek Prasoon
Recently uploaded
GDSC Cloud Lead Presentation.pptxAbhinavNautiyal8
Chandrayaan 3.pptxPrasunJha12
"The Intersection of architecture and implementation", Mark RichardsFwdays
GDSC ZHCET Google Study Jams 23.pdfAbhishekSingh313342
Machine learning with quantum computersSpeck&Tech
Safe Community Call #12.pdfLornyPfeifer
Embrace Uncertainty: Building a UX Process for an Agile World
- 1. Embrace Uncertainty Building a X Process for an Agile World Tom Illmensee |
- 8. Product < Process < Culture < Leadership Balanced Team Meeting, Chicago 2012
- 12. .
- 14. #29 Being process-oriented, not product-driven, is the most important and difficult skill for a designer to develop.
- 18. 2. Outputs, outcomes and goals
- 21. 4. Share, debate, and critique.
- 23. We reviewed frameworks from Lean thinkers.
- 24. We reviewed frameworks from Lean thinkers.
- 25. 5. Re-organize and re-sequence.
- 26. Think Make Check Personas Sketches Research planning Charter (who, what, why) Design studio Session facilitation Business model Prototyping Analytics Stories & scenarios Information architecture A/B testing Story mapping Content and copy Critique
- 27. The process was a prototype. So we tested it.
- 28. Trial run: “Save Jobs”
- 29. Think Monday Personas Charter (who, what, why) Stories & scenarios
- 32. Think / Make Thursday Stories & scenarios Revise prototypes Visual design
- 34. A/B testing
- 35. A B
- 36. Think Make Check Test plan Develop variant design Monitor data Requirements (variant design) Configure testing tool Review results Review results of previous tests
- 37. Product < Process < Culture < Leadership Balanced Team Meeting, Chicago 2012
- 44. Three things you should do with your team next week: 1. Review UX processes via interviews 2. Map each process on cards 3. Review, critique and revise as a group
- 45. Product < Process > Culture > Leadership Balanced Team Meeting, Chicago 2012
- 46. Gratitude Jeff Patton Marty Cagan Hugh Beyer Jeff Gothelf Janice Fraser Anders Ramsay Desiree Sy Stefan Klocek Eric Reis
Editor's Notes
- Hello. I’m Tom. I’m thrilled to be here at Innovate Virginia. What a great day! I lead the User Experience team at Snagajob, here in Richmond. My team has been working for the past 6 months to refine our UX processes, and today I’ll share with you some highlights from this journey.
- Before we get started, we have a small favor to ask. Each person should have a plain white index card. We would appreciate your feedback on this presentation. So please write down 2 things you like and 2 things we could improve about the talk. We’ll collect cards at the end.
- Who here has ever had a crappy piece of frozen pizza from a box? These “pizzas” are stamped out in a factory. There is little craftsmanship here – and the product tastes manufactured and industrial. The ingredients are extruded and squirted – and not very healthy. The cheese may or may not originate from a cow. You can’t taste much love. The quality of frozen pizza is inconsistent at best. The product is a reflection of the process used to make it.
- Who has made one of these homemade pies? The pizza dough comes from a can! The ingredients are combined on a cookie sheet and baked. A good word for this process is “shortcuts.” The results are a little better than the frozen options. If the cook uses some fancier ingredients, then you might taste a little more love in the final product but it’s not going to win any awards. Again, the product is a reflection of the process used to make it.
- The next level requires making the crust from scratch, preparing the sauce and selecting quality toppings. The cook uses a special technique to toss the dough. Pizzas are cooked in a special oven. This pizza tastes better than the other options. The product is a reflection of the process used to make it.
- Neapolitan pizza is on an entirely different level. Richmond, Virginia is also home to Stuzzi, among the few elite pizzerias in the US authorized to make D.O.C. – following the Naples formula. D.O.C. translates to controlled designation of origin – which requires that a food product be produced within a specified regionusing defined methods and that it satisfy a defined quality standard. Part artist, part scientist - the chef follows an elaborate procedure to make the perfect dough. The tomatoes are crushed by hand. The cheese is note grated, it’s torn and placed strategically around the pie. Pies are fired in a special oven at 1000 F in 60 seconds. The final product is sublime. The product is a reflection of the process used to make it.
- Many types of pizza. Many recipes. Many processes. The shortcut methods produce indigestion. The D.O.C. method produces rapture.
- At Stuzzi, the product reflects the meticulous process outlined in the recipes. This process is a reflection of the chef’s commitment to a culture of quality and craftsmanship. This culture is a reflection of the restaurant owner’s leadership and vision. Software follows the same path. As designers, the things we help produce reflect our processes. We cannot produce high-quality work with a weak process. And our processes reflect the cultures we help create within our companies and agencies. Culture reflects leadership and vision. This talk will focus specifically on the process area.
- Snagajob was founded in 2000. We’re a small software-as-a-service company focused on connecting hourly workers with employers, and we provide tools to help businesses manage human resources in restaurants, stores, hotels, and warehouses. Our products are good – over 35 million job seekers have accounts with us, and sales to employers are up year over year. But we want to be much better than “good.” Our UX team pushes for excellence.
- At Snagajob, product discovery and definition is the responsibility of a “core team.” Each team is lead by a product manager – who functions as the product owner. The PM works with an interaction designer and engineer to explore new product ideas, enhancements and features. The core team was responsible for evaluating value, usability and feasibility of each idea, and then describing validated items in backlogs. They built prototypes and validated these ideas with users ahead of the larger development teams.
- We recognized that our Agile culture was very good at getting products to market. But we also recognized that releases sometimes arrived a little “burned” around the edges – and sometimes with toppings customers didn’t really want. What do people do with toppings they don’t like? They pick them off. This happens in every organization. Even recently with Apple’s Maps. Part of the problem was that each of our product teams used different recipes, tools, and ovens. Our results were inconsistent because our processes were inconsistent. We devised a process to help improve our process.
- There were several challenges with this model: 1. Discovery and construction sometimes overlapped. This, of course results in wasted effort because the team might commit resources to build too soon. This is what happens when teams plan, design and construct all at once. Hazardous!
- 2. With as many as 6 product teams, product discovery experience and practice varied widely between teams. PMs with stronger project management skills were also stronger with delivery than with discovery. Right arms were stronger than left arms. Left arms were stronger than right arms. Agile principles facilitate efficient construction practices, but do little to support UX practices. This imbalance is easy to spot when teams spend more time prototyping solutions than defining problems. The key to delivering value is understanding what customers’ actual problems are.
- Let me take a moment to talk about the P-word. Why is process sometimes a dirty word in Agile settings? Jeff Patton told me once that process is sometimes the shield we reach for when things get out of control. It’s a way to slow…things…down. But there are some great benefits to being process-oriented. In his book, 101 Things I Learned in Architecture School, Matthew Frederick describes what being process-oriented means. Tell me if the following points ALL sound like they would work for Agile teams…
- Products are reflections of the processes we use to make them. Here’s what our team did to focus on improving our processes…
- This was our team: 4 interaction designers, 2 visual designers, an information architect, and me. From previous gigs each of us brought a collection of recipes and techniques. We were good cooks in the kitchen, but our styles were different. And since we all were shared resources across departments, we were all working on separate teams.
- Our first step was to outline our issues and assumptions about our UX practice. For example, we felt we invested significantly more time creating prototypes than doing foundational research and measuring outcomes. Another key assumption: a leaner process would help us eliminate wasted time and effort building things users didn’t value or couldn’t use easily.
- The second step was to articulate outputs, outcomes and goals. One example of an output: a sequenced list of tools, methods and techniques for interaction and visual designers. An outcome: a consistent framework for developing hypotheses and answering questions about our users. We had 3 goals: (1) APPLY our UX tools consistently across product teams, (2) INTEGRATE our UX process with product management and engineering processes, and (3) EVANGELIZE user-centered thinking and design within the company through a common vocabulary.
- So in order to assess our own knowledge we interviewed each designer about their teams, tools and techniques. We deconstructed our recipes to reveal our processes. The goal was to learn how everyone was working, how many people got mixed in the UX kitchen, what departments, and what deliverables (if any) were produced as a result.If you try our approach, we recommend using (2x4) cards to document each task and tool. They are easy to sort and re-arrange. Here’s Chip, our Visual Design manager, taking pictures of his process.
- When we were finished with the interviews we had 7 stacks with over 500 cards! Pictured here is the final mega stack.
- Our next step was to share and discuss our processes and recipes with each other. We organized a workshop away from the office and made a day of it. This was expected to be a session of fast prototyping followed by debate and critique.
- First, each designer presented his or her process. So cards made each story visible. We noticed gaps and overlaps. The good news was that nothing on the cards was unknown or surprising. We had all the building blocks laid out – the ingredients were pretty good. Here, Archie describes how interaction design is applied on his product team. The other interaction designers and visual designers look on.
- After the designers presented their processes, we discussed frameworks that could help us organize everything. We gravitated to Lean thinkers because of their focus on experimentation and emphasis on customer value. For example, Eric Ries makes a great case for building something small, measuring how customers and users respond to it, then making time to learn from the experience. Insights feed back into the build phase, are measured and learned from, until the team has established value, usability, and feasibility.
- Janice Fraser re-worked this model slightly to focus on understanding customers and users as a first step. From there, the team designs something they’ll validate. The loop repeats until the team discovers an idea that is valuable, usable and feasible. The team thinks in terms of hypotheses and problem statements instead of solution definitions. We experiment iteratively, quickly and with just enough design to answer research questions. This is the framework our team decided to model our process upon.
- At this point none of the designers saw the whole breakdown… Which we could barely fit on four tables. We learned through debate as we sorted process components. And with card sorting magic everything became manageable and surprisingly aligned.
- Much like a recipe for pizza dough, we sequenced our steps and organized our tools in a way that we hoped would produce better design outcomes. Here is how the interaction designers sequenced their tools and techniques. We recognized that we sometimes invested too much time in the “Make” stage developing prototypes. From this point on, we would try to balance our time and attention in all three areas.
- Just like any new recipe, the process itself was an experiment. We needed to validate and iterate.
- One team chose a relatively small piece of functionality from the backlog: a system to allow job seekers a way to save jobs to their account and apply later. The core team was comprised of a product manager, interaction designer, and lead engineer.
- On Monday we described our primary persona, then placed her at the center of a story in which the feature helps her accomplish a task. In the charter, we defined how we’d measure success, outlined requirements and dependencies, and articulated the business case.
- On Tuesday we sketched solutions collaboratively using the design studio technique. The team created 12 different ways to solve the problem. From there, we took the best ideas from the sketches and wireframed the flow on a whiteboard and quickly built an interactive prototype with Axure.
- On Wednesday we validated the first prototype with users. We do this via remote usability testing and screensharing. This photo shows a typical session led by the interaction designer and observed by the entire team.
- On Thursday we went back to the drawing board – changing the prototype based on what we learned the previous day. At this point we involved the visual designer to help put a bit more polish on the prototype.
- On Friday we validated our refined prototype with moreusers. The UX team also critiqued the design. Pictured here is one of the feedback sheets from the critique session. Our design principles served as lenses that help us evaluate the work.
- Another example: A/B tests. These are simple experiments where 2 or 3 designs are put into production at once and we measure to see which one performs best.
- The team wanted to know if jobs should be listed by company name or job title. We applied the Think>Make>Check framework to guide us through the experiment.
- Here’s how we used the process to facilitate the experiment. This also helped us introduce the new UX process to other teams and more engineers.Perhaps the best outcomes yet have come from these tests because the PM, UX and engineers collaborated and communicated consistently through the think > make > check loop. Even when the outcome of an experiment is failure, the thrill of building something fast, measuring it well, and debating the results and planning next steps make the whole effort worthwhile. We learn quickly what users do with the design (quantitative) and have a pretty good theory about why (because we also do usability tests). Morale suffers when the engineers (and UX and PMs) only experience “make.” Frustration develops fast when we skip over think and check. We stopped doing that.
- 6 months after we started this journey together, our team has a process that is malleable, a little messy, but made out of the right ingredients and combined in a better sequence.
- And our recipe and technique is getting better because we work on it every day.
- This is the napkin-drawing version of the discovery / delivery framework our product teams are using. Your mileage may vary. Here’s how this works: core teams grow the opportunity backlog based on ideas and inputs from stakeholders and generated through research with customers and users. The team prioritizes the backlog and chooses which ideas merit further study. Questions and problems are explored through multiple trips through the TMC loop. When the team feels like it has a solution that is valuable, usable and feasible, they describe the feature on cards and grow the product backlog. Ideas that need further validation (live data prototypes) are released in BETA, where they continue through the loop at a larger scale. The team understands that these live data prototypes will need to be validated qualitatively and quantitatively.
- We use our process as framework for performance reviews. We measure how and when we apply tools and focus on areas we should improve.
- We made our process visible and tangible in our projects and through consistent UX vocabulary. But process is not something we just talk about. We live it and do it. UX process has helped connect the team and bring renewed focus on customers and users.
- Our process is a work-in-progress. It’s not perfect. We invite discussion and debate. We let it change. We know it’s a recipe we will perfect only by trial and error.
- There are many good UX processes and recipes out there. Ultimately, we learned that our process had to be tailored to our unique culture. We had to roll our own.
- Three easy things you can do with your team this week.
- Of course, customers don’t line up around the block for a solid process. They want a great product. The process helps you get there. But a culture of quality and visionary leadership are also essential components. A logical, compelling UX process can affect these areas, too. Our experience is showing that it can help reinforce and even change culture.One of our product managers told us recently: “The processes you guys were able to implement changed our culture to being more product focused with an emphasis on validated learning.”
- There are many people who are working to understand the intersections of UX, Agile and Lean thinking. We have been lucky to work with Jeff Patton – his guidance and support have helped us tremendously.
- We appreciate your feedback. Please jot some notes on the cards.