If you have been to other workshops you should understand that no amount of time really enough time. We’ll only have time to simulate exercises that take hours or days to do. So, I’ll talk a bit, you guys will work a bit, and then I will interrupt you just as it gets interesting, talk to you more, and so on. There are 4 exercises… I think... So I will stop you from doing the exercises, so we can talk about the next phase. I have arranged it so there’s no absolutely critical last lecture. If we run out of time, so be it. We’ll get through what we can as I want you to try some of this stuff yourself. … A couple years ago, this…
… is what everyone meant when they said “ecosystem.” But that was a passing fad. It’s never been really true, and Apple certainly didn’t invent the ecosystem.
Everything is an ecosystem, and good designers know this. Not just digital things either. The world is a complex, inter-related system, and physical products—especially information products—have been multi-channel for decades, at least. Newspapers and magazines are designed not just to be readable, but to be appealing to the newsstand. And, to have room for mailing labels for home delivery. Print is, in some ways, responsive.
Today, a popular magazine sells around 3 million subscriptions a week. Well, from the 60s or so through the 1990s, TV guide was vastly, vastly popular. They sold 20 million copies a week of this [IMAGE]. What is all but a database dump you pay for, and which is also full of ads. You can see why the Internet did so well. The guys who actually gathered this data realized early on they didn’t make a magazine though. They had a data product. And when the first EPGs came along, the TV-Guide channel that is the first thing that came up when you turned on the early Cable TV systems, the data they had was all ready for it. They had already been storing short and long descriptions (reruns within a week don’t get the full description, or maybe space is an issue), as well as the concept of meta-data well before anyone called it that. Derived, with some improvements for history, from http://karenmcgrane.com/category/content-strategy/ though I saw it presented instead.
Simplifying the story some, what with mergers and acquisitions over the decades, the content we see today on our much more high tech cable, satellite or streaming systems is not just the same basic format, but in some cases is the SAME EXACT CONTENT. There’s no need to write a new description for that Lawrence Welk show every time it airs, so the original 1979 description can still be pulled from the database and used. It’s all data.
OKAY,I am not going to keep talking all day. We’re going to try this out, and you will group up into teams to design your own little digital, multi-channel product. You will probably be approaching design from whatever way you are used to that meaning. Database design. Software design. Process design. But I am often going to show you stuff that looks a little different. I’ve been all sorts of things, like a FED, and a bad DBA, but you’d probably most identify me as a UX guy now. Which means design to me is something else. If we mean different things by design, how can we work together? Well, I don’t think we mean different things.
UX design is about building systems, too. We just include users as part of the system and acknowledge that they have quantifiable needs and constraints. That’s crucial to understand. Because users are the least-changing part of your system. The ones most prone to error and inconsistency, but also the one least easy to force into any particular behavior.
Making your product futureproof is about understanding the users. User centric design requires backing up, approaching problems fresh, really from the user’s point of view, and then considering what is really unique and powerful about the platforms that you are led to. It is pretty much platform-agnostic, and today with all these devices around us, we’re beginning to be freed from constraints of platforms. We are on the cusp of being able to assume the technology exists to do what we need, and we can approach this from the real-world point of view, from the user and business perspective alone.
Cities are already wired up, with utility and roadway monitoring, and stores tracking inventory, and you, at increasingly real time speeds.
People are wired. We all carry a handset, I have a smartwatch, and how many of you wore a fitness tracker at least a little in the last few days. I won’t ask you to reveal yourselves, but almost everyone with a biomedical implant (from hearing aids to insulin pumps) in the last 5 years has a wirelessly-connected device inside their body.
These are increasingly visible. This is a wired sidewalk. Okay, it’s just projected onto the sidewalk, but this is real. Subway information is projected onto New York sidewalks as a sort of ambient information radiator to remind pedestrians of their travel options. How can and will your data be repurposed, now or in the future?
Thinking of systems means thinking about how information works agnostic of channels. It means thinking of Information Architecture, data architecture, even storage technologies and what’s in the API, first. Think about some really successful products. Twitter, which is SMS based, but you use as a Website, as an app on your computer or handset, or tablet… Or take Evernote. Who used some competing tool to keep track of stuff like that a few years ago? SimpleNote, Springpad, WizNotes, Google Keep, FetchNotes, Onenote, NotionNote, SimpleNote,… ? Well, most of those disappeared or are much smaller. Why? They were great websites. Or apps. On iOS only. But Evernote is a platform, more than anything. They built not a website or an app but a product around people’s needs to capture, store and retrieve. Their platform even has a name (Trunk) and teams work on that as a product in and of itself. Thinking of their product as a service meant they could move into new platforms immediately, without re-thinking their business model, or re-architecting their systems. Being user-centric means they made decisions good for their product, and their business instead of what is comfortable or easy to build.
Some people have trouble moving past TheWeb always equaling The Internet. We use http for lots of data transfer, and what with webviews almost all apps are actually a little hybrid.If that helps you, just think of it this [QUOTE] way. At least in digital output, your Web content is going to be re-used in many, many ways, not just on your Web Site.Content is how many of us live and breathe. If you aren't asking hard questions about your content strategy, and that means reuse, tagging, promotion, archiving and use in multiple channels, you are going to die.Quote from here: http://daringfireball.net/2014/04/rethinking_what_we_mean_by_mobile_web
Even if you find it hard to think you have anything to do with traditional publishers, you do. This leaked strategy report from the NYT is FASCINATING. You all want to take the time to read it, or at least the analysis of it that’s actually at the link here. Write this down. Read it tonight. Full article link: http://www.niemanlab.org/2014/05/the-leaked-new-york-times-innovation-report-is-one-of-the-key-documents-of-this-media-age
Okay, how do we do this? How do you move from “Software Driven Design” to product-centered, or user centered design?
Things change, but keeping it simple and for now this is a good set of touchpoints to hit on to make sure you are designing for users, and for use on every platform and interface needed. Today, we’ll treat it as a process, and step through it one by one, but many of the steps are done over and over, repeatedly, or constantly.
This is a workshop. We’re going to do actual hands-on work, to start the design of a digital/physical product of some sort. And we’re going to do it by teams. BREAK UP BY TEAMS. 5-6 ON A TEAM. PM/PO + DEV + UX + ??? WRITERS? VIZD?Instead of giving you something, I want you to come up with the product we’re going to design. If you are working on one, tell your team about it. If not, or you /really/ cannot share it, then everyone throw out ideas. Remember, you can just hate your smartwatch or smart thermostat and have a better idea. We can explore existing niches. Do that now. Give an elevator pitch, no more than about 30 seconds, of your product idea. More than one is fine, then the team votes.
What did you do while deciding on your projects? One of you talked, and the others listened. You asked questions, I hope. You had a conversation, but you didn’t really spend time designing. You didn’t draw (I hope???). Keep doing this. Even when you have a week, instead of two minutes, don’t get ahead of yourself. Listen, ask, listen more. Research, read, analyze. I came to this not from some philosophical understanding, but from failing in the distant past. I am one of those “genius designers” (as Jared Spool described it more or less in an article, though I hate that term) who did really well actually interrupting explanations and coming up with solutions on the whiteboard. But, when I stop, listen, and think, I always come up with better solutions. I have done both ways at places with robust success measures. It doesn’t just make you more friendly and popular, but yields better outcomes. Jared Spool article: http://www.uie.com/articles/five_design_decision_styles/
If you get a team together who have been working on similar products, or have been talking to customers on the first or failed version, they have a lot of knowledge in their heads. Annoyingly, it’s inside their heads. And people are bad at analyzing and sharing really. But, we can use this technique to extract the information and find really, really interesting answers as a result. And, quickly. Ideally, we have a couple hours, but you can do something in a few minutes. Which is all the time we have.
These are some questions I like to ask, though I don’t always find them by this process. If some like “what is the product” seem simple, I have never asked this of a team — even a team three years into a project — without getting multiple answers. You have to know what you are working on, and make sure everyone is on the same page.
Incidentally, you can gather this without getting everyone in the room. Send out emails, internet surveys, or get people on calls. http://shoobe01.blogspot.com/2012/02/client-questionnaire.html
But we’re going to do it live. And we’ll shorten it. Let’s just answer two questions- What ONE problem does it solve. - How is it solved today?Everyone write that answer on a post-it, and put it on the whiteboard. Now I said ONE, but I mean one per Post-It. Have more ideas? Write two. Or five, or fifteen answers. That’s fine. Start doing those now, as I keep talking at you. Assume that you are the project team, and everyone’s answers are valid, so whatever you got from the elevator pitch earlier is what the project is. 20 MINUTES IF POSSIBLE.
Mark data/systems with a box or star or something. I often use different colored Post-Its, stickers, etc. but I forgot to ask for such supplies. We’ll make do. Question the truth. We are getting to core needs, so is your technology or system needed, or just convenient? Remember to judge based on ecosystem impacts. If using something due to familiarity, not so much. But because you are extending an old system to new platforms, that is good!Data and content are the same things to me. Whether someone wrote it or its something like weather data, it needs to be understood and planned for in similar ways. 20 MINUTES IF POSSIBLE.
LET THEM WORKSTOP WHEN I SEE THEM ABOUT TO CIRCLE, ETC. DONE: When you do this for a client, or your team, pick a moderator to make sure you are doing the exercise right and to keep track of time. You can’t spend more than about 5 minutes per step. You can in reality discuss this stuff for hours. If you are the designer for a real project, you won’t generally write on cards, but will just organize and moderate.
One of the things I like about this exercise, or any exercise I use now, is that it isn’t a dead end. You don’t congratulate yourself on the team building exercise, but transition it directly into the next step, and eventually to actionable or testable items. You should be able to define and measure if you meet these goals when you have launched the product, and work towards them as you progress to specify, design and build it.
Objectives and key results, or OKRs provide a nice guidance on the sorts of goals we're looking for, but they are found though a formalized process that could be seen as competing with or duplicating the exercise we just did. If it helps, think of this as a product-level version of success criteria. You will evolve this to each feature, or define features to assure you are building the right thing, as you go along. But we do NOT mean the TDD centric sorts of technical measures you may be used to if you are an Agile shop. These are much higher level.OKRs can be read about here http://www.eleganthack.com/an-okr-worksheet/
Another thing I think is critical is knowing your audience. And, It’s crucial to define the user as broadly as possible. Not just all the types of end users who will use your product, but internal users like administrators, content creators, and the people responsible for system maintenance. Whether you’re porting legacy systems to new platforms or creating all-new technologies from scratch, beyond the digital technology, there are business processes and objectives that you need to address as well. …Now, classic slow-as-molasses UCD types of teams can spend literally years and hundreds of thousands of dollars creating personas. Forget budgets, speed alone makes this impossible. By then, the world has changed. There are many ways to speed this up.
This term I got from Kelly Goto, and I quite like it. You have in mind for your project — and I mean all your projects, and by “you” I mean everyone on the project — what type of users it will address. You do, but mostly you don’t actually say that out loud. Simply running an exercise like the KJ, with post-its and everyone in a room (or questionnaires, etc.) can gather all the information everyone presumes, and much that is actually known from previous projects and monitoring user behavior like sales. You can get an 80% persona with information on hand in a few hours.
I don’t mean that Google has started offering a persona service. I mean using simple internet search to fill one out. I have done this, so successfully that others pointed it out in a meeting where one of these 18 month efforts was being presented: “but Steven made 4 pretty good personas in 3 days last week.” I did it by taking that presumptive persona information, then searching on terms that came up. This was a professional role, so I used LinkedIn groups, job postings and resumes. But you can do the same by stalking forums, blogs, and other internet properties where people of the role you expect hang out. A couple days can immerse you in the culture so much you can start to understand it quite well.
There’s a lot to be said for taking even a little bit of time to do real research. Seeing how people really work in their natural environments is enlightening. This is a good example. As part of just testing a product related to this [MFV dongle] I was on site with these truck maintenance guys. Years into this project, I found this. They are, after many failed attempts like the thing he’s holding, disappointed with digital recordkeeping and data collection, so have a room of cubbyholes with paperwork. That had never come up. They didn’t think it was worth mentioning. I just stumbled across it. What are you missing by making assumptions about your users?
The exercise we did that’s all over these walls, is supposed to be basically technology-agnostic. It’s about users, and to some degree benefits your organization can gain. Well the next step is to start modeling the process. But I want you to stay technology-agnostic, as much as possible. There are assumptions about the scope of the work that are based on technology, how your info is stored, what sort of wearable it is. Ignore those. Don’t throw them away, but assume the previous exercise covered them,…
And build a user task flow. This is about designing truly from the user’s point of view. Think about them, their motivations, their goals and the real, broader environment. About what they already do with their day. If it is raining, or they are driving vs. walking.
There are many ways to draw this. But most of all, make sure you talk about services, data, sensors, networks and users. Not screens, pages, buttons.
It doesn’t have to be pretty. Just start with boxes, or even literally take post-its off the wall and start the diagram from the previous exercise work.
If more comfortable with thinking about the user and their environment directly, this is a good place to use the storyboard. And, feel free to use several methods. Not just in your work, but now. Someone can draw the flow of all the data and all possible processes at the same time someone else is drawing a storyboard. But collaborate. Everyone talk, and grab the pen to take over if you must.30 MINUTES, BUT LIMITED!
But you may be wondering, how do we think about data, and systems and connectivity? Well, do that also. None of this is just about users, or just about any other one facet. Ideally, you never again make a document that disregards other layers of the stack. In this case, your data and interconnectivity is critical. It might be hard to see, or to make everyone understand that a box is not about UI but process. You may need to extract it more explicitly like this.
Or, boil it down to the fundamentals. At the data model level I find lots of developers missing out on doing deliberate design, and assuming things that may not be the best choice. This describes an integrated system, with synching to remote servers, and only using the capabilities each device type has.
20 – 30 MINUTES
The next design artifact you will build is going to get much more specific. And if we have the time, we’ll even start making one. But before you can get there, you have to stop to understand again, and make sure you are designing the details the way the user will actually consume them.
First, always work at device scale. Don’t do all the drawing on the computer. Don’t show off your designs on PPT to a conference room. A simple way is to draw on printouts the same size as your device. Any device. The Pebble guys had printouts the size of the expected watch screen.For mobile interfaces, take your digital drawings as you progress through the design, and stick them on the handset. [SHOW EXAMPLE ON PHONE]
I assume you all know the story of Jeff Hawkins, even if you don’t recognize the name. One of the founders of Palm. He carried around a block of wood — supposedly this one, wrapped in tape and a paper printout — and pretended to take notes on it when he would want to use it. He got to try out the concept, in context, in much the way a real user would. He specifically focused on the form factor as a previous project had failed, he believed because it was too big to be pocketable.
Me, I would have actually written on it or something. Which I did. This is me working on an eReader, before the iPad. Using my windows tablet wasn’t great because it was so big, and without a general-purpose computer to put screenshots on, I made a plywood tablet instead. We wrote on it, taped printed designs to it, and it was used when presenting early concepts to the clients even. Recently, working on a wearable I cannot otherwise discuss, we make sure there are mockups of the wearable around all the time. Mockups. They do NOTHING. Chunks of metal and plastic. But it provides not just an anchor for discussions, but you can really get into the mind of the user and think about how they will wear it, and how they will refer to it when it blinks and buzzes at them.
Behind everything, there are wires, radio antennas, batteries, sensors, ICs, resistors, films, adhesives. If you are building for the real world, and don’t plan for when the world intrudes, you will fail.
Understand your technology, and check yourself. There are many technologies you have to use. And all of them I see misunderstood very often. You will probably never use GPS for your app, but location, which is derived from many sources. Or displays. We like to pretend they are made of pixels, like when you zoom in to Photoshop, each one a square of a different color. But display technologies are not like that, and they vary. Even simple technologies often matter a lot. Do you want to use all of the phone’s battery in an hour? Better understand the impacts of what you do on the technologies.
Connectivity may be such an issue that no client level work can solve it. Here is a diagram I made describing how the only way we can get the needed product to work in Africa is to build (or lease space on) local data centers and get them piped straight into the mobile network operators.
In case you think this is all on the implementation side, so you’ll worry later, wrong. The software design, the interaction design, and even the information architecture and data concepts can induce failure if you are not careful. I’ve already mentioned Evernote’s competitors, but there are many other cases. Twitter keeps the numbers close, but thought they were a website for a while, to the detriment of use rates on other platforms. Right now I am working on a revision of the system that uses this [MFV dongle]. This is a prototype, and the real one will be much, much smaller. Anyway, we built the product to live read a remote system. Much, much, much later we found out it cannot be live, but gets snapshots of the remote system. Like, after it was built.
We tested it, and even with a few tweaks, it didn’t work. The concept of the app design didn’t make a bit of sense in fairly fundamental ways. We have video, which I cannot show you as the app is secret, of people failing to do what we expect. I am just finishing a moderate redesign of the interface to reflect the way the system works. These are non-trivial differences, as it turns out. You can’t add a button or explain it, but have to build the interaction around the way the system works. Tab bars went away entirely, the menu items all changed. The first several pages we show people every time they enter the app… gone. Serious changes.
So with those fears in mind, let’s design a system. Not an ecosystem, but a single system. You are going to take the task flow, and now we’ll make it platform specific. In reality, we’d branch it to the many platforms needed, and should even make as part of this exercise identifying the various platforms. Here, go ahead and assume one exists. Whichever was posited during the elevator talk at the beginning, whether smart shoes, or a thermostat better than Nest, that’s the platform we are using, not the desktop website or the mobile app. Go through the task flow, and mark all the points where you are sure the connected device will have some input or output. Then, pick two or three of these vertices, as again we only have a little bit of time, sit down, and design the interface and interaction that goes with these.
Draw at least two platform interfaces. Three is better. I mean different platforms; desktop, mobile, TV, or different interfaces that use the same data in different ways. Print or email or packing labels are fine also. Think of where content appears.Only do about one page for each. Make it the same one for each platform.Then draw, next to it, by marking with letters on the drawings themselves, the data. I want you to get a sense of the content or data you are using by scale or context.
Then, and mostly, I want you to describe what is happening. You may also draw. Not just flows, but interfaces and interactions. But then describe the interactions. This is where I get ranty about not prototyping, but drawing and describing. As we move to these all-new and arbitrary devices, you can’t code them up. They might not exist as you create the hardware. You have to be able to draw and specify things like the lights, the blink rate, the way the user shakes or waves at it.
Remember when I said you need to design in multiple ways? Well this is where it’s time to start thinking really seriously about things like content and data design.If you don’t integrate this thinking into the design early on, the implementation teams for each platform will go off and create their own solutions. You will end up with a fragmented and expensive to maintain product at best, and much more likely a broken one. Do you think it’s reasonable that your bank’s website takes up to 24 hours to reflect a deposit you made at an ATM? It’s the same system, isn’t it? Don’t be like that. 20 MINUTES
When you get out in the world and start drawing like this, remember to think of this step in the process as an extension of the previous drawings you did. This is a mobile phone app, but that’s the only one I can show you now. You don’t have to, ever, jump to “now we’re doing wireframes” or worse “comps.” Keep with the user-centric methods. Here, it’s screens, but the user is shown walking down the streets of Nairoi. And it’s not all the app. The first screen is SMS, then he goes to the Website, where he’s enticed to download the app. You can show flows and interactions and user context. You probably should.
And, to think of both what is a good feature, and what is an anti-good feature. Are you making your users do too much work you can do for them? Are you making them mistrust the system? Are you actually scaring them with how your system works? Origin of the quote: https://readmill.com/admitonetojl/reads/microinteractions/highlights/ggn3gg
The developers here especially may be familiar with something called SIT, or some variation of System Integration Testing. It’s an end to end technical test of the system. First, it’s increasingly a lie. Are you testing on the real hardware, on mobile networks other than in your corporate office? Outside? With noise and glare and distractions? Because those all matter. The system includes people. We’ve traditionally broken than out as a usability test, if we even bother to test them. But users and their context is the most troublesome part of the experience it turns out
First, yes, you should understand. Make sure the system basically works. Check the hardware and make sure your fundamental assumptions are correct. Here, we’re trying out the UI on an ePaper device. There’s very little info out there on design for ePaper, so we had to do it all experimentally. Using this view camera and trial and error to make just the colors readable. There were many other parts to test of this.
Test, test, test. Test on real devices, not just emulators. First, in your office, like this maybe. This is me trying out a responsive app, to make sure it works right on as many devices as I can. If you have a multi-screen experience, get the phone, desktop and smart watch out at the same time, and do end to end testing to make sure it doesn’t just work, but makes sense. Wear it around, try it. Does it work in reality?
Test in the field. With real hardware, in real circumstances. With real users if you can. And, you can. User testing is not that hard. In fact, I find that not having a lab makes it easier and cheaper. You have to know how to test — how not to mess up the results with your opinions — but you go to the users, so they are easier to get ahold of. Here, were testing this dongle on trucks. The photo is from the user’s point of view. He wore these video glasses so we can review the work later. Even the data gathering is literally from the users POV. That helps, and you see stuff you couldn’t get later. As you see here, we also are getting off the screen. We capture everything the user sees, not just the handset.
Test in the context of their environmentWatch what they do, not what they sayGetting people to talk is easier than you would thinkThe best ideas come from unscripted threadsA complete prototype isn’t needed to test your conceptshttp://www.usabilitycounts.com/2013/08/20/five-things-ive-learned-from-usability-testing/
If you miss these addresses, just Google my name and you’ll find me. QUESTIONS – WHAT DID I NOT COVER
Just trust me on this one: Everything you do is too complex to adequately model and map. Assume you have always missed something, so you are prepared to deal with the unexpected, both in design and so you can modify your product over time to take advantage of new ways you find people using your learning information. You could just remember instead…
I say there’s something called Resilience Design. Or there should be. Here’s a simple example… I also still wear normal watches. One is a dive watch, because it’s shiny, not because I am a diver or anything. It is one of those with a twisty ring around the outside. That part with the numbers twists around. If you don't know, and I didn't until recently, this is used as a simple timer. But on mine, and on all dive watches (vs. Aviators watches), the ring only goes one way. The clicky detent lets it go counter-clockwise, only. WHY? … Because it's for timing remaining air. The ring might get bumped and change it's setting. Having it show less time might be inconvenient, but going the other way might kill you. And, you don't even need to know this. It just works. That's the sort of brilliantly-simple answer I am talking about with resilient design.
You need to make your designs resilient because users will never, ever do what you expect. You, or others in your organization, probably draw diagrams that assume everyone starts at the home page, drills down through a preferred path and gets their information. It’s not true. People bookmark, share, and search. They use your process in unexpected ways, and your system returns errors or data you didn’t expect. If you try to design to accommodate these, or to test for them in the traditional use case model, you CAN’T. For one project I worked on, I did some quick math and to create the use cases for all variations would take approximately the remaining life of the universe. Really. These are arbitrarily complex products, so don’t lament, embrace the complexity.
On a typical website, I find that home page as the entry point is rarely over 10%, and is often so low as to be ignored; hundreds of visits a month when hundreds of thousands visit the site. This is real data.
This is even more important with the way people engage on mobile devices. People seek out and consume content sometimes a few seconds at a time. They get interrupted, and come back to read a bit again so it has to be ready for them. Does your information work well and make sense if set aside for a few minutes? A few hours? Make sure session expiry and other technical things don’t get in the way of users coming back. And, remind them to come back. Use SMS, and app notifications or reminders within the site or app of items that may be interesting or that they didn’t complete. Or emails. Or postcards. Or whatever fits the way you have a conversation with your customers or users.
YOUR PLATFORMS ARE INHERENTLY FRAGMENTEDMany U.S. companies assume or extrapolate their experiences onto the rest of the world. Not only is the rest of the world different, parts of it are radically different. I worked in San Francisco for a year, and it is not the US, so is sort of terrible that this is where everything is supposed to come from. This chart is smartphone OS penetration by region. The legend hardly matters (but is obvious for some… Android is green, iOS is white), but instead look at the wild variations between regions. Local variations, by region, or by type of user within a region, are radically different.
An astonishing 1.8 billion people regularly use the mobile internet. Mostly, that means web browsing, But look at this.There are 7 billion humans on earth. Remember, that’s everyone who is alive. So, essentially everyone who can feed and clothe themselves uses SMS. FIVE BILLION users. If you think Facebook is the killer app of the internet, that’s SIX TIMES bigger. And SMS is a great engagement engine, with ten times the response rate and 100 times the response speed of reliable things like email. Oh, and nowhere near that many people actually have an email address. More people subscribe to SMS news alerts than get a newspaper. Does re-engagement mean SMS? Yes, probably.
API First: Creating ecosystems instead of products
―Sadly, no decision about
architecture is a decision, one that
will determine your success or
failure as a company.‖
– Michael Sharkey
―I publish a website, but tens of
thousands of my most loyal readers
consume it using RSS apps. What
should they count as, ―app‖ or
I say: who cares? It’s all the web.‖
– John Gruber
KJ (Post-It®) Exercise:
• “What one problem does it solve?”
• “How is it solved today?”
• Put up answers
• Group answers (draw a circle)
• Label groups (write the label)
• Pick the top three (at most) groups