Successfully reported this slideshow.
Your SlideShare is downloading. ×

Map quest 2015_denverstartupweek_spectrum

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 69 Ad

More Related Content

Slideshows for you (20)

Similar to Map quest 2015_denverstartupweek_spectrum (20)

Advertisement

Map quest 2015_denverstartupweek_spectrum

  1. 1. A Celebration of Everything Entrepreneurial in Denver September 28 – October 2, 2015 #DENStartupWeek
  2. 2. Title Sponsors
  3. 3. Designer Track Sponsor
  4. 4. Our Spectrum of Testing & Research When do we research and test? When don’t we? September 30, 2015 4
  5. 5. Hi! Lindsay Moore 
 Lead UX Research Scientist SuAnne Hall 
 Director of Design
  6. 6. 01 Why are we talking to you about research?
  7. 7. 01 “Research is a luxury we don’t have time for.”
  8. 8. 8
  9. 9. 9 You can never. EVER. Guess how people will use your product based on your own experiences and assumptions. — image by Whitney Hess
  10. 10. Assumption: 
 When a user looks up a POI, they’ll be delighted to get all of our awesome POI data like hours, menus, amenities, reviews, tickets, ability to book a nearby hotel, what’s nearby, etc Reality: 
 When we literally put all this great info center stage, users were very angry that we were blocking the map, as they could no longer get the context for where the POI was! Example 1 — Creating features based on partnerships, revenue, data availability, etc
  11. 11. Assumption: 
 If a user doesn’t choose a result from predictive search, they haven’t seen the result they wanted, so we should show them different results Reality: 
 Actually, many users just hit enter or search, bypassing search ahead for “no good reason” Example 2 — Designing based on common sense
  12. 12. Assumption: 
 We should optimize our public transit routing to minimize the amount of walking. Reality: 
 “If I'm not in a rush to get somewhere, I would think that a 30-45 min walk leg would be ok, beyond that and I would really have to be wanting to walk to actually do it. Also, I personally am more likely to walk in Manhattan as it's easy to hop on alternate transportation if I get tired or have a change in plans.” —User Quote Example 3 — Engineering based on behavior assumptions
  13. 13. Assumption: 
 When users search, they search for a what & a where (Pizza, Denver) Reality: 
 Actually a minority of users search this way. Much more common to just specify what and assume that we’ll find what’s nearby, or to put the where at the beginning (eg. Anaheim Airport) Example 4 — Letting assumptions become a cultural meme
  14. 14. 01 – Simon Sinek “People don’t buy what you do, they buy why you do it.”
  15. 15. We believe in saving the day by helping and empowering our users, in both small and big ways. It’s our job to remove the obstacles that make life anything short of easy, fun — and flexible, too. When the day requires a hundred little wins, we’re there. 15 We Believe…
  16. 16. 01 - Ken Jennings “We’re accustomed to only using maps when we’re frazzled and annoyed and when things have gone wrong. It’s like a bottle of Immodium.  By the time you pull out a bottle of Immodium, things have already gone very wrong with your day.”
  17. 17. OK, so maybe research isn’t just a luxury… 17
  18. 18. 18 Ifwe ship quickly without validating the experience, we’ll end up spending up to 100x more to fix what we got wrong than it would have cost in the design stage.* YOU COULD SPEND $200 AND 4 HOURS FIXING THAT LEAKY PIPE, OR IT WILL COST YOU $200,000 IN 2 MONTHS TO REPAIR YOUR FLOODED BASEMENT. HMMMM. I REALLY WANTED TO TAKE THAT CAT SWEATER KNITTING CLASS THIS WEEKEND. I THINK I’LL JUST WAIT FOR MY BASEMENT TO FLOOD AND PAY $200,000. * IEEE Spectrum, Why Software Fails, http://spectrum.ieee.org/computing/software/why-software-fails WallStreet & Technology, Can UX Deliver 1000% ROI? http://www.wallstreetandtech.com/wealth-management/can-ux-deliver-1000--roi/a/d-id/1268572?
  19. 19. 01 – CEO of Jaguar “If you think good design is expensive, you should look at the cost of bad design.”
  20. 20. 20 HOWEVER
  21. 21. 21 Ifwe spend time researching, validating, and testing every little decision we make, we’ll become an overly-cautious culture that promotes the immobilization of innovation and agility. WE TEST NOW. MMMM Nom nom ~burrrrrp TIMELINE EATO-MATIC 3000
  22. 22. 22 OK then. So WHEN do we research and test?
  23. 23. 23 There are two main schools of thought in which we tend to fall in one bucket or the other. But we don’t have to take sides! When we combine both schools, we can use an effective model that saves time and money while still making smart, customer-centric decisions. WOOT
  24. 24. We need a tactical answer (like a button label) We need a North Star-type answer (like a Product Narrative for a new product) WEEEEEEE BAGOINK. YES YES Can we anwer our questions quickly & easily? If we get the answers wrong, will it be expensive & time-consuming to fix post-dev? Label or button changes Major changes to the user’s workflow Brand new product Small iterations to existing features Lean MVP date-driven project Project where we’re not reinventing the wheel Finding the best placement or layout Introducing important new data or services Redesign of a product Any updates aren’t feasible
  25. 25. We need a tactical answer (like a button label) We need a North Star-type answer (like a Product Narrative for a new product) WEEEEEEE BAGOINK. YES YES Can we anwer our questions quickly & easily? If we get the answers wrong, will it be expensive & time-consuming to fix post-dev? GENERATIVE RESEARCH EVALUATIVE RESEARCH A/B TESTING DESIGN EXPERTISE QUANTITATIVE ASSESSMENT CLICK/EYE-TRACKING TESTS ETHNOGRAPHIC RESEARCH MINI-TESTING DIARY STUDIES USABILITY TESTS QUALITATIVE INTERVIEWS HEURISTIC EVALUATION COMPETITIVE ANALYSIS
  26. 26. • Instead of saying ‘we can’t afford to gather any customer insight/research’, think about how much it could cost NOT to do a bit of up-front research. • Not too bad? Great! Let’s learn and iterate along the way. • Would getting this implementation wrong be devastating to fix? Would it alienate our users significantly? 
 Re-think that gamble. How to use the spectrum
  27. 27. • Factor in the realities of your company and your situation. • Time • Team capacity • Available funding • Ease of dev change • Existing userbase • Skin in the game Things to consider
  28. 28. • Minimal customer insight + dependence on a high ROI = 
 incredibly risky. • Significant personal investment ($, time, emotional) can lead to a loss of perspective on your product Watch out for these!
  29. 29. • Don’t confuse your MVP (minimum viable product) with a MP (minimal product) • Confusing your MVP with your EP (the eventual product) Watch out for these!
  30. 30. 01 – Laura Klein, VP of Product at Hint Health “An MVP is not an excuse for a crappy product. MVPs are all about learning, and you don’t learn anything from a crappy product except that people don’t like crappy products.”
  31. 31. 01 Case Studies
  32. 32. 32 MapQuest.com Redesign
  33. 33. Web Revolution. 33
  34. 34. SERP, DERP 35 ???
  35. 35. We need a tactical answer (like a button label) We need a North Star-type answer (like a Product Narrative for a new product) WEEEEEEE BAGOINK. YES YES Can we anwer our questions quickly & easily? If we get the answers wrong, will it be expensive & time-consuming to fix post-dev?
  36. 36. We need a tactical answer (like a button label) We need a North Star-type answer (like a Product Narrative for a new product) WEEEEEEE BAGOINK. YES YES Can we anwer our questions quickly & easily? If we get the answers wrong, will it be expensive & time-consuming to fix post-dev? What do we call Search? Mini-testing Mental Models Cross-device usability testing of resposive designs Search testing: where are the problems?
  37. 37. We need a tactical answer (like a button label) We need a North Star-type answer (like a Product Narrative for a new product) WEEEEEEE BAGOINK. YES YES Can we anwer our questions quickly & easily? If we get the answers wrong, will it be expensive & time-consuming to fix post-dev? What do we call Search? Mini-testing Mental Models Cross-device usability testing of resposive designs Search testing: where are the problems?
  38. 38. We need a tactical answer (like a button label) We need a North Star-type answer (like a Product Narrative for a new product) WEEEEEEE BAGOINK. YES YES Can we anwer our questions quickly & easily? If we get the answers wrong, will it be expensive & time-consuming to fix post-dev? What do we call Search? Mini-testing Mental Models Cross-device usability testing of resposive designs Search testing: where are the problems?
  39. 39. When can you stop researching? 40
  40. 40. 41 MapQuest.com’s Mental Models
  41. 41. 01 – Indi Young “Mental models give you a deep understanding of people’s motivations and thought-processes, along with the emotional and philosophical landscape in which they are operating.”
  42. 42. Mental Model Example: Getting to Work 43
  43. 43. mapquest Mental Spaces for users of MapQuest.com (Yogi) MENTALSPACEMINDSETGOALSTASKSOBJECTS + I’m going someplace and need to get my bearings + I need to figure out the specifics of how to get from point A to point B + I want to figure out a route between multiple places + I need to plan my transit: how long will it take me to get there? how far away am I? what are my options for getting there? Finding a Place & Consuming Results can be children of this mental space + I’m gathering info about my options so I can make a decision + I’m applying my preferences and criteria to determine the best route for me (explicitly or mentally) + I’m evaluating tradeoffs between routes + I want to scan an overview of the route/figure out what’s in store + I want an overview and will dive in to details as needed + I need to understand the narrative step-by-step + I want to understand my backup options should something go wrong Find out the best route to get me there. Familiarize myself with the choices. Make a route decision. Find out how to get somewhere that I need to go. See tasks for ‘Finding a Place’ See tasks for ‘Consuming Results’ See tasks for ‘Browse an Area/ Category’ Apply criteria (one time vs sticky) See tasks for ‘Consuming Directions’ Take a screenshot Jot down directions to take with me Configure print page (remove steps, remove ads, write notes, mini maps) Bookmark/ send my directions (to phone/ email/GPS) Copy/paste directions into another program Close browser Print directions Apply criteria & knowledge (one time vs sticky, ‘I just know’) Select a route (suggested, custom, make mental selection) Change a route (dragging, applying options) Read thru narrative Scan map/route for overview Compare routes (time, distance, road type, etc) Set a starting location Select a route type (driving, transit, walking) Set a destination Add stop(s) to route (actually or mentally) Set desired info level (ignore/hide steps, show more info, turn on layers) Interact with map (pan, zoom in for detail, zoom out for context, look at turns, etc) Drill further into details (zoom to step, miles from one step to another, etc) Re-route on the fly Reference printed/ written directions Use phone/GPS to give turn-by-turn directions Refine search (by typing different words, by selecting addt’l criteria) Type in place (address, name, name & area, neighborhood, district, intersection, city, state, etc) Google for address Use geo- targeting to target current location Select choices from search- ahead Interact with map (pan, zoom in/out, click on pins, turn on layers, etc) Confirm desired result found (location/details — is this the right one?) Scan list of results/pins on map for overview Print resultsScan list and compare results (ratings, hours, location, etc) Select individual result to view more info Take another action on a result outside MQ (call, visit website, etc) Get directions to a result Share results (mobile, URL, email, social media, etc) + I’ve decided on my route and now I need to get my bearings before being in transit + I’ve decided on my route and I need to learn the specifics before being in transit + I need to get this information out of mapquest.com and into the type of tool/artifact I’ll use while in transit Consuming Directions can be revisited here Prepare artifacts and knowledge to take with into transit. + I’m referencing my previously-aquired information and knowledge while in transit + I’m referencing my previously-prepared artifacts while in transit + I need to be able to adjust on the fly, if possible Use my previous activities and artifacts to get there. + I want to find a specific place (out of curiosity or to get more info/get bearings/etc) + I want to find something centered around a specific place (targeted exploration) + I’m getting directions and don’t know the exact destination so I’d better find that first + I want to verify that MQ can find this place before getting my directions + I need to get directions and am just used to starting with finding a place Find the thing I’m searching for. + I’m verifying that the place I was looking for comes up as a result + I need understand the context of where my result(s) are on a map, in relation to one another, or in relation to a specific place + I want to scan the set of results to compare and figure out the best fit or best choice + I want to take action on my result(s) at MQ (get more info, print, get directions, share, etc) + I want to take action on my result(s) outside of MQ (visit website, call, etc) Check if the results fit what I was looking for. Make a decision about a choice/set of choices. Take action. Inputting My Start & End Consuming Directions Preparing for Transit Inputting a SearchIn Transit Consuming Results ROUTE SINGULAR PLACE SINGULAR PLACE AREA SET OF PLACES
  44. 44. mapquest Mental Spaces (Cont.) for users of MapQuest.com (Yogi) MENTALSPACE Researching Place Info Browsing an Area/Category Exploring Nearby (find near a place) Setting Up Tools & Bookmarks Trip Planning (find near a route) Utilizing Tools & Bookmarks MINDSETGOALSTASKSOBJECTS + I want to get more info on a result from my finding a place inquiry + I’m researching a specific place out of curiosity + I’m researching a specific place that I am at/that I will be at + I’m researching the individual places in a set of results and comparing them + I want to take action on my result(s) at MQ (get more info, print, get directions, share, etc) + I want to take action on my result(s) outside of MQ (visit website, call, etc) + I’m looking for something to do or somewhere to go + I have a general area in mind, rather than a specific place + I need to find something within a geographic boundary (strict, loose, or unstated — could be a neighborhood, district, city, state, etc) + I want to see what options are available within a category of place (restaurants, activities, etc) + I want to do open-ended exploration + I want to browse a map to get context for what’s there + I want to browse a map or category out of curiosity Find out what’s out there for me. Gather a set of choices. Get the lay of the land. Get more detailed info about a place. Make a decision about a choice/set of choices. Take action. Search for a category/ keyword nearby Refine by selecting additional criteria (layers/filters) Turn on layers on a map after finding a place Expand criteria to trump vincinity or expand area Type in category/ keyword and area Pan/zoom my default map view Type in area (neighborhood, district, city, state, etc) Refine search terms (Italian restaurants, banks open now) Turn on layers on a map (category or area, eg. neighborhoods) Type in category & assume area (based on current map view) Refine by selecting additional criteria (layers/filters) Re-route to stops/side trips (actually or mentally) Evaluate tradeoffs (stopping ealier vs later, exit A vs exit B, etc) Search for a category/ keyword or turn on layers (without wiping out directions) Utilize other sites for research purposes Utilize other sites for research purposes Expand area to search within (”play areas” along route) Find best stopping points (by time, available services, distance, convenient exits) Intentionally save a query (search terms, categories/ layers, addt’l criteria/filters) Have MapQuest remember my recents (passive saving of places/settings ) Intentionally save a place (singular) Intentionally save a route Share my saved things (places, queries, routes, custom maps) Create collections of places/ custom maps Set up my preferences Set up my home/work locations Have MapQuest default to my settings (sticky from last time) Pull up my saved things (places, queries, routes, custom maps) Have MapQuest present my recents (places) Continue a task I started on .com on mobile or in app Have MapQuest default to my preferences + I’m looking for something to do or somewhere to go + I have a specific place in mind that I want to look near + I want to see what options are available within a category of place (restaurants, activites, etc) nearby a specific place + I want to do targeted exploration + I want to browse a map to get context for what’s near a place + I want to plan what to do at a destination (place or area) Find out what’s out there for me near a given place. Gather a set of choices. Get the lay of the land. + I’m looking for something to do or somewhere to go along my way as I travel + I have a route in mind that I want to look near + I want to find things I’ll need along a route (gas, food, rest stops, etc) + I want to plan sensible route stops (for convenience, time, distance, availability of services, etc) + I want to do targeted exploration either before I go or while in transit + I want to browse to get context for what’s along my route + I want to get a sense of how stops/side trips affect my overall route Find out what’s out there for me along a route. Gather a set of choices. Get the lay of the land. + I want to save/retain information for later use + I want to take an action now that will make me be able to use MapQuest more efficiently in the future Let me come back to what I’m doing. Make me more efficient. + I want to recall/reference the information I found earlier + I’m using MapQuest and it remembers things that are useful to me + I’m using MapQuest and it remembers my prefer- ences + I’m using MapQuest and it remembers my settings from last time Let me come back to what I’m doing. Make me more efficient. SET OF PLACES ROUTE SINGULAR PLACE SINGULAR PLACE SINGULAR PLACE AREA AREA Interact with map (pan, zoom in/out, click on pins, etc) Consume place details (ratings, hours, location, etc) View nearby/ similar/related things (places, events, activities, lodging, etc) Get directions to the place Share place (mobile, URL, email, social media, etc) Print place info Take another action on a result outside MQ (call, visit website, etc) Take another action at MQ (book hotel, make reservation, etc) See tasks for ‘Finding a Place’ See tasks for ‘Consuming Results’ See tasks for ‘Researching Place Info’ See tasks for ‘Getting Directions’ See tasks for ‘Consuming Directions’ See tasks for ‘Researching Place Info’
  45. 45. mapquest GETTING DIRECTIONS Mental Model for users of MapQuest.com (Yogi) Inputting My Start & End Consuming Directions Preparing for Transit In Transit Inputting a Search Inputting a Search Consuming Results Consuming Results (in Search Ahead) Browsing an Area/Category EXAMPLE SCENARIOS POSSIBLE NEXT STEPS BEARINGS –OR– SPECIFICS BEARINGS –OR– SPECIFICS EVALUATE & ADJUST TRY & REFINE DECISION MADE! “I need to drop off a floral delivery to a client at 10135 W. San Juan Way, Littleton, CO 80127 (Re/Max Professionals)” “I need to go to my tennis match at Glenmoore Country Club.” “I’m meeting friends at the bowling alley on 16th Street Mall.” “I’ve got my route!” Researching Place Info Trip Planning Researching Place Info ARRIVING FROM ORGANIC SEARCH RESULT “I need to figure out how to get somewhere.” I want to get my bearings for how to get there —OR— I want to figure out the specifics of how to get from point A to point B
  46. 46. Phew. 47
  47. 47. beta.mapquest.com
  48. 48. 49 MapQuest Compare
  49. 49. “We think we need to add public transit to our apps…” 50
  50. 50. 01 “Hmmm, there’s something more here than just public transit…”
  51. 51. Oh, dear. 52 Directions Scenario — Route Options Page-level findings: • Anything green looks clickable, so people assumed they could click the busses here for more info • In general people were able to make a decision about which route they’d want fairly quickly • But some wanted more info: • the 41 is coming at 11:22 which is in 2 min. If I miss it, is there another one right behind it, or should I pick the other route? • Where exactly does that 41 bus pick up? Can I make it there in time? Do I need to run? • Which busstop is closest? • How many blocks is this walk? Not noticed by some – they did more of a manual calculation One person interpreted this as “1 minute” Some wanted intersection/stop info all the way up here in the flow. People felt they’d be able to click here for more info on that bus line People wanted to be able to see the next few departures here, too Generally noticed People wondered why we’re showing them a ridiculous route with so many transfers. But the icons help them deduce # of transfers quickly. ‘Next’ was easily found Find Nearby Scenario — Bus-Stop Infosheet Page • Peo ref the • Re pe ab som sho un wh be no key no • Wi to mi • So fre tim These icons are confusing, given the bus-stop context. Unclear Save To Favs Who is this phoning? The transit authority? interpreted as “…” or “more” Info Difference is unclear when this is interpreted as “…” Unclear to some how to get more times or what to do next… Do they hit ‘Go’? Do they tap on the card for the N bus? Can they swipe the card to show more times? One person wanted a ‘More Times’ button Blue highlight not clear but label noticed This needs to be accurate based on context: Downtown, Uptown, Inbound, Outbound Find Nearby Scenario — Bus Line Details Page-level findings: • This view is not what people expect to see when they get more info from the previous page. I think that’s because their lens is “get me more information about the N-bus pickups at this bus stop”, not “show me everything about the N-bus line” • With RT arrival info it’s useful to communicate how many min delayed, too • Some people try to deduce frequency from the listed times, partially because there’s an expectation that RT is not accurate anyway Highlight interpreted as “leaving next” not as RT Dot and arrow indicating the stop you’re focused on is understood by some, but it takes a minute. Others miss that altogether. Unclear to some how to get more times. Can they swipe to show more times? Several people wanted an arrow showing there were more times or a button to view later trains
  52. 52. Ok, so this Compare thing… Maybe there’s something to that… 53
  53. 53. 14
  54. 54. “It’s raining but it’s cheaper to take the train. I’m trying to see how convenient it would be since Ubers are more expensive when it's raining. I wasn't at home, but at a restaurant, so I used google maps to see how easy it would be. Seemed like a pretty easy route!”
  55. 55. Ah, yes, Compare. That’s more like it. 56
  56. 56. 57
  57. 57. 59 Takeaways
  58. 58. 60 Takeaway #1: Figure out the right research, not ask whether to do it at all.
  59. 59. 61 Takeaway #2: Don’t take sides.
  60. 60. 62 Takeaway #3: Build the right product without becoming immobilized by process.
  61. 61. 63 Takeaway #4…
  62. 62. 14 Stakeholder Interviews 1 Quantitative Assessment 2 Revenue Analyses 1 Brand Research Review* *FutureBrand studies 300+ Customer Service Logs Reviewed 10 In-Depth Qualitative User Interviews (with concept testing) April-May 2014 01
  63. 63. 8 Core Scenarios Identified 12 Mental Spaces Identified 3 Main Flows Identified 10 Primary Page States Identified 416+ Survey Responses about Labeling 12 User Interviews + Concept Testing June 2014 01
  64. 64. 15 Rounds of Wireframing 10 Interactive Prototypes for Responsive Design Testing 11 Rounds of Visual Design Wireframe Minitests in Local Coffee Shops 14 Visual Design Minitests in Local Coffee Shops July-October 2014 01 11 Usability Sessions on Responsive Designs 31
  65. 65. 1 Card Sort 6 Usability Sessions 34 Rounds of Visual Design* *We think?? We lost count! Wireframe Minitests in Local Coffee Shops 14 Visual Design Minitests in Local Coffee Shops November 2014-Today 01 5 Rounds of Wireframes* *We think?? We lost count! 17
  66. 66. 68 Takeaway #4: If we can do it, you can do it too.
  67. 67. 69 Thanks!

×