Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

User Experience Testing in Your RFP Process - David Rosen - highedweb 2014

1,762 views

Published on

Learn how user experience testing can save your university money, reputation and headaches. This session will cover the University of Minnesota’s efforts to incorporate user feedback into the software purchasing process. The session will cover several use cases that illustrate the considerations and challenges faced by teams at UMN making major software investments, and will show how user feedback helped those teams make data-driven decisions, avoid choosing the wrong tool, and prepare for the tools’ rough edges ahead of rollout.

You wouldn’t dream of buying a car without taking a few options for a test drive. Why would you spend what could be millions of dollars on a software solution for your campus without taking the opportunity to test it?

Published in: Technology
  • Be the first to comment

  • Be the first to like this

User Experience Testing in Your RFP Process - David Rosen - highedweb 2014

  1. 1. Try Before You Buy: User Experience Testing in Your RFP Process Can Save You Time and Money David Rosen, User Experience Analyst
  2. 2. WHO IS THIS GUY? David Rosen User Experience Analyst University of Minnesota Twitter: @DRosen_UX Hashtag: #heweb14 Track hashtag: #UAD8
  3. 3. #UAD8 WHAT’S AN RFP? Sandwich RFP Requirement Points Vendor 1 Vendor 2 Vendor 3 Does the sandwich have ham on it? 3 Yes (3) Yes (3) No (0) What condiments are available for the sandwich? 3 Mayo, Mustard (2) Mustard, Jalapenos (1) All condiments (3) Can the sandwich be toasted? 2 Yes (2) No (0) Yes (2) How quickly will the sandwich be made? 5 10 minutes (3) 5 minutes (5) 15 minutes (1)
  4. 4. #UAD8 EXPERIENCE CHECK http://www.cnet.com/au/news/supercomputer-runs-longest-simulation-of-brain-activity-to-date/
  5. 5. HOW DID WE GET THIS STUFF? Some common reasons software vendors are chosen: • They were the least expensive • They gave the most polished demonstration of their product • They checked the most boxes on the RFP • They were one of the committee member’s favorites before the process began • We have already been using their product #UAD8
  6. 6. “It’s a very sobering feeling to be up in space and realize that one’s safety factor was determined by the lowest bidder on a government contract.” Alan Shepard https://creativecommons.org/licenses/by-nc/2.0/legalcode
  7. 7. https://www.flickr.com/photos/barry_b/76055201/ #UAD10
  8. 8. #UAD8 REENACTMENT OF THAT MEETING
  9. 9. “No” http://corporate.ford.com/our-company/heritage/company-milestones-news-detail/ commemorating-150-years-of-henry-ford-visionary #UAD8
  10. 10. #UAD8 ANOTHER CHALLENGE Going with the product as-is • Customization is hard to maintain and expensive https://www.flickr.com/photos/stevendepolo/3797233888/
  11. 11. #UAD8 NOT “VANILLA” https://www.flickr.com/photos/ginnerobot/3156117675/
  12. 12. #UAD8 IMPLEMENTING “VANILLA” “Vanilla” can be tasty… if you let your users test it https://www.flickr.com/photos/stevendepolo/3797233888/
  13. 13. AVOIDING NON-FAT, UNFLAVORED ICE MILK Case Study: Survey Tool #UAD8
  14. 14. ASKING USERS IF THEY WANT FEATURES “Who Wants a Pony?” image from http://voiceofthemonkey.com/wp-content/uploads/2011/11/Who-Wants-A-Pony.jpg #UAD8
  15. 15. #UAD8 THE OLD TOOL LOOKED LIKE THIS
  16. 16. #UAD8 GETTING INCLUDED https://www.flickr.com/photos/glynlowe/7183121469/
  17. 17. #UAD8 VENDOR DEMONSTRATIONS
  18. 18. NO MORE SMOKE AND MIRRORS Because we were dictating the terms of the demonstration, we could see when the vendors were trying to skip steps, or avoid less pleasant interactions. https://www.flickr.com/photos/xtream_i/316644980/ #UAD8
  19. 19. PREPARATION FOR USABILITY TESTING “We’re reluctant to let you try this without at least a three-hour training session” #UAD8
  20. 20. #UAD8 PREPARATION FOR USABILITY TESTING
  21. 21. #UAD8 PREPARATION FOR USABILITY TESTING
  22. 22. SOMETIMES IT’S HARD TO WATCH They made it look so EASY! #UAD8
  23. 23. GATHERING ISSUES • Identifying pain points • Quantity of issues is not a comparison tool #UAD8
  24. 24. #UAD8 NEXT STEPS
  25. 25. #UAD8 NEXT STEPS
  26. 26. #UAD8 NEXT STEPS
  27. 27. WHAT WE GAVE TO THE VENDOR Recommendations – A description of what we wanted to either understand better or have changed Alert Training/Support – Training and Help Desk teams will need to be aware of this issue Prepare Vendor Request – A request for a change in order for the solution to fit the need Response Required – We are expecting an explanation of this issue Submit OIT Request – This may require configuration on our part Hold for Further Review #UAD8
  28. 28. #UAD8 WHAT WE GAVE TO THE VENDOR We understand that some of these tasks may or may not be standard, but they were included as a means to evaluate the usability and intuitiveness of the system. In this way, we are able to gain some insight into configuration setting issues, areas that may require extra training, and/or items that need to addressed by you, the vendor. Attached is a workbook outlining the issues that were experienced and that we would like you to respond to. Please review and enter your response in the space provided. In the columns to the right, you may indicate if any of these options are applicable.
  29. 29. #UAD8 WHAT WE HEARD FROM ONE OF THE VENDORS “A response was required from us on each item, and since we were motivated to win the contract we made commitments to address the vast majority of issues. Now that all items targeted for the short term have been addressed, we know our product has become easier to use.”
  30. 30. WHAT WE HEARD FROM SOME OTHER VENDORS http://memecrunch.com/meme/15QYJ/uhhhh-no #UAD8
  31. 31. #UAD8 UX HELPED BUILD A BUSINESS CASE
  32. 32. #UAD8 LESSONS LEARNED http://educationalaspirations.com/category/assessment-data/
  33. 33. LESSONS LEARNED Vendors don’t understand what we are doing by “usability testing” They also think that we’re “difficult” #UAD8 http://www.flickr.com/photos/79538062@N00/3732589905
  34. 34. LESSONS LEARNED Access to the finalists’ software cannot be optional #UAD8
  35. 35. LESSONS LEARNED Vendor Demonstrations need a strong moderator from the selection team #UAD8 https://www.flickr.com/photos/leshaines123/6909108380/
  36. 36. LESSONS LEARNED Testing is best after vendor demonstrations are complete #UAD8
  37. 37. IF YOU REMEMBER NOTHING ELSE 1. WE. SAVE. MONEY. – We’ve had a preview of implementation – Our training and help desk teams are prepared ahead of time for known issues 2. There are fewer surprises in a situation where surprises are expensive 3. This makes vendors better vendors and partners #UAD8
  38. 38. Go do it! 1. Use the stuff you’d ask the vendors anyway 2. You can do this on the cheap with limited resources #UAD8
  39. 39. #UAD8 BOOKS TO GET STARTED
  40. 40. THANK YOU! Questions? David Rosen User Experience Analyst University of Minnesota Email: rose0414@umn.edu Twitter: Drosen_UX Web: z.umn.edu/usability

×