Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
UX for Machine Learning
Presented by Bonnie Cook
THE (WO)MAN AND THE MACHINE
STARTER UX ML QUESTIONS
I. Where do I find information about UX for ML?
II. Are there resources like tutorials? Demos? Thou...
ML RENAISSANCE
I. “Machine learning is in the midst of a renaissance that will
transform countless industries and provide ...
UX FOR MACHINE
LEARNING
SECTION ONE
1
LET’S START WITH “WHAT’S UX?”
I. “User Experience (UX) design is the process of creating
products that provide meaningful ...
THEN, MACHINE LEARNING?
I. “Machine learning is a field of artificial intelligence that
uses statistical techniques to give ...
THE EVOLUTION OF HCI
SECTION TWO
2
USES OF HCI OVER TIME
I. 1980’s - 1990’s: Desktop and Mental Models
II. 1990’s - Early 2000’s: Collaboration and Communica...
ML - A DIFFERENT KIND OF LOGIC
I. Fuzzy logic - logic formed on approximations versus exact
a. Need auxiliary knowledge of...
“PRIOR” TO MACHINE LEARNING
I. Systems are used to Boolean logic where every expression
has to ultimately evaluate as eith...
FACE VALUE
I. Infinite number of combos, genders, vantages,
environments, races, etc…
II. Examples of how we cannot reach p...
20 QUESTIONS
I. Ways we network with others
a. Not sure what things about you will overlap (job etc)
II. For conversationa...
DESIGN THINKING
SECTION THREE
3
DESIGN THINKING
“Design thinking is a human-centered approach to
innovation that draws from the designer’s toolkit to inte...
UX & DESIGN THINKING
I. Design Thinking is a process or methodology. It is about
applying a specific way of thinking to a s...
DT & ML STAGES
I. Design Thinking Stages
a. Empathize
i. Who is my user?
ii. Pain points
b. Define
i. Point of View
c. Idea...
WHO’S LISTENING?
SECTION FOUR
4
YOU AND ML ON THE DAILY
I. Siri - taught to understand the nuances of our language
II. Facebook - uses algorithms to recog...
Phoneme - Any of the perceptually distinct units of sound in a specified language that distinguish one
word from another, f...
Image: Sam Drozdov
CURATION VS ALGORITHMS
I. Apple Music has DJs who custom create playlists rather
than how Spotify or Pandora works
a. Desi...
ML LANGUAGE
SECTION FIVE
5
UNDERSTANDING ML LANGUAGE
I. Machines like Siri understanding spoken language is a big
deal because it allows development ...
“GAMING THE ALGORITHM”
I. Pamela Pavilscak, author of Emotionally Intelligent Design, talks about the
strategic evasion of...
OTHER ALGORITHMIC SETBACKS
I. They’re working with the big picture of what you do
a. Topics based on behavior
II. Incomple...
ML EMPATHY
SECTION SIX
6
A BEHAVIORAL PROBLEM
I. “We are more than just the sum of our behaviors. We don’t
behave rationally, maybe not even predic...
ALGORITHMIC EMPATHY
I. Algorithms change the way we explore and understand
empathy with users
II. Pamela Pavilscak’s proce...
DATA ROLE PLAY
I. It acknowledges the abstraction of algorithms
a. Good way to start the conversation about this new type
...
ALGORITHM SWAP
I. We have very private experiences with our devices
II. Might change once we start interacting with voices...
ALGORITHMIC PERSONAS
I. Design teams chose a base persona on a combination of:
a. Demographics
b. Interviews
c. Behavioral...
SHARED MYTHOLOGIES
I. When we reach an algorithmic disconnect, people start to
speculate as to WHY it’s no longer working
...
ML iOT DEVICES
SECTION SEVEN
7
DESIGNERS AND iOT DEVICES
I. “Allows designers to discover implicit patterns within
numerous facets of a users behavior. T...
DISNEY AND MACHINE LEARNING
I. Eventually Disney wants “a system that has gathered so
much data about a user over such a l...
GO WITH THE FLOW
SECTION EIGHT
8
TRADITIONAL UX FLOW Graphic: VladcDesign
DESIGN THINKING FLOW Graphic: IDEO
ML/PREDICTIVE MODELING FLOW
Graphic: Class Central
DATA SCIENCE
I. Look like familiar UX pattern/flows?
a. Divergence and convergence
II. Predictive modeling is the scientific...
CONCEPT TO CREATION
SECTION NINE
9
USING DEDUCTIVE & INDUCTIVE
REASONING
I. Helps designers with holistic understanding of actions
II. General rules for obse...
USING DEDUCTIVE & INDUCTIVE
REASONING
I. “Inductive reasoning starts with a group of specific
observations that look for pa...
STORYTELLING
I. Storytelling for developers and designers to visualize something together
a. More than just screens that u...
AIRBNB
I. Were creating a model to answer question “what will the booked
price of a listing be on any given day in the fut...
SKETCHES AND DIAGRAMMING
Amber
Cartwright,
AirBnB
*Smart Price
Regression
model next to a
visualization
explaining the
mod...
UX ML BEST
PRACTICES
SECTION TEN
10
ML UX BEST STEPS - PATRICK H.
1. Design tasks explicitly so that users can catch errors and redirect system
behavior.
2. F...
ML PROTOTYPING TOOLS
SECTION ELEVEN
11
WHY PROTOTYPE?
I. To understand - “freedom to think through all the different
ways you could solve the problem, discover n...
ML PROTOTYPING - WEKINATOR
I. ML challenges rapid prototyping because of architectures and training
II. Requisite of code ...
QUICK DEMO - WEKINATOR
http://www.wekinator.org/walkthrough/
MATHEMATICA
I. “Tool features a polished UI and does not require deep
understanding of programming, though some basic
fami...
MATHEMATICA CONTINUED
I. “Provides turnkey support for range of common ML tasks
like image recognition, text classification...
QUICK DEMO - MATHEMATICA
http://www.wolfram.com/broadcast/video.php?c=89&v=358
WRAPPING IT UP
I. Time to make that prototype and test with people
a. Are they happy? Are they freaked out? Do they trust
...
KEEP EXPLORING
I. HCI designers prototyping process post product release
II. UX for Machine Learning is growing at a rapid...
THAT’S ALL, FOLKS!
Bonnie Cook
bonnie@smashingboxes.com
TAKEAWAYS
READ
Patrick Hebron - @patrickhebron - www.patrickhebron.com
Pamela Pavliscak - @paminthelab - Emotionally Intel...
UX and Machine Learning
UX and Machine Learning
UX and Machine Learning
UX and Machine Learning
UX and Machine Learning
UX and Machine Learning
UX and Machine Learning
UX and Machine Learning
Upcoming SlideShare
Loading in …5
×

UX and Machine Learning

144 views

Published on

Learn how Machine Learning influences UX in this presentation by Smashing Boxes Designer Bonnie Cook.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

UX and Machine Learning

  1. 1. UX for Machine Learning Presented by Bonnie Cook
  2. 2. THE (WO)MAN AND THE MACHINE
  3. 3. STARTER UX ML QUESTIONS I. Where do I find information about UX for ML? II. Are there resources like tutorials? Demos? Thought leaders? III. How is machine learning going to change the future of UX? a. Opportunities and setbacks IV. Revolutions will take hold when tools become highly mature and more accessible (for designer/dev/user needs) a. We have not reached this level of expertise yet
  4. 4. ML RENAISSANCE I. “Machine learning is in the midst of a renaissance that will transform countless industries and provide designers with a wide assortment of new tools for better engaging with and understanding users. These technologies will give rise to new design challenges and require new ways of thinking about the design of user interfaces and interactions.” - Patrick Hebron, Machine Learning for Designers
  5. 5. UX FOR MACHINE LEARNING SECTION ONE 1
  6. 6. LET’S START WITH “WHAT’S UX?” I. “User Experience (UX) design is the process of creating products that provide meaningful and relevant experiences to users. This involves the design of the entire process of acquiring and integrating the product, including aspects of branding, design, usability, and function.” -The Interaction Design Foundation
  7. 7. THEN, MACHINE LEARNING? I. “Machine learning is a field of artificial intelligence that uses statistical techniques to give computer systems the ability to “learn” (e.g., progressively improve performance on a specific task) from data, without being explicitly programmed.” -Wikipedia II. Just in case: Algorithms are a process or set of rules to be followed in calculations or other problem-solving operations, especially by a computer.
  8. 8. THE EVOLUTION OF HCI SECTION TWO 2
  9. 9. USES OF HCI OVER TIME I. 1980’s - 1990’s: Desktop and Mental Models II. 1990’s - Early 2000’s: Collaboration and Communication III. Mid 2000’s - 2010: Self Expression and Social Change IV. Now - Future: Crafting technologies to the human experience
  10. 10. ML - A DIFFERENT KIND OF LOGIC I. Fuzzy logic - logic formed on approximations versus exact a. Need auxiliary knowledge of range of values i. Car sizes 1. Approximation of range 2. Can meaningfully discuss with other people II. Computers have “lived a life of experiential deprivation.” - Patrick Hebron
  11. 11. “PRIOR” TO MACHINE LEARNING I. Systems are used to Boolean logic where every expression has to ultimately evaluate as either true or false II. Machine Learning challenges this logic by giving machines experiential knowledge, which then helps deal with the fuzzier/human logic a. “Design challenges the problem of working with imprecise technology and unpredictable behavior.” - Ibid
  12. 12. FACE VALUE I. Infinite number of combos, genders, vantages, environments, races, etc… II. Examples of how we cannot reach perfect recognition with faces, despite a lifetime of examples
  13. 13. 20 QUESTIONS I. Ways we network with others a. Not sure what things about you will overlap (job etc) II. For conversational user interfaces a. Make sure you have the ability to create common ground with the users and the things the system can do to become better acquainted III. Works as a learning decision tree
  14. 14. DESIGN THINKING SECTION THREE 3
  15. 15. DESIGN THINKING “Design thinking is a human-centered approach to innovation that draws from the designer’s toolkit to integrate the needs of people, the possibilities of technology, and the requirements for business success.” - Tim Brown, president and CEO, IDEO
  16. 16. UX & DESIGN THINKING I. Design Thinking is a process or methodology. It is about applying a specific way of thinking to a situation. II. Design Thinking methods can be utilized not only by UX designers, but also developers, product owners and marketers. III. Creative insight isn’t limited to just designers a. Designer’s emphasis on criticism
  17. 17. DT & ML STAGES I. Design Thinking Stages a. Empathize i. Who is my user? ii. Pain points b. Define i. Point of View c. Ideate i. Brainstorming d. Prototype i. Working and ready e. Test i. Monitor Use ii. Effectiveness I. Machine Learning Stages a. Analyze i. Capture key decisions to find out variables/metrics b. Synthesize i. Combo of separate elements to create new c. Ideate i. Small sample of data applied to various analytical models/algorithms for insight d. Tuning i. Additional data capture e. Validate i. UX and analytic model tuning perspectives Source: Abhay Pandey - Medium
  18. 18. WHO’S LISTENING? SECTION FOUR 4
  19. 19. YOU AND ML ON THE DAILY I. Siri - taught to understand the nuances of our language II. Facebook - uses algorithms to recognize faces from contact list III. Google Maps - traffic speed/time and best possible route IV. Google Search - recommendations based on searches a. 2012 - Introduced Knowledge Graph - algorithm used to decipher semantic content of a search query V. PayPal - ML to fight off fraud. Analyzes tons of consumer data and evaluate risks VI. Netflix - video recommendation engine VII. Uber - algorithms to determine arrival times, pick up locations, and UberEATS’ VIII. Lyst - match customer searches with relevant rec. Meta-data tags for visual comp. IX. Spotify - ML to establish likes and dislikes and provides list of related tracks
  20. 20. Phoneme - Any of the perceptually distinct units of sound in a specified language that distinguish one word from another, for example p, b, d, and t in the English words pad, pat, bad, and bat.
  21. 21. Image: Sam Drozdov
  22. 22. CURATION VS ALGORITHMS I. Apple Music has DJs who custom create playlists rather than how Spotify or Pandora works a. Designers need to be able to design BOTH algorithmic data and curated data - they’re equally important personas b. What data do you have? c. What data do you want?
  23. 23. ML LANGUAGE SECTION FIVE 5
  24. 24. UNDERSTANDING ML LANGUAGE I. Machines like Siri understanding spoken language is a big deal because it allows development of more natural interaction paradigms II. Diverse vocals/speech patterns III. Speech-to-text sometimes struggles for humans and computers alike (Shazam and SoundHound)
  25. 25. “GAMING THE ALGORITHM” I. Pamela Pavilscak, author of Emotionally Intelligent Design, talks about the strategic evasion of computers knowing too much about us a. Then we’re disappointed that we get a bunch of info not related to us i. Private browsing, ads vs ad Google search b. Algorithms face backwards, not forward i. “Making predictions on past behaviors doesn’t take into account what is essentially human. We change a little every day. We get interested in new things. We change our minds. We feel a little conflicted. Algorithms have a hard time with that.”
  26. 26. OTHER ALGORITHMIC SETBACKS I. They’re working with the big picture of what you do a. Topics based on behavior II. Incomplete information - algorithms keep up with big purchases but other data points can be inaccurate III. Resources - history of sites (anticipatory) but doesn’t understand personalization completely
  27. 27. ML EMPATHY SECTION SIX 6
  28. 28. A BEHAVIORAL PROBLEM I. “We are more than just the sum of our behaviors. We don’t behave rationally, maybe not even predictable irrationally. A human presence can improve the algorithmic experience.” - Pamela Pavilscak
  29. 29. ALGORITHMIC EMPATHY I. Algorithms change the way we explore and understand empathy with users II. Pamela Pavilscak’s process to “Cultivate Algorithmic Empathy” a. Research practices: Data Role Play, Algorithm Swap, Data Doubles, Algorithmic Personas, and Shared Mythologies (broken out on next slides)
  30. 30. DATA ROLE PLAY I. It acknowledges the abstraction of algorithms a. Good way to start the conversation about this new type of empathy b. Would it be awkward to ask a stranger for private information like email and phone number? i. Even worse - getting their info without them knowing it
  31. 31. ALGORITHM SWAP I. We have very private experiences with our devices II. Might change once we start interacting with voices rather than hands a. Right now it feels unnatural to spend time with someone else’s private self i. YET, can be meaningful to see that side of the person you’re designing for
  32. 32. ALGORITHMIC PERSONAS I. Design teams chose a base persona on a combination of: a. Demographics b. Interviews c. Behavioral data that is collected
  33. 33. SHARED MYTHOLOGIES I. When we reach an algorithmic disconnect, people start to speculate as to WHY it’s no longer working II. So INSTEAD of letting myth after myth float around a. Design to reveal the algorithm i. “See yourself through the ads you encounter” like on Facebook etc…
  34. 34. ML iOT DEVICES SECTION SEVEN 7
  35. 35. DESIGNERS AND iOT DEVICES I. “Allows designers to discover implicit patterns within numerous facets of a users behavior. These patterns carry inherent meanings, which can be learned from and acted upon, even if the user is not expressly aware of having communicated them. In this sense, these implicit patterns can be thought of as input modalities that, in practice, serve a very similar purpose to more tangible input modes.” - Patrick Hebron
  36. 36. DISNEY AND MACHINE LEARNING I. Eventually Disney wants “a system that has gathered so much data about a user over such a long period of time that the anticipatory design creates a completely different experience for each person, despite the fact that all the users are in the same physical location.” -Bryon Houwens
  37. 37. GO WITH THE FLOW SECTION EIGHT 8
  38. 38. TRADITIONAL UX FLOW Graphic: VladcDesign
  39. 39. DESIGN THINKING FLOW Graphic: IDEO
  40. 40. ML/PREDICTIVE MODELING FLOW Graphic: Class Central
  41. 41. DATA SCIENCE I. Look like familiar UX pattern/flows? a. Divergence and convergence II. Predictive modeling is the scientific process within machine learning that is incredibly important for an aligned UX methodology and practice III. We can only do but so much with current material a. Present opportunities for growth
  42. 42. CONCEPT TO CREATION SECTION NINE 9
  43. 43. USING DEDUCTIVE & INDUCTIVE REASONING I. Helps designers with holistic understanding of actions II. General rules for observing and collecting data a. “Deductive - broad theory about rules governing a system, distill theory into more specific hypotheses, gather specific observations and test against our own hypotheses to confirm if original theory was right or wrong.”- Patrick Hebron
  44. 44. USING DEDUCTIVE & INDUCTIVE REASONING I. “Inductive reasoning starts with a group of specific observations that look for patterns in those observations, formulate tentative hypotheses, and ultimately try to produce a general theory that encompasses original observations.” - Patrick Hebron
  45. 45. STORYTELLING I. Storytelling for developers and designers to visualize something together a. More than just screens that user interacts with, but understand why things are happening behind the curtains II. Importance of storytelling in the beginning of product creation a. The experience impacting the user i. Storyboards ii. Prototypes iii. Strategy Decks iv. Diagrams III. Empowerment as a shared vision and language with fewer steps
  46. 46. AIRBNB I. Were creating a model to answer question “what will the booked price of a listing be on any given day in the future?” a. Developers were talking alphas and betas - foreign material to many designers II. Asked developers to sketch out the idea in a diagram and do a walk-through a. THIS is where the connection was made III. Language barrier broken down and then created a shared language about the product and moving forward (storytelling)
  47. 47. SKETCHES AND DIAGRAMMING Amber Cartwright, AirBnB *Smart Price Regression model next to a visualization explaining the model is made of three parts that vary by host.
  48. 48. UX ML BEST PRACTICES SECTION TEN 10
  49. 49. ML UX BEST STEPS - PATRICK H. 1. Design tasks explicitly so that users can catch errors and redirect system behavior. 2. Fallback mechanisms - circumvent in ML functionality and perform tasks with explicit logic. 3. Test in as many environments and limited audience release. 4. Use metrics—confidence scores—to assess feature. Realistic expectations. 5. Consider impressive sounding feature if unreliable. Failure vs feature. 6. Make risks obvious. Allow users to decide if benefit outweighs risk. 7. Serious consequences—even death—in system failure. Take extreme caution to assess risk/liabilities.
  50. 50. ML PROTOTYPING TOOLS SECTION ELEVEN 11
  51. 51. WHY PROTOTYPE? I. To understand - “freedom to think through all the different ways you could solve the problem, discover new problems that need to be addressed, and help you refine your ideas with the feedback you receive.” II. To test and improve - “the main reason to prototype” III. To communicate - “invest team, stakeholders, or end users” IV. To advocate - “for the design or direction” - Kathryn McElroy, Prototyping for Physical and Digital Products
  52. 52. ML PROTOTYPING - WEKINATOR I. ML challenges rapid prototyping because of architectures and training II. Requisite of code or data sets limits tools (evolving) III. Currently, there are some existing tools we can use a. Wekinator - it’s free and open source i. Allows development of experimental gesture recognition and interface controllers for microphones, webcams, Kinect etc... ii. Programming free! iii. “Walks designer through process where defines particular gesture by demoing it to machine and then associates gesture with the desired output action.” -Ibid
  53. 53. QUICK DEMO - WEKINATOR http://www.wekinator.org/walkthrough/
  54. 54. MATHEMATICA I. “Tool features a polished UI and does not require deep understanding of programming, though some basic familiarity with text-based scripting is helpful II. Machine learning features applied to wide range of data types and auto data preprocessing and model selection features will help users get good results without a great deal of trial-and-error or deep knowledge of a particular model’s training parameters.” -Patrick Hebron
  55. 55. MATHEMATICA CONTINUED I. “Provides turnkey support for range of common ML tasks like image recognition, text classification, and classification or regression of generic data. II. Datasets can be loaded through interactive, visual interface III. Extremely well documented and embeds assistive tools like feature suggestion and autocompletion directly to its interface” -Ibid IV. Not free www.wolfram.com/mathematica
  56. 56. QUICK DEMO - MATHEMATICA http://www.wolfram.com/broadcast/video.php?c=89&v=358
  57. 57. WRAPPING IT UP I. Time to make that prototype and test with people a. Are they happy? Are they freaked out? Do they trust you? Do they trust the device(s)? b. Observe, listen, chronicle. c. Why? What can you do? What can’t you do?
  58. 58. KEEP EXPLORING I. HCI designers prototyping process post product release II. UX for Machine Learning is growing at a rapid pace and keep in mind we’re all figuring it out—experts included III. Use tried and true methods where it is appropriate IV. Remember, YOU already use machine learning—think like another user V. Errors are opportunities
  59. 59. THAT’S ALL, FOLKS! Bonnie Cook bonnie@smashingboxes.com
  60. 60. TAKEAWAYS READ Patrick Hebron - @patrickhebron - www.patrickhebron.com Pamela Pavliscak - @paminthelab - Emotionally Intelligent Design LEARN Wekinator - http://www.wekinator.org/ Mathematica - http://www.wolfram.com/mathematica/ LET US HELP! SB Design Sprints - 2-hour to 2-day discovery sprint to help identify your needs, personas and design schemas. Prototype Workshop - Come with an idea, leave with a clickable prototype Dave Shepley - dave.shepley@smashingboxes.com @smashingboxes

×