PROTOTYPING AND FIELD EVALUATION AS RISK MITIGATION IN MOBILE PRODUCT DEVELOPMENT Frank Bentley Motorola Applied Research ...
In fewer words… <ul><li>How to decide which experiences to  productize with minimal effort. </li></ul>
Outline <ul><li>Who we are/were </li></ul><ul><li>How we get ideas </li></ul><ul><li>What we build </li></ul><ul><li>How w...
Preface <ul><li>Meant to encourage lively discussion </li></ul><ul><li>Please ask questions at any point! </li></ul>
Social Media Research Lab <ul><li>Charter: Invent things that help make people feel more connected </li></ul><ul><li>12 “t...
How we get ideas… <ul><li>We spend time with people! </li></ul><ul><li>Contextual Inquiry </li></ul><ul><li>Home tours </l...
We collect and analyze data… <ul><li>From a study on music use: </li></ul><ul><ul><li>“ Some of my CDs remind me of a time...
Design ideas based in real data
Research questions to mitigate risk <ul><li>How does (or doesn’t!) the experience fit into everyday life? </li></ul><ul><l...
What we build: <ul><li>Several rapid functional prototypes of most promising design ideas from ethnographic work </li></ul...
Examples:
How we test: <ul><li>Small field-research teams (2-3 people) </li></ul><ul><li>Rapid data collection and analysis </li></u...
Case Study: Ambient Mobile Communications <ul><li>How to use ambient context to aid mobile  </li></ul><ul><li>communicatio...
Ambient, defined… <ul><li>Data received in the environment, without specific effort </li></ul><ul><li>For mobile: data obs...
Case study: Ambient Mobile Communications <ul><li>Project goal: Explore how people can communicate passively through ambie...
Location Sharing Study <ul><li>Objective: To use current location sharing practices as inspiration for the design of new m...
Motion Presence (2006) <ul><li>Micro-coordination: The conversation and planning that occurs around planning to meet </li>...
Motion Presence <ul><li>Build only what you need </li></ul><ul><ul><li>Built simple phonebook (call, text, name, number, m...
Motion Presence: Findings <ul><li>Participants were able to infer: Location, Activity, Availability, Arrival Time, Destina...
Risk Mitigated… <ul><li>While research hypotheses confirmed, business case not there </li></ul><ul><ul><li>Application not...
Music Context Study <ul><li>Objective: Understand how context plays a role in music selection, playback, and sharing </li>...
Music Presence (2005) <ul><li>Research question:  </li></ul><ul><ul><li>How will an ambient awareness of  friend’s music p...
Music Presence <ul><li>Build only what you need </li></ul><ul><ul><li>No code on phone: web service that monitors last.fm ...
Music Presence Findings <ul><li>Used music playing to infer other presence information (at home, bored, etc.) </li></ul><u...
Risk Mitigated… <ul><li>Concept successful </li></ul><ul><ul><li>Further iterations performed </li></ul></ul><ul><ul><li>L...
Photo Sharing Study <ul><li>Objective: To understand how and why people share photos </li></ul><ul><li>Methods:  </li></ul...
Photo Presence (2007) <ul><li>Research questions:  </li></ul><ul><ul><li>How can photos be used to ambiently share an expe...
Photo Presence <ul><li>Build only what you need </li></ul><ul><ul><li>J2ME application built by Sydney startup </li></ul><...
Photo Presence Findings <ul><li>Telepresence </li></ul><ul><ul><li>Being able to log on and see what she was doing, like w...
Risk Mitigated… <ul><li>Concept successful </li></ul><ul><ul><li>Startup continued on path </li></ul></ul><ul><ul><li>Chan...
Overall guiding principles: <ul><li>Keep prototyping simple and focused on your research questions </li></ul><ul><li>Evalu...
Benefits: <ul><li>Validate main user value without building full application/service </li></ul><ul><li>Introduce several c...
Principles <ul><li>Build only what you need </li></ul><ul><li>Build the experience, not the technology </li></ul><ul><li>S...
Upcoming SlideShare
Loading in …5
×

Prototyping and Field Evaluation as Risk Mitigation in Mobile Product Development

3,014 views

Published on

In order to develop a successful mobile application, it is necessary that the experience fits into people’s lives and solves real problems that they face. In Motorola Labs, we often finish exploratory ethnographic work with hundreds of design ideas for new mobile experiences. In order to identify those with the most value, we prototype key components of these ideas and test them with real people in their daily lives. I’ll talk about our strategies for rapid prototyping and field evaluation using examples from our work on the Ambient Mobile Communications program.

Published in: Technology
0 Comments
15 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,014
On SlideShare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
0
Comments
0
Likes
15
Embeds 0
No embeds

No notes for slide

Prototyping and Field Evaluation as Risk Mitigation in Mobile Product Development

  1. PROTOTYPING AND FIELD EVALUATION AS RISK MITIGATION IN MOBILE PRODUCT DEVELOPMENT Frank Bentley Motorola Applied Research and Technology Center
  2. In fewer words… <ul><li>How to decide which experiences to productize with minimal effort. </li></ul>
  3. Outline <ul><li>Who we are/were </li></ul><ul><li>How we get ideas </li></ul><ul><li>What we build </li></ul><ul><li>How we test </li></ul><ul><li>Case Study: Ambient Mobile Communications </li></ul>
  4. Preface <ul><li>Meant to encourage lively discussion </li></ul><ul><li>Please ask questions at any point! </li></ul>
  5. Social Media Research Lab <ul><li>Charter: Invent things that help make people feel more connected </li></ul><ul><li>12 “transdiciplinary” researchers – anthropology, human computer interaction, cognitive psychology, computer science, electrical engineering, business </li></ul><ul><li>Time horizon of 2-5 years to product </li></ul><ul><li>Work closely with business groups throughout process </li></ul>
  6. How we get ideas… <ul><li>We spend time with people! </li></ul><ul><li>Contextual Inquiry </li></ul><ul><li>Home tours </li></ul><ul><li>Diary studies </li></ul><ul><li>Activity logs </li></ul><ul><li>Shadowing </li></ul><ul><li>Observation </li></ul>
  7. We collect and analyze data… <ul><li>From a study on music use: </li></ul><ul><ul><li>“ Some of my CDs remind me of a time I had and I like to put it on and remember that time I had with it. Fall always gets me in the mood to play music I always listen to. Me and my sisters sitting on the porch and talking in our Nike sweatshirts. We used to play this song over and over when we went to her house” </li></ul></ul><ul><ul><li>“ I often don’t like listening to the old stuff because…it sometimes takes me back to somewhere I don’t want to be…” </li></ul></ul><ul><ul><li>“ She made this CD of music that related to them, she had this basket of CDs at the wedding and everyone took one and the bubbles…” </li></ul></ul><ul><ul><li>Has CDs in a stack with no cases – sorts like playing cards </li></ul></ul>
  8. Design ideas based in real data
  9. Research questions to mitigate risk <ul><li>How does (or doesn’t!) the experience fit into everyday life? </li></ul><ul><li>What existing solutions do people have to meet this needs? What are existing workarounds? </li></ul><ul><li>What aspects of our prototype solution need to be optimized in a real product? (is it too slow, not integrated enough, etc.) </li></ul>
  10. What we build: <ul><li>Several rapid functional prototypes of most promising design ideas from ethnographic work </li></ul><ul><li>Goal: to decide on experiences that fit into people’s lives and are worth turning into a product </li></ul><ul><li>Principles: </li></ul><ul><li>Build only what you need </li></ul><ul><li>Build the experience, not the technology </li></ul>
  11. Examples:
  12. How we test: <ul><li>Small field-research teams (2-3 people) </li></ul><ul><li>Rapid data collection and analysis </li></ul><ul><li>8-10 users, 2-4 weeks (go until you see repetitions in data) </li></ul><ul><li>Principles: </li></ul><ul><li>Social groups for social technologies </li></ul><ul><li>Real context of use </li></ul><ul><li>Primary device </li></ul><ul><li>Field-based data collection </li></ul>
  13. Case Study: Ambient Mobile Communications <ul><li>How to use ambient context to aid mobile </li></ul><ul><li>communication and enhance existing </li></ul><ul><li>social communities </li></ul>
  14. Ambient, defined… <ul><li>Data received in the environment, without specific effort </li></ul><ul><li>For mobile: data observable while performing normal tasks on device </li></ul><ul><ul><li>e.g. integrated into phone book, messaging application, media player </li></ul></ul>
  15. Case study: Ambient Mobile Communications <ul><li>Project goal: Explore how people can communicate passively through ambient cues </li></ul><ul><li>Initial field work </li></ul><ul><ul><li>Location Sharing Study </li></ul></ul><ul><ul><li>Music Sharing Study </li></ul></ul><ul><ul><li>Photo Context Study </li></ul></ul><ul><li>Concepts: </li></ul><ul><ul><li>Motion Presence </li></ul></ul><ul><ul><li>Music Presence </li></ul></ul><ul><ul><li>Photo Presence </li></ul></ul>
  16. Location Sharing Study <ul><li>Objective: To use current location sharing practices as inspiration for the design of new mobile social applications </li></ul><ul><li>Methods: </li></ul><ul><ul><li>Recorded phone calls of 7 participants for one week </li></ul></ul><ul><ul><li>Analyzed for purpose of location disclosure, place disclosed, social relationships involved, truth </li></ul></ul><ul><li>Finding: People often know location of close friends and family, but are unaware of exact transitions </li></ul>
  17. Motion Presence (2006) <ul><li>Micro-coordination: The conversation and planning that occurs around planning to meet </li></ul><ul><li>Research questions: </li></ul><ul><ul><li>Will knowing if someone is in motion or at a place help people micro-coordinate? </li></ul></ul><ul><ul><li>Does not sharing the location itself mitigate privacy concerns? </li></ul></ul>
  18. Motion Presence <ul><li>Build only what you need </li></ul><ul><ul><li>Built simple phonebook (call, text, name, number, motion only) </li></ul></ul><ul><ul><li>1 person-week of development </li></ul></ul><ul><li>Build the experience, not the technology </li></ul><ul><ul><li>Sent presence over port-based SMS </li></ul></ul><ul><li>Social groups for social technologies </li></ul><ul><ul><li>4 groups of friends/family (10 total) </li></ul></ul><ul><li>Real context of use </li></ul><ul><ul><li>Used in their everyday lives for two weeks </li></ul></ul><ul><li>Primary device </li></ul><ul><ul><li>Moved SIM and phonebook into our phone for duration of trial </li></ul></ul><ul><li>Field-based data collection </li></ul><ul><ul><li>Voicemails and interviews + usage logs </li></ul></ul>
  19. Motion Presence: Findings <ul><li>Participants were able to infer: Location, Activity, Availability, Arrival Time, Destination </li></ul><ul><li>Participants used the application to: </li></ul><ul><ul><li>Moderate availability </li></ul></ul><ul><ul><li>Micro-coordinate </li></ul></ul><ul><ul><li>Arrive at the same time </li></ul></ul><ul><ul><li>Get more time at their current activity </li></ul></ul><ul><ul><li>See people were following through on commitments </li></ul></ul><ul><ul><li>Check on other’s safety </li></ul></ul><ul><ul><li>Social awareness – know what’s going on with others </li></ul></ul><ul><li>Privacy not a major concern given plausible deniability </li></ul>
  20. Risk Mitigated… <ul><li>While research hypotheses confirmed, business case not there </li></ul><ul><ul><li>Application not heavily used </li></ul></ul><ul><ul><li>Participants did not want to continue using service </li></ul></ul><ul><ul><li>Calling others seen as sufficient solution </li></ul></ul><ul><li>Concepts of ambient micro-coordination useful </li></ul><ul><ul><li>Lessons learned to design future applications in the business </li></ul></ul><ul><ul><li>Concept of inference from ambiguous social data </li></ul></ul>
  21. Music Context Study <ul><li>Objective: Understand how context plays a role in music selection, playback, and sharing </li></ul><ul><li>Methods: </li></ul><ul><ul><li>12 participants </li></ul></ul><ul><ul><li>Home tour, contextual inquiry, semi-structured interviews </li></ul></ul><ul><li>Findings: </li></ul><ul><ul><li>Music knowledge often comes from friends </li></ul></ul><ul><ul><li>Music choice often depends on occasion, who is around </li></ul></ul>
  22. Music Presence (2005) <ul><li>Research question: </li></ul><ul><ul><li>How will an ambient awareness of friend’s music playing be used? </li></ul></ul><ul><li>Application: </li></ul><ul><ul><li>View friend’s initials, music metadatain SMS inbox </li></ul></ul><ul><ul><li>2 nd round, J2ME application with lightweight feedback, 30 second clips </li></ul></ul>
  23. Music Presence <ul><li>Build only what you need </li></ul><ul><ul><li>No code on phone: web service that monitors last.fm feeds and sends SMS to friends </li></ul></ul><ul><li>Build the experience, not the technology </li></ul><ul><ul><li>Sent updates over SMS, not fully integrated with last.fm </li></ul></ul><ul><li>Social groups for social technologies </li></ul><ul><ul><li>1 group of 4 friends (initial study) </li></ul></ul><ul><li>Real context of use </li></ul><ul><ul><li>Used in their everyday lives for 7 days </li></ul></ul><ul><li>Primary device </li></ul><ul><ul><li>Never had to switch phones (just ensured they added SMS plans) </li></ul></ul><ul><li>Field-based data collection </li></ul><ul><ul><li>Voicemails and interviews + server logs </li></ul></ul>
  24. Music Presence Findings <ul><li>Used music playing to infer other presence information (at home, bored, etc.) </li></ul><ul><li>Explicitly played music so that other people would notice (e.g. a song a friend had given) </li></ul><ul><li>SMS + IM around playback (“that rocks!”) </li></ul><ul><li>“ I was looking at her [music updates] to see when she’d gotten back from going out.” </li></ul>
  25. Risk Mitigated… <ul><li>Concept successful </li></ul><ul><ul><li>Further iterations performed </li></ul></ul><ul><ul><li>Learnings shared with business teams </li></ul></ul><ul><li>Helped to create more social experience </li></ul><ul><ul><li>Added lightweight responses (thumbs up/down) </li></ul></ul><ul><ul><li>Added 30 second clips to help with unfamiliar songs </li></ul></ul>
  26. Photo Sharing Study <ul><li>Objective: To understand how and why people share photos </li></ul><ul><li>Methods: </li></ul><ul><ul><li>5 users </li></ul></ul><ul><ul><li>Home tours, semi-structured interviews </li></ul></ul><ul><li>Findings </li></ul><ul><ul><li>Desire to share photos in-the-moment </li></ul></ul><ul><ul><li>Sharing photos to particular (and changing) groups of people </li></ul></ul>
  27. Photo Presence (2007) <ul><li>Research questions: </li></ul><ul><ul><li>How can photos be used to ambiently share an experience in real time? </li></ul></ul><ul><ul><li>How can we encourage conversations around real-time media? </li></ul></ul>
  28. Photo Presence <ul><li>Build only what you need </li></ul><ul><ul><li>J2ME application built by Sydney startup </li></ul></ul><ul><li>Build the experience, not the technology </li></ul><ul><ul><li>Application integrated with existing web service </li></ul></ul><ul><li>Social groups for social technologies </li></ul><ul><ul><li>3 social groups (10 participants) </li></ul></ul><ul><li>Real context of use </li></ul><ul><ul><li>Used in their everyday lives for three weeks </li></ul></ul><ul><li>Primary device </li></ul><ul><ul><li>Participants were given devices with application preloaded, switched SIM and phonebook </li></ul></ul><ul><li>Field-based data collection </li></ul><ul><ul><li>Voicemails and interviews </li></ul></ul>
  29. Photo Presence Findings <ul><li>Telepresence </li></ul><ul><ul><li>Being able to log on and see what she was doing, like when we were at work and couldn’t talk – B2 </li></ul></ul><ul><ul><li>There’s something kind of satisfying about the immediacy of right now in California my brother is doing this – B1 </li></ul></ul><ul><li>Automatic </li></ul><ul><ul><li>I like it cause I can just take pictures and I don’t have to fuss with it and it gets uploaded. –C3 </li></ul></ul><ul><li>Communication </li></ul><ul><ul><li>Looking at people’s tiles and looking at people’s comments on mine. It shows that they were interested in what I liked. Took enough time out of their day to make a comment. To some people that might make their day or brighten up their day – A2 </li></ul></ul>
  30. Risk Mitigated… <ul><li>Concept successful </li></ul><ul><ul><li>Startup continued on path </li></ul></ul><ul><ul><li>Changes to focus on small-group sharing </li></ul></ul><ul><li>Learnings helped clarify value prop </li></ul><ul><ul><li>Automatic, ambient sharing throughout day </li></ul></ul><ul><ul><li>Small groups of close friends and family </li></ul></ul><ul><ul><li>Simple communication around media </li></ul></ul>
  31. Overall guiding principles: <ul><li>Keep prototyping simple and focused on your research questions </li></ul><ul><li>Evaluate with real users in their real situations of interaction </li></ul><ul><li>Try a lot of ideas and pick ones that stick </li></ul><ul><li>Iterate! </li></ul>
  32. Benefits: <ul><li>Validate main user value without building full application/service </li></ul><ul><li>Introduce several cycles of iteration before committing to product </li></ul><ul><li>Get initial feedback for market segmentation </li></ul>
  33. Principles <ul><li>Build only what you need </li></ul><ul><li>Build the experience, not the technology </li></ul><ul><li>Social groups for social technologies </li></ul><ul><li>Real context of use </li></ul><ul><li>Primary device </li></ul><ul><li>Field-based data collection </li></ul><ul><li>Questions? Discussion? </li></ul>

×