PROTOTYPING AND FIELD EVALUATION AS RISK MITIGATION IN MOBILE PRODUCT DEVELOPMENT Frank Bentley Motorola Applied Research and Technology Center

In fewer words… How to decide which experiences to productize with minimal effort.

How to decide which experiences to productize with minimal effort.

Outline Who we are/were How we get ideas What we build How we test Case Study: Ambient Mobile Communications

Who we are/were

How we get ideas

What we build

How we test

Case Study: Ambient Mobile Communications

Preface Meant to encourage lively discussion Please ask questions at any point!

Meant to encourage lively discussion

Please ask questions at any point!

Social Media Research Lab Charter: Invent things that help make people feel more connected 12 “transdiciplinary” researchers – anthropology, human computer interaction, cognitive psychology, computer science, electrical engineering, business Time horizon of 2-5 years to product Work closely with business groups throughout process

Charter: Invent things that help make people feel more connected

12 “transdiciplinary” researchers – anthropology, human computer interaction, cognitive psychology, computer science, electrical engineering, business

Time horizon of 2-5 years to product

Work closely with business groups throughout process

How we get ideas… We spend time with people! Contextual Inquiry Home tours Diary studies Activity logs Shadowing Observation

We spend time with people!

Contextual Inquiry

Home tours

Diary studies

Activity logs

Shadowing

Observation

We collect and analyze data… From a study on music use: “ Some of my CDs remind me of a time I had and I like to put it on and remember that time I had with it. Fall always gets me in the mood to play music I always listen to. Me and my sisters sitting on the porch and talking in our Nike sweatshirts. We used to play this song over and over when we went to her house” “ I often don’t like listening to the old stuff because…it sometimes takes me back to somewhere I don’t want to be…” “ She made this CD of music that related to them, she had this basket of CDs at the wedding and everyone took one and the bubbles…” Has CDs in a stack with no cases – sorts like playing cards

From a study on music use:

“ Some of my CDs remind me of a time I had and I like to put it on and remember that time I had with it. Fall always gets me in the mood to play music I always listen to. Me and my sisters sitting on the porch and talking in our Nike sweatshirts. We used to play this song over and over when we went to her house”

“ I often don’t like listening to the old stuff because…it sometimes takes me back to somewhere I don’t want to be…”

“ She made this CD of music that related to them, she had this basket of CDs at the wedding and everyone took one and the bubbles…”

Has CDs in a stack with no cases – sorts like playing cards

Design ideas based in real data

Research questions to mitigate risk How does (or doesn’t!) the experience fit into everyday life? What existing solutions do people have to meet this needs? What are existing workarounds? What aspects of our prototype solution need to be optimized in a real product? (is it too slow, not integrated enough, etc.)

How does (or doesn’t!) the experience fit into everyday life?

What existing solutions do people have to meet this needs? What are existing workarounds?

What aspects of our prototype solution need to be optimized in a real product? (is it too slow, not integrated enough, etc.)

What we build: Several rapid functional prototypes of most promising design ideas from ethnographic work Goal: to decide on experiences that fit into people’s lives and are worth turning into a product Principles: Build only what you need Build the experience, not the technology

Several rapid functional prototypes of most promising design ideas from ethnographic work

Goal: to decide on experiences that fit into people’s lives and are worth turning into a product

Principles:

Build only what you need

Build the experience, not the technology

Examples:

How we test: Small field-research teams (2-3 people) Rapid data collection and analysis 8-10 users, 2-4 weeks (go until you see repetitions in data) Principles: Social groups for social technologies Real context of use Primary device Field-based data collection

Small field-research teams (2-3 people)

Rapid data collection and analysis

8-10 users, 2-4 weeks (go until you see repetitions in data)

Principles:

Social groups for social technologies

Real context of use

Primary device

Field-based data collection

Case Study: Ambient Mobile Communications How to use ambient context to aid mobile communication and enhance existing social communities

How to use ambient context to aid mobile

communication and enhance existing

social communities

Ambient, defined… Data received in the environment, without specific effort For mobile: data observable while performing normal tasks on device e.g. integrated into phone book, messaging application, media player

Data received in the environment, without specific effort

For mobile: data observable while performing normal tasks on device

e.g. integrated into phone book, messaging application, media player

Case study: Ambient Mobile Communications Project goal: Explore how people can communicate passively through ambient cues Initial field work Location Sharing Study Music Sharing Study Photo Context Study Concepts: Motion Presence Music Presence Photo Presence

Project goal: Explore how people can communicate passively through ambient cues

Initial field work

Location Sharing Study

Music Sharing Study

Photo Context Study

Concepts:

Motion Presence

Music Presence

Photo Presence

Location Sharing Study Objective: To use current location sharing practices as inspiration for the design of new mobile social applications Methods: Recorded phone calls of 7 participants for one week Analyzed for purpose of location disclosure, place disclosed, social relationships involved, truth Finding: People often know location of close friends and family, but are unaware of exact transitions

Objective: To use current location sharing practices as inspiration for the design of new mobile social applications

Methods:

Recorded phone calls of 7 participants for one week

Analyzed for purpose of location disclosure, place disclosed, social relationships involved, truth

Finding: People often know location of close friends and family, but are unaware of exact transitions

Motion Presence (2006) Micro-coordination: The conversation and planning that occurs around planning to meet Research questions: Will knowing if someone is in motion or at a place help people micro-coordinate? Does not sharing the location itself mitigate privacy concerns?

Micro-coordination: The conversation and planning that occurs around planning to meet

Research questions:

Will knowing if someone is in motion or at a place help people micro-coordinate?

Does not sharing the location itself mitigate privacy concerns?

Motion Presence Build only what you need Built simple phonebook (call, text, name, number, motion only) 1 person-week of development Build the experience, not the technology Sent presence over port-based SMS Social groups for social technologies 4 groups of friends/family (10 total) Real context of use Used in their everyday lives for two weeks Primary device Moved SIM and phonebook into our phone for duration of trial Field-based data collection Voicemails and interviews + usage logs

Build only what you need

Built simple phonebook (call, text, name, number, motion only)

1 person-week of development

Build the experience, not the technology

Sent presence over port-based SMS

Social groups for social technologies

4 groups of friends/family (10 total)

Real context of use

Used in their everyday lives for two weeks

Primary device

Moved SIM and phonebook into our phone for duration of trial

Field-based data collection

Voicemails and interviews + usage logs

Motion Presence: Findings Participants were able to infer: Location, Activity, Availability, Arrival Time, Destination Participants used the application to: Moderate availability Micro-coordinate Arrive at the same time Get more time at their current activity See people were following through on commitments Check on other’s safety Social awareness – know what’s going on with others Privacy not a major concern given plausible deniability

Participants were able to infer: Location, Activity, Availability, Arrival Time, Destination

Participants used the application to:

Moderate availability

Micro-coordinate

Arrive at the same time

Get more time at their current activity

See people were following through on commitments

Check on other’s safety

Social awareness – know what’s going on with others

Privacy not a major concern given plausible deniability

Risk Mitigated… While research hypotheses confirmed, business case not there Application not heavily used Participants did not want to continue using service Calling others seen as sufficient solution Concepts of ambient micro-coordination useful Lessons learned to design future applications in the business Concept of inference from ambiguous social data

While research hypotheses confirmed, business case not there

Application not heavily used

Participants did not want to continue using service

Calling others seen as sufficient solution

Concepts of ambient micro-coordination useful

Lessons learned to design future applications in the business

Concept of inference from ambiguous social data

Music Context Study Objective: Understand how context plays a role in music selection, playback, and sharing Methods: 12 participants Home tour, contextual inquiry, semi-structured interviews Findings: Music knowledge often comes from friends Music choice often depends on occasion, who is around

Objective: Understand how context plays a role in music selection, playback, and sharing

Methods:

12 participants

Home tour, contextual inquiry, semi-structured interviews

Findings:

Music knowledge often comes from friends

Music choice often depends on occasion, who is around

Music Presence (2005) Research question: How will an ambient awareness of friend’s music playing be used? Application: View friend’s initials, music metadatain SMS inbox 2 nd round, J2ME application with lightweight feedback, 30 second clips

Research question:

How will an ambient awareness of friend’s music playing be used?

Application:

View friend’s initials, music metadatain SMS inbox

2 nd round, J2ME application with lightweight feedback, 30 second clips

Music Presence Build only what you need No code on phone: web service that monitors last.fm feeds and sends SMS to friends Build the experience, not the technology Sent updates over SMS, not fully integrated with last.fm Social groups for social technologies 1 group of 4 friends (initial study) Real context of use Used in their everyday lives for 7 days Primary device Never had to switch phones (just ensured they added SMS plans) Field-based data collection Voicemails and interviews + server logs

Build only what you need

No code on phone: web service that monitors last.fm feeds and sends SMS to friends

Build the experience, not the technology

Sent updates over SMS, not fully integrated with last.fm

Social groups for social technologies

1 group of 4 friends (initial study)

Real context of use

Used in their everyday lives for 7 days

Primary device

Never had to switch phones (just ensured they added SMS plans)

Field-based data collection

Voicemails and interviews + server logs

Music Presence Findings Used music playing to infer other presence information (at home, bored, etc.) Explicitly played music so that other people would notice (e.g. a song a friend had given) SMS + IM around playback (“that rocks!”) “ I was looking at her [music updates] to see when she’d gotten back from going out.”

Used music playing to infer other presence information (at home, bored, etc.)

Explicitly played music so that other people would notice (e.g. a song a friend had given)

SMS + IM around playback (“that rocks!”)

“ I was looking at her [music updates] to see when she’d gotten back from going out.”

Risk Mitigated… Concept successful Further iterations performed Learnings shared with business teams Helped to create more social experience Added lightweight responses (thumbs up/down) Added 30 second clips to help with unfamiliar songs

Concept successful

Further iterations performed

Learnings shared with business teams

Helped to create more social experience

Added lightweight responses (thumbs up/down)

Added 30 second clips to help with unfamiliar songs

Photo Sharing Study Objective: To understand how and why people share photos Methods: 5 users Home tours, semi-structured interviews Findings Desire to share photos in-the-moment Sharing photos to particular (and changing) groups of people

Objective: To understand how and why people share photos

Methods:

5 users

Home tours, semi-structured interviews

Findings

Desire to share photos in-the-moment

Sharing photos to particular (and changing) groups of people

Photo Presence (2007) Research questions: How can photos be used to ambiently share an experience in real time? How can we encourage conversations around real-time media?

Research questions:

How can photos be used to ambiently share an experience in real time?

How can we encourage conversations around real-time media?

Photo Presence Build only what you need J2ME application built by Sydney startup Build the experience, not the technology Application integrated with existing web service Social groups for social technologies 3 social groups (10 participants) Real context of use Used in their everyday lives for three weeks Primary device Participants were given devices with application preloaded, switched SIM and phonebook Field-based data collection Voicemails and interviews

Build only what you need

J2ME application built by Sydney startup

Build the experience, not the technology

Application integrated with existing web service

Social groups for social technologies

3 social groups (10 participants)

Real context of use

Used in their everyday lives for three weeks

Primary device

Participants were given devices with application preloaded, switched SIM and phonebook

Field-based data collection

Voicemails and interviews

Photo Presence Findings Telepresence Being able to log on and see what she was doing, like when we were at work and couldn’t talk – B2 There’s something kind of satisfying about the immediacy of right now in California my brother is doing this – B1 Automatic I like it cause I can just take pictures and I don’t have to fuss with it and it gets uploaded. –C3 Communication Looking at people’s tiles and looking at people’s comments on mine. It shows that they were interested in what I liked. Took enough time out of their day to make a comment. To some people that might make their day or brighten up their day – A2

Telepresence

Being able to log on and see what she was doing, like when we were at work and couldn’t talk – B2

There’s something kind of satisfying about the immediacy of right now in California my brother is doing this – B1

Automatic

I like it cause I can just take pictures and I don’t have to fuss with it and it gets uploaded. –C3

Communication

Looking at people’s tiles and looking at people’s comments on mine. It shows that they were interested in what I liked. Took enough time out of their day to make a comment. To some people that might make their day or brighten up their day – A2

Risk Mitigated… Concept successful Startup continued on path Changes to focus on small-group sharing Learnings helped clarify value prop Automatic, ambient sharing throughout day Small groups of close friends and family Simple communication around media

Concept successful

Startup continued on path

Changes to focus on small-group sharing

Learnings helped clarify value prop

Automatic, ambient sharing throughout day

Small groups of close friends and family

Simple communication around media

Overall guiding principles: Keep prototyping simple and focused on your research questions Evaluate with real users in their real situations of interaction Try a lot of ideas and pick ones that stick Iterate!

Keep prototyping simple and focused on your research questions

Evaluate with real users in their real situations of interaction

Try a lot of ideas and pick ones that stick

Iterate!

Benefits: Validate main user value without building full application/service Introduce several cycles of iteration before committing to product Get initial feedback for market segmentation

Validate main user value without building full application/service

Introduce several cycles of iteration before committing to product

Get initial feedback for market segmentation

Principles Build only what you need Build the experience, not the technology Social groups for social technologies Real context of use Primary device Field-based data collection Questions? Discussion?

Build only what you need

Build the experience, not the technology

Social groups for social technologies

Real context of use

Primary device

Field-based data collection

Questions? Discussion?