Proactive Displays: Bridging the Gaps between Online Social Networks and Shared Physical Spaces Joe McCarthy Principal Instigator Strands Labs, Seattle

Outline A few shameless plugs Overview: A brief history of Proactive Displays An example: the Context, Content & Community Collage Related work: commercial & academic Future plans (Slides available online at http://www.slideshare.net/gumption )

A few shameless plugs

Overview: A brief history of Proactive Displays

An example: the Context, Content & Community Collage

Related work: commercial & academic

Future plans

Strands: social recommendation & discovery http://www.strands.com/corp http://blog.strands.com

Strands: social recommendation & discovery Big announcement today: BBVA + Strands Tú cuentas (You count, in Spanish)

Strands: social recommendation & discovery “ What you play counts!”

Recommender System Startup Contest Deadline: 15 September 2008 ACM RecSys 2008 23-25 October Lausanne, Switzerland

Strands Labs Seattle Mission: “ To design, develop and deploy technologies that weave together the various strands of our activities, interests and passions to bridge the gaps between the digital and physical worlds and help people relate to the other people, places and things around them in ways that offer value to all participants .” [http://gumption.typepad.com/blog/2008/03/innovating-at-m.html] Location:

Proactive Displays: Definition ( ˌ )prō- ˈ ak-tiv adjective acting in anticipation of future problems, needs, or changes di- ˈ splā noun a setting or presentation of something in open view

Proactive Displays + Online Profiles Physical Tokens + Large Displays Better Real-world Interactions Bridging the gaps between people by bridging the gaps between the online and offline worlds = Large visual or aural displays that can sense & respond to the people and activities taking place in their vicinity

1st Generation Proactive Displays MusicFX MusicFX: An Arbiter of Group Preferences for Computer-Supported Cooperative Workouts Joseph F. McCarthy and Theodore Anagnost 1998 ACM Conference on Computer-Supported Cooperative Work (CSCW ‘98) A Multi-Agent System for Meting Out Influence in an Intelligent Environment M. V. Nagendra Prasad and Joseph F. McCarthy Eleventh Innovative Applications in Artificial Intelligence Conference (IAAI ‘99) http://www.youtube.com/watch?v=WLgXLl4uGYk a fitness center environment that knows who is working out, what they like to listen to, and dynamically adjusts the music to best suit the group of exercisers at any given time.

2nd Generation Proactive Displays Visual Awareness Location Tools (ActiveMap & EventManager) ActiveMap: A Visualization Tool for Location Awareness to Support Informal Interactions Joseph F. McCarthy and Eric S. Meidel First International Symposium on Handheld and Ubiquitous Computing (HUC '99) EventManager: Support for the Peripheral Awareness of Events Joseph F. McCarthy and Theodore D. Anagnost. Second International Symposium on Handheld and Ubiquitous Computing (HUC 2000) a suite of tools for creating greater awareness of the location and activities of colleagues within a workgroup, with an aim to provide more opportunities for informal communication

3rd Generation Proactive Displays UniCast, OutCast, GroupCast: Ubiquitous Peripheral Displays UniCast, OutCast & GroupCast: Three Steps Toward Ubiquitous Peripheral Displays Joseph F. McCarthy, Tony J. Costa and Edy S. Liongosari Third International Conference on Ubiquitous Computing (UbiComp 2001) Promoting Awareness of Work Activities through Peripheral Displays Elaine M. Huang, Joe Tullio, Tony J. Costa and Joseph F. McCarthy 2002 ACM Conference on Human Factors in Computer Systems (CHI 2002)

4th Generation Proactive Displays AutoSpeakerID Ticket2Talk Neighborhood Window Augmenting the Social Space of an Academic Conference Joseph F. McCarthy, David W. McDonald, Suzanne Soroczak, David H. Nguyen and Al M. Rashid ACM 2004 Conference on Computer Supported Cooperative Work (CSCW 2004) Proactive Displays: Supporting Awareness in Fluid Social Environments David W. McDonald, Joseph F. McCarthy, Suzanne Soroczak, David H. Nguyen and Al M. Rashid ACM Transactions on Computer-Human Interactions (TOCHI), Vol. 14, No. 4, January 2008

4th Generation Proactive Displays (v 2)

5th Generation Proactive Displays The Context, Content & Community Collage @ Nokia Research Center Palo Alto The Context, Content & Community Collage: Sharing Personal Digital Media in the Physical Workplace Joseph F. McCarthy, Ben Congleton, F. Maxwell Harper To appear: ACM 2008 Conference on Computer Supported Cooperative Work (CSCW 2008)

The Latest Generation of Proactive Displays Shortcomings of previous efforts Special-purpose sensors (IR badges, RFID tags) Special-purpose profiles Special-purpose installations Goals of present / future efforts Multi-purpose sensors (BT mobiles … NFC) Multi-purpose profiles Multi-purpose installations

Shortcomings of previous efforts

Special-purpose sensors (IR badges, RFID tags)

Special-purpose profiles

Special-purpose installations

Goals of present / future efforts

Multi-purpose sensors (BT mobiles … NFC)

Multi-purpose profiles

Multi-purpose installations

The Problem Employees feel un[der]appreciated #1 reason people leave jobs (US Labor Dept) 2/3 of workers receive no recognition (Gallup) Active disengagement costs US$300B / year Employees feel disconnected Lack of community in workplace Us vs. them Managers have less time, broader responsibilities Difficult to manage frequent, sustained “personal touch”

Employees feel un[der]appreciated

#1 reason people leave jobs (US Labor Dept)

2/3 of workers receive no recognition (Gallup)

Active disengagement costs US$300B / year

Employees feel disconnected

Lack of community in workplace

Us vs. them

Managers have less time, broader responsibilities

Difficult to manage frequent, sustained “personal touch”

Friendships at Work 30% of employees have a best friend at work (BF@W) 20% of employees dedicate time to building relationships at work Employees with a BF@W are 50% more satisfied with their company Employees with a BF@W are 7 times more likely to be engaged in their work

Maintaining Friendships through Social Media ambient intimacy “ being able to keep in touch with people with a level of regularity and intimacy that you wouldn’t usually have access to” Leisa Reichert http://www.disambiguity.com/ambient-intimacy/ http://www.slideshare.net/leisa/ambient-intimacy continuous partial friendship David Weinberger http://www.hyperorg.com/backissues/joho-may04-07.html

ambient intimacy

“ being able to keep in touch with people with a level of regularity and intimacy that you wouldn’t usually have access to”

Leisa Reichert

http://www.disambiguity.com/ambient-intimacy/

http://www.slideshare.net/leisa/ambient-intimacy

continuous partial friendship

David Weinberger

http://www.hyperorg.com/backissues/joho-may04-07.html

Friendships through Social Media @ Work? @

The Context, Content & Community Collage: A proactive display application Opening physical windows into online content in workplace contexts to foster a greater sense of community

A New Generation of Proactive Displays Goal: increase the sense of community in the workplace Method: open windows in shared physical spaces into the social media co-workers are creating Situated Social Software (SiSoSo) [vs. Mobile Social Software (MoSoSo)] Hypotheses Adding a physical dimension of audience will promote greater social media usage Social media sharing in the workplace will enhance personal relationships Enhanced personal relationships will lead to more productive professional relationships Challenges Privacy / benefit tradeoff (usability vs. security) Sustainable engagement (after novelty factor has worn off) How to measure the impact? (measuring community, relationships)

Goal: increase the sense of community in the workplace

Method: open windows in shared physical spaces into the social media co-workers are creating

Situated Social Software (SiSoSo) [vs. Mobile Social Software (MoSoSo)]

Hypotheses

Adding a physical dimension of audience will promote greater social media usage

Social media sharing in the workplace will enhance personal relationships

Enhanced personal relationships will lead to more productive professional relationships

Challenges

Privacy / benefit tradeoff (usability vs. security)

Sustainable engagement (after novelty factor has worn off)

How to measure the impact? (measuring community, relationships)

A User-Generated Video on Proactive Displays http://www.youtube.com/watch?v=m5rU9cAiYsY

Signup / Login

Select / Configure Module

Select Bluetooth Device(s)

Locations (mock-up)

Hardware & Software

Interactions with / through the displays

Interactions with displays

The Highest Rated Content (5 up votes)

Edge detection http://www.snopes.com/horrors/robbery/kidney.asp

Everything in moderation

Moderation: NSFW [ N ot S afe F or W ork]

Moderation: SFW

Evaluation: interactions through displays Context 8 displays, deployed July 19, evaluated after 4 weeks 75 full-time / part-time residents 44 proactive display accounts (34 non-empty) Perpetual alpha (location sensing, new modules, new features) Hypotheses Adding a physical dimension of audience will promote greater social media usage Social media sharing in the workplace will enhance personal relationships Enhanced personal relationships will lead to more productive professional relationships Web-based Survey Mix of multiple choice and open text questions 32 responses

Context

8 displays, deployed July 19, evaluated after 4 weeks

75 full-time / part-time residents

44 proactive display accounts (34 non-empty)

Perpetual alpha (location sensing, new modules, new features)

Hypotheses

Adding a physical dimension of audience will promote greater social media usage

Social media sharing in the workplace will enhance personal relationships

Enhanced personal relationships will lead to more productive professional relationships

Web-based Survey

Mix of multiple choice and open text questions

32 responses

Positive / Negative Impact (Personal / Professional)

A few choice comments I was surprised by some of the photos taken by [a colleague]. I had no idea of his very diversified and, in some cases, adventurous interests, and his photos showed me a side of him I would have never realized. The photos also give you an appreciation of other people's interests and unique travels. I find the photos absolutely fascinating and a very strong method of bringing all of us closer. I[t] was nice to see people putting up "themes". One person had a Star Wars theme, actually happening just by tags, but appeared to me as if a themed series of photos. Somewhat negative: I was surprised to see a photo of my family (my husband and my two kids) with another female friend. They were all sprawled on the grass. It reminded me of a picnic we all had together. But it also made me self-conscious: what would other people think of seeing this scene. Who is the "mystery lady"? Positive: walking into the kitchen and spotting my daughter's happy face on the screen. It was a nice greeting! Positive: I am delighted to see other peoples pics of exotic places they visited. Negative: seeing a picture of a guy in his underwear. The face was not in the pics so I couldn't tell who it is. Somebody esle in the room "killed" the photo. Positive: seeing people cluster around displays. I like that!

I was surprised by some of the photos taken by [a colleague]. I had no idea of his very diversified and, in some cases, adventurous interests, and his photos showed me a side of him I would have never realized. The photos also give you an appreciation of other people's interests and unique travels. I find the photos absolutely fascinating and a very strong method of bringing all of us closer.

I[t] was nice to see people putting up "themes". One person had a Star Wars theme, actually happening just by tags, but appeared to me as if a themed series of photos.

Somewhat negative: I was surprised to see a photo of my family (my husband and my two kids) with another female friend. They were all sprawled on the grass. It reminded me of a picnic we all had together. But it also made me self-conscious: what would other people think of seeing this scene. Who is the "mystery lady"? Positive: walking into the kitchen and spotting my daughter's happy face on the screen. It was a nice greeting! Positive: I am delighted to see other peoples pics of exotic places they visited. Negative: seeing a picture of a guy in his underwear. The face was not in the pics so I couldn't tell who it is. Somebody esle in the room "killed" the photo. Positive: seeing people cluster around displays. I like that!

Impact of Displays on Social Media

Influence of Rank & Role

Influence of Team Membership

Influence of competition Wii Tournaments

Future Work: Algorithms Location / Proximity detection Near / far Proxemics Incorporating meta-data Folksonomy, Semiotics Priority queue management People, streams Recommender System Module management Space, time, displays

Location / Proximity detection

Near / far

Proxemics

Incorporating meta-data

Folksonomy, Semiotics

Priority queue management

People, streams

Recommender System

Module management

Space, time, displays

Future Work: Modules Other sources of content Other interaction modalities Other dimensions of experience

Other sources of content

Other interaction modalities

Other dimensions of experience

Future Work: Interfaces & Interactivity Interfaces Signup page Display interface Administrator interface Interactivity Authentication Phones Touchscreens Other?

Interfaces

Signup page

Display interface

Administrator interface

Interactivity

Authentication

Phones

Touchscreens

Other?

Future Work: Replicability and Sustainability Deployments Nokia: NY, Dallas, Helsinki, LA … Academia: UMich, UMN, Stanford, … ?: Yahoo!, Flickr, … Mobiquitous Marketing (Mobile + Ubiquitous) Holistic approach to engagement marketing Co-promotional opportunities Contextual advertising in physical world

Deployments

Nokia: NY, Dallas, Helsinki, LA …

Academia: UMich, UMN, Stanford, …

?: Yahoo!, Flickr, …

Mobiquitous Marketing (Mobile + Ubiquitous)

Holistic approach to engagement marketing

Co-promotional opportunities

Contextual advertising in physical world

Future Work @ Strands Labs Seattle New Online Sources (http://beta.strands.com) New Physical Spaces (“third places”)

Related Work: Commercial Awareness / Interactions at Events Digital Signage Captivate TV ClearChannel, et al. RippleTV

Related Work: OneKeyAway / MatchlinC OneKeyAway Singles mixers in LA / SF Bay area; since May 2004 Questionnaire: 64 true-false questions about sex, religion, drug use, how you spend your spare time, etc “ I just want to get people together and talk about relationships … to discuss the questions, talk about their habits and personality traits. I think the device facilitates that.” – Edwin Duterte MatchlinC: personality compatibility device a mini “relationship advisor” Infrared “zapping” Three color codes (stoplight): red, amber, green) Reminiscent of Lock & Key, Lovegety, Meme Tags

OneKeyAway

Singles mixers in LA / SF Bay area; since May 2004

Questionnaire:

64 true-false questions about sex, religion, drug use, how you spend your spare time, etc

“ I just want to get people together and talk about relationships … to discuss the questions, talk about their habits and personality traits. I think the device facilitates that.” – Edwin Duterte

MatchlinC: personality compatibility device

a mini “relationship advisor”

Infrared “zapping”

Three color codes (stoplight): red, amber, green)

Reminiscent of Lock & Key, Lovegety, Meme Tags

Related Work: partyStrands (1) New Year’s Eve Times Square, NYC Paul Van Dyk Crystal Ballroom, Portland Buda del Mar Bar Madrid http://www.partystrands.com http://blog.mystrands.com/category/partystrands

Related Work: partyStrands (2)

Related Work: Research (1) Meme Tags (MIT) Interpersonal display : display for other people RF + LEDs + programming CityWall (Helsinki IIT) Multi-touch display in city center Flickr photos tagged with “helsinki” Interactions with display vs. through display Twitterspaces (Indiana U) Large display in campus lounge Dynamic visualization of group “tweets” (Twitter)

Meme Tags (MIT)

Interpersonal display : display for other people

RF + LEDs + programming

CityWall (Helsinki IIT)

Multi-touch display in city center

Flickr photos tagged with “helsinki”

Interactions with display vs. through display

Twitterspaces (Indiana U)

Large display in campus lounge

Dynamic visualization of group “tweets” (Twitter)

Related Work: Research (2) PlasmaPoster (FXPAL) Interactive community bulletin board Corporate, conference, café contexts Opinionizer (Sussex) Shared display at social events Interaction through typed input Dynamo (Sussex) High school setting Interaction via USB disk AgentSalon (ATR) Interaction via PalmGuides (PDAs) Conversations mediated by animated agents Public and Situated Displays O’Hara, Perry, Churchill, Russell

PlasmaPoster (FXPAL)

Interactive community bulletin board

Corporate, conference, café contexts

Opinionizer (Sussex)

Shared display at social events

Interaction through typed input

Dynamo (Sussex)

High school setting

Interaction via USB disk

AgentSalon (ATR)

Interaction via PalmGuides (PDAs)

Conversations mediated by animated agents

Thanks! For more information: [email_address] http://interrelativity.com/joe http://gumption.typepad.com http://www.slideshare.net/gumption C4 made possible by Ben Congleton (U. Mich), Max Harper (U. Minn) Nokia Context, Content & Community (C3) team All my former colleagues at Nokia Future work being made possible by Strands [Labs [Seattle]] Yogi Patel Sameer Ahuja Shelly Farnham Richie Hazlewood Josh Lind

For more information:

[email_address]

http://interrelativity.com/joe

http://gumption.typepad.com

http://www.slideshare.net/gumption

C4 made possible by

Ben Congleton (U. Mich), Max Harper (U. Minn)

Nokia Context, Content & Community (C3) team

All my former colleagues at Nokia

Future work being made possible by Strands [Labs [Seattle]]

Backup

Related Work: Alone Together Two Hours of Joint Solitude http://www.coffeegeek.com/opinions/cafestage/10-19-2005 Alone Together http://blogs.parc.com/playon/

Related Work: Addictive Games Amy Jo Kim, Social Architect, Shufflebrain “ Putting the Fun in Functional: Applying Game Mechanics to Functional Software” Incentives for games, communities Collecting Points Feedback Exchanges Customization http://www.shufflebrain.com/etech2007.html

Amy Jo Kim, Social Architect, Shufflebrain

“ Putting the Fun in Functional: Applying Game Mechanics to Functional Software”

Incentives for games, communities

Collecting

Points

Feedback

Exchanges

Customization

Early proactive displays Dangling String (PARC) Bus Mobile (UC Berkeley)

Proactive Displays in the large Sunset @ 200MHz (PARC) Love Board (Hachiko Crossing)

Proactive Displays on the road Alaris E-boards (www.alaris.net)

Proactive displays, circa 2054

Details Location / proximity detection Bluetooth sampling (MAC addresses, BT names) Location server (separate component) Priority queue management New / old, near / far, 70/30 Display management Firefox + Javascript Flickr module Ruby API Remote administration Process, power Modular, flexible framework Front-end (time / space sharing) Back-end

Location / proximity detection

Bluetooth sampling (MAC addresses, BT names)

Location server (separate component)

Priority queue management

New / old, near / far, 70/30

Display management

Firefox + Javascript

Flickr module

Ruby API

Remote administration

Process, power

Modular, flexible framework

Front-end (time / space sharing)

Back-end

Flickr API