Frank Bentley Senior Staff Research Engineer Motorola Labs Truly Mobile Applications: Location, Media Capture, and Connectivity

What is unique to mobile? http://www.flickr.com/photos/artamnesia/135744020/

Rich Mobile Applications Yahoo! Go ZoneTag Radar ShoZu

Overview Location Media Capture Social Connectivity Putting it all together: ZoneTag

Location

Media Capture

Social Connectivity

Putting it all together: ZoneTag

Mobile Location Cell ID Available on all GSM phones Localization to zip-code level Requires translation to semantically meaningful location GPS Needs special hardware Not available indoors/in cities Minutes to get first fix

Cell ID

Available on all GSM phones

Localization to zip-code level

Requires translation to semantically meaningful location

GPS

Needs special hardware

Not available indoors/in cities

Minutes to get first fix

Cell ID In GSM networks, all cells in the world have a globally unique ID made up of four numbers: cell id, lac, mnc, and mmc Databases are being made that map cell IDs into locations Cells can be manually mapped to semantic locations (e.g. Campus, Home, Downtown Boston, etc.)

In GSM networks, all cells in the world have a globally unique ID made up of four numbers: cell id, lac, mnc, and mmc

Databases are being made that map cell IDs into locations

Cells can be manually mapped to semantic locations (e.g. Campus, Home, Downtown Boston, etc.)

Cell Density Cell topology scales with population density http://www.intel-research.net/Publications/Seattle/100920061625_366.pdf

Cell topology scales with population density

Uses of Cell ID Cell ID has been used to: Determine when someone was at a given place (e.g. home) (see Sohn et al ‘05) Distinguish between walking/driving/stationary (see Sohn et al ’06) Provide accurate positioning (e.g. Chen et al ’06) Down to 95% error of 163m with all visible cells Determine when people are moving between locations (e.g. Bentley et al ’07) Databases such as ZoneTag and Place Lab can map Cell ID to physical locations

Cell ID has been used to:

Determine when someone was at a given place (e.g. home) (see Sohn et al ‘05)

Distinguish between walking/driving/stationary (see Sohn et al ’06)

Provide accurate positioning (e.g. Chen et al ’06)

Down to 95% error of 163m with all visible cells

Determine when people are moving between locations (e.g. Bentley et al ’07)

Databases such as ZoneTag and Place Lab can map Cell ID to physical locations

GPS 30 satellites in earth orbit 6 satellites always in line of site of any place on earth Receivers must find 4 or more satellites for a fix First fix usually in 1.5 minutes Typical phone can last for four hours with GPS turned on Can get you accurate position within 10m with altitude People in GPS coverage 4.5% of the day

30 satellites in earth orbit

6 satellites always in line of site of any place on earth

Receivers must find 4 or more satellites for a fix

First fix usually in 1.5 minutes

Typical phone can last for four hours with GPS turned on

Can get you accurate position within 10m with altitude

People in GPS coverage 4.5% of the day

GPS Continued… Some phones have built in GPS All Motorola phones on the iDEN (Nextel) network Some GSM phones (including European version of the a780, n95 from Nokia, etc.) Many CDMA phones (e.g. RAZR on Verizon) Phones without GPS can use a bluetooth unit sends a NMEA stream to the phone phone can listen through a BT serial connection

Some phones have built in GPS

All Motorola phones on the iDEN (Nextel) network

Some GSM phones (including European version of the a780, n95 from Nokia, etc.)

Many CDMA phones (e.g. RAZR on Verizon)

Phones without GPS can use a bluetooth unit

sends a NMEA stream to the phone

phone can listen through a BT serial connection

Place Its (Sohn et al 2006) The problem: getting reminders in particular locations (e.g. call my mom when I get home, stop at the grocery store on the way home, etc.) Uses Cell ID to determine when the user is in a given place Manually connects sets of Cell IDs to places in people’s lives When user enters a given ‘place,’ if there are any alerts for them, notifies them (also can have leaving alerts)

The problem: getting reminders in particular locations (e.g. call my mom when I get home, stop at the grocery store on the way home, etc.)

Uses Cell ID to determine when the user is in a given place

Manually connects sets of Cell IDs to places in people’s lives

When user enters a given ‘place,’ if there are any alerts for them, notifies them (also can have leaving alerts)

Motion Presence (Bentley et al 2007) Augmented phone book See when close friends and family are moving or in one place Users used this data to infer location, activity, destination, time to destination, etc. Used GSM Cell ID changes to determine when people transitioned between moving and at a place

Augmented phone book

See when close friends and family are moving or in one place

Users used this data to infer location, activity, destination, time to destination, etc.

Used GSM Cell ID changes to determine when people transitioned between moving and at a place

Media Capture Phone is a window into a person’s world Data-connected phones allow people to share experiences with others 29 billion mobile images captured/year http://www.flickr.com/photos/scmtngirl/109216105/

Phone is a window into a person’s world

Data-connected phones allow people to share experiences with others

29 billion mobile images captured/year

Media Capture Distant Closeness: knowing about others, keeping others informed about oneself, without direct interaction - Nancy VanHouse (Berkeley) Mobile photos taken for: Memory, Relationship Creation and Maintenance, Self-Presentation, Self-Expression, Functional, Social Documentary - Nancy VanHouse (Berkeley) and Marc Davis (Yahoo!) http://www.flickr.com/photos/yorkie/459299857/

Distant Closeness: knowing about others, keeping others informed about oneself, without direct interaction - Nancy VanHouse (Berkeley)

Mobile photos taken for: Memory, Relationship Creation and Maintenance, Self-Presentation, Self-Expression, Functional, Social Documentary

- Nancy VanHouse (Berkeley) and Marc Davis (Yahoo!)

Other uses of camera To control a cursor To set a level in a game To determine if the phone is in a pocket/purse QR Codes Makinde Adeagbo, Ben Lu, Chris Moses, Kah Seng Tay (MIT 21W.780 ’07)

To control a cursor

To set a level in a game

To determine if the phone is in a pocket/purse

QR Codes

Radar Service of TinyPictures Inc. Lightweight Visual Communication Share media with close circle throughout the day J2ME application to see photo streams and comment on the go

Service of TinyPictures Inc.

Lightweight Visual Communication

Share media with close circle throughout the day

J2ME application to see photo streams and comment on the go

ShoZu Allows for photo/ video blogging to: Blogger Flickr Kodak YouTube Etc…

Allows for photo/ video blogging to:

Blogger

Flickr

Kodak

YouTube

Etc…

Social Connectivity Not just about access to data Think access to people and places http://www.flickr.com/photos/martinkreyness/301315926/

Not just about access to data

Think access to people and places

Connectivity: SMS Simple way to send information between phones Ports allow data to go straight to J2ME applications instead of to Messaging application No server necessary No data plan necessary SMS can auto-start application with MIDlet-Push

Simple way to send information between phones

Ports allow data to go straight to J2ME applications instead of to Messaging application

No server necessary

No data plan necessary

SMS can auto-start application with MIDlet-Push

Connectivity: Data Data services open up phone to the world of the Internet Connections with social networks, information, and places HTTP/HTTPS supported in J2ME Web services supported on latest handsets ~10% of US subscribers have data plans

Data services open up phone to the world of the Internet

Connections with social networks, information, and places

HTTP/HTTPS supported in J2ME

Web services supported on latest handsets

~10% of US subscribers have data plans

Connectivity: Bluetooth Discovery and communication with nearby devices Sharing media with co-located people Discovering who is around for media tagging Multi-player mobile gaming without need for a server or data plan

Discovery and communication with nearby devices

Sharing media with co-located people

Discovering who is around for media tagging

Multi-player mobile gaming without need for a server or data plan

Putting it all together A walk through the ZoneTag application Client for Motorola phones developed by Motorola Labs Server developed by Yahoo! Research Berkeley Allows users to: capture photos apply location information (derived from Cell ID) apply relevant tags based on location upload tagged photos to Flickr

A walk through the ZoneTag application

Client for Motorola phones developed by Motorola Labs

Server developed by Yahoo! Research Berkeley

Allows users to:

capture photos

apply location information (derived from Cell ID)

apply relevant tags based on location

upload tagged photos to Flickr

ZoneTag: Location On starting application, gets cell id Resolves this using Yahoo! Research Berkeley’s ZoneTag server Keeps current location and relevant tags cached Updates location and tags when cell id changes

On starting application, gets cell id

Resolves this using Yahoo! Research Berkeley’s ZoneTag server

Keeps current location and relevant tags cached

Updates location and tags when cell id changes

ZoneTag: Media Capture Integrated viewfinder, when supported File system scanner that runs on launch to find new media Automatic suggestion to upload photos taken in the past 10 minutes

Integrated viewfinder, when supported

File system scanner that runs on launch to find new media

Automatic suggestion to upload photos taken in the past 10 minutes

ZoneTag: Social Uploads media to Flickr with tags Allows use of Flickr privacy levels Allows browsing of photos taken in vicinity or with a given tag

Uploads media to Flickr with tags

Allows use of Flickr privacy levels

Allows browsing of photos taken in vicinity or with a given tag

Mobility – More than just on-the-go Location- Based Media-Centric Socially Connected http://flickr.com/photos/pingnews/272805477/

Location- Based

Media-Centric

Socially Connected

References… Practical Metropolitan-Scale Positioning for GSM Phones (Chen et al ’06) http://www.intel-research.net/Publications/Seattle/100920061625_366.pdf Place-Its: A Study of Location-Based Reminders on Mobile Phones (Sohn et al ’05) http://www.cse.ucsd.edu/users/k2li/placeits-ubicomp05.pdf Mobility Detection Using Everyday GSM Traces (Sohn et al ’06) http://www.placelab.org/publications/pubs/mobility-ubicomp2006.pdf Distant Closeness, Cameraphones and public image sharing (VanHouse ’06) http://groups.sims.berkeley.edu/pics/vanhouse_distant_closeness.pdf Sharing Motion Information with Close Family and Friends (Bentley and Metcalf ’07) http://web.mit.edu/bentley/www/papers/paper724-bentley.pdf ZoneTag: Designing Context-Aware Mobile Media Capture to Increase Participation (Ahern at al ’06) http://groups.sims.berkeley.edu/pics/papers/Ahern_et_al_zonetag_pics06.pdf Location Disclosure to Social Relations: Why, When, & What People Want to Share (Consolvo et al ’05) Place Lab: Device positioning using radio beacons in the wild (LaMarca et al ’05) http://www.placelab.org/publications/pubs/pervasive-placelab-2005-final.pdf MIT Class 21w.780 http://web.mit.edu/21w.780/www/spring2007

Practical Metropolitan-Scale Positioning for GSM Phones (Chen et al ’06) http://www.intel-research.net/Publications/Seattle/100920061625_366.pdf

Place-Its: A Study of Location-Based Reminders on Mobile Phones (Sohn et al ’05) http://www.cse.ucsd.edu/users/k2li/placeits-ubicomp05.pdf

Mobility Detection Using Everyday GSM Traces (Sohn et al ’06) http://www.placelab.org/publications/pubs/mobility-ubicomp2006.pdf

Distant Closeness, Cameraphones and public image sharing (VanHouse ’06) http://groups.sims.berkeley.edu/pics/vanhouse_distant_closeness.pdf

Sharing Motion Information with Close Family and Friends (Bentley and Metcalf ’07) http://web.mit.edu/bentley/www/papers/paper724-bentley.pdf

ZoneTag: Designing Context-Aware Mobile Media Capture to Increase Participation (Ahern at al ’06) http://groups.sims.berkeley.edu/pics/papers/Ahern_et_al_zonetag_pics06.pdf

Location Disclosure to Social Relations: Why, When, & What People Want to Share (Consolvo et al ’05)

Place Lab: Device positioning using radio beacons in the wild (LaMarca et al ’05) http://www.placelab.org/publications/pubs/pervasive-placelab-2005-final.pdf

MIT Class 21w.780 http://web.mit.edu/21w.780/www/spring2007