Exploiting incidental interactions between mobile devices
Exploiting incidental interactions between mobile devices Jamie Lawrence Terry R. Payne Raul V. Kripalani Intelligence, Agents and Intelligence, Agents and Electronics and Computer Multimedia Group Multimedia Group Science Department Electronics and Computer Electronics and Computer University of Southampton, UK Science Department Science Department University of Southampton, UK University of Southampton, UK email@example.com firstname.lastname@example.orgABSTRACT for the majority of computer interfaces but it has the dis-Not all human–computer interactions are directly derived advantage of requiring the user’s attention for the planning,from the concious intention of a user. There are a class of execution and evaluation phases. Whilst this may not be ainteractions which occur as a side eﬀect of the user’s cur- problem when the user is fully occupied by the task, thererent goals but which nevertheless, provide a useful modality are circumstances where the user is neither capable, norfor interacting with mobile applications. This paper dis- required, to switch their attention to the computer. Thiscusses the use of these ‘incidental interactions’ in mobile ‘attention scarcity’ is more prevalent with mobile phones,applications, particularly those that rely on the notion of where the user is typically required to devote at least partco-presence. This paper presents three projects that ex- of their attention to other tasks. In contrast, applicationsploit incidental interactions: The ﬁrst, Amigo, is a appli- on a desktop computer are assumed to be the main andcation that uses Bluetooth to construct a representation of primary focus of the user.a user’s social network and associate these with calendarevents; the second project, Co-presence Communities, takes This paper advocates an interaction model for mobile de-the Amigo concept further by mining the co-presence data to vices that co-opts the user’s intentional actions for one task,discover reoccurring group meetings; and ﬁnally, BluScreen as inputs towards an entirely diﬀerent goal. An example,is a pervasive public display that utilises co-presence data elaborated throughout this paper, is the goal of building ato provide feedback to an agent-based marketplace, which representation of your social network. A traditional systemis responsible for allocating the time slots for each presen- would allow a user to input the names of their contacts andtation. perhaps note when, where and how they maintained that re- lationship1 . Our proposal, explored throughout this paper, is that the goal of building the social network representationCategories and Subject Descriptors should be a by-product of the actual social interactions. So,H.5.2 [User Interfaces]: Interaction styles; H.4 [Information the action of a user meeting someone would automaticallySystems Applications]: Miscellaneous update the corresponding entries in their proﬁle.Keywords This paper begins with an introduction to ‘incidental inter-Incidental interactions, Co-presence, Mobile social applica- actions’ (in Section 2) and speciﬁcally to Co-presence as ations, Bluetooth form of interaction (Section 3). Section 4 provides a brief description of three current research projects which employ incidental interactions. In particular, Section 4.1 presents1. INTRODUCTION Amigo, a web-based social networking site, that uses in-Traditional literature in the ﬁeld of human–computer inter- formation derived from incidental interactions. Section 4.2action assumes that the user has a particular goal in mind introduces Co-presence Communities as an extension of thewhen they start interacting with a computer. This is of- Amigo project that provides a richer analysis of a user’s so-ten characterised by Norman’s interaction model , which cial network. BluScreen, in Section 4.3, is a public displayconsists of two primary phases: execution (including plan- that uses the incidental interactions of viewers to inﬂuencening) and evaluation. That is, the user begins with a goal the future content. Finally Section 5 concludes this paper.in mind, executes a plan to achieve this and then evaluatesthe results against the intended state. This model accounts 2. INCIDENTAL INTERACTIONS In contrast to the Norman model of intentional actions, Dix has proposed ‘incidental interactions’ to describe those ac- tions that are co-opted by a system to serve a diﬀerent goal from that which the user was currently undertaking . Dix deﬁnes incidental interaction as the situation “where actions performed for some other purpose, or unconscious signs, are 1 indeed, this is the model for all ‘social networking services’ currently available on the Internet
interpreted in order to inﬂuence/improve/facilitate the ac-tors’ future interaction or day-to-day life” This emphasises the user’s focus on one task, whilst the sys-tem is surreptitiously using those interactions to actuallyfulﬁl a diﬀerent goal. The canonical example of an inciden-tal interaction is the interior light of a car: When the useropens the car door, the interior lights are activated. Theuser’s goal is to enter or exit the car, but the system usesthe interaction with the car door to fulﬁl the unstated goalof illuminating the car interior. Note that ‘incidentalness’does not imply that an action or subsequent goal is uninten-tional, just that the user may have previously speciﬁed it tothe system and not be consciously thinking about it whilstperforming the actions. Incidental interactions also diﬀer inthe feedback that the user receives. Whereas a traditionalapplication would provide direct, explicit feedback to theuser regarding their actions, feedback from an incidental in-teraction will often be minimal, unobtrusive and delayed.The de-emphasis of explicit user interaction and feedback Figure 1: The Amigo architecture consisting of Blu-provides incidental applications with a strong requirement Mobile client, BluDesktop and the Amigo web ser-for autonomous software, and an obvious connection to the viceﬁeld of agent-based computing . of real-world interactions, the user can participate in co-3. CO-PRESENCE presence systems, without resorting to activities outside ofCo-presence refers to the spatio-temporal conditions under their normal daily routine. This aligns well with Dourish’swhich people can interact with each other. Goﬀman deﬁned argument that embodiment “does not simply mean ‘physicalit as the condition when people “sense that they are close manifestation.’ Rather, it means being grounded in every-enough to be perceived in whatever they are doing, including day, mundane experience... [embodiment] is the property oftheir experiencing of others, and close enough to be perceived our engagement with the world that allows us to make itin this sensing of being perceived ” . Co-presence (or more meaningful ”. The real–world embodiment facilitated by aformally, corporeal co-presence ) is the condition under co-presence modality (i.e., that the interaction is “groundedwhich two or more people are in the same place, at the same in everyday, mundane experience”) neatly matches the char-time. In general, co-presence is not an intentional action by acteristics of an incidental interaction, which should resultitself but rather a by-product of some other goal, which from some other real–world goal (that “mundane experi-makes it a useful form of incidental interaction. ence”).Humans experience co-presence through one or more sen-sory inputs but, since computational systems lack that ca- 4. CO-PRESENCE PROJECTSpability, it is necessary to develop a technological solution The three recent projects described in this section utilise ato co-presence detection. If we take Goﬀman’s view of co- co-presence interaction modality in a similar way to previ-presence, then any technological co-presence sense should be ous applications (such as the selection in ). In particular,roughly equivalent to human sensory inputs. The class of the Serendipity application  detects co-presence betweentechnologies called Personal Area Networks (PAN) provide two individuals using Bluetooth-enabled phones, attemptsa good match with Hall’s work on our sensory abilities . to match their two proﬁles and, if successful, provides an in-In particular, the eﬀective range of PANs corresponds to the troductory service. This has obvious dating applications butdistance at we can perceive individual characteristics in peo- the actual usage scenario was to enhance corporate collab-ple. Most wireless PANs, such as Bluetooth, are eﬀectively oration. Moving beyond the immediate interaction, Eagleconﬁned to a single room due to their short-range and lim- & Pentland have used Bluetooth phones as mobile sensorsited wall-penetration abilities. Bluetooth also the advantage from which to determine a user’s activity and location .that it has been widely adopted in mobile devices, from lap- In another direct example of incidental interactions fromtops to PDA’s and mobile phones. For the purposes of this co-presence, the Jabberwocky device used Bluetooth to de-work, the wireless ‘bubble’ formed by a Bluetooth device is termine a user’s ‘Familiar Strangers’ . This also demon-used as a co-presence sensor and the presence of that device strated how a user might come to subvert the co-presenceis assumed to indicate the presence of its owner. Whilst it interaction by deliberately straying away from their usualis also assumed that there is one-to-one correlation between patterns to seek out new pastures and avoid their familiara mobile device and human owner, it is also accepted that strangers.individuals without mobile phones (or without Bluetoothactivated) will not be discoverable. 4.1 Amigo Amigo is an social networking service built upon real inter-A Bluetooth sensor attached to an individual allows the action data collected by Bluetooth phones. It keeps a historycomputation of co-presence to be embodied within the real– of co-presence encounters and a calendar of events, and usesworld . This embodiment means that through the medium these to infer social relationships and attach occurrences of
co-presence with particular diary entries. Amigo consistsof three main components: a client application, BluMobile;a desktop application, BluDesktop; and the central Amigoweb service. The architecture is sumarised in Figure 1.The BluMobile client is a J2ME application that can be in-stalled on a Bluetooth-equipped mobile phone (a W800i wasused in our trials), and regularly scans for other Bluetoothdevices in the vicinity. These encounters are logged to thephones internal memory and may be displayed in a textualor graphical form. However, the main purpose of BluMo-bile is not the visualisation of these co-presence encounters,but as a means of collecting the co-presence data. Due tothe storage and processing limitations of mobile phones, the Figure 2: The BluScreen installationco-presence data is then transferred to the user’s PC forprocessing by the BluDesktop client. BluDesktop acts asthe user’s main store of co-presence data, calendar entries tine and conceptual clusterer . However, the details of(in RDF iCal format2 ), and social network representation that algorithm are outside the scope of this paper.(in FOAF format3 ). The representation of the user’s so-cial network is built from the co-presence data, using the A Co-presence Community is formed by the repeated co-simple rule that any person who has been co-present with presence between a group of individuals at approximately thethe user for more than a speciﬁed total amount of time (de- same time period ; a Monday morning project meeting or afault: 1 hour) is considered a ‘friend’ 4 . Anyone who doesn’t daily coﬀee break at 3:00 would be two examples. These co-meet the criteria of ‘friendship’ is just considered to be a ca- presence communities are probabilistic representations thatsual encounter. A mapping between calendar events and the cluster together co-presence events on both temporal (startpeople associated with it is constructed by considering any and end times) and membership dimensions. A commu-person who was co-present with the user for more than a nity may be stable to a varying degree across these two di-speciﬁed percentage of the event’s duration (default: 20%) mensions. For example, the co-presence community formedto be related to that event. These event-person connections by weekly project meetings would have stable membershipare used to form a representation of the event’s social group. and temporal dimensions (the same group of people at theThe resulting user proﬁle, which incorporates the social rela- same time interval). In contrast, the community formed ontionships and event groups, is created as an RDF document a commuter bus might have a stable temporal dimensionand uploaded to the Amigo web service. This proﬁle is used (because you always catch the same bus) but an unstableby the Amigo site to provide a mechanism for browsing the membership (because it’s not always the same people onproﬁles of those members of the user’s social network (where that bus). By obtaining these community memberships viaauthourised) or the co-presence history by events, time or wireless sensors, it is be possible to build a dynamic, self-device. updating model of the people that a user is normally in the proximity oﬀ at any given time.Amigo is an example of an application based upon inciden-tal interactions because the act attending of a social event AIDE, the Ambient Information Dissemination Environment,(e.g. Salsa dancing) has the incidental eﬀect of building is one possible application of co-presence communities. AIDEup a representation of the social group around that event. allows users to distribute content in a peer-to-peer manner,Another functionality of BluMobile is the ability to be noti- using their co-presence communities as a self-updating dis-ﬁed, through an audio or vibration alert, when one of your tribution list. The advantage of co-presence communities for‘friends’ comes into range. Admittedly, this isn’t a terribly this application is that they allow dissemination in diﬀer-useful feature with the typical 10 m range of Bluetooth, how- ent contexts: sharing jokes with strangers on the commuteever it does highlight the incidental eﬀect of a co-presence home (i.e., a time-stable community) or disseminating theinteraction: as the user approaches a friend (for whatever latest call-for-proposals to a research group (a fully stableintentional reason), their phones will incidentally vibrate. community formed by weekly meetings). AIDE ambiently determines which community a detected device belongs to and transfers the selected content in the background, based4.2 Co-presence Communities on the source user’s speciﬁed preferences. With the AIDECo-presence Communities is a project that aims to discover application, the user is expected to pursue their normal goalssocio-temporal patterns from co-presence data (much like and the dissemination of their content (being a longer-termAmigo) but relies on the user to add contextual information goal) is carried out as a side-eﬀect of their daily routine.(whereas Amigo utilises calendar information). An initial al-gorithm for discovering these co-presence communities hasbeen developed that incorporates a feature extraction rou- 4.3 BluScreen BluScreen is another project that derives its functionality2 http://www.w3.org/2002/12/cal/ical.rdf from the incidental interaction between co-present devices.3 http://xmlns.com/foaf/0.1/ However, in this case, the co-presence activity is used to4 A ‘friend’ in this sense is the simple binary relationship auction oﬀ display time for a pervasive public display to au-provided for by the FOAF representation rather than any tonomous advertising agents . Figure 2 shows one of thedeeper sociological construct BluScreen installations mounted near a major thoroughfare.
The user’s interaction with the screen (such as pausing to  P. Dourish. Where the Action Is. MIT Press, 2001.view the contents) is co-opted by the BluScreen to inciden-tally provide feedback to the agent marketplace about your  N. Eagle. Can serendipity be planned? MIT Sloanimplied preferences, and thereby change the content you re- Management Review, 46(1):10–14, Fall 2004.ceive in the future.  N. Eagle and A. Pentland. Reality mining: Sensing complex social systems. Journal of Personal and5. CONCLUSIONS Ubiquitous Computing, pages 1–14, February 2006.This paper has concentrated on a class of interaction whichis not normally considered interesting in HCI research: those  E. Goﬀman. Behaviour in Public Places. The Freeinteractions that are incidental rather than being explicitly Press, 1966.intentional. These incidental interactions are more impor-  E. T. Hall. The Hidden Dimension. Anchor Books,tant to mobile applications since the user’s attention can 1969.be productively directed at other tasks. The examples andprojects discussed in this paper are all based on co-presence  J. Lawrence, T. R. Payne, and D. D. Roure.interactions: Amigo uses them to build up a semantic rep- Co-presence communities: using pervasiveresentation of a user’s social network; Co-presence Commu- communities to support weak social networks. Innities exploit the interactions to build a more comprehen- Proceedings of the Distributed and Mobilesive social model and, through the AIDE application, al- Collaboration Workshop (DMC). IEEE, June 2006.low content to be disseminated without repeated, explicit,intentional actions; ﬁnally, the BluScreen project uses the  P. Maes. Agents that reduce work and informationco-presence interaction between viewers and a pervasive dis- overload. Commun. ACM, 37(7):30–40, 1994.play device to enhance the quality of the displayed content.  D. A. Norman. The Psychology of Everyday Things.For some mobile applications, the focus needs to be moved Basic Books, 1988.away from the traditional modalities of the mobile phoneand towards less attention-demanding, more autonomous,  E. Paulos and E. Goodman. The familiar stranger:and more context-aware, interactions. anxiety, comfort, and play in public places. In Proceedings of the 2004 conference on Human factors6. REFERENCES in computing systems, pages 223–230. ACM Press,  R. Beale. Supporting social interaction with smart 2004. phones. IEEE Pervasive Computing, 4(2):35–41,  T. R. Payne, E. David, N. R. Jennings, and April–June 2005. M. Shariﬁ. Auction mechanisms for eﬃcient  A. Dix. beyond intention - pushing boundaries with advertisement selection on public displays. In incidental interaction. In Proceedings of Building Proceedings of European Conference on Artiﬁcial Bridges: Interdisciplinary Context-Sensitive Intelligence, page in press, Trentino, Italy, 2006. Computing, Sept 2002.  S. Zhao. Toward a taxonomy of copresence. Presence,  A. Dix, J. Finlay, G. D. Abowd, and R. Beale. Human 12(5):445–455, October 2003. Computer Interaction. Prentice Hall, 2004.