NFC-based User Interfaces


Published on

Talk given at the NFC 2012 Workshop, March 13 2012, Helsinki Finland. The talk presents the research performed on NFC-based user interfaces at the University of Oulu, Department of Computer Science and Engineering, Oulu, Finland.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

NFC-based User Interfaces

  1. 1. NFC-Based User Interfaces Jukka Riekki, Ivan Sanchez, Mikko Pyykkönen Department of Computer Science and Engineering University of Oulu Oulu, Finland NFC 2012 4th International Workshop on Near Field Communication March 13th, 2012 Helsinki, FinlandThese slides were modified slightly after the presentation. Three videos were replaced with text andfigures. Two of the videos are available at YouTube, links are provided on the corresponding slides.
  2. 2. Table of Contents• Introduction• Previous work• Interaction model• Advertising NFC tags• Conclusions
  3. 3. Introduction• NFC can have more profound effect on our everyday lives than has been discussed in public – Transit and Ticketing, Payment, and Advertising applications are just the tip of the iceberg• NFC-based user interfaces can be built to any Internet or mobile phone application; to any device connected to Internet – Generally, NFC can be used to replace manual typing, menu selections, and other user interface actions with acts of touching. – For example: household appliances with an Internet connection; see the next slide
  4. 4. IntroductionWhen will this application be published?
  5. 5. Introduction• We see significant potential in interactive spaces – Environments equipped with rich user interfaces and offering a large variety of services for users – Focus on user control – instead of intelligent environments observing the users and activating services autonomously• The challenge is to provide easy-to-use user interfaces – The available services need to be communicated clearly to users; how they can be controlled, and the devices the services use – The actions for controlling the services need to be easy to perform.
  6. 6. Introduction• NFC excels in tackling this challenge – Graphical icons advertise services, commands & devices – NFC tags are placed under icons – Available services can be identified by visual scanning – Commands performed by touching icons with NFC phones• We study interactive spaces – We have built many prototypes containing NFC-based user interfaces• This talk presents some of these prototypes and our current work on: – Interaction model for NFC-based user interfaces – Graphical language for advertising NFC tags
  7. 7. Previous Work• Advertising NFC tags• Prototypes: – Positioning Prototype 2006: NFC + WLAN positioning – Information Board Prototype 2007: Information board map + tags – Touch & Control Prototype 2007: Controlling a wall display – Touch & Run Prototype 2008: Outdoor game – Touch & Collect Prototype 2008: Pupils collect content from a museum – EPG Control Prototype 2009: UI for Electronic TV Program Guide – PlaceMessaging Prototype 2009: Leaving messages on a place – Touch & Learn Prototype 2009: Learning languages – iPoster Prototype 2010: Poster presentation with NFC UIs – Touch & Learn to Read Prototype 2010: Children learn to read• Next, some of this work is presented in more detail
  8. 8. Previous Work: Icons• Research on NFC icons is quite rare• We have designed several sets of NFC icons
  9. 9. Previous Work: IconsEPG control board: over 30 icons & tags iPoster: a cartoon contains the icons and gives instructions
  10. 10. Touch & Control Prototype 2007• NFC tag is placed next to a wall display• Touching the tag starts a multimedia player on the wall display• The player can be controlled – With phone UI (BOTTOM LEFT) ControlCube – By touching icons on the Control Panel (BOTTOM MIDDLE) – By touching icons on the Control Cube (TOP RIGHT)• Minimalistic phone UI suffices for the panel and the cube – a single image (BOTTOM RIGHT)• Video:
  11. 11. Previous Work: Lessons learnt• Carefully designed graphical icons facilitate recognizing: • The services available in the local environment • How the services can be controlled• But not much has been done to develop the visual appearance of NFC user interfaces• Touching is a natural way of interacting with our everyday environment • Users can focus on their activities (calm computing!) • This interaction model results in users being present in the environment • And interacting directly with other users • Instead of each user focusing on own mobile terminals display (or on PCs’ or tablet PCs’ display)
  12. 12. Previous work: Lessons learnt cont’d• Most of our prototypes have been tested with real users • Feedback has been very positive• In fact, phones with very modest resources can be used to build impressive user interfaces • Large touch displays are not mandatory • Rather, a small phone easy to hold is better!• Lots of different user interfaces can be built with the current technology • Many applications need software only for phones (no server); the tag is the database! • So applications can be kept simple and robust • And using the applications is cheap (no communication costs)
  13. 13. Interaction model: Motivation• Interactive space UIs differ from traditional GUIs – UI is spread in the environment – Tangible UI, i.e. physical objects are handled• An interaction model can be used to – Identify and define the UI components (objects, devices,…) – Structure and model the interaction – Study alternatives• In addition – The model might be used to present recommended interaction patterns, for example
  14. 14. Interaction model: Elements• User: a person in an interactive space• Token: a physical object in the space – Users can control and observe the states of tokens• Service: a digital object consisting of a set of functionalities and policies controlling their usage – Users achieve their goals with these functionalities• Resource: a device in the physical space• Services use resources to realize their functionalities – To recognize the states of tokens – To change the states of tokens and other environment – To present information to users – To get input from a user (e.g. touch display)
  15. 15. Interaction model: Tokens & Resources• The mapping between tokens and resources determines – How services experience the user-token interaction – How users experience the service-resource interaction• An object (e.g. an NFC phone) can be – From the user perspective a token (handled as physical object) – From the service perspective a resource• NFC tags are resources as well• When NFC readers are placed in the environment – User handles tokens equipped with NFC tags
  16. 16. Interaction model: Stages• Discovery Stage: a user scans visually the space to find the tokens to interact with• Composition Stage: the user composes the application he or she wants to use – By selecting services and tokens – E.g. by touching tokens with a phone; NFC tags in tokens• Usage Stage: The tokens are handled – User communicates with the application by interacting with the tokens in the physical space – Services interact with resources in the digital space
  17. 17. The model elements in the environment
  18. 18. Advertising NFC tags• We advertise NFC tags with service advertisements – Graphical advertisements communicating information about interaction possibilities – And offering the means for realizing the interaction
  19. 19. Advertising NFC tags• A service advertisement specifies – The exact position to touch – All details a user needs to know when making a decision on whether to touch a tag or not• Attention element: draws a users attention to the advertisement – This element is included into each advertisement• Technology element: indicates the technology utilized in the interaction – Also other interaction technologies, like 2D barcodes and Bluetooth, can be used
  20. 20. Advertising NFC tags• Interaction element: advertises the exact point to touch – NFC tags are placed under these elements• Action element: indicates the action the system performs when a user touches the interaction element – The related NFC tag stores the command to execute that action, or the data the system maps to the command• Context element: can be used to present, for example, the service to be commanded
  21. 21. Advertising NFC tags• Instruction element: explains how the interaction is to be performed; text, cartoon, etc – Separate instruction elements can give general instructions about services in a certain area• A service advertisement does not have to contain all these elements• When users become familiar with NFC and some details are known otherwise, less details suffice in service advertisements
  22. 22. Advertising NFC tags: Examples• Service advertisement for a photo album service presenting photographs on a wall display:
  23. 23. Advertising NFC tags: Examples• Advertisement for a voting service:
  24. 24. Advertising NFC tags: Examples• Multimedia player service for a tourist attraction:
  25. 25. Advertising NFC tags: Examples• Service advertisement for connecting a phone to a photo printer. Left: printing is free. Right: printing costs 2,5 Euros:
  26. 26. Advertising NFC tags: Examples• Service ad for downloading and joining a game shown on a wall display:
  27. 27. Advertising NFC tags: Examples• Maps and contact information by touching:
  28. 28. Conclusions• NFC can revolutionize the way we use applications in our everyday life – a single touch to give a command• We have good experiences on: • Applications related to a place and/or using local resources • Using objects in environment as data containers • Learning applications• NFC is ready for large scale usage – for a rich set of different applications• Simple phones are sufficient• We continue building prototypes and developing the model and the language • The model can be used when designing applications and to communicate good practices • The visual language can advance common understanding
  29. 29. Contact information and acknowledgements• Contact information: – –• Acknowledgements – This presentation was enabled by numerous researchers from the University of Oulu and from our partners. The EPG application was developed at the University of Tampere. The work has been funded by Tekes, EU, regional funding agencies and companies
  30. 30. SELECTED PUBLICATIONSRiekki J, Sanchez I, Pyykkönen M (2012) NFC-Based User Interfaces. 4th International Research Workshop with focus on Near FieldCommunication (NFC2012), March 13th, 2012, Helsinki, Finland. Invited paper. (THIS TALK)Riekki J, Cortés M, Hytönen M, Sánchez I and Korkeamäki RL (2012) Touching Nametags with NFC Phones: A Playful Approach toLearning to Read. LNCS Transactions on Edutainment. Springer Verlag. In press.Sanchez I, Cortes M, Riekki J & Pyykkönen M (2011) NFC-Based Physical User Interfaces for Interactive Spaces. In: Near FieldCommunications Handbook (Internet and Communications). Ed Syed a. Ahson and Mohammad Ilyas. Auerbach Publications. 175-230.Pyykkönen M, Riekki J, Alakärppä I, Sánchez I, Cortés M & Saukkonen S (2012) Designing Tangible User Interfaces for NFC Phones. InKiyokawa, K. (Ed.) Advances in Human Computer Interaction. Hindawi Publishing Corporation. Volume 2012. J, Sanchez I & Pyykkönen M (2010) Remote control for pervasive services, Int. J. Autonomous and Adaptive CommunicationsSystems, Vol. 3, No. 1, pp.39–58., J. (2007) RFID and smart spaces, Int. J. Internet Protocol Technology, Vol. 2, Nos. 3/4, pp.143–152. J, Salminen T & Alakärppä I (2006) Requesting Pervasive Services by Touching RFID Tags. IEEE Pervasive Computing, Jan.-Mar. 2006. 5(1)40-46. J, Cortés M, Hytönen M, Sánchez I & Korkeamäki R-L (2011) Learning to Read by Touching Nametags with NFC Phones. TheFourth International eLearning Baltics Conference (eLBa 2011), 26-27 May 2011, Rostock, Germany.Sanchez I, Davidyuk O & Riekki J (2009) Towards User-Oriented Application Composition, In Proc. the IEEE Workshop on PervasiveService Computing and Applications (PSCA09), December 17-19, Shanghai, China.Sanchez I, Riekki J, Rousu J & Pirttikangas S (2008) Touch & Share: RFID Based Ubiquitous File Containers. In Proceedings of 8thInternational Conference on Mobile and Ubiquitous Multimedia (MUM08), December 3-5 2008, Umeå, Sweden. 57-63. I, Riekki J, Pyykkönen M (2008) Touch & Control: Interacting with Services by Touching RFID Tags. In Proceedings of the 2ndInternational Workshop on RFID Technology - Concepts, Applications, Challenges (IWRT 2008), In conjunction with ICEIS 2008.Barcelona, Spain, June 12-13, 2008. pp 53-62.Riekki J, Sanchez I, Pyykkönen M (2008) Universal Remote Control for the Smart World. F.E. Sandnes et al. (Eds.): UIC 2008, LNCS5061, Springer-Verlag, pp. 563-577. 10.1007/978-3-540-69293-5_44Sánchez I, Cortés M & Riekki J (2007) Controlling Multimedia Players using NFC Enabled mobile phones. In Proceedings of 6thInternational Conference on Mobile and Ubiquitous Multimedia (MUM07), Oulu, Finland, December 12-14 2007.