Successfully reported this slideshow.

Audio-Only Augmented Reality System for Social Interaction

1

Share

Loading in …3
×
1 of 25
1 of 25

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Audio-Only Augmented Reality System for Social Interaction

  1. 1. Audio-Only Augmented Reality System for Social Interaction TOM GURION1 AND NORI JACOBY1,2 1BAR-ILAN UNIVERSITY, RAMAT-GAN 2HEBREW UNIVERSITY, JERUSALEM
  2. 2. Augmented Reality Google Glasses www.google.com/glass/ Decolabs http://www.decolabs.com/ Wikitude http://www.wikitude.com/
  3. 3. Augmented reality in the audio domain?
  4. 4. Introduction and framework: Interactive music systems ADA: The Intelligent Space (2002) http://specs.upf.edu/installation/547 Reactable (2007) http://www.reactable.com/
  5. 5. Introduction and framework: Interactive music systems AutoRap by Smule (2012) http://www.smule.com/autorap RjDj – Sonic Experiences (2008) http://www.smule.com/autorap
  6. 6. Introduction and framework: Social context “A flash mob is a group of people who assemble suddenly in a public place, perform an unusual and seemingly pointless act for a brief time, then quickly disperse, often for the purposes of entertainment, satire, and artistic expression. Flash mobs are organized via telecommunications, social media, or viral emails.” (Wikipedia)
  7. 7. Introduction and framework: Social context “Silent disco has become a common name for a disco where people dance to music listened to on wireless headphones. Rather than using a speaker system, music is broadcast via an FM-transmitter with the signal being picked up by wireless headphone receivers worn by the participants.” (Wikipedia)
  8. 8. Research targets Propose and implement an audio-only augmented reality system for social interaction. Evaluate the system in the context of silent disco party. More generally, explore the potential of new ways of music consumption through mobile devices.
  9. 9. System Description
  10. 10. Overview The six balloons bundles Bluetooth beacon
  11. 11. Overview App Screenshot The partyApplication page in Google Play
  12. 12. Demonstration http://www.youtube.com/watch?v=2kJoeD2iWBA
  13. 13. Our positioning requirements What we need: Accuracy of about one meter Mobile No infrastructure What we don’t need: Exact positioning The solution: Bluetooth Based Relative Positioning system
  14. 14. Bluetooth discovery routine Repeatedly search for Bluetooth beacons. Extract received signal strength indicator (RSSI) from the Bluetooth device discovery and use it as an estimation of the distance between the user and the beacon. Send the device address and RSSI value to Pure Data patch using libpd API (Brinkmann, 2011).
  15. 15. Player patch Each one of the six beacons correspond to pre defined sound zone player. The patch receives RSSI values from the app and route them to sound zone player according to Bluetooth address. Each sound zone uses the RSSI value differently to manipulate the music in real-time (E.g. volume, music filtration, etc.)
  16. 16. Evaluation: preliminary results
  17. 17. Experiment design Participants were randomly assigned to two groups. Both groups started the experiment together. In the control blocks participants heard pre composed music based on the music of the interactive system. They were informed that the experiment consists of interactive and control segments. Group A consists of 8 participants (4 females and 4 males) with mean age of 36.7 (s.d = 12.3); group B consists of 10 participants (3 females and 7 males) with mean age of 29.6 (s.d = 10.2). Participants had a diverse musical background with 4.7 mean years of musical training (s.d = 5.2).
  18. 18. Measurements Surveys Each participant filled pre/post party surveys that included questions regarding their musical background and preferences as well as system evaluation feedback. Bluetooth discoveries count Counting the number of Bluetooth device discoveries made by the participants’ phones during the interactive and the control blocks. In order to eliminate edge effects, we analyzed only the two middle blocks of the experiment. Positioning tracking Video capturing of the party from above the dance floor. Analyzed offline.
  19. 19. Results: Surveys We found that using the interactive system participants tends to dance less with people they knew in advance. Participants self-reported significantly higher levels of movement using the system. We didn’t find supportive results to show that participants dance more with strangers. Dancing with known people: paired t-test, t(14) = -2.5, p_value = 0.01 Changing location in space: paired t-test, t(15) = 3.9, p_value < 0.01
  20. 20. Results: Bluetooth discoveries count We found higher counts during the interactive blocks of the party compared with the control blocks. paired t-test, t(16) = 1.7, p_value = 0.06, non-significant
  21. 21. Video tracking Tracking participants manually in Pure Data to generate positioning tables.
  22. 22. Results: Positioning tracking We didn’t find significant difference between the position in the interactive and control blocks. However position was indicative to the social relations between participants! We asked participants for their social relationships to other participants. We compared the resulted map with video correlation index. The results shows that the p_value < 0.001
  23. 23. Conclusions Our system demonstrate a simple way to use Bluetooth technology for relative indoor positioning. Our preliminary results demonstrate the potential for audio-only augmented reality to significantly enrich the experience of music consumption and social interaction. We show that the social implications of the system can be validated in a controlled experiment. We show that positioning tracking can be used to obtain social relationships within large group of people.
  24. 24. Future research We will use the devices accelerometers to asses entrainment. Larger and more comprehensive experiments with a different design.
  25. 25. Thank you!

Editor's Notes

  • “Master student… Thesis”
  • “We are all familiar with the concept of augmented reality and you are probably know much more about that then me. But note that most of the different project in this area are in the visual domain.”
  • “I was interested if it possible to extends our perception and interaction with the real world using audio-based experience instead of visual.”
  • “Lets start with an introduction.Music technology changed music creation and consumption in the last 60 years. I will focus on the last decade.On the left you can see… Switzerland”
  • “…it is called ‘Scenes’ in the RjDj glossary”
  • “Now I want to talk about some social phenomena that relate to my project. The first one is the flash mob… Note that this social phenomena is highly dependent on new technology, especially on social media… Commercial use of this idea.”
  • After 1:“Using the system, participants will be able to interact with one another as well as with system components to affect the music in real time.”
  • “So how does the system work&quot;
  • “There are six balloons bundles on the dance floor. Every one of them… Inside every one of the bundles there is… This is a completely standard 10$ beacon but any other Bluetooth enables device will work just fine” NOKIA JOKE “…except for these beacons, all the other equipment required for the system is the mobile devices of the participants.”
  • “Participants come to the party with their own Android devices and headphones. They install our application from Google Play. After turning Bluetooth on… Each bundle creates a virtual sound zone around it… in your headphones. In addition, participants can move the sound zones… Imagine…”
  • “Here is a video from a party using the system. You can’t hear anything because… The system is here with me and some of you have already tried it. For now I will show you a short simulation that uses the same music we use in our system. You will see how participants can interact with the sound zones and actually choose which one to listen to. In addition you will hear the influence of different participants on one another when one virtual participant takes different sound zones with him during this simulation.”
  • “As you saw in the demonstration, a crucial component of the system is the ability to find positioning of participants on the dance floor. When we started to consider different options for indoor positioning we realized that we…”“We don’t need the absolute positioning of the participants in space, just the distances between the participants and the sound zones.”“…That we developed.”
  • “RSSI is part of the Bluetooth protocol and therefore implemented on every mobile device.”“Pure data is both programming language and a powerful sound engine. We use Pd for the audio part of the application.”
  • PHASE CHANGE“Most of the research in augmented reality nowadays does not describe formal user evaluation of the system. We wanted to check if we could evaluate the system in a controlled experiment. Again, we want to discover the properties of the social behavior using the system as opposed to standard silent disco party.”
  • After 1: Describe the experiment in details. About the blocks etc.
  • What I want to get from the different measurements. Find differences in social behavior. More counts shows that participants interact more with system components.
  • “…Future experiment”
  • Regarding the right graph:“In the right graph we see the counts during the control compared with interactive blocks. As we can see, most of the participants have higher counts during interactive blocks and therefore are bellow the main diagonal (marked with red X’s).”
  • After 3:“We used Monte Carlo reshuffling to check how similar those two maps are and we found…Actually, participant 7 and 8 are a married couple…There is a lot of data in this slide but the bottom line is this:…”
  • “We will use this measure as an implicit measurements of engagement.”“We hope that with better experiment design we will find much more significant results.”
  • ×