Web Facilitated Playin the Real WorldSketching in HardwarePortland – 20 July, 2012@ajfisher
Play enhances our livesImage (CC) flickr – Duncan / kkseema / Adam Jones / mzagozda
How we play todayImage (CC) flickr – Henry Stratford
Civic spaces need an upgradeImage (CC) flickr – Jesus Leon
Time to level upImage (CC) flickr – ipdegirl
1. Web Sockets overview2. Real time web in a real time real world3. Applications & demos
WebsocketsImage (CC) flickr – InertiaCreeps
PubSub messaging                        Publisher    Produces Messages{a:event, x:123,...}                       Web Socke...
Common pattern                 PublishersBroker                 Subscribers
Websocketing thingsImage (CC) flickr – rfranklinaz
Real time data capture
Architecture for playInterface                                DjangoConsolidation / processing                       Webso...
BenefitsWeb interfaces have wide reachNo “app” downloadsMobile devices have numerous sensorsCan access other parts of the ...
Time to playImage (CC) flickr – kalexanderson
Tank Tag
Tug of war
Tug of war
Multiplayer lighting
Play with the lightsPoint a browser atajf.io/sketch12Live demoed at conference
Multiplayer lighting
ResourcesExamplesgithub.com/ajfisher/sketching-conf-demoWebsocket Client for Arduinogithub.com/krohling/ArduinoWebsocketCl...
Web Facilitated Playin the Real WorldSketching in HardwarePortland – 20 July, 2012@ajfishergithub.com/ajfisherslideshare.n...
Upcoming SlideShare
Loading in …5

Web Facilitated Play in the Real World


Published on

Presented at Sketching in Hardware 2012, this presentation outlines how combining mobile devices & network connected physical devices via real time web technologies can change the way we play.

Outlining the current state of technology for connecting physical devices to the real time web using Web Sockets and messaging, three examples are shown to illustrate different aspects of play.

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Hi. My name's Andrew Fisher and my background is as an interaction developer and today I want to talk to about web facilitated play in the real world.
  • Play is fundamental to our lives. From our earliest childhood through to being an adult. Play is a core part of what defines our humanity. Technology and play are somewhat symbiotic and each one helps to drive the other forward and evolve. http://www.flickr.com/photos/duncan/79106711/sizes/o/ http://www.flickr.com/photos/kkseema/2042946052/sizes/o/ http://www.flickr.com/photos/adam_jones/3794676268/sizes/o/ http://www.flickr.com/photos/mzagozda/4242006639/sizes/l/
  • So I was on a train the other day going to work and saw someone that looked like this. Headphones in, iPhone out, Totally immersed in what was obviously a multiplayer game. This is a familiar view of the modern world. At times I'm this person. The technologist in me loves the fact we have the technical capability for someone to play a multiplayer game from their phone in Melbourne with others in Madrid and Moscow. However, I can't help but feel we've had to give something up as a result as we tune out the world around us so totally. Image: http://www.flickr.com/photos/henry_stradford/5348910688/sizes/o/
  • I think our shared spaces are causing some of this as well. How often do you stand on a train station platform and see advertising 6 months or in some cases 6 years out of date. Our civic environment hasn't kept pace with technology and is becoming ever more boring just as our phones become ever more interesting – is it any wonder we tune out? Image: http://www.flickr.com/photos/jesusleon/6932806609/sizes/o/
  • We have hit an interesting time though. With a nearly billion smartphones in people's pockets and a billion more not far away. With physical computing devices increasing in capability and increasingly networked with all the benefits that confers And all of this coupled with the web! Now we have the technologies at our disposal to reshape the nature of play or at least of interaction in our shared spaces using digital tooling. So what could this do for play as a result? Image: http://www.flickr.com/photos/ipdegirl/5327066487/sizes/o/
  • So today I want to show you how this can work. I'll give a brief intro to web sockets, then show you the technical architecture for doing this and then round up with some examples and a demo at the end.
  • So the technology that enables real world play is called web sockets. Is anyone here familiar with it? Has anyone actually used it in a project? Okay so very quickly web sockets is part of the suite of tech that makes up HTML 5. Thought it's not actually HTML. http://www.flickr.com/photos/inertiacreeps/1414845543
  • Web sockets is an upgrade to the standard HTTP connection Getting rid of the traditional request / response method. Instead it opens a long lived, bi directional network connection that data can be passed along. This lends itself well to scenarios where you are passing small packets of data very frequently over connections open for long periods like minutes or hours. Often we use this for messaging use cases and the most frequent pattern is pub / sub.
  • As you can see we have a publisher that produces a message and sends it to the web sockets server which acts as a broker and it passes that on to subscribers who consume the message and do something with it. Now in web sockets world most messages are passed around using JSON because it's lightweight, many libraries exist, it's expressive and flexible.
  • So this architecture scales well and you can have many publishers and subscribers just producing and consuming messages. Many topologies exist for different needs but this is one of the most common. Now you understand a bit about web sockets let's talk about how we can use it to connect objects in the real world.
  • So what's interesting about web sockets is that it's a protocol and as such anything that can talk it can become a publisher or a subscriber and work with messages. http://www.flickr.com/photos/88548643@N00/139617711/
  • For example we could have an arduino with a temperature sensor on it that broadcasts temperature readings every 100 milliseconds to the broker. The subscribers could be a real time logging database for later processing and a web client for real time graphing of the data. Now scale the arduino out and have 10 sensor nodes, 100, 1000 all producing data in real time that could be used by a dozen subscribers all doing different things with it. As interesting as that facet is, I came here to talk about play. For play, then, we can turn all of this on it's head. So here's a view of the stack I use to do this.
  • At the top we have web browsers across phone, tablet or desktop with interfaces that create messages. These get sent to the web sockets server for processing. I use Django Socket IO as I'm a python guy but there's things like Node.JS which are conceptually similar. The broker then forwards on messages to the subscribers which in this case is an arduino running Kevin Rohling's excellent WebSockets library and it then turns that data into some type of physical control.
  • There's a lot of benefits to using this stack which you can see here, however the main one I want to call out is that as the system is web based and messages are JSON it means I can prototype and simulate EVERYTHING in the browser. This means I can get the web interaction rock solid and the networking totally debugged BEFORE going to hardware which then becomes a decoupled and more straightforward implementation on the physical side.
  • Now you understand how the architecture works I have three examples I want to share, including one I've brought with me that you can play with. http://www.flickr.com/photos/kalexanderson/5315471388/sizes/o/
  • My first iteration of this was purely digital in order to assess the viability of the stack under real world conditions. Tank tag uses the tilt sensor in a phone to drive a tank around on a screen; and the touch screen to fire, so you can tag other players. The key is that the phone is now the controller and everyone plays together on a shared space on a screen or via a projector. The largest scale I've had about 100 people playing this on a 40ft projected screen at once. I tried for bigger once and got nearly 150 before the network seized up for volumes of data being shifted.
  • For my next iteration, I wanted to incorporate something much more physical but also create a sense of team work and focus in the space amongst the players. The tug of war concept was used for this and again players interact with their mobile phones. After hitting a web page the players are divided into a red and green team with their own goals to pull a magnet towards.
  • This time the accelerometer is used for input and players have to shake their phones as hard as possible. The team that is collectively pulling the hardest at any time will pull the magnet towards their goal and eventually light it up. This was a lot of fun and really starts to create a sense of shared use in the space given the focus and requirements for team work and it's all facilitated by web enabled devices that people already have in their pockets.
  • The third example I've actually brought with me from Australia. It's quite simple so it was travel proof but hopefully it will serve to illustrate to you the direction this can all go.
  • If you want to play with this then point a web browser on any device to This won't work in Internet Explorer however so use something else. Click on the circles and away you go. As you all can have different colours they should mix somewhat if you hit the same lights at once. As you can see the interaction is near instant and again there's almost no barrier to someone interacting with this. ---- HOPEFULLY PEOPLE CLICKING NOW!
  • So if you consider my original points about creating interactive spaces by bringing mobile, physical computing and web together this is how we can do it. Imagine something like this scales up to say 10 metres across and hung on the side of a building and all you'd need is a phone with a web browser in order to interact with it.
  • This stack will work in any context where we want to create interaction on a semi mass scale but where physical proximity is important. So it could be used for interactive advertising for shared user experiences, general civic space upgrades or for spaces like museums to create points of interaction with exhibits via smartphones without having to worry about downloaded apps.
  • In summary, I've walked you through a tech stack that gives you the ability to harness the real time web and allow it to interact with the real world. I've got some resources here which I'll send around as a link and all the code is in my git hub for the demos I've shown today. Obviously come and talk to me about this if you're interested and I can show you code etc if you want to see it.
  • I hope this gives a slightly different view on how we can bring together mobile with physical devices and use the web to facilitate play in the real world.
  • Web Facilitated Play in the Real World

    1. 1. Web Facilitated Playin the Real WorldSketching in HardwarePortland – 20 July, 2012@ajfisher
    2. 2. Play enhances our livesImage (CC) flickr – Duncan / kkseema / Adam Jones / mzagozda
    3. 3. How we play todayImage (CC) flickr – Henry Stratford
    4. 4. Civic spaces need an upgradeImage (CC) flickr – Jesus Leon
    5. 5. Time to level upImage (CC) flickr – ipdegirl
    6. 6. 1. Web Sockets overview2. Real time web in a real time real world3. Applications & demos
    7. 7. Websockets
    8. 8. WebsocketsImage (CC) flickr – InertiaCreeps
    9. 9. PubSub messaging Publisher Produces Messages{a:event, x:123,...} Web Sockets Processor / Broker Server{a:event, x:321,...} Subscriber Consumes Messages
    10. 10. Common pattern PublishersBroker Subscribers
    11. 11. Websocketing thingsImage (CC) flickr – rfranklinaz
    12. 12. Real time data capture
    13. 13. Architecture for playInterface DjangoConsolidation / processing Websockets server SocketIO Something to control Network connected
    14. 14. BenefitsWeb interfaces have wide reachNo “app” downloadsMobile devices have numerous sensorsCan access other parts of the web app(eg Authentication)Web scaling well known problem with solutionsDebug in browser BEFORE going to hardware
    15. 15. Time to playImage (CC) flickr – kalexanderson
    16. 16. Tank Tag
    17. 17. Tug of war
    18. 18. Tug of war
    19. 19. Multiplayer lighting
    20. 20. Play with the lightsPoint a browser atajf.io/sketch12Live demoed at conference
    21. 21. Multiplayer lighting
    22. 22. Applications
    23. 23. ResourcesExamplesgithub.com/ajfisher/sketching-conf-demoWebsocket Client for Arduinogithub.com/krohling/ArduinoWebsocketClientDjango Socket IOgithub.com/stephenmcd/django-socketioSocket IO Librarysocket.ioWebSockets Backgrounddev.w3.org/html5/websockets/
    24. 24. Web Facilitated Playin the Real WorldSketching in HardwarePortland – 20 July, 2012@ajfishergithub.com/ajfisherslideshare.net/andrewjfisher