Supersense! Context Massive Sensing and its ImplicationsPhilip van Allen, Media Design Practices, Art Center College of Design
Supersense! Context Massive Sensing Context - Now/ Future Machine Sensing Massive Sensing Internet of Things (IoT) Human Perception & Science
Supersense! Context Machine Sensing Sensing Now What sensors are in your phone?Discussion: Identify all the sensors in your phone
Supersense! Context Machine Sensing Sensing Now What sensors are in your phone? multi-touch internal moisture/temperature GPS sound accelerometer – motion in x/y/z gyro – orientation and position ambient light proximity compass camera still/video buttons
Supersense! Context Machine Sensing What is a sensor? Interfaces with an environment Measures a physical property Transduces that property into a numeric value (typically) Has a certain range and resolution
Supersense! Context Machine Sensing Sensing Now Types location: GPS, RFID environmental: moisture, temp, humidity, light, sound, altitude, etc. position/motion: acceleration, distance, pressure, orientation, compass, etc. energy consumption: Kill-A-Watt vision – facial recognition, objects language – Siri gesture - Kinect, LeapMotion
Supersense! Context Machine Sensing Sensing FutureMajor corporations are actively investigating possible applications of massive sensing. Here’s one perspective fromIBM.
Supersense! Context Machine Sensing Sensing Future Types location inside buildings: people and things – micro location touch: input & output vision: scene & object recognition smell: trace of a person or thing that was recently in a place medical: cancer, health status, old/young, etc. chemicals of any sort taste: food, water, nutrition, soil, any chemical sound: fuller language generation/recognition mimic sensors in nature
Supersense! Context Massive Sensing What is Massive Sensing? Billions of connected sensors Public/Private Low cost It’s here already
Supersense! Context Massive Sensing Massive Sensing NowTraffic displays are an integration of data speed and location data from driver cell phones + sensors in the roads +feeds from caltrans & news
Supersense! Context Massive Sensing Massive Sensing NowWeb Analytics gather “clickstreams” from millions of people every minute as they browse the web
Supersense! Context Massive Sensing Massive Sensing Now How The "Internet Of Things" Is Turning Cities Into Living Organisms When city services can autonomously go online and digest information from the cloud, they can reach a level of performance never before seen. First up, water systems that automatically know when it will rain and react accordingly.This is an example of massive, AUTONOMOUS BEHAVIOR as well as sensing.What are some other examples of massive sensing?
Supersense! Context Massive Sensing Massive Sensing Future Enabling Technologies/Contexts sensors smaller, cheaper, more energy efﬁcient low-power wireless mesh networks energy harvesting techniques (e.g. Powercast Wireless Sensors) economies of scale: sensors in phones, cars internet infrastructure growth nano-technology enables embedding in everything clothes, paint, our bodies, ??? language recognition, AI, big data
Supersense! Context Massive Sensing Massive Sensing Future Enabling Technologies/Contexts social acceptance: life-logging, cameras everywhere, etc. corporate r&d - IBM, Intel need for cost efﬁciencies environmental monitoring/mitigation climate change earthquakes/tsunamis
Supersense! Context The Internet of Things (IoT) Internet of Things Overview Everything has an ID and generates data It’s all sent/received via the cloud Data access in & out is asynchronous Range of models: one-to-one one-to-many many-to-manyIDs = RFID, IPv6 address, custom IDIoT used to be thought of as being based on RFIDs (see Shaping Things), now it’s active devicesData becomes a service
Supersense! Context The Internet of Things (IoT) IoT Data Services COSM “Turn these ﬂows of information into action by ThingSpeak triggering online Applications. Sen.se Things become smart, versatile and interact in real time with remote things.” --Sen.seDEMO: Show IOBridge on web display, as well as in NETLab Toolkit
Supersense! Context The Internet of Things (IoT) IoT Action ServicesExercise: Everyone pair up & create an IFTTT recipe
Supersense! Context The Internet of Things (IoT) IoT Hardware IOBridge NinjaBlocks Twine Belkin WeMo Insteon Almond+DEMO: IOBridge, Belkin WeMo
Supersense! Context The Internet of Things (IoT) Most IoT Applications Suck Mashups: turning your data into something useable and useful. “We believe that looking at or processing data feeds one at a time doesn’t help you solve real life problems. Most often you need to combine data from several different sources in order to understand what is really going on, take smart decisions, make really useful, useable and non annoying applications.” --Sen.se Blog
Supersense! Context Human Perception Engaging with Human Perception Why work with scientists? different perspective interesting new methodologies creative synthesis an interesting context to work with need to maintain expertise/independence of design Understand how people will perceive sensing data to make sophisticated systems Can use it as a metaphor for design - biomemtics, machine human interfacesHow do people perceive the world?
Supersense! Context Human Perception Perception is different than sensing Multiple Inputs Knowledge Context Actively probing env. Apple or Potato? ReverberationEat an apple slice and potato slice while holding your nose - can you tell the difference in taste?
Supersense! Context Human Perception Seeing with your skin Sensory Substitution Paul Bach-y-Rita 1972Video pixels are turned into vibrators on person’s back - note that the person is able to move the camera and learnwhat things “look” like.