This document provides an overview of RoomWare, which allows users to interact with their environment using sensors, mobile devices, and online services. The key concepts discussed are: 1. Record the moment - bookmark objects, events, and locations using RFID, QR codes, and geo-tagging. 2. Connect to others - link people using identifiers, proximity detection, social networks, and a RoomWare server. 3. Connect to your environment - interact using mobile devices, handshakes between apps and sensors, active devices, and software. Tools that enable these functions include computers, RFID readers, touchscreens, projectors, input devices, and controllers like the Wii remote

1. RoomWare
Workshop
August 29 / 30, 2009

2. INTRO
HOME BREW GALORE

3. Glorious stuff you can make
at home

4. RoomWare for:

7. (notice they are regular kids)

8. Three Keys
Anyone can do it
Simple to setup
No special knowledge
required

10. Workshop basic goals
Create concrete solutions
Using off the shelf materials
Where anything is remote controllable
via sensors or a mobile phone

11. Three design rules
Easy to install
Easy to expand / extend
Up and running in minutes

12. Means and Materials
Microphones
(Web) cams
Paper & Printers
Mobile phones
Online web services
RoomWare Web Applications

13. Fields
1. Online: Web applications
2. Mobile: RoomWare applications
3. Local: User input via sensors
4. Server: Extending the RoomWare
server
5. Conceptual: Use cases for RoomWare
use

14. 1: Online
1. Identity Broker
2. Visualization of social data
(Flikr, Twitter, .. in Flash / HTML)
3. QR-code reader (Flash)
4. Connection to RoomWare Server

15. 2: Mobile
1. Augmented (web) browser
1. QR-code reader
2. Direct call to RW service
2. QR-code / RoomWare handshake
3. RoomWare Remote control

16. 3: Local (input / sensors)
1. Scratch input
2. Sensors
3. Camera: gestures / fiducials / QR / AR

17. 4: RW Server
1. Installer
2. Bridges
3. Communication / interaction via
Sockets
4. Definition / description of interfaces
5. Config panel

18. 5: Conceptual
How would you like to use
RoomWare?

20. PART ONE
BASIC CONCEPTS OF
ROOMWARE

21. A quick flip through
presentation
Basic concepts of RoomWare

22. Basic concepts
Record the moment
Enrich the present *
Connect to others
Connect to your environment
Re-live the past
Discover the future
Basic concepts of RoomWare

23. CONCEPT ONE
Record the moment
Basic concepts of RoomWare

24. Using
Record the moment

25. RFID cards / tags
Record the moment

26. QR-codes
Record the moment

27. Local readers
Record the moment

28. Geo tagging
Record the moment

29. RoomWare Server
Record the moment

30. Record the moment

31. To bookmark
Record the moment

32. An object
Object + moment + location
Record the moment

33. Record the moment

34. Record the moment

35. A point
Moment + location
Record the moment

38. An event
Location + [start time | end time]
Record the moment

40. Record the moment

41. CONCEPT TWO
Connect to others
Basic concepts of RoomWare

42. Using:
Connect to others

43. Input devices
Connect to others

44. Identifiers & readers
Connect to others

45. RoomWare Server
Connect to others

46. Identity Brokers
Profile Identity
<XML>
IN Social OUT <identity></identity>
Identifier <identity></identity>
Profile </XML>
Access
Location
Grant
Connect to others

47. Proximity
You Proximity levels
Close Around the Around the block
corner
Connect to your environment

48. Social networks
Connect to others

49. Software
Connect to others

50. To:
Connect to others

51. Link people
• Showing relationships / connections
• Showing meta-data (Data Cloud)
• Linking people using Social Services
Connect to others

52. Connect to others

53. CONCEPT THREE
Connect to your
environment
Basic concepts of RoomWare

54. Using:
Connect to your environment

55. (Mobile) devices
Connect to your environment

56. Identifiers & readers
Connect to your environment

57. RoomWare Server
Connect to your environment

58. Proximity
You Proximity levels
Close Around the Around the block
corner
Connect to your environment

59. Handshakes
RW
RW Mobile
Handshake
App
Web RoomWare
interface Server
YOU
QR-code
reader
Connect to your environment

60. Active devices
• BlueTooth (MAC address)
• WIFI
• Active RFID
• (anything with a radio transmitter)
Connect to your environment

61. Pattern recognition
• Biometric face recognition
Connect to your environment

62. Software
• Layar
• RoomWare Remote mobile applications
• ...
• (Web) Applications
Connect to your environment

63. Online services
• Remote RoomWare Controller
Connect to your environment

64. - Interacting with your environment

65. For:

66. - Interacting with the
environment
- Interacting with your environment

67. Using:
- Interacting with your environment

68. Projectors
- Interacting with your environment

69. RoomWare Remote
• (Mobile) Web Application
• Accessible via phone and laptop
- Interacting with your environment

70. QR / RFID handshake
• To pair you to the Remote RoomWare
Controller application
- Interacting with your environment

71. Controllers
• Wii-mote (whiteboard, laser pointer)
• Arduino + input sensors
• Scratch Input (sound + pattern recogn.)
• Touch sensitivity via sound (“Sensitive
Object”)
• ...
- Interacting with your environment

72. Sensors
• Heat
• Movement
• Humidity
• Pressure
• Proximity
• ....
- Interacting with your environment

73. Devices
• Door openers / magnetic locks
• Heating / Cooling
• Lighting
• Audio / video players
• ....
- Interacting with your environment

74. Online preferences
• Music
• Books
• Video’s
• Artists
• Brands
• ....
- Interacting with your environment

75. Online content
- Interacting with your environment

76. Sophisticated stuff
- Interacting with your environment

77. To:
- Interacting with your environment

78. Set music
- Interacting with your environment

79. Share your photos
- Interacting with your environment

80. Influence lighting
- Interacting with your environment

81. Influence lighting
- Interacting with your environment

82. Influence lighting
- Interacting with your environment

83. Influence projections
- Interacting with your environment

84. - Interacting with your environment

85. - I am here...
Connect to your environment

86. Mark your presence
- I am here...

87. Mark your presence
- I am here...

88. Mark your presence
- I am here...

89. ...here
- I am here...

90. ...here
- I am here...

91. ...or here
- I am here...

92. - I am here...

93. - Where is .. ?
Connect to your environment

94. This person?
- Where is...

95. That place?
- Where is...

96. My friends & places?
- Where is...

97. My events?
- Where is...

98. Connect to your environment

99. - When will... ?
Connect to your environment

100. ..this happen?
- When will...

101. - When will...

102. - What else is here?
Connect to your environment

103. Exits, people I know
- What else is here?

104. Places to go
- What else is here?

105. Anything of interest
- What else is here?

106. - What else is here?

107. CONCEPT FOUR
Relive the past
Basic concepts of RoomWare

108. Relive the past

109. Using
Relive the past

110. Logging
• Identity Broker
• Date / Time / Location based on
RoomWare feedback
Relive the past

111. Relive the past

112. Replay your recorded life
Relive the past

113. Your activity stream
Relive the past

114. Relive the past

115. Relive the past

116. Relive the past

117. Relive the past

118. Relive the past

119. CONCEPT FIVE
Discover the future
Basic concepts of RoomWare

120. Discover the future

121. Agenda’s of venues and places
Discover the future

123. PART TWO
TOOLS TO MAKE IT WORK

124. Conceptual / in practice
Tools to make it work

125. ONE
Enabling the room
Tools to make it work

126. Computers
Enabling the room

127. RW Server
Identification (inter)action
Data Source
ID Broker Web App
RoomWare Server
Bridge A ID Bridge B
Adapter
ID Reader
Sensors Devices
Enabling the room

128. ZigBee networks
Enabling the room

129. RFID readers
Enabling the room

130. Active devices
• BlueTooth (MAC address)
• WIFI
• Active RFID
• (anything with a radio transmitter)
Enabling the room

131. Touch screens/tables
Enabling the room

132. Projectors
Enabling the room

133. Flat screens
Enabling the room

134. Touch screens
Enabling the room

135. Enabling the room

136. TWO
Interacting with the environment
Tools to make it work

137. Input devices
Interacting with the environment

138. Identifiers & readers
Interacting with the environment

139. Controllers
• Wii-mote (whiteboard, laser pointer)
• Arduino + input sensors
• Scratch Input (sound + pattern recogn.)
• Touch sensitivity via sound (“Sensitive
Object”)
• ...
Interacting with the environment

140. Sensors
• Heat
• Movement
• Humidity
• Pressure
• Proximity
• ....
Interacting with the environment

141. Devices
• Door openers / magnetic locks
• Heating / Cooling
• Lighting
• Audio / video players
• ....
Interacting with the environment

142. Interacting with the environment

143. THREE
Connecting with others
Tools to make it work

144. Identity Brokers
Profile Identity
<XML>
IN Social OUT <identity></identity>
Identifier <identity></identity>
Profile </XML>
Access
Location
Grant
Connecting to others

145. Software
Connecting to others

146. Connecting to others

147. PART THREE
THE FRAMEWORK

148. Basic setup
• Identity Broker
• RoomWare Server
• ID Readers (RFID, BT, QR-code)
• (Web) Applications
• Sensors and Devices
• Bridges
The framework

149. Identification (inter)action
Data Source
ID Broker Web App
RoomWare Server
Bridge A ID Bridge B
ID Reader Sensors Devices
The framework

150. Identification (inter)action
Data Source
ID Broker Web App
Pointer to
RoomWare Server
Bridge A ID Bridge B
Adapter
ID Reader
Sensors Devices
The framework

151. Identity Reader
• Is used to identify people in a location
• Can be any device: BlueTooth, RFID, ...
• Can be software: Biometric Face
recognition, QR codes
• Can be [one] to [one] (RFID, QR code)
• Can be [one] to [many] (BlueTooth)
The framework

152. Identity Broker
• Is an external service
• Can be any provider
• Is called by the RoomWare Server
• Translates an abstract number to one
or more (online) identities
The framework

153. Basic ID Broker setup
Profile
[one] [one]
[many] [many]
<XML>
IN OUT <identity></identity>
Identifier Identity
<identity></identity>
</XML>
- QR code - Flickr
- RFID card - Twitter
- BlueTooth ID - FaceBook
- BlueTooth MAC address - LinkedIn
- ... - ...
Identity Broker

154. Protecting Identities
• Basis: distrust + roles (professional/private)
• User defines which social identities are
public (visible for anyone)
• Per location the user sets access grants to
specific social identities
• Each set of grants can be put in a social
profile
Identity Broker

155. Using QR codes
QR-code
• Each of your social profiles can be
represented by a unique QR-code
• Each of these QR-codes is a key
• By showing the QR-code, the user creates
a connection between a social profile and a
location
Identity Broker

156. Using specific identifiers
• Social profiles can also be connected to
specific Identifiers (RFID card, BlueTooth
device, QR-code)
Identity Broker

157. ID Broker: access grant
[many]
Profile [one] Identity
[one] [many]
[many] [many]
<XML>
IN [many] Social OUT <identity></identity>
Identifier <identity></identity>
[many]
Profile </XML>
[one]
[many]
[one] Access
Location [many]
Grant
Identity Broker

158. Identity Broker

159. (Web) Applications
• Run outside RoomWare Server
• Uses external Web Services to create
(data) visualizations related to present
people
• RoomWare Server has pointer to App
• RoomWare server acts as a bridge for
the app to interact with the space
The framework

160. Adapters
• Are pieces of software to communicate
with RoomWare and Sensors and Devices
• Plugs into RoomWare via XML stream or
using Sockets
The framework

161. Sensors
• Read temperature, rotation, speed,
humidness, etcetera
The framework

162. Devices
• Perform actions: like opening / closing
doors, setting temperature
The framework

163. Bridges
• Pass information from A to B
• Have a limited intelligence
• Have no awareness of what is sent
• Connect (Web) Applictions to Readers,
Sensors and Devices
The framework

164. Identification (inter)action
Data Source
ID Broker Web App
RoomWare Server
Bridge A ID Bridge B
Adapter
ID Reader
Sensors Devices
The framework

165. Conclusion
• RoomWare aggregates Identities and passes
these to (Web) Applications
• Communication is directly between Web
Applications and Devices
• RoomWare Server functions as a bridge
The framework

167. PART FOUR
THE WEB INTERFACE

168. Room Software Presentation
Mobile
Web
Application
ID readers Laptop /
desktop
Sensors
RoomWare Beamer
Devices
The web interface

169. Room Software Presentation
Mobile
Remote
Control
ID readers Laptop /
desktop
Sensors
RoomWare Beamer
Devices
The web interface

170. The web interface

171. RoomW
Remote are
Control
The web interface

172. Room Software Presentation
Mobile
Visualize
ID
Laptop /
readers
desktop
Sensors
RoomWare Beamer
Devices
The web interface

173. The web interface

174. Parts
• Remote calls to RoomWare server
• Webinterface in Flash / Ajax / HTML
• Communication via HTTP POST /
Sockets
• Security via handshake / pairing
The web interface

175. Local Web
User ID
ID Reader
ID Broker
RW Server Identities
Web App
The web interface

176. Local Web App Web Client
RW Conn
Event
User actions
handler
RW Server
Data Retr.
Visualization
Send
Instructions Room
Sensors
Adapter
Devices
The web interface

178. PART FIVE
AUTHENTICATION
PROCESSES

179. Means
• RFID
• BlueTooth
• QR-code
• Mobile Phone
Authentication process

180. Basic assumptions #1
• RFID / BlueTooth: Preferred Identifiers
• QR-code = easy to copy = “unsafe”
• QR-code = identifier = easy to produce
Authentication process

181. ID Broker + QR-code
Create ID Broker
account
Account ID
Identity
Print out
Authentication process

182. RFID / BlueTooth
• Using QR-code to link RFID / BT to my
Identity
Authentication process

183. Register RFID | BT
ID Reader Handler
√ Registered
1 QR-code QR + RFID
Application
reader QR + BTID
X Unregistered 3.
2.a RFID Register
Reader
RoomWare
Server
2.b BlueTooth ID Broker
Reader
RFID / BlueTooth

184. Register Process
ID Reader Application
√ Registered
1 Read Show identity
QR-code based on QR-code
X Unregistered
2 Read [xxID] “Please confirm”
?
Feedback “Register [xxID]
3 ID Broker failed | success”
RFID / BlueTooth

185. Use RFID | BT
ID Reader Handler
√ Registered
1.a QR-code Application 3: visualiza-
reader tions
√ Registered
1.b RFID
ID Broker
Reader
RoomWare
Server
1.c BlueTooth
Reader 2: Social
Identities
RFID / BlueTooth

186. Possible Scenarios
• Single ID Broker: I register once. Every
venue will recognize me
• Multiple ID Brokers: At each new
venue I do basic registration process. My
[xxID] will be locally stored & linked to
specific broker
• Central IDB Switch: All RW Servers
call Central IDB Switch. Stored is
[xxID] + link to “Personal ID Broker”
RFID / BlueTooth

187. QR-code
• Using only QR-codes to identify and
connect
Authentication process

188. Authentication
• RoomWare requires affirmation of your
presence
• Authentication via handshake
QR-code

189. RW QR handshake
RW QR
RoomW
are
+ =Access Mobile
Applicat
ion
YOUR QR
QR-code

190. RW QR handshake
QR-code Device RoomWare
RW
RW Mobile
Handshake
App
Web RoomWare
interface Server
YOU
QR-code
reader
QR-code

192. PART SIX
PROXIMITY CONCEPTS

193. Basis

194. Possible means
• Active devices
• Remote sensing
• BlueTooth, WIFI, active RFID
• Face recognition
• Geo location
• ...

195. Proximity
You Proximity levels
Close Around the Around the block
corner

196. Proximity
You Proximity levels
Close Around the Around the block
corner

197. Proximity
Location
Location
Service
You Proximity levels
Close Around the Around the block
corner
Service
Location
Service

198. Feedback
Location
Location
Service
You Proximity levels
Close Around the Around the block
corner
Service
Event
Service

199. Feedback
• Who else is here?
• Where am I?
• What can I do/find here?

200. Actions
• Direct manipulation of RW connected
devices
• Projection of my public data to my
surroundings

202. Concepts in action

203. Close proximity
• Connect to other people
• Find common interests
• Share information from public space

204. In another room
• Where is...?
• Find people interesting to you
• Find objects close by

205. Around the corner
• Where is...?
• Find people interesting to you
• Find objects close by

206. Around the block
• Where is...?
• Find locations and people interesting to you
• Find objects close by

209. END

210. PART SIX
DATA CLOUDS

211. Basis #1
Your data
cloud
You

212. Venue
Interaction
Your data
cloud
You

213. Ambient
Service
Venue
Ambient
Service
Interaction
Your data
cloud
You

214. Basis #3
PULL
NOT PUSH!

215. PART SEVEN
REMOTE ROOMWARE
CONTROLLER

216. Hitting 2 recievers
You

217. Phone: Seeing 2 services

218. Hitting more recievers
You

219. Phone: Seeing more services

220. And more services

221. Acessing a service
Location Based
Service Info
Lorum ipsum
Use >

222. Ambient
Service
Basic Setup
“Open ID” Online
DataCloud
connector services
Server Server
reciever reciever
item ID
You item ID
Feed back
Web from venue
connection

224. Usage

225. Finding people
Me

226. Finding stuff

227. Making stuff happen

229. Conclusion

230. Location / time based Bookmarks
Location based Applications
Location based Interactions

231. Bookmarks - AnyWhere Server
Applications - Any provider
Interactions - RoomWare

232. Parts

233. AnyWhere offers: Anyone offers:
Identification
Central access
• Proxies
Data Shielding
• Services
Bookmarking
• Controls