The document proposes merging physical and digital worlds by creating Augmented Worlds (AWs) where physical objects and spaces are augmented with digital entities. It introduces the concept of Augmented Things (ATs) which represent augmented entities in the Web of Things. ATs have properties like location and holograms. The document envisions an infrastructure with nodes hosting AWs, tools to couple digital and physical worlds, and AR libraries interacting with AT holograms. A case study of an augmented hospital is discussed. Future work includes prototyping an AW platform and validating the AT/AW concepts in applications like augmented reality case studies.

1. Towards
the Web of Augmented Things
Angelo Croatti , Alessandro Ricci
{a.croatti | a.ricci}@unibo.it
IOT-ASAP Workshop @ ICSA 2017
April 4th – Gothenburg, Sweden
ALMA MATER STUDIORUM – UNIVERSITY OF BOLOGNA
COMPUTER SCIENCE AND ENGINEERING DEPARTMENT (DISI)

2. |
Outline
A. Croatti (Unibo) Towards The Web of Augmented Things 2
1) Merging the Physical and the Digital
 Computation as Augmentation
 Cyber-Physical Systems and Augmented Reality
2) Augmented Worlds Vision
 Spatial Coupling, Holograms, Physical Embedding, Human users Interaction
3) The Web of Augmented Things
 Augmented Thing Model
 Towards a first infrastructure
4) A (significant) case study
5) Future Directions

3. |
Merging the Physical and the Digital
A. Croatti (Unibo) Towards The Web of Augmented Things 3
DIGITAL
“MATTER”
PHYSICAL
MATTER

4. |
On the notion of Augmentation
A. Croatti (Unibo) Towards The Web of Augmented Things 4
Mobile Computing
HUMAN
AUGMENTATION
Wearable Technologies
Smart Glasses
Pervasive Computing
ENVIRONMENT
AUGMENTATION
Internet of Things
… …
COMPUTATION-AS-AUGMENTATION
… computer programs should be designed to be an extension of
the physical environment where people live and work!

5. |
 The aim is to transform
interactions for humans
with the physical world
Cyber-Physical Systems
A. Croatti (Unibo) Towards The Web of Augmented Things 5
Augmented Reality plays a fundamental role
Wearable Devices and Technologies
 Hands-free systems
“Computation into the environment”
 Pervasive/Ubiquitous computing (IoT)
 Context-Awareness
 A class of systems that deeply embed cyber/digital capabilities in the physical
world

6. |
Augmented Worlds
A. Croatti (Unibo) Towards The Web of Augmented Things 6
Agent
AE
- location
Physical
Object
coupled
with
AE
- location
Physical
Location
perceive/act
perceive/act
observe region
Agent
observe
Hologram
- geometry
represented by
perceive/act
AE
- location
represented by
User
Location
Object
Location
DIGITALWORLDPHYSICALWORLD
AGENTS LEVEL
ENVIRONMENT LEVEL
AE REGION
interact
with
 A software application that enriches the
functionalities of a particular physical
environment exploiting “augmented entities”
 full-fledged computational object
 located in the space
 perceivable by humans and software agents
 an interface with actions to act upon the entity
 i.e to change its internal state
 a set of observable properties, to perceive and
observe the current state of the entity
 enabling event-driven publish/subscribe form of interaction

7. |
 The location of an AE is not necessarily based on
some physical computational device placed in
that location
 An AE can be created/instantiated in any point of
a region covered by the AW
 with respect to the local reference system
 An AE could be useful to:
 express a kind of enriching for the physical world’s
objects and inhabitants
 propose and offer complex services and/or
functionalities
Augmented Worlds
Spatial Coupling
A. Croatti (Unibo) Towards The Web of Augmented Things 7
Agent
AE
- location
Physical
Object
coupled
with
AE
- location
Physical
Location
perceive/act
perceive/act
observe region
Agent
observe
Hologram
- geometry
represented by
perceive/act
AE
- location
represented by
User
Location
Object
Location
DIGITALWORLDPHYSICALWORLD
AGENTS LEVEL
ENVIRONMENT LEVEL
AE REGION
interact
with

8. |
Augmented Worlds
Holograms
A. Croatti (Unibo) Towards The Web of Augmented Things 8
Agent
AE
- location
Physical
Object
coupled
with
AE
- location
Physical
Location
perceive/act
perceive/act
observe region
Agent
observe
Hologram
- geometry
represented by
perceive/act
AE
- location
represented by
User
Location
Object
Location
DIGITALWORLDPHYSICALWORLD
AGENTS LEVEL
ENVIRONMENT LEVEL
AE REGION
interact
with
 An AE exist in an AW even if it is not perceivable
in terms of visual impact
 an AE can be just a computational object doing
something somewhere
 A geometry can be associated to an AE to
provide a representation for the real world
 i.e. to perceive the entity through a smart glass
 Holograms for Augmented Entities
 a set of 2D/3D shapes
 part of the internal (perceivable) state of the AE

9. |
 Agents living in an AW can exploit proper spatial
lookup functions to discover the ID of the AE
within a particular space region
 Agents can also continuously track/observe a
region to be notified when an AE enter in the
region or exit from the region
 Asynchronous Event-Oriented Style
Augmented Worlds
Discovery and Observability
A. Croatti (Unibo) Towards The Web of Augmented Things 9
Agent
AE
- location
Physical
Object
coupled
with
AE
- location
Physical
Location
perceive/act
perceive/act
observe region
Agent
observe
Hologram
- geometry
represented by
perceive/act
AE
- location
represented by
User
Location
Object
Location
DIGITALWORLDPHYSICALWORLD
AGENTS LEVEL
ENVIRONMENT LEVEL
AE REGION
interact
with

10. |
Augmented Worlds
Physical Embedding
A. Croatti (Unibo) Towards The Web of Augmented Things 10
Agent
AE
- location
Physical
Object
coupled
with
AE
- location
Physical
Location
perceive/act
perceive/act
observe region
Agent
observe
Hologram
- geometry
represented by
perceive/act
AE
- location
represented by
User
Location
Object
Location
DIGITALWORLDPHYSICALWORLD
AGENTS LEVEL
ENVIRONMENT LEVEL
AE REGION
interact
with
 An AE can extend a physical object for human
users interaction
 i.e. exploiting holograms as visual interfaces to
control/inspect object state
 An AE can offer the medium to dynamically
extend physical objects functionalities
 Is necessary a bidirectional coupling between
the object and the AE
 considering also a “degree of freshness” to infer the
current state of coupling

11. |
Augmented Worlds
Human Users Interaction
A. Croatti (Unibo) Towards The Web of Augmented Things 11
Agent
AE
- location
Physical
Object
coupled
with
AE
- location
Physical
Location
perceive/act
perceive/act
observe region
Agent
observe
Hologram
- geometry
represented by
perceive/act
AE
- location
represented by
User
Location
Object
Location
DIGITALWORLDPHYSICALWORLD
AGENTS LEVEL
ENVIRONMENT LEVEL
AE REGION
interact
with
 Is a main aspect of Augmented Worlds
 an AW is a multi-user application
 multiple users perceive, share and interact within
the same augmented entity
 An user can be associated to an AE
(“augmented body”) with specific features
 useful to controlling user I/O and to reifying the
dynamic information about the user

12. |
 Defining a common application level for
the IoT based on web and related
protocols and tools
 important advantages in terms of software
development and engineering of complex
systems
 allows to avoid problems due to
heterogeneity of IoT-related technologies
and communication protocols
 Devices and services are fully integrated
in the Web
 web APIs through RESTful architectures
The Web of Things (in a nutshell)
A. Croatti (Unibo) Towards The Web of Augmented Things 12

13. |
WoAT : the Web of Augmented Things
Applying the WoT to Augmented Worlds
A. Croatti (Unibo) Towards The Web of Augmented Things 13
WoT Architecture Stack
WoAT
Extension
Smart
Glasses
AR Marker
Vuforia
AR Toolkit
Unity 3D
Augmented Thing
Model
Space Model
Agents
Digital Shared
Space
Augmented
Mashups
Social Virtual
Environments
1) Exploit WoT principles and stack for
Augmented Worlds
2) Integrate Augmented Worlds concept
and AR into the WoT
The Core Idea
A Thing in the WoT perspective that
represents an Augmented Entity living in
an Augmented World
AT: Augmented Thing

14. |
Augmented Thing Model
Common elements
A. Croatti (Unibo) Towards The Web of Augmented Things 14
 Like Things in WoT, an AT:
 has a root URL (its network address)
 it features a RESTful web interface
 self-descriptive messages
 HTTP operations (GET, POST, PUT, DELETE, HEAD)
 event-oriented mechanisms (i.e. Web Socket)
 The AT model can be considered a specialization
of the WoT Resource/Data according to AW (and
AE) concepts:
 actions and observable properties of an AE can be
naturally mapped on WoT concepts
 GET can be used to retrieve observable properties
 POST can be used to send actions to an AT

15. |
Augmented Thing Model
Specific elements for Space Coupling
A. Croatti (Unibo) Towards The Web of Augmented Things 15
 Every AT has a location in the physical space
 /properties/location
 can include information about both the position and
the orientation of the entity
 Every AT has an extension in space (possibly a
single point)
 /properties/extension

16. |
Augmented Thing Model
Specific elements for Holograms and Physical Embedding
A. Croatti (Unibo) Towards The Web of Augmented Things 16
 An AT may have one or more holograms defining
how the entity is perceived by human users and
how they can interact with it
 /properties/holograms
 Information about the physical embedding can
be specified reusing the /thing resource of
the WoT model

17. |
 An extension of the
WoT architecture with:
 specific infrastructural
supports for
implementing WoAT
abstractions
 client-side libraries to
enable the AR-based
experience for human
users
 It is logically built upon
the WoT layer
An Infrastructure for WoAT
A first sketch
A. Croatti (Unibo) Towards The Web of Augmented Things 17
WoAT
AW Node
WoAT
WoAT WoAT
IoT
WoT
IoT
AW
Application
WoAT
Infrastructure
AR Library
AT
AT
AT
Physical Things Software Components Infrastructure Node Humans
perceive/act
preceive
region
perceive/act
Agent
Agent
Coupling/Sync
Toolkit
Hologram
interact
with
related to
Human
User
Wearable/Eyewear
Devices

18. |
An Infrastructure for WoAT
Details
A. Croatti (Unibo) Towards The Web of Augmented Things 18
 Hosts the AW and provides the System of
Reference
 coupled with the physical world
 Provides functionalities to lookup and
register ATs
The Infrastructure Node
 Proposes abstraction to guarantee to
agents capabilities to
 join the AW
 interact with ATs
Autonomous Software Components
 Provides capabilities to keep the coupling
with real things updated
 performing state synchronization
 Rests in turn on the IoT layer
 Adds the support of AR libraries to
manage the visualization and interaction
with holograms for humans
 exploiting proper devices
Holograms Interaction
Physical Things

19. |
 In the Healthcare 4.0 concept, an hospital can be conceived as
an Augmented World
 “Augmented Hospital”
 Currently, we are working with the purpose to build an
Augmented Emergency Room
 To support trauma team and trauma leader in managing emergencies
 TraumaTracker Case Study @ Bufalini Hospital in Cesena (Italy)
 Infrastructural smart support and augmentation
 beacon to detect rooms and facilities
 smart displays
 smart glasses and wearables
A (significant) case study
A. Croatti (Unibo) Towards The Web of Augmented Things 19
AUGMENTED
HOSPITAL
EMERGENCY
AND
TRAUMA
MANAGEMENT
HEALTHCARE
4.0
SMART
ASSISTANCE

20. |
 The idea of WoAT may bring important benefits to the engineering of Augmented Worlds
 more generally, of any systems integrating pervasive computing and AR technologies
 The availability of WoAT model (and supporting infrastructures) offers the possibility to
develop AWs featuring different levels of openness
 several computational parts developed using different languages
 hosted by heterogeneous runtime platform
 To validate the idea
 a wide range of explorations are necessary
 a first prototype of a platform/infrastructure for WoAT is mandatory
 first concrete applications could concern well-known existing AR-based case studies
Future Directions
A. Croatti (Unibo) Towards The Web of Augmented Things 20

21. Towards the Web of Augmented Things
Angelo Croatti , Alessandro Ricci
{a.croatti | a.ricci}@unibo.it
thanks for your attention!