Physical-Cyber-Social Computing involves the integration of observations from physical sensors, knowledge and experiences from cyber systems, and social interactions from people. This will allow machines to understand contexts, correlate multi-domain data, and provide personalized solutions by leveraging background knowledge spanning physical, cyber, and social domains. Semantic computing plays a key role in bridging differences between domains to derive insights. The vision is for systems that can proactively initiate information needs with minimal human involvement.

1. Physical-Cyber-Social
Computing
Presented at SW2022 @ ISWC2012
AmitSheth
LexisNexis Ohio Eminent Scholar,
Director, Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing
Wright State University, Dayton, OH, USA
Ack: PramodAnanthram, Cory Henson, and Kno.e.sisSemantic Sensor Web team
1

2. 1999 - 2002
2

3. Building knowledge base by entity extraction, use of this knowledge Semantics & Semantic Web
for search, annotation, and personalization. in 1999-2002
Semantic Web: Early Realizations to Wide
Semantic search and knowledge browsing
Adoption
Semantic categorization, association, and inference
Creating a “Web of related Information”
Semantic enrichment using domain knowledge
http://knoesis.wright.edu/library/download/HSK02-SEE.pdf
http://knoesis.wright.edu/library/download/SBA+02-IC.pdf
3
http://www.google.com/insidesearch/features/search/knowledge.html

4. Now…12 years later
4

5. Google’s knowledge graph allows exploration of interconnected things
5

6. Semantic Web: 10 Years from Now
6

7. Physical-Cyber-Social
Computing
Presented at SW2022 @ ISWC2012
AmitSheth
LexisNexis Ohio Eminent Scholar,
Director, Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing
Wright State University, Dayton, OH, USA
Ack: PramodAnanthram, Cory Henson, and Kno.e.sisSemantic Sensor Web team
7

8. Cyber-Physical Systems
The computational and communication components closely intera
with physical components. Enables better cyber-mediated observ
of and interaction with physical components.
CPS involves sensing, computing, and actuating components
8
http://www.nsf.gov/pubs/2013/nsf13502/nsf13502.htm

9. Physical-Cyber Systems
Physical Systems Cyber Systems
Health applications and tools that monitor
a person physically and connect them
to care providers (e.g. doctors)
Offers sensor solutions for monitoring cardiovascular ailments
Remote heath monitoring
Mobile cardiac outpatient telemetry and real time analytics Provides solutions to monitor cardio activity
Non-invasive wearable monitoring of vital signals
Applications integrating telemedicine service
Cardiovascular patient monitoring
Remote real-time patient monitoring of vitals
Continuous monitoring of physiological parameters
9
http://quantifiedself.com/2010/01/non-invasive-health-monitoring/

10. Cyber-Social Systems
Cyber Systems Social Systems
Observations spanning Cyber and Social
world – people share their activities,
knowledge, experiences, opinions, and
perceptions.
Sharing experiences and management of conditions and their treatments
Managing risks and making informed decisions based on gene sequencing
10

11. Physical-Cyber-Social Systems
Physical Systems Cyber Systems Social Systems
Social aspect of sharing and friendl
Involves interactions between all the three competition
components.
This data is stove piped due to fragmentation in sensor
data collection services.
Integration and interaction between physical, cyber, and
social components for computation is brittle.
Needs significant human involvement in interpretation of
Sensors collecting observations
physiological observations using their knowledge of the
form the physical world domain and social experiences.
Self knowledge through numbers
Data collected from physiological sensors
analyzed in the social context of “similar” people. QS conference where experiences of
analyzing and visualizing data from
physiological sensors are presented.
11
http://www.fitbit.com/product/features#social

12. What if?
Computations leverages observations form
sensors, knowledge and experiences from
people to understand, correlate, and personalize
solutions.
Physical- Social-Cyber
Cyber
Physical-Cyber-Social
12

13. Search vs. Solution
Conventional search returns a set of documents for serving
the information need expressed as a search query.
Answer engine like WolframAlpha provides answers for a
query.
Chances of finding heartburn
cases in a year along with their
age group
Chances of finding heartburn
cases based on ethnicity.
Analyzing this data with sensor observations collected for Mark, who is a white, 65 years of age,
190 pounds, 5’ 10’’ has relative low chances of having a heartburn Distribution of heartburn cases based
13
on age, weight, height and BMI

14. Physical-Cyber-Social Computing
Physical Systems Cyber Systems
Semantics play a crucial role in bridging the
semantic gap between different sensor types,
modalities, and observations to derive insights
leading to a holistic solution.
Social Systems
14

15. Physical-Cyber-Social Computing
Physical, Cyber We experience the world through
and Social perceptions and actions
Observes
Influences
Experiences
Determines
Influences Perceptual
Inference
Influences
Evolves
Experiences evolve our
background knowledge
Experience
Background Knowledge
(spanning Physical-Cyber-
Background knowledge + new 15
Social)
http://www.yourdictionary.com/brain observations will enhance our

16. Physical-Cyber-Social Computing
2
Diastolic blood pressure
(BP) between 86 and 90
Observes
Influences
Experiences
Determines 1
6
Experiences in managing blood Asian
pressure male
4
Shared physiological Corrective actions to be 5
Influences Perceptual How are my peers of the
observations from sensors 7 taken
Increase the use of herbs
Inference same socio-economic-
and spices instead of salt cultural background doing
Influences w.r.t. BP?
3
Asian male has lower thresholds Evolves
for hypertension
Experience
Background Knowledge
(spanning Physical-Cyber-
Social) 16
http://www.yourdictionary.com/brain

17. Cyber-Physical Social Systems
Landscape
Cyber There are silos of knowledge on the cyber
Rich world which are under utilized.
Cyber world connects
the physical world EMR and knowledge of
which consists of PHR the medical
other machines or
domain Physiological
humans sensor data from
human population
Current CPS is focused on , Sensing, processing, Social
and actuating components along with Health related
communication, power consumption, security. Humans take experiences
decisions based shared by
on insights
provided by other
humans
humans and
machines in the
physical world
Physical Sensors around, on, and in humans will bridge the physical
and cyber world.
Social networks bridge the social interactions
in the physical and cyber world.
Physical systems being
instrumented with
sensors, we have
deeper view of
physical world
17

18. Cyber Physical Systems: Now
Cyber There are silos of knowledge on the cyber
Rich world which are under utilized.
EMR and knowledge of
PHR the medical
domain Physiological
sensor data from
human population
Current CPS is focused on , Sensing, processing, Social
and actuating components along with Health related
communication, power consumption, and security. experiences
shared by
humans
Sensing
Mark’s discomfort sensed by:
galvanic skin response, heart rate, fit bit, and Microsoft kinect
Computing
Physical His phone computes the possible cause being increased intake of fried food. Social networks bridge the social interactions
Sensors around, on, and in humans will bridge the physical in the physical and cyber world.
and cyber world.
Actuating
Physical Mark advised to go low on fried food.
Mark is experiencing heartburn.
Should Mark be convinced by this?
What if fatty food intake was just a coincidence with something serious?
18

19. Physical Cyber Social Computing: A Vision
Cyber There are silos of knowledge on the cyber
Rich world which are under utilized.
EMR and knowledge of
PHR the medical
domain Physiological
sensor data from
human population
We believe that current CPS should view the physical world Social
by incorporate solutions form (knowledge) cyber world Health related
with a lens of social context. experiences
shared by
humans
Sensing
Mark’s discomfort sensed by:
galvanic skin response, heart rate, fit bit, and Microsoft kinect
Computing
Physical Physical Cyber Social Computing involves: (1) Comparing physiological observationsSocial networks bridgeto him (age, weight, lifestyle,
from people similar the social interactions
Sensors around, on, and (2)humans will health the physical of similar people reporting heartburn in the physical and history of ailments of Mark
ethnicity, etc.) in Analyzing bridge experiences (3) Incorporating cyber world.
and cyber world.
(4) Leveraging medical domain knowledge of diseases and symptoms.
•He is advised to visit a doctor since he had a heart condition (from EMR) in the past and heartburns in similar people (social) was a
symptom of arterial blockage
Physical
Mark is experiencing heartburn.
Actuating
Alert to contact his doctor.
19

20. Toward the Vision:
Semantic Perception in Khealth
A Khealth application:
leveraging low cost
sensors toward
reducing hospital
readmissions for
ADHF patients
Khealth will continue toward
the vision of Physical-Cyber-
Social Computing to
understand, correlate, and
personalize solutions
Paper at ISWC: C. Henson, K. Thirunarayan, A. Sheth, An Efficient Bit Vector Approach to
20
Semantics-based Machine Perception in Resource-Constrained Devices
http://knoesis.org/healthApp/

21. Semantic Perception as an example
component of PCS Computing
Convert large number of observations to semantic
abstractions that provide insights and translate
into decisions
1
Translating low-level signals Explanation
into high-level knowledge
Observe Perceive
Property Feature
Prior Knowledge 2
Focusing attention on those
aspects of the environment that
Discrimination provide useful information

22. CPS Current State of Art: Limitations
• CPS are stovepipe systems with narrow set of
observations of the real world.
• Current CPS do not possess the knowledge
support for decision making with Mark’s case.
• Social aspects plays an important role in
decision making of CPS.
• The vision of Cyber Physical Social Computing
is to provide solutions for these limitations.
22

23. Conclusions
• Transition form search to solution engines for actionable
information.
• Seamless integration of technology involving minimal human
involvement.
• Transition form reactive systems (humans initiating
information need) to proactive systems (machines initiating
information need).
• Sharing of knowledge, experiences, and observations across
physical-cyber-social worlds lead to informed decision
making.
• Physical Cyber Social Computing plays an important role in
this vision. Semantic computing will provide integration and
reasoning capabilities needed for PCS computing.
26

24. A bit more on this topic
Influential visions by Bush, Licklider, Eaglebert, and Weiser.
Amit Sheth,”Computing for Human Experience: Semantics-Empowered Sensors,
Services, and Social Computing on the Ubiquitous Web," IEEE Internet Computing, vol.
14, no. 1, pp. 88-91, Jan.-Feb. 2010, doi:10.1109/MIC.2010.4
A. Sheth, Semantics empowered Cyber-Physical-Social Systems, Semantic Web in
2012 workshop at ISWC 2102.
Cory Henson, AmitSheth, KrishnaprasadThirunarayan, 'Semantic Perception:
Converting Sensory Observations to Abstractions,' IEEE Internet Computing, vol. 16,
no. 2, pp. 26-34, Mar./Apr. 2012, doi:10.1109/MIC.2012.20
Cory Henson, KrishnaprasadThirunarayan, AmitSheth. An Ontological Approach to
Focusing Attention and Enhancing Machine Perception on the Web. Applied Ontology,
vol. 6(4), pp.345-376, 2011.
Cory Henson, KrishnaprasadThirunarayan, and AmitSheth, 'An Efficient Bit Vector
Approach to Semantics-based Machine Perception in Resource-Constrained Devices,'
In: Proceedings of 11th International Semantic Web Conference (ISWC 2012), Boston,
Massachusetts, USA, November 11-25, 2012.
27