Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Amit Sheth
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing:
Wright State University, Dayton, Ohio
Sem...
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7433358
Credit: Looi Consulting (http://www.looiconsulting.com/home/enterprise-big-data/)
● In 2008, data generated > storage avai...
How would an enterprise get actionable information?
http://www.slideshare.net/NamrataChatterjee/nokias-supply-chain-manage...
http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/
The Patient of the Future
MIT Technology R...
First used in 2004 redefined 2013: http://wiki.knoesis.org/index.php/Smart_Data.
Smart Data
● The astounding bandwidth of your senses is 11
million bits of information every second.
● In conscious activities like r...
● How can we take inspiration from human brain and derive an intelligent processing of big data
○ for service enterprise n...
http://www.livescience.com/1863-theory-intelligence-works.html, http://www.wired.com/2015/10/scientists-can-now-predict-in...
Brain
Neuroscience
Cognitive
Science
Linguistics
Smart Data and
Intelligent Computing
● Semantic
Computing
● Cognitive
Com...
http://www.research.ibm.com/cognitive-computing/brainpower/, https://newtonsapplevce.wordpress.com/2016/01/06/neuromorphic...
A Second Approach to Intelligence
Prior Medical Knowledge
D1
D3
D1
Medical History and
Past observations
S1
S2
S3
Sn
..
..
D1
D2
D3
S1
S2
S3
..
..
..
..
..
...
ACTIONS
situation awareness useful
for decision making
ABSTRACTIONS
make sense to humans
KNOWLEDGE
for interpretation of o...
1Marcus, Philip, Kevin R. Murphy, Abid Rahman, and Christopher D. O’Brien. "Intrapatient symptom variability in
adults and...
How do we solve problems with real-world complexity, gather vast amounts of data,
diverse knowledge, and come up with inte...
Semantics, perception, and cognition interact seamlessly.
● Semantic Computing can deal with big data challenges.
● Cognit...
http://www.mezzacotta.net/garfield/?comic=1470, https://in.pinterest.com/pin/222435669066482336/
● Semantics is the “meani...
Semantics attaches meaning to observation by providing a definition within a system context
or the knowledge that people p...
Population Level
Personal
Wheeze – Yes
Do you have tightness of chest? –Yes
ObservationsPhysical-Cyber-Social System Healt...
Cognition is the process by which an autonomous system perceives its
environment, learns from experiences, anticipates the...
DARPA launched a Cognitive Computing project in 2002. Cognitive Computing was defined as
the ability to “reason, use repre...
http://www.rogerschank.com/fraudulent-claims-made-by-IBM-about-Watson-and-AI
Need to be Cautious with Claims (Roger Schank)
“where we see the world we do not decide to see it” Daniel Kahneman
Perception is an active cyclical process of exploratio...
http://www.ted.com/talks/blaise_aguera_y_arcas_how_computers_are_learning_to_be_creative#t-345661
Image of a bird and a co...
Perceptual computing is the ability for a computer to recognize what is going on around it.
Computer can perceive the envi...
http://www.jfsowa.com/pubs/cogcycle.pdf, http://www.jfsowa.com/talks/interop.pdf
Peirce’s Cycle of Pragmatism
John Sowa’s ...
Perceptual Computing supports the ability to:
● Ask contextually relevant and personalized questions.
● Complement Semanti...
Hyperthyroidism
Elevated
Blood
Pressure
Systolic blood pressure of 150 mmHg
“15
0”
... ...
Levels of Abstraction
Making Sense of Sensor Data With
Henson, et al. An Ontological Approach to Focusing Attention and Enhancing Machine Percep...
1
2
Perception Cycle
Prior Knowledge on the Web
Explanation is the act of choosing the objects or events that best account for a set of
observations; often referred to as...
Explanatory Feature: a feature that explains the set of observed properties
elevated blood pressure Hypertension
Hyperthyr...
Discriminating Property: is neither expected nor not-applicable
clammy skin
Hypertension
Hyperthyroidism
Pulmonary EdemaPa...
Orders of magnitude resource savings for generating and storing relevant abstractions vs. raw observations.
Relevant Abstr...
Asthma
Providing actionable information in a timely manner is
crucial to avoid information overload or fatigue
Sleep data Communi...
Personal level
Signals
Public level
Signals
Population level
Signals
Domain
Knowledge
http://www.tuberktoraks.org/managete...
Sensordrone
(Carbon monoxide,
temperature, humidity)
Sensor Platforms
Android Device
(w/ kHealth
App)
Total cost: ~ $550
A...
For collecting observations from both machine sensors
and from patients in the form of a questionnaire.
kHealth Kit: Andro...
kHealth Kit: Android Application
Personal level
Signals
Public level
Signals
Population level
Signals
Domain
Knowledge
Risk Model
Events from
social stream...
Risk assessment
model
Semantic
Perception
Personal level
Signals
Public level
Signals
Domain
Knowledge
Population level
Si...
Risk assessment
model
Semantic
Perception
Personal level
Signals
Public level
Signals
Domain
Knowledge
Population level
Si...
Sensordrone – for monitoring
environmental air quality
Wheezometer – for monitoring
wheezing sounds
Can I reduce my asthma...
Domain Knowledge
ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-a...
Did you cough more
than 20 times today?
- Yes
Semantic Sensor
Network Ontology
Did you cough more than 20
times today?
- Y...
Another Example
with a Focus on a Richer Model
We are still working on the simpler representations of the real world!
http://artint.info/html/ArtInt_8.html, http://en.wi...
We need computational paradigms to tap into the rich pulse of the human
populace, and utilize diverse continuous stream of...
By 2001 over 285 million Indians lived in cities, more than in all North American
cities combined (Office of the Registrar...
• What time to start?
• What route to take?
• What is the reason for traffic?
– Wait for some time or re-route?
Questions ...
Complementary Data Sources
Image credit: http://traffic.511.org/index
Multiple Events
Varying influence
Interact with Each Other
Challenge: Non-linea...
7 × 24
LDS(1,1), LDS(1,2) ,…., LDS(1,24)
LDS(7,1), LDS(7,2) ,…., LDS(7,24)
.
.
.
di
hj
Mon.
Tue.
Wed.
Thu.
Fri.
Sat.
Sun.
...
Compute Log Likelihood for
each hour of observed data
(di,hj) LDS(hj,di)
7 × 24
Lik(1,1), Lik(1,2) ,…., Lik(1,24)
Lik(7,1)...
Hourly Traffic Dynamics Over a Day
Public Safety
Urban Planning
Gov. & Agency
Admin.
Energy & water
Environmental
TransportationSocial Programs
Healthcare
Ed...
Pramod Anantharam, Payam Barnaghi, Krishnaprasad Thirunarayan, and Amit Sheth. 2015. Extracting City Traffic Events from S...
Complementary Events
Traffic Incident; road-construction
Textual Events from Tweets vs. 511.org: Complementary
Corroborative Events
Fog visibility-air-quality; fog
Textual Events from Tweets vs. 511.org: Corroborative
Timeliness
Concert Concert
Textual Events from Tweets vs. 511.org: Timeliness
Image Credit: http://traffic.511.org/index
slow-moving-traffic
Domain knowledge in the
form of traffic vocabulary
Domain k...
This example demonstrates use of:
– Multimodal data streams (types of events from text - signature from sensor data).
– Mu...
Domain Knowledge
Historical Data
Annotation
511.org
SC
CC
Slow traffic due
to football match.
Slow traffic due
to accident...
Thank you, and please visit us at http://knoesis.org
For more information on kHealth, please visit us at
http://knoesis.or...
Part of my group @ Kno.e.sis
Backup/discussion
slides
73
Perception can be split into two processes:
1.Processing sensory input, which transforms these low-level information to
hi...
Scott Kelso uses “Circular Causality” to describe the situation in dynamic systems where
the cooperation of the individual...
Perception is an active, cyclical
process of exploration and
interpretation.
“Perception and action, if these
unifying mod...
http://www.simplypsychology.org/perception-theories.html, http://www.youramazingbrain.org/supersenses/necker.htm
Psycholog...
http://www.people-clipart.com/people_clipart_images/clip_art_illustration_of_a_young_woman_with_her_hand_on_her_hip_wearin...
1. Cognition is a general term for all forms of knowing (e.g. attending,
remembering, reasoning and understanding concepts...
Cognition enables individuals to understand their environment and paves
the way for the application of perception to explo...
The system’s understanding of it’s worlds is inherently specified to the
form of the systems’ embodiment and is dependent ...
From http://philosophyofbrains.com/2015/12/14/surfing-uncertainty-prediction-action-and-the-embodied-mind.aspx
Top-Down an...
http://www.fil.ion.ucl.ac.uk/~karl/Whatever%20next.pdf, http://libertymotive.com/truth/
Top-Down Approach model emphasizes...
Attempts to build an artificial cognitive system that can be positioned in a two-dimensional space, with one axis
defining...
Understanding traffic flow variations
Histogram of speed values
collected from June 1st 12:00 AM to June 2nd 12:00 AM
Histogram of travel time values
collected ...
This distribution resembles a Gaussian Mixture
Model (GMM)
Multiple Gaussian Distributions: A Better Fit for Speed Observa...
Assume Normalcy to be uninterrupted traffic flow
July 2014 has no events so, we
hypothesize higher log-likelihood
score
Ju...
Root Cause Analysis Action Recommendation
Find Triggers of Asthma
Derive the cause of asthma attacks
for a given patient u...
Sensordrone
• Precision gas sensor
• Reducing gas sensor
• Oxidizing gas sensor
• Non-contact
thermometer
• Humidity senso...
• If an anomaly is detected on a link L and during time period [tst, tet], then the
anomaly is explained by an event if th...
Brain to Computing
Second Approach to Intelligence
How can we get inspiration
from brain to computing?
One approach, recreate the brain in
silicon. A second approach, different
disciplines learn about the human brain,
get ins...
"The domain of cognitive science occupies the intersection of philosophy, neuroscience,
linguistics, cognitive psychology,...
https://mitpress.mit.edu/books/artificial-cognitive-systems
Two hallmarks of cognition:
● Prediction out of the past inter...
Contextual
https://en.wikipedia.org/wiki/Cognitive_computing http://edtechreview.in/dictionary/174-what-is-adaptive-learni...
Reduced
CO level =>
better asthma
control
High CO influences
Wheezing Level (Low/High)
High CO High Luminosity
High Wheeze...
http://phys.org/news/2014-10-nobel-importance.html https://student.societyforscience.org/article/nobel-goes-finding-brain%...
https://mitpress.mit.edu/books/soar-cognitive-architecture, http://www.hindawi.com/journals/mpe/2012/530561/fig1/
Soar Cog...
http://my.ilstu.edu/~jrbaldw/370/Meaning.htm
Semiotics
Brain GPS - Grid Cell/ Place Cells
● in 1971 John O’Keefe of University College London discovered “place cells” found in
H...
http://www.scientificamerican.com/article/10-big-ideas-in-10-years-of-brain-science/
In the future optogenetics will allow...
http://www.scientificamerican.com/sciam/assets/media/multimedia/110514-MindListicle/1.jpg http://www.scientificamerican.co...
Philanthropist Paul Allen, co-founder of Microsoft
● In 2003 , Allen Institute of Brain Science began mapping gene activit...
Neurocognitive functions are cognitive functions linked with the functions of
particular areas, neural pathways, or cortic...
The term 'cognitive neuroscience' was coined by George Miller and Michael Gazzaniga toward the end of the 1970s.
Cognitive...
http://www.scientificamerican.com/article/10-big-ideas-in-10-years-of-brain-science/
http://blogs.scientificamerican.com/t...
Semantic, Cognitive and Perceptual Computing – three intertwined strands of a golden braid of intelligent computing
Semantic, Cognitive and Perceptual Computing – three intertwined strands of a golden braid of intelligent computing
Semantic, Cognitive and Perceptual Computing – three intertwined strands of a golden braid of intelligent computing
Semantic, Cognitive and Perceptual Computing – three intertwined strands of a golden braid of intelligent computing
Upcoming SlideShare
Loading in …5
×

Semantic, Cognitive and Perceptual Computing – three intertwined strands of a golden braid of intelligent computing

475 views

Published on

See http://grammars.grlmc.com/WebST2016/courseDescription.php#Amit_keynote

While Bill Gates, Stephen Hawking, Elon Musk, Peter Thiel and others engaged in OpenAI discuss whether or not AI, robots, and machines will replace humans, proponents of human-centric computing continue to extend work in which humans and machine partner in contextualized and personalized processing of multimodal data to derive actionable information. In this talk, we discuss how maturing paradigms such as semantic computing (SC), cognitive computing (CC), complemented by the emerging perceptual computing (PC) paradigm provide a continuum through which to exploit the ever-increasing and growing diversity of data that could enhance people’s daily lives. SC and CC sift through raw data to personalize it according to context and individual user, creating abstractions that move the data closer to what humans can readily understand and apply in decision-making. PC, which interacts with the surrounding environment to collect data that is relevant and useful in understanding the outside world, is characterized by interpretative and exploratory activities, that is supported by use of prior/background knowledge. Using the examples of personalized digital health and smart city, we will demonstrate how SC, CC and PC form complementary capabilities that will enable development of next generation of intelligent systems.

References:
Amit Sheth, "Computing for Human Experience: Semantics-Empowered Sensors, Services, Social Computing on the Ubiquitous Web," IEEE Internet Computing, 14 (1), January/February 2010.

Amit Sheth, Pramod Anantharam, Cory Henson, Semantic, Cognitive, and Perceptual Computing: Advances toward Computing for Human Experience,IEEE Computer, March 2016. http://online.qmags.com/CMG0316/default.aspx?pg=67&mode=2#pg67&mode2

Amit Sheth, Internet of Things to Smart IoT Through Semantic, Cognitive, and Perceptual Computing, IEEE Intelligent Systems, March/April 2016.

Published in: Data & Analytics

Semantic, Cognitive and Perceptual Computing – three intertwined strands of a golden braid of intelligent computing

  1. 1. Amit Sheth Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing: Wright State University, Dayton, Ohio Semantic, Cognitive, and Perceptual Computing: three intertwined strands of a golden braid of intelligent computing Keynote @ International Summer School on Web Science & Technology (WebST), Bilbao, Spain, 19 May 2016.
  2. 2. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7433358
  3. 3. Credit: Looi Consulting (http://www.looiconsulting.com/home/enterprise-big-data/) ● In 2008, data generated > storage available. Less than 0.5% of data get analyzed. ● Vast variety of data: text > images > A/V > genome sequencing > IoT ● Of all the data generated, which data is relevant, and why? Which data to analyze? Which data can offer insight? Who cares for what data? How to get attention to a human decision maker? What we need is intelligent processing to get actionable, smart data. A Big Challenge and Opportunity in Recent Times
  4. 4. How would an enterprise get actionable information? http://www.slideshare.net/NamrataChatterjee/nokias-supply-chain-management-case-study, http://www.economist.com/node/7032258 ● Weak crisis judgement. ● Failure to take prompt action. ● Single supplier reliability. Fire at Royal Philips electronic semiconductor plant, New Mexico in March 2000. 8 trays of wafers containing the miniature circuitry to make several thousand chips for mobile phones was destroyed. The expected time to recover was estimated to be a week. ● Fire breakout in clean room. ● Inability to determine the exact damage to the clean room. ● Lack of emergency preparation. ● Early speculation of possible crisis. ● Preparedness against supply crisis. ● Finding alternative source of chip supply.
  5. 5. http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/ The Patient of the Future MIT Technology Review, 2012 How would an individual get actionable information?
  6. 6. First used in 2004 redefined 2013: http://wiki.knoesis.org/index.php/Smart_Data. Smart Data
  7. 7. ● The astounding bandwidth of your senses is 11 million bits of information every second. ● In conscious activities like reading, the human brain distills approximately 40 bits of information per second. http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/mobile-white-paper-c11-520862.html The Brain: Inspiration for Intelligent Processing: What if we could automate such interpretation of data?
  8. 8. ● How can we take inspiration from human brain and derive an intelligent processing of big data ○ for service enterprise need to serving individual needs ● In particular, take inspiration from cognition and perception, and for pedagogy, outline three computing paradigms for building future intelligent systems focused on converting data into abstractions that humans act upon for making decision and taking action ○ not all aspects of capturing human intelligence are addressed ○ not focused on making machines that replace humans - instead focus is on CHE ● CHE: Humans and machines partner to enhance human experience and better informed decision making with humans in the Loop. * 2010 article: http://wiki.knoesis.org/index.php/Computing_For_Human_Experience; distinct from “Human Experience of Computing by Booch: Dec. 9, 2011 at http://computingthehumanexperience.com/category/computing/. Sheth - 2008+ * What is this talk about?
  9. 9. http://www.livescience.com/1863-theory-intelligence-works.html, http://www.wired.com/2015/10/scientists-can-now-predict-intelligence-brain-activity/ Human Intelligence is not confined to a single area in the brain, but is the result of multiple brain areas working in coordination. Human Intelligence is related to how information travels through the brain. According to a review of 37 imaging studies in Journal Behavioral and Brain Science, intelligence is related not so much to brain size or brain structure, but to how efficiently information travels through the brain. Some brains are better than others at certain things because of the way they are wired. “The more certain regions are talking to one another, the better you are able to process information quickly and make inference.” Intelligence Research
  10. 10. Brain Neuroscience Cognitive Science Linguistics Smart Data and Intelligent Computing ● Semantic Computing ● Cognitive Computing ● Perceptual Computing Cognitive Psychology …. …. Brain Inspiration
  11. 11. http://www.research.ibm.com/cognitive-computing/brainpower/, https://newtonsapplevce.wordpress.com/2016/01/06/neuromorphic-technology/ Recreating the brain in silicon One Approach to Intelligence
  12. 12. A Second Approach to Intelligence
  13. 13. Prior Medical Knowledge D1 D3 D1 Medical History and Past observations S1 S2 S3 Sn .. .. D1 D2 D3 S1 S2 S3 .. .. .. .. .. .. .. .. D1 D1 Doctor Patient Q1 Q2 Qn A1 A2 An Blood Pressure Heart Rate Breathing Rate Body Temperature Multi-model Observations Current: Observing a Snapshot of the Patient
  14. 14. ACTIONS situation awareness useful for decision making ABSTRACTIONS make sense to humans KNOWLEDGE for interpretation of observations Contextualization Personalization DATA Observations from machine and social sensors Converting Data to Actions
  15. 15. 1Marcus, Philip, Kevin R. Murphy, Abid Rahman, and Christopher D. O’Brien. "Intrapatient symptom variability in adults and children with asthma: Results of a survey." Advances in therapy 22, no. 5 (2005): 488-497. “ … survey indicates that adult patients and caregivers of pediatric patients report variability in asthma symptoms over time, even when asthma medications are taken.”1 Personal level Signals Public level Signals Population level Signals Future: Analyzing a Multifaceted Continuous Stream of Diverse Data
  16. 16. How do we solve problems with real-world complexity, gather vast amounts of data, diverse knowledge, and come up with intelligent decisions and timely actions? Next, a pedagogical approach.
  17. 17. Semantics, perception, and cognition interact seamlessly. ● Semantic Computing can deal with big data challenges. ● Cognitive Computing can use relevant knowledge to improve data understanding for decision-making. ● Perceptual Computing can provide personalized and contextual abstractions over massive amounts of multimodal data from the physical, cyber, and social realms. https://www.linkedin.com/pulse/perceptual-computing-third-strand-golden-braid-amit-sheth Semantic Computing, Perceptual Computing, Cognitive Computing
  18. 18. http://www.mezzacotta.net/garfield/?comic=1470, https://in.pinterest.com/pin/222435669066482336/ ● Semantics is the “meaning or relationship of meanings, or relating to meaning ” (Webster). ● Meaning and use of data (Information System). ● It is concerned with the relationship between the linguistic symbols and their meaning or real-world objects. ● Semantics important role is to make data meaningful to people. Semantics
  19. 19. Semantics attaches meaning to observation by providing a definition within a system context or the knowledge that people possess [Sheth 2016]. Semantic Computing encompasses the technology required to represent concepts and their relationships in an integrated semantic network that loosely mimics the brain’s conceptual interrelationships. Web of data Semantics attached to objects in the world Semantic Computing
  20. 20. Population Level Personal Wheeze – Yes Do you have tightness of chest? –Yes ObservationsPhysical-Cyber-Social System Health Signal Extraction Health Signal Understanding <Wheezing=Yes, time, location> <ChectTightness=Yes, time, location> <PollenLevel=Medium, time, location> <Pollution=Yes, time, location> <Activity=High, time, location> Wheezing ChectTightness PollenLevel Pollution Activity Wheezing ChectTightness PollenLevel Pollution Activity RiskCategory <PollenLevel, ChectTightness, Pollution, Activity, Wheezing, RiskCategory> <2, 1, 1,3, 1, RiskCategory> <2, 1, 1,3, 1, RiskCategory> <2, 1, 1,3, 1, RiskCategory> <2, 1, 1,3, 1, RiskCategory> . . . Expert Knowledge Background Knowledge tweet reporting pollution level and asthma attacks Acceleration readings from on-phone sensors Sensor and personal observations Signals from personal, personal spaces, and community spaces Risk Category assigned by doctors Qualify Quantify Enrich Outdoor pollen and pollution Public Health Well Controlled - continue Not Well Controlled – contact nurse Poor Controlled – contact doctor SSN Semantic Computing for Our Personalized Digital Health (Asthma) Application
  21. 21. Cognition is the process by which an autonomous system perceives its environment, learns from experiences, anticipates the outcome of the events, acts to pursue goals and adapt to the changing environment. https://mitpress.mit.edu/books/artificial-cognitive-systems PerceptionAction Anticipate Adapt Assimilate Cognition as a cycle of anticipation, assimilation, adaptation: embedded in, contributing to, and benefitting from a continuous process of action and perception. What is Cognition?
  22. 22. DARPA launched a Cognitive Computing project in 2002. Cognitive Computing was defined as the ability to “reason, use represented knowledge, learn from experience, accumulate knowledge, explain itself, accept direction, be aware of its own behavior and capabilities, [and] respond in a robust manner to surprises.” IBM describes the components used to develop, and behaviors resulting from, “systems that learn at scale, reason with purpose and interact with humans naturally.” According to them, while sharing many attributes with the field of artificial intelligence, it differentiates itself via the complex interplay of disparate components, each of which comprise their own individual mature disciplines [1]. Our Take Cognitive computing interprets annotated observations obtained from Semantic computing , or raw observations from diverse sources and presents actionable information to humans. Cognitive computing systems learns from their experiences and improve when performing repeated tasks. [1] https://en.wikipedia.org/wiki/Cognitive_computing Cognitive Computing
  23. 23. http://www.rogerschank.com/fraudulent-claims-made-by-IBM-about-Watson-and-AI Need to be Cautious with Claims (Roger Schank)
  24. 24. “where we see the world we do not decide to see it” Daniel Kahneman Perception is an active cyclical process of exploration and interpretation. Perception enables individual to focus on most promising course of action by incorporating background knowledge that provided a comprehensive contextual understanding Perception
  25. 25. http://www.ted.com/talks/blaise_aguera_y_arcas_how_computers_are_learning_to_be_creative#t-345661 Image of a bird and a computer says it’s a bird. Perception: Google
  26. 26. Perceptual computing is the ability for a computer to recognize what is going on around it. Computer can perceive the environment and the users in that environment. The computer determines what needs a user might have and react to those needs without giving or receiving any additional information. Perception: Intel
  27. 27. http://www.jfsowa.com/pubs/cogcycle.pdf, http://www.jfsowa.com/talks/interop.pdf Peirce’s Cycle of Pragmatism John Sowa’s Perception Cycle
  28. 28. Perceptual Computing supports the ability to: ● Ask contextually relevant and personalized questions. ● Complement Semantic Computing and Cognitive Computing by providing the machinery to ask the next question or derive a hypothesis. It uses machine perception together with available background knowledge to explore and interpret observations. Perceptual Computing system enables the personalization of information provided by the cognitive computing system. For example, it minimizes the uncertainty by providing a personalized and contextualized understanding of a patient’s environment and symptomatic variations. Perceptual Computing: Our View
  29. 29. Hyperthyroidism Elevated Blood Pressure Systolic blood pressure of 150 mmHg “15 0” ... ... Levels of Abstraction
  30. 30. Making Sense of Sensor Data With Henson, et al. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web, Applied Ontology, 2011. Our First Approach to Implement Perceptual Computing
  31. 31. 1 2 Perception Cycle
  32. 32. Prior Knowledge on the Web
  33. 33. Explanation is the act of choosing the objects or events that best account for a set of observations; often referred to as hypothesis building Discrimination is the act of finding those properties that, if observed, would help distinguish between multiple explanatory features 1 2 Explanation and Discrimination
  34. 34. Explanatory Feature: a feature that explains the set of observed properties elevated blood pressure Hypertension Hyperthyroidism Pulmonary Edema Clammy Skin Palpitations Explanation
  35. 35. Discriminating Property: is neither expected nor not-applicable clammy skin Hypertension Hyperthyroidism Pulmonary EdemaPalpitations elevated blood pressure Discrimination
  36. 36. Orders of magnitude resource savings for generating and storing relevant abstractions vs. raw observations. Relevant Abstractions Raw Observations Resource Savings of Abstracting Sensor Data
  37. 37. Asthma
  38. 38. Providing actionable information in a timely manner is crucial to avoid information overload or fatigue Sleep data Community dataPersonal Schedule Activity data Personal health records Data Overload for Patients/Health Aficionados
  39. 39. Personal level Signals Public level Signals Population level Signals Domain Knowledge http://www.tuberktoraks.org/managete/fu_folder/2011-03/html/2011-3-291-311.html Contextual ActionablePersonalized OR How is my Asthma control? Should I take additional medication today? How can I reduce my asthma attacks at home? Asthma: Challenges in Heterogeneity, Variability, and Personalization
  40. 40. Sensordrone (Carbon monoxide, temperature, humidity) Sensor Platforms Android Device (w/ kHealth App) Total cost: ~ $550 Along with sensor platforms in the kit, the application uses a variety of population level signals from the web: Pollen level Air Quality Temperature & Humidity Node Sensor (exhaled Nitric Oxide) Fitbit ChargeHR (Activity, sleep quality) kHealth Kit for the Application for Asthma Management
  41. 41. For collecting observations from both machine sensors and from patients in the form of a questionnaire. kHealth Kit: Android Application
  42. 42. kHealth Kit: Android Application
  43. 43. Personal level Signals Public level Signals Population level Signals Domain Knowledge Risk Model Events from social streams Take medication before going to work. Avoid going out in the evening due to high pollen levels. Contact a doctor. Analysis Personalized Actionable Information Data Acquisition & aggregation kHealth: Health Signal Processing Architecture
  44. 44. Risk assessment model Semantic Perception Personal level Signals Public level Signals Domain Knowledge Population level Signals GREEN – Well controlled YELLOW – Not well controlled Red – Poor controlled How controlled is my asthma? Patient Health Score (Diagnostic)
  45. 45. Risk assessment model Semantic Perception Personal level Signals Public level Signals Domain Knowledge Population level Signals Patient health Score How vulnerable* is my control level today? *considering changing environmental conditions and current control level Patient Health Score (Prognostic)
  46. 46. Sensordrone – for monitoring environmental air quality Wheezometer – for monitoring wheezing sounds Can I reduce my asthma attacks at night? What are the triggers? What is the wheezing level? What is the propensity toward asthma? What is the exposure level over a day? Commute to work Luminosity CO Level CO in gush during the daytime. Actionable Information Personal level Signals Public level Signals Population level Signals What is the air quality indoors? Close the window at home during day to avoid CO2 inflow, to avoid asthma attacks at night Decision Support for Doctors and Patients: A Scenario
  47. 47. Domain Knowledge ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-agonist; *consider referral to specialist Asthma Control and Actionable Information Asthma Domain Knowledge
  48. 48. Did you cough more than 20 times today? - Yes Semantic Sensor Network Ontology Did you cough more than 20 times today? - Yes Did you cough more than 20 times today? - Yes Did you cough more than 20 times today? - Yes Did you cough more than 20 times today? - Yes Did you cough more than 20 times today? - No Raw Data Historical Data from a Person 20 times cough is http://knoesis.org/asthma/high-coughing 672 steps is http://knoesis.org/asthma/low-activity 1 h 17 mins REM Sleep is http://knoesis.org/asthma/disturbed- sleep Annotated Data 672 is http://knoesis.org/asthma#steps 1 h 17 mins is http://knoesis.org/asthma#REM_Sleep 20 is http://knoesis.org/asthma#Cough_Incident Personalizati on Well Controlled Very Poorly Controlled Not Well Controlled Contextualization Abstractions (Actionable Information) Knowledge Base and Unstructured Data The symptoms (high-coughing, low-activity, disturbed- sleep) are interpreted with respect to a person with severity level “Mild Asthma” Interpretation Exploration Understanding CC PC SC Example from kHealth Project
  49. 49. Another Example with a Focus on a Richer Model
  50. 50. We are still working on the simpler representations of the real world! http://artint.info/html/ArtInt_8.html, http://en.wikipedia.org/wiki/Traffic_congestion Solve Represent Interpret Real world Simplified representation Compute What did not change in data processing for quite some time?
  51. 51. We need computational paradigms to tap into the rich pulse of the human populace, and utilize diverse continuous stream of data Represent, capture, and compute with richer and fine-grained representations of real-world problems Solve Represent Interpret Real world Richer representation Compute + Richer representation of traffic observations. Effective solutions People interpreting a real-world event. What should change?
  52. 52. By 2001 over 285 million Indians lived in cities, more than in all North American cities combined (Office of the Registrar General of India 2001) 1. 1 The Crisis of Public Transport in India 2 IBM Smarter Traffic Modes of Transportation in Indian Cities Texas Transportation Institute (TTI) Congestion report in U.S. Severity of the Traffic Problem
  53. 53. • What time to start? • What route to take? • What is the reason for traffic? – Wait for some time or re-route? Questions Asked Daily
  54. 54. Complementary Data Sources
  55. 55. Image credit: http://traffic.511.org/index Multiple Events Varying influence Interact with Each Other Challenge: Non-linearity in Traffic Dynamics
  56. 56. 7 × 24 LDS(1,1), LDS(1,2) ,…., LDS(1,24) LDS(7,1), LDS(7,2) ,…., LDS(7,24) . . . di hj Mon. Tue. Wed. Thu. Fri. Sat. Sun. Mon. Tue. Wed. Thu. Fri. Sat. Sun. Speed/travel-time time series data from a link. Time series data for each hour of day (1-24) for each day of week (Monday – Sunday). Mean time series computed for each day of week and hour of day along with the medoid. 168 LDS models for each link; Total models learned = 425,712 i.e., (2,534 links × 168 models per link). Step 1: Index data for each link for day of week and hour of day utilizing the traffic domain knowledge for piece-wise linear approximation Step 2: Find the “typical” dynamics by computing the mean and choosing the medoid for each hour of day and day of week Step 3: Learn LDS parameters for the medoid for each hour of day (24 hours) and each day of week (7 days) resulting in 24 × 7 = 168 models for each link Learning Context-specific LDS Models
  57. 57. Compute Log Likelihood for each hour of observed data (di,hj) LDS(hj,di) 7 × 24 Lik(1,1), Lik(1,2) ,…., Lik(1,24) Lik(7,1), Lik(7,2) ,…., Lik(7,24) . . . Train? Yes (Training phase) Tag Anomalous hours using the Log Likelihood Range No (di,hj) (min. likelihood) Anomalies L = Partition based on (di,hj) Speed and travel-time time Observations from a link Log likelihood min. and max. values obtained from five number summary Partition based on (di,hj) 7 × 24 LDS(1,1), LDS(1,2) ,…., LDS(1,24) LDS(7,1), LDS(7,2) ,…., LDS(7,24) . . . di hj (Input) (Output) Tagging Anomalies with LDS Models
  58. 58. Hourly Traffic Dynamics Over a Day
  59. 59. Public Safety Urban Planning Gov. & Agency Admin. Energy & water Environmental TransportationSocial Programs Healthcare Education Twitter as a Source of City Events
  60. 60. Pramod Anantharam, Payam Barnaghi, Krishnaprasad Thirunarayan, and Amit Sheth. 2015. Extracting City Traffic Events from Social Streams. ACM Trans. Intell. Syst. Technol. 6, 4, Article 43 (July 2015), 27 pages. DOI=10.1145/2717317 http://doi.acm.org/10.1145/2717317 Last O night O in O CA... O (@ O Half B-LOCATION Moon I-LOCATION Bay B-LOCATION Brewing I-LOCATION Company O w/ O 8 O others) O http://t.co/w0eGEJjApY O Extracting City Events from Textual Data
  61. 61. Complementary Events Traffic Incident; road-construction Textual Events from Tweets vs. 511.org: Complementary
  62. 62. Corroborative Events Fog visibility-air-quality; fog Textual Events from Tweets vs. 511.org: Corroborative
  63. 63. Timeliness Concert Concert Textual Events from Tweets vs. 511.org: Timeliness
  64. 64. Image Credit: http://traffic.511.org/index slow-moving-traffic Domain knowledge in the form of traffic vocabulary Domain knowledge of traffic flow synthesized from sensor data Explained-by Horizontal operator: relating/mapping data from different modality to a concept (theme) within a spatio-temporal context; Spatial context even include what it means to have a slow traffic for the type of road Understanding: Semantic Annotation of Sensor + Textual Data Utilizing Background Knowledge
  65. 65. This example demonstrates use of: – Multimodal data streams (types of events from text - signature from sensor data). – Multiple sources of knowledge/ontologies. – Semantic annotations and enrichments. – Use of rich representation (PGM). – Statistical approach to create normalcy models and understand anomalies using historical data. – Explain anomalies using extracted events. – Provide actionable information. How traffic analysis captures complexity of the real-world?
  66. 66. Domain Knowledge Historical Data Annotation 511.org SC CC Slow traffic due to football match. Slow traffic due to accident. Slow traffic due to construction. raw numbers Multimodal Data Anomalous Traffic Pattern Learned Models Abstractions (Actional Information) PC
  67. 67. Thank you, and please visit us at http://knoesis.org For more information on kHealth, please visit us at http://knoesis.org/projects/khealth Cognitive Computing Semantic Computing Perceptual Computing Contributors and collaborators for this talk: Pramod Anantharam Cory Henson Dr. T.K. Prasad Sujan Perera Utkarshani Jaimini Thank You
  68. 68. Part of my group @ Kno.e.sis
  69. 69. Backup/discussion slides 73
  70. 70. Perception can be split into two processes: 1.Processing sensory input, which transforms these low-level information to higher-level information (e.g., extracts shapes for object recognition). 2. Processing which is connected with a person's concepts and expectations (knowledge) and selective mechanisms (attention) that influence perception. https://en.wikipedia.org/wiki/Perception Perception
  71. 71. Scott Kelso uses “Circular Causality” to describe the situation in dynamic systems where the cooperation of the individual parts of the system determine the global system’s behavior, which in turn governs the behavior of the individual parts. This is related to Andy Clark’s concept of continuous reciprocal causation, which occurs when some system S is both continuously affecting and simultaneously being affected by activity in some other system O. https://mitpress.mit.edu/books/artificial-cognitive-systems Component Dynamics Global System Behavior InfluencesDetermine Circular Causality and Autonomous Cognitive Systems
  72. 72. Perception is an active, cyclical process of exploration and interpretation. “Perception and action, if these unifying models are correct, are intimately related and work together to reduce prediction error by sculpting and selecting sensory inputs” Perception-Action Cycle (Neisser, 1976) https://www.amazon.com/Cognition-Reality-Principles-Implications-Psychology/dp/0716704773 Neisser Perceptual-Action Cycle
  73. 73. http://www.simplypsychology.org/perception-theories.html, http://www.youramazingbrain.org/supersenses/necker.htm Psychologist Richard Gregory (1970) argued that perception is a constructive process which relies on top-down processing. A lot of information reaches the eye, but much is lost by the time it reaches the brain (Gregory estimates about 90% is lost). The brain has to guess what a person sees based on past experiences. Our perceptions of the world are hypotheses based on past experiences and stored information (prior knowledge). Sensory receptors receive information from the environment, which is then combined with previously stored information about the world which we have built up as a result of experience. The formation of incorrect hypotheses will lead to errors of perception Richard Gregory’s Perception
  74. 74. http://www.people-clipart.com/people_clipart_images/clip_art_illustration_of_a_young_woman_with_her_hand_on_her_hip_wearing_jeans_and_a_half_sleeve_shirt_0071-0908- 3022-3627.html Zadeh, Lotfi A. "Toward a perception-based theory of probabilistic reasoning with imprecise probabilities." Journal of statistical planning and inference 105, no. 1 (2002): 233-264 http://sweetclipart.com/cute-little-girl-holding-daisy-719 http://www.keyword-suggestions.com/aGlzcGFuaWMgYXJ0/ Lotfi Zadeh and Perception • Measurements are crisp; perceptions are fuzzy. • Perception-based information is drawn from natural language. • Humans use perceptions of time, direction, speed, shape, possibility, likelihood, truth, and other attributes of physical and mental objects. • Perceptions are imprecise. • Example: not very high, about 0.8
  75. 75. 1. Cognition is a general term for all forms of knowing (e.g. attending, remembering, reasoning and understanding concepts, facts, propositions, and rules). 2. Cognitive processes are how you manipulate your mental contents. 3. Cognitive psychology is the study of cognition. 4. Cognitive science is an interdisciplinary field that extends the principles of cognitive psychology to other systems that manipulate information. Cognition and Brain-Non-invasive brain scanning allows correlations to be made between human conscious experiences and patterns of brain activity. Studies of both visual and auditoryperception allow distinctions to be made between brain regions that do and do not show activity patterns that correlate with conscious experiences. Results from study of brain lesions, application of drugs, and electromagnetic disruption of the function of specific brain regions can be interpreted in combination with results from brain scans. 1. Neural correlates of the visual vertical meridian asymmetry by Taosheng Liu, David J. Heeger, and Marisa Carrasco in Journal of Vision (2007) Volume 6: 1294–1306. 2. Jump up Hierarchical Processing of Auditory Objects in Humans by Sukhbinder Kumar, Klaas E Stephan, Jason D Warren, Karl J Friston and Timothy D Griffithsin in PLoS Comput Biol. (2007) Volume 3:e100. https://en.wikiversity.org/wiki/Cognition What does cognition mean? (from Wikipedia)
  76. 76. Cognition enables individuals to understand their environment and paves the way for the application of perception to explore and deepen the understanding The network of concepts and relationships enables the cognition and perception needed to interpret daily experiences. What does cognition mean? (from Wikipedia)
  77. 77. The system’s understanding of it’s worlds is inherently specified to the form of the systems’ embodiment and is dependent on the system’s history of interactions and it’s experiences. So this property of making sense of it’s environmental interactions is one of the foundations of a branch of cognitive science called “Enaction” From https://mitpress.mit.edu/books/artificial-cognitive-systems Emergent Paradigm of Cognitive Science
  78. 78. From http://philosophyofbrains.com/2015/12/14/surfing-uncertainty-prediction-action-and-the-embodied-mind.aspx Top-Down and Bottom-Up (Andy Clark)
  79. 79. http://www.fil.ion.ucl.ac.uk/~karl/Whatever%20next.pdf, http://libertymotive.com/truth/ Top-Down Approach model emphasizes the use of background knowledge to predict content. The information that needs to be communicated ‘upward’ is just the prediction error: the divergence from the expected signal. Top-down Expectation/Prediction firm, not squishy red and green, not just red Prediction Error fruit stem Bottom-up Observation Andy Clark
  80. 80. Attempts to build an artificial cognitive system that can be positioned in a two-dimensional space, with one axis defining a spectrum running from purely computational techniques to techniques strongly inspired by biological models, and with another axis defining the level of abstraction of the biological model. Adapted from https://mitpress.mit.edu/books/artificial-cognitive-systems. Inspiration Abstraction Level Modelling decomposition of hypothetical model of brain Cognitive system modelled on the microscopic organization of the brain Cognitive system based on statistical learning of specific domain rules Cognitive system based on artificial neural networks Computational Biological Low High IBM Brain Chip Aspects of Modelling Cognitive Systems
  81. 81. Understanding traffic flow variations
  82. 82. Histogram of speed values collected from June 1st 12:00 AM to June 2nd 12:00 AM Histogram of travel time values collected from June 1st 12:00 AM to June 2nd 12:00 AM Traffic Data: First Peek
  83. 83. This distribution resembles a Gaussian Mixture Model (GMM) Multiple Gaussian Distributions: A Better Fit for Speed Observations?
  84. 84. Assume Normalcy to be uninterrupted traffic flow July 2014 has no events so, we hypothesize higher log-likelihood score June 2014 has many events so, we hypothesize lower log-likelihood score -115655.8 (Closer to Normalcy) -125974.3 88 Golden Gate Fields: Comparing Months with Varying Event Occurrences
  85. 85. Root Cause Analysis Action Recommendation Find Triggers of Asthma Derive the cause of asthma attacks for a given patient using statistical techniques + knowledge of asthma and its triggers. Minimize Asthma Attacks Model actions based on the utility theory (cost of actions & its rewards) + knowledge of action consequences. Two Research Directions for kHealth Asthma with More Ddata…
  86. 86. Sensordrone • Precision gas sensor • Reducing gas sensor • Oxidizing gas sensor • Non-contact thermometer • Humidity sensor • Temperature sensor • Light sensor • Color sensors • Pressure • Proximity • Expansion connector Node Sensor • Exhaled Nitric Oxide Fitbit ChargeHR • Heart Rate • All-Day Activity • Sleep Monitoring 90 kHealth Kit: Sensor Platforms
  87. 87. • If an anomaly is detected on a link L and during time period [tst, tet], then the anomaly is explained by an event if the event occurred in the vicinity within 0.5km radius and during [tst-1, tet+1]. • CAVEAT: An anomaly may not be explained because of missing data. Thanks to Dr. Krishnaprasad Thirunarayan for sharing this slide. Spatio-temporal Co-occurrence Criteria
  88. 88. Brain to Computing Second Approach to Intelligence
  89. 89. How can we get inspiration from brain to computing?
  90. 90. One approach, recreate the brain in silicon. A second approach, different disciplines learn about the human brain, get inspiration and learn things for computing. (Slide on mapping from brain to computing.)
  91. 91. "The domain of cognitive science occupies the intersection of philosophy, neuroscience, linguistics, cognitive psychology, and computer science (artificial intelligence)." Gerrig, R. J., Zimbardo, P. G., Campbell, A. J., Cumming, S. R., & Wilkes, F. J. (2008). Psychology and life (Australian edition). Sydney: Pearson Education Australia, p. 248.https://en.wikiversity.org/wiki/Cognitive_science. Cognitive Science
  92. 92. https://mitpress.mit.edu/books/artificial-cognitive-systems Two hallmarks of cognition: ● Prediction out of the past interactions. ● Learning new knowledge by making sense of its interactions with the world around it. The dependency on exploration and development is one of the reasons why an artificial system requires a rich sensory-motor interface with its environment and why embodiment plays such a pivotal role. Hallmarks of Cognition
  93. 93. Contextual https://en.wikipedia.org/wiki/Cognitive_computing http://edtechreview.in/dictionary/174-what-is-adaptive-learning http://www.ibm.com/developerworks/rational/library/may05/bittner-spence/ http://www.inspiring-visuals.com/interactive-multimedia-presentations.shtml http://leadingagent.net/blog/an-awesome-contextual-real-estate-crm-solution-is-critical-to-smart-real-estate-practice-growth/ Features that cognitive systems express are: Adaptive Interactive Iterative and stateful Cognitive Systems
  94. 94. Reduced CO level => better asthma control High CO influences Wheezing Level (Low/High) High CO High Luminosity High Wheeze Low Luminosity Low Wheeze Carbon Monoxide Wheeze Luminosity 1Amit Sheth, Pramod Anantharam, Cory Henson, 'Physical-Cyber-Social Computing: An Early 21st Century Approach,' IEEE Intelligent Systems, vol. 28, no. 1, pp. 78-82, Jan.-Feb., 2013. http://doi.ieeecomputersociety.org/10.1109/MIS.2013.20 Horizontal Operators (Semantic Integration) operates on data from heterogeneous sources to create Integrated or correlated data streams. Vertical Operators (Semantic abstraction) operates on artifacts at each level and transcends them to the next level. “a holistic treatment of data, information, and knowledge from the PCS worlds to integrate, correlate, interpret, and provide contextually relevant abstractions to humans.”1 Physical-Cyber-Social Computing for Actionable Insights from Multimodal Data
  95. 95. http://phys.org/news/2014-10-nobel-importance.html https://student.societyforscience.org/article/nobel-goes-finding-brain%E2%80%99s-%E2%80%98gps%E2%80%99 http://www.scilogs.be/neuropolis/hoe-helpen-de-kanaaltjes-in-ons-oor-ons-om-te-navigeren/
  96. 96. https://mitpress.mit.edu/books/soar-cognitive-architecture, http://www.hindawi.com/journals/mpe/2012/530561/fig1/ Soar Cognitive Architecture
  97. 97. http://my.ilstu.edu/~jrbaldw/370/Meaning.htm Semiotics
  98. 98. Brain GPS - Grid Cell/ Place Cells ● in 1971 John O’Keefe of University College London discovered “place cells” found in Hippocampus. ● In 2005 May-Britt and Edvard Moser of Norwegian University of Science and Technology added a new discovery : the existence of “grid cells” in the nearby cortex. ● The grid and place cells work together to constitute an internal navigation system. ● Grid cells provide a set of coordinates that enable a rat to navigate through its environment in conjunction with other cells that recognize the positioning of the head and the borders of a room. ● Grid cell systems give you immediate sense of recognition of, yes, this is where I am. "That's my home, there's my office, there's the entrance to the metro." ● Alzheimer's patients wander off into the night, as a consequence of the dying of neurons in the entorhinal cortex and the hippocampus. https://www.kth.se/blogs/prasanth/tag/india-and-sweden/ http://www.dallasnews.com/opinion/sunday-commentary/20130920-gps-may-reroute-the-brain.ece http://www.scientificamerican.com/sciam/assets/media/multimedia/110514-MindListicle/1.jpg http://www.activebeat.com/health-news/nobel-prize-awarded-to-scientists- who-discovered-brain-gps-system/ http://www.scientificamerican.com/article/10-big-ideas-in-10-years-of-brain-science/
  99. 99. http://www.scientificamerican.com/article/10-big-ideas-in-10-years-of-brain-science/ In the future optogenetics will allow us to decipher both how various brain cells elicit feelings, thoughts and movements—as well as how they can go awry to produce psychiatric disorders. ● Stanford scientists presented a technique for switching individual neurons on or off with light in 2005. ● To probe how a certain class of neurons helps mice navigate mazes, scientists insert electrodes into brain tissue and stimulate thousands of neurons at a time. ● Scientists can tuck light-sensitive molecules into specific brain cells to manipulate only those selected neuron types or networks. ● Shining a light makes those neurons either more or less active and can elucidate their role in a behavior or disease.Mouse with optogenetic tools in operation, including implanted fiberoptic and light-sensitive molecules produced in the brain, all representing technologies developed in the Deisseroth lab at Stanford University by graduate students Raag Airan, Feng Zhang, Ed Boyden, and Lief Fenno. Credit: Raag Airan, Feng Zhang, Ed Boyden, and Lief Fenno “Over the past decade hundreds of research groups have used optogenetics to learn how various networks of neurons contribute to behavior, perception and cognition,” wrote Ed Boyden, a co-inventor of optogenetics. Optogenetics
  100. 100. http://www.scientificamerican.com/sciam/assets/media/multimedia/110514-MindListicle/1.jpg http://www.scientificamerican.com/article/10-big-ideas-in-10-years-of-brain-science/ http://www.nature.com/nrn/journal/v5/n7/fig_tab/nrn1429_F1.html Woman at the Human Genome exhibit at the National Museum of Natural History in Washington, D.C. Human Genome Project (HGP) Mapping and Identification of clusters of disease-related human genes for boosting our understanding of genetic pathways that spawn neurological and psychiatric disorders like schizophrenia, Alzheimer’s disease, depression and autism, among other disorders. “Neurogenetics is the role of genetics in the development and function of the nervous system” Neurogenetics
  101. 101. Philanthropist Paul Allen, co-founder of Microsoft ● In 2003 , Allen Institute of Brain Science began mapping gene activity in mouse brain ● Genetic activity help researchers discover genes relevant to certain diseases or behaviors. ● Institute launched a 10-year plan to examine where specific genes are active and how these genetic circuits process the vast flow of information into the brain. ● For example, the brain map showed that genes associated with autism appear to be acting on a specific type of brain cell in neocortex. That suggests "we should be looking at this particular type of cell in the neocortex, and furthermore that we should probably be looking very early in the prenatal stages for the origin of autism". A top-down 3-D view of the cortico-connections originating from multiple distinct cortical areas, visualized as virtual tractography using Allen Institute Brain Explorer software. Credit: Allen Institute for Brain Science. http://www.scientificamerican.com/sciam/assets/media/multimedia/110514-MindListicle/1.jpg http://www.scientificamerican.com/article/10-big-ideas-in-10-years-of-brain-science/ Brain Mapping
  102. 102. Neurocognitive functions are cognitive functions linked with the functions of particular areas, neural pathways, or cortical networks in the brain substrate layers of neurological matrix at the cellular molecular level. Their understanding is closely linked to the neuropsychology and cognitive neuroscience, two disciplines that broadly seek to understand how the structure and function of the brain relates to perception, de-fragmentation of concepts, memory embed, association and recall both in the thought process and behavior. https://en.wikipedia.org/wiki/Neurocognitive Neurocognitive
  103. 103. The term 'cognitive neuroscience' was coined by George Miller and Michael Gazzaniga toward the end of the 1970s. Cognitive neuroscience began to integrate the newly laid theoretical ground in cognitive science, that emerged between the 1950s and 1960s. Cognitive neuroscience is concerned with the scientific study of the biological processes and aspects that underlie cognition, with a specific focus on the neural connections in the brain which are involved in mental processes. It addresses the questions of how psychological/cognitive activities are affected or controlled by neural circuits in the brain. Cognitive neuroscience relies upon theories in cognitive science coupled with evidence from neuropsychology, and computational modeling. Neurons play the most vital role, since the main point is to establish an understanding of cognition from a neural perspective, along with the different lobes of the cerebral cortex. David Marr (neuroscientist) concluded that one should understand any cognitive process at three levels of analysis: • Computational. • Algorithmic/representational. • Physical levels of analysis. https://en.wikipedia.org/wiki/Cognitive_neuroscience Cognitive Neurocognitive
  104. 104. http://www.scientificamerican.com/article/10-big-ideas-in-10-years-of-brain-science/ http://blogs.scientificamerican.com/talking-back/discovery-of-brain-s-navigation-system-wins-2014-nobel-prize-in-medicine/ References

×