Computing for Human Experience [v4]: Keynote @ OnTheMove Federated Conferences
Upcoming SlideShare
Loading in...5
×
 

Computing for Human Experience [v4]: Keynote @ OnTheMove Federated Conferences

on

  • 3,304 views

"Computing for Human Experience: Semantics empowered Cyber-Physical, Social and Ubiquitous Computing beyond the Web" Keynote at On the Move Federated Conferences, Crete, Greece, October 18, 2011.

"Computing for Human Experience: Semantics empowered Cyber-Physical, Social and Ubiquitous Computing beyond the Web" Keynote at On the Move Federated Conferences, Crete, Greece, October 18, 2011.
http://www.onthemove-conferences.org/

Abstract: http://www.onthemove-conferences.org/index.php/keynotes/amitsheth




Statistics

Views

Total Views
3,304
Views on SlideShare
2,814
Embed Views
490

Actions

Likes
3
Downloads
34
Comments
1

6 Embeds 490

http://knoesis.org 477
http://thinkery.me 6
http://www.twylah.com 3
https://si0.twimg.com 2
http://a0.twimg.com 1
https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Talk abstract: http://www.onthemove-conferences.org/index.php/keynotes2011/204

    Two examples given during the talk for physical(sensor)-cyber-social systems: http://www.slideshare.net/knoesis/secure-semantics-empowered-rescue-environment and http://twitris.knoesis.org

    Related Vision Paper: http://wiki.knoesis.org/index.php/Computing_For_Human_Experience
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Imageshttp://blogs.ebrandz.com/tag/social-networks/http://news.cnet.com/8301-13579_3-10141672-37.htmlhttp://www.thehindu.com/opinion/columns/sainath/article123884.ecehttp://www.samplestuff.com/2011/06/11/frugal-family-fun-nights/
  • Imagehttp://www.naturewalls.org/category/stream/
  • Imageshttp://www.cbi.umn.edu/about/babbage.htmlhttp://news.cnet.com/8301-13579_3-10141672-37.htmlhttps://fpm-www3.fpm.wisc.edu/safety/occupationalHealth/Ergonomics/LaptopErgonomics/LaptopErgonomicsQuickReference/tabid/106/Default.aspxHow many personal computers (PC) in the world -- http://www.gartner.com/it/page.jsp?id=703807
  • Imageshttp://news.cnet.com/8301-13579_3-10141672-37.htmlhttps://fpm-www3.fpm.wisc.edu/safety/occupationalHealth/Ergonomics/LaptopErgonomics/LaptopErgonomicsQuickReference/tabid/106/Default.aspxhttp://jimilocker.info/?p=165
  • Images
  • Images
  • Imagehttp://www.healthsciencestrategy.com/2011/04/will-mhealth-apps-and-devices-empower-epatients-for-wellness-and-disease-management-a-case-study-2/
  • Images:http://www.fitbit.com/http://www.relativitycorp.com/socialnetworkmarketing/http://heinrich.house.gov/index.cfm?sectionid=11&parentid=2&sectiontree=2,11&itemid=131
  • http://www.slideshare.net/IQLab/social-media-101-for-pharma-3494462
  • http://www.slideshare.net/IQLab/social-media-101-for-pharma-3494462
  • http://www.slideshare.net/IQLab/social-media-101-for-pharma-3494462
  • http://www.slideshare.net/IQLab/social-media-101-for-pharma-3494462
  • Use-Case for People-Content-Network analysis - Not limited to user-community engagement, but it can answer many questions, exploiting potential of citizen-sensingPresentation- http://www.slideshare.net/knoesis/understanding-usercommunity-engagement-by-multifaceted-features-a-case-study-on-twitterReferencesH. Purohit, Y. Ruan, A. Joshi, S. Parthasarathy, A. Sheth.Understanding User-Community Engagement by Multi-faceted Features: A Case Study on Twitter, SoME 2011, Workshop on Social Media Engagement, in conjunction with WWW 2011.M. Nagarajan, H. Purohit, A. Sheth. A Qualitative Examination of Topical Tweet and Retweet Practices , 4th Int'l AAAI Conference on Weblogs and Social Media, ICWSM 2010A. Sheth, H. Purohit, A. Jadhav, P. Kapanipathi, L. Chen. Understanding Events Through Analysis Of Social Media , Kno.e.sis Technical Report, 2010
  • There is an event oriented community being formed during an eventDynamic domain model would give enhanced understanding of what is going on in the event, and hence, the communityCommunity has people, sub-community of users in the community will form an implicit network for ‘resource sharing’ and ‘resource provider’ How do we get to know such people? Semantic analysis of user profile description, with help of external knowledges bases to tell us, ‘what is a user interest I’ will tell us, ‘who is what’ – Blogger, news journalist, trustee, Red Cross Member etc. and also, ‘who’ is interested in ‘what’This information of mined user interests and types, from users profiles, can be leveraged to perform semantic association with dynamic domain model of the event
  • Images:http://www.fitbit.com/ http://quantifiedself.com/https://foursquare.com/
  • Images:http://www.fitbit.com
  • Images:http://depositphotos.com/4946293/stock-photo-Abstract-earth-puzzle.html
  • Images:http://massthink.wordpress.com/2007/06/10/husserl-in-indubitable-response-to-descartes-and-kant/
  • Four characteristics of perceptual inference
  • Perception is an abductive process
  • PCT is a well known abductive logic framework
  • Single disorder assumption says that an explanation is parsimonious if it covers, or explains, all observations.Abductive reasoning is generative– which has performance hit.Single entity/abstraction can account for all properties, so we can have deductive approach for finding explanation adequate.
  • Encoding background knowledge in OWL (bipartite model is converted into RDF)
  • Can work with incomplete informationAs new observation comes, explanation is updated
  • Images:http://www.ida.liu.se/~eriho/COCOM_M.htmhttp://www.idemployee.id.tue.nl/g.w.m.rauterberg/lecturenotes/ucd%20lecture-3/sld019.htm
  • 20,000 weather stations (with ~5 sensors per station)Real-Time Feature Streams - live demo: http://knoesis1.wright.edu/EventStreams/ - video demo: https://skydrive.live.com/?cid=77950e284187e848&sc=photos&id=77950E284187E848%21276
  • Automated detection of different types of fires, which each require different extinguishing methodsYouTubeSECURE Demo: http://www.youtube.com/user/knoesisCenter?blend=1&ob=5
  • Images:http://gigaom.com/cloud/sensor-networks-top-social-networks-for-big-data-2/
  • Images:http://thewvsr.com/doctors.htm
  • - Such machines (and average/everyday people) could take some responsibilities, thus freeing up a doctors time to be more productive
  • Body sensors: thermometer, optical blood-flow sensor, galvanic skin response monitor, accelerometerObservations: e.g., pulse-rate is high, activity is lowHistory: previous heart attack, genetic predisposition to heart diseaseAbstraction: current condition is worrisomePotential explanation for observations = …Example current technologies: quantified-self, my life bits, Basis, fitbit, etc.
  • Personal sensors: self, blood-pressure monitorFocus: using knowledge of domain, patient history, and current observations, ask patient pointed questions:about symptoms only they can observe, e.g., chest painabout symptoms that can be observed by sensors the patient has current access to (i.e., blood-pressure monitor at home)about taking medications that have been prescribed for known disorders (history) that are also in current explanation (diagnosis)Observations: self observations, observations from at-home sensors (e.g., blood-pressure monitor)Abstraction: continuously update explanations based on new observations (guided by asking questions through focus)Treatment: based on current explanation, prescribe treatment:tell patient to see doctor (perhaps syncing patient and doctor calendars to recommend appointment), and send information to doctortell patient to take medication, either over-the-counter or medication that has previously been prescribed to patientCurrent example technologies: WebMD, Medlineplus, etc.
  • Background Knowledge (i.e., Ontologies, knowledge-bases)Domain ontology (e.g., cardiology) describing medical and health knowledgeSensor ontology describing characteristics of medical equipmentPerception ontology describing process of translating observed symptoms to actionable knowledge (i.e., diagnoses, treatments)Social/Community SupportPersonal ContextPersonal Medical HistoryIndividual medical historyRecord types: EHR, PHR, PVR, etc.Systems: MS Health VaultFamily Genetic HistoryE.g., genetic predisposition to heart disease
  • Going through perception cycle (regular font observation: sensors can detect; bold observation: sensors cannot detect)
  • Background Knowledge (i.e., Ontologies, knowledge-bases)Domain ontology (e.g., cardiology) describing medical and health knowledgeSensor ontology describing characteristics of medical equipmentPerception ontology describing process of translating observed symptoms to actionable knowledge (i.e., diagnoses, treatments)Social/Community SupportPersonal ContextPersonal Medical HistoryIndividual medical historyRecord types: EHR, PHR, PVR, etc.Systems: MS Health VaultFamily Genetic HistoryE.g., genetic predisposition to heart disease
  • Imagehttp://www.thehindu.com/opinion/columns/sainath/article123884.ecehttp://www.samplestuff.com/2011/06/11/frugal-family-fun-nights/

Computing for Human Experience [v4]: Keynote @ OnTheMove Federated Conferences Computing for Human Experience [v4]: Keynote @ OnTheMove Federated Conferences Presentation Transcript

  • Computing for Human Experience Keynote at On-the-Move Federated Conference, October 2011: http://www.onthemove-conferences.org/ Amit ShethKno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing Wright State University, Dayton, OH, USA Special thanks & contributions: Cory Henson 1
  • Our 1 world with 7 billion peopleliving 1 experience at a time 2
  • with a stream of experiences to be … shared 3
  • Today, technology increasingly engagesindividuals, society and humanity with … 1-2 billion computers 5 billion 40+ billion mobile phones mobile sensors http://www.gartner.com/it/page.jsp?id=703807 4
  • With constant connectivity enabled through global networks 1-2 billion computers 5 billion 40+ billion mobile phones mobile sensors 5
  • So, how can we leverage this tech to improve our experiences without losing ourselves in the process? 6
  • Or drown in data? 7
  • From machine-centric to human-centric designMachines to accommodate our experiences, as opposed to the other way around Computing to liberate 8
  • To accomplish this requires a fundamentalshift in how we interact and communicatewith computational machines We must take a more holistic view of computation, as a shared universe, populated by people and machines working in harmony to achieve our highest aspirations 9
  • Ubiquitous Computing – M. Weiser We have caught glimpses of this vision … Memex– V. Bush 10
  • The ways in which technology and humansinteract are fundamentally changing In turn, our (human) experiences are changingOur activities, decisions, thoughts, and feelingsare affected by the ubiquitous integration oftechnology into the fabric of our lives Let this affect be positive 11
  • Computing for Human Experience AmitShethKno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing Wright State University, Dayton, OH, USA Special thanks & contributions: Cory Henson 12
  • Alan Smith Vinh Michael HemantP Nguyen Cooney urohit Matthan Sink Sujan Perera Wenbo WangPramod Koneru Cory Henson Amit ShethMaryam Panahiazar Kalpa GunaratnaAshutoshJadhav Sanjaya Wijeratne Pramod Prateek PavanKapanip Delroy Sarasi Lalithsena Ajith Anantharam Jain athi Lu Chen Cameron Ranabahu
  • CHE is an approach to improving the humancondition through computational means, andwith minimal burden This may be achieved through the contextual assistance, augmentation, and absolution of human capabilities Human capabilities such as sensing, perception, attention, memory, decision making, control, etc. 14
  • Consider the following example … 15
  • A cross-country flight from New York to Los Angeles on aBoeing 737 plane generates a massive 240 terabytes of data - GigaOmni Media 16
  • But, how much data is generated regarding thehealth and well-being of the pilot or passengers? zero, none, zilch! 17
  • Image the ability to monitor and control our health with the same care and precisionthat goes into the 737.And not just providingdoctors with suchcontrol, but you and me. 18
  • Health information is now available from multiple sources • medical records • background knowledge • social networks • personal observations • sensors • etc. 19
  • Sensors, actuators, and mobile computing are playing anincreasingly important role in providing data for early phases ofthe health-care life-cycleThis represents a fundamental shift:• people are now empowered to monitor and manage their own health;• and doctors are given access to more data about their patients 20
  • Unfortunately, when personal health data is collectedand presented, it often looks like this … gibberish. 21
  • What is needed is a more intuitive and intelligent representation of our health.Personal Health Dashboard Health Metrics with Meaning Image: http://bit.ly/lV2V73 22
  • How is this accomplished? 23
  • Key Enablers of CHE• Integration of heterogeneous, multimodal data• Bridging the physical-cyber-social divide• Elevating abstractions that machines and people understand• Semantics at an extraordinary ScaleThese enablers are brought together throughSemantic Web technologies 24
  • Foundation on which these enablers stand 25
  • Foundation on which these enablers stand 26
  • Integration of heterogeneous, multimodal data 27
  • Integration of heterogeneous, multimodal data Background Knowledge: ontologies, knowledge bases, LOD, databases, etc. Social/Community Data:social network data, wisdom of the crowds, etc. Sensor Data: observations from machine sensors, citizen sensors (i.e., patients, doctors), laboratory experiments, etc. Personal Context: location, schedule, items (e.g., accessible sensors), etc. Personal Medical History: Electronic Medical Records, Personal Health Records, Patient Visit Records, etc. 28
  • What is Social Media?Communications using online technologiesto share opinions, insights, experiences andperspectives with each other. 29
  • Popular types of Healthcare Social MediaBlogs – DiabetesMine, HealthMatters, WebMD, NYT HealthBlog, etc.Microblogs –Livestrong, Stupid Cancer, etc.Social Networks –OrganizedWisdom, PatientsLikeMe, DailyStrength, NursesRecommendDoctors, CureTogether, etc.Podcasts – John Hopkins Medical Podcasts, Mayo Clinic, etc.Forums – Revolution Health Groups, Google Health Groups, etc. 30
  • HCPs aren’t waiting to be detailed, they’re turning to the social web to educate themselves 60% ofof physicians either or are interested in using social networksnetworks 60% physicians either use use or are interested in using social 112,000 docs talk to eachother on Sermo. This doc-to-doc blogger has 53,000 readers this month + 20,000 Twitter followers 65% of docs plan to use social media for professional development Manhattan Research 2009, 2010 Sermo,com Compete.com http://www.slideshare.net/IQLab/social-media-101-for-pharma-3494462 31
  • People are turning to each other online to understand their health http://www.slideshare.net/IQLab/social-media-101-for-pharma-3494462 32
  • 83% of online adults search for health information http://www.slideshare.net/IQLab/social-media-101-for-pharma-3494462 33
  • 83% of online adults search for health information 60% of them look for the experience of “someone like me” http://www.slideshare.net/IQLab/social-media-101-for-pharma-3494462 34
  • "I dont know, but I can try to find out" is the default setting for people with health questions."I know, and I want to share my knowledge"is the leading edge of health care. Savannah Fox, The Social Life of Health Information, Pew Internet Report, May 12, 2011. Available at http://www.pewinternet.org/Reports/2011/Social-Life-of-Health-Info.aspx 35
  • People-Content-Network Analysis Intra Community Activity and connectivity – How well connected are individual nodes (People) – What keeps them strongly connected over time (Relationship types - Knowledge of Content) Will the two communities coordinate well during an event- crisis or disaster? • Interplay between all three dimensions – P, C, N Inter-Community Connectivity • Any bridges to connect to the other community? (People) • Any Similarity in actions with the other communityImage: http://themelis-cuiper.com (Can Content help?) For more info: http://www.slideshare.net/knoesis/understanding-usercommunity-engagement- by-multifaceted-features-a-case-study-on-twitter 36
  • People-Content-Network Analysis Event oriented Community ExternalKnowledge bases Dynamic Domain Model SEMANTIC for the ASSOCIATION TO Social Network event UNDERSTAND ENHANCED ENGAGEMENT LEVEL User Mined User Interests Profiles and User Types 37
  • Bridging the physical-cyber-social divideComputation is no longer confined to pure symbol manipulation. The previously strictrelations between the digital and physical world are blurring. 38
  • Bridging the physical-cyber dividePsyleron’s Mind-Lamp (Princeton U),connections between the mind and thephysical world. MIT’s Fluid Interface Group: wearable device with a projector for deep interactions with the environment Neuro Skys mind-controlled headset to play a video game. 39
  • Bridging the physical-cyber-social divide FitBit Community allows the automated collection and sharing of health-related data, goals, and achievementsFoursquare is an online application whichintegrates a persons physical location and Community of enthusiasts that share experiences ofsocial network. self-tracking and measurement. 40
  • Bridging the physical-cyber-social divide Tweeting Sensors sensors are becoming social 41
  • Bridging the physical-cyber-social divideFor more info: http://twitris.knoesis.org/ 42
  • Dynamic Model Creation Continuous Semantics 43
  • Dynamic Model Creation 44
  • The design and building of physical-cyber-social systems requires effectiveconceptualization and communication between people and machines. To reach this vision requires advancement in the area of machine perception, enabling machines the ability to abstract over low-level observations. 45
  • Abstraction provides the ability to interpret and synthesize information in a way thataffords effective understanding and communication of ideas, feelings, perceptions, etc.between machines and people. Abstraction 46
  • People are excellent at abstraction; ofsensing and interpreting stimuli tounderstand and interact with the world. The process of interpreting stimuli is called perception; and studying this extraordinary human capability can lead to insights for developing effective machine perception. 47
  • Sensor “real-world” observation Physical Social Cyber perceptionconceptualization of “real-world” Sensor Data / Social Data 48
  • Both people and machines are capable of observing qualities, such as redness. observes Observer Quality * Formally described in a sensor/observation ontology 49
  • Sensor and Sensor Network (SSN) Ontology http://www.w3.org/2005/Incubator/ssn/XGR-ssn/ 50
  • The ability to perceive is afforded through the use of backgroundknowledge, relating observable qualities to entities in the world. Quality * Formally described in inheres in domain ontologies (and knowledge bases) Entity 51
  • http://linkedsensordata.com 52
  • With the help of sophisticated inference, both people and machines arealso capable of perceiving entities, such as apples. perceives Perceiver Entity • the ability to degrade gracefully with incomplete information • the ability to minimize explanations based on new information • the ability to reason over data on the Web •fast (tractable) 53
  • Abductive Logic Deductive Logic (e.g., OWL) high complexity (relatively) low complexity minimize explanations tractabledegrade gracefully Web reasoning Perceptual Inference (i.e., abstraction) Cory Henson, KrishnaprasadThirunarayan, AmitSheth, Pascal Hitzler. Representation of Parsimonious Covering Theory in OWL-DL. Proceedings of the 8th International Workshop on OWL: Experiences and Directions (OWLED 2011), San Francisco, CA, USA, June 5-6, 2011. 54
  • Parsimonious Covering Theory disorder causes manifestation m1• Goal is to account for observed symptoms d1 with plausible explanatory hypotheses m2 (abductive logic) d2 m3 d3 m4• Driven by background knowledge modeled explanation observations as a bipartite graph causally linking disorders to manifestations YunPeng, James A. Reggia, "Abductive Inference Models for Diagnostic Problem-Solving" 55
  • PCT Parsimonious Cover• coverage: an explanation is a cover if, for each observation, there is a causal relation from a disorder contained in the explanation to the observation• parsimony: an explanation is parsimonious, or best, if it matches some criteria of suitability (i.e., single disorder assumption) 56
  • Convert PCT to OWLGivenPCT problem P is a 4-tuple ⟨D, M, C, Γ⟩• D is a finite set of disorders• M is a finite set of manifestations• C is the causation function [C : D ⟶ Powerset(M)]• Γ is the set of observations [Γ ⊆ M ]Δ is a valid explanation (i.e., is a parsimonious cover)GoalTranslate P into OWL, o(P), such that o(P) ⊧ Δ Cory Henson, KrishnaprasadThirunarayan, AmitSheth, Pascal Hitzler. Representation of Parsimonious Covering Theory in OWL-DL. Proceedings of the 8th International Workshop on OWL: Experiences and Directions (OWLED 2011), San Francisco, CA, USA, June 5-6, 2011. 57
  • disorders (D) PCT Background for all d ∈ D, write d rdf:type Disorder Knowledge in OWL ex: flu rdf:type Disorder cold rdf:type Disorderdisorder causes manifestation fever manifestations (M) headache extreme exhaustion for all m ∈ M, write m rdf:type Manifestation flu severe ache and pain ex: fever rdf:type Manifestation mild ache and pain stuffy nose headache rdf:type Manifestation … sneezingcold sore throat causes relations (C) severe cough mild cough for all (d, m) ∈ C, write d causes m ex: flu causes fever flu causes headache … Cory Henson, KrishnaprasadThirunarayan, AmitSheth, Pascal Hitzler. Representation of Parsimonious Covering Theory in OWL-DL. Proceedings of the 8th International Workshop on OWL: Experiences and Directions (OWLED 2011), San Francisco, CA, USA, June 5-6, 2011. 58
  • observations (Γ) for mi∈ Γ, i =1 … n, writePCT Observations and Explanation owl:equivalentClassExplanations in OWL causes value m1 and … causes value mn ex: Explanation owl:equivalentClass causes value sneezing and causes value sore-throat and causes value mild-cough explanation (Δ) Δrdf:type Explanation, is deduced ex: cold rdf:type Explanation flu rdf:type Explanation Cory Henson, KrishnaprasadThirunarayan, AmitSheth, Pascal Hitzler. Representation of Parsimonious Covering Theory in OWL-DL. Proceedings of the 8th International Workshop on OWL: Experiences and Directions (OWLED 2011), San Francisco, CA, USA, June 5-6, 2011. 59
  • The ability to perceive efficiently is afforded through the cyclicalexchange of information between observers and perceivers. Observer sends sends observation focus Traditionally called the Perception Cycle (or Active Perception) Perceiver 60
  • Nessier’s Perception Cycle 61
  • Cognitive Theories of Perception (timeline)1970’s – Perception is an active, cyclical process of exploration andinterpretation. - Nessier’s Perception Cycle1980’s – The perception cycle is driven by background knowledge inorder to generate and test hypotheses. - Richard Gregory (optical illusions)1990’s – In order to effectively test hypotheses, some observations aremore informative than others. - Norwich’s Entropy Theory of Perception 62
  • Key Insights• Background knowledge plays a crucial role in perception; what we know (or think we know/believe) influences our perception of the world.• Semantics will allow us to realize computational models of perception based on background knowledge. Contemporary Issues• Internet/Web expands our background knowledge to a global scope; thus our perception is global in scope• Social networks influence our knowledge and beliefs, thus influencing our perception 63
  • Integrated together, we have an general model – capable of abstraction –relating observers, perceivers, and background knowledge. observes Observer Quality sends sends observation inheres in focus perceives Perceiver Entity 64
  • Modeled in set-theoretic notation with components mapped to Parsimonious Covering Theory and OWLCory Henson, KrishnaprasadThirunarayan, AmitSheth. An Ontological Approach to Focusing Attention and EnhancingMachine Perception on the Web. Applied Ontology, 2011 (accepted). 65
  • Applications of Weather Rescue Healthcare 66
  • Weather ApplicationWeather Application Detection of events, such as blizzards, from weather station observations on LinkedSensorData Demos: Real-Time Feature Streams 67
  • SECURE: Semantics Empowered Rescue Application Weather EnvironmentRescue robots detect different types of fires, which may require differentmethods/tools to extinguish, and relays this knowledge to first responders. Demo:SECURE: Semantics Empowered Rescue Environment 68
  • Healthcare Application Weather Application EMR Detection of errors in Electronic Medical Records and missing knowledge in a cardiology domain model 69
  • Healthcare Application Weather ApplicationEMR: "Her prognosis is poor both short term and long term, however, wewill do everything possible to keep her alive and battle this infection." without IntellegO with IntellegO Problem Problem SNM:40733004_infection SNM:68566005_infection_urinary_tract A syntax based NLP extractor By utilizing IntellegO and cardiology (such as Medlee) can extract background knowledge, we can more this term and annotate accurately annotate the term as asSNM:40733004_infection SNM:68566005_infection_urinary_tract 70
  • Healthcare Application Weather ApplicationEMR: ”The patient is to receive 2 fluid buloses." without IntellegO with IntellegO Problem Treatment SNM:32457005_body_fluid Fluid is part of buloses treatment, not a problem A syntax based NLP extractor By utilizing IntellegO and cardiology (such as Medlee) can extract background knowledge, we can determine this term and annotate that this is an incorrect annotation. asSNM:32457005_body_fluid 71
  • Semantic ScalabilityIn 2008, the rate of data generation surpassed storage capacity. With 7 billion people, and agrowing number of sensors, how can such a such a system scale? By shining a light onrelevant human experience, supported by knowledge, while dimming the minutia of data. http://gigaom.com/cloud/sensor-networks-top-social-networks-for-big-data-2 72
  • Semantic Scalability It is clear that purely syntax-based solutions will not scaleex. – keyword based search/index, non-textual data, multimodal data for an event 73
  • Semantic Scalability Path to Web scale semantics1. Focusing attention on important information and ignoring irrelevant data2. Converting low-level data (observations) to high-level knowledge (abstractions)3. Utilizing CHE technology to more evenly distribute responsibility and activities among people and machines 74
  • 1. Focusing attention on important information and ignoring irrelevant data We were able to demonstrate 50% savings in sensing resource requirements during the detection of a blizzard. Cory Henson, KrishnaprasadThirunarayan, AmitSheth. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web. Applied Ontology, 2012. (accepted) 75
  • 2. Converting low-level data to high-level knowledge (observations to abstractions)Experiment – during a blizzard, we utilized Intelleg0 to collectand analyze over 110,000 sensor observations, from:• 800 weather stations (~5 sensors per station)• across 5 states (Utah, Nevada, Colorado, Wyoming, and Idaho)• for 6 days (April 1 – 6, 2003) Cory Henson, KrishnaprasadThirunarayan, AmitSheth. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web. Applied Ontology, 2012. (accepted) 76
  • 2. Converting low-level data to high-level knowledge (observations to abstractions) We were able to demonstrate an order of magnitude resource savings between storing observations vs. relevant abstractions Cory Henson, KrishnaprasadThirunarayan, AmitSheth. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web. Applied Ontology, 2012. (accepted) 77
  • 2. Converting low-level data to high-level knowledge (observations to abstractions) While this is a good result, the benefit provided for a single person – a single experience – is far more dramatic. Cory Henson, KrishnaprasadThirunarayan, AmitSheth. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web. Applied Ontology, 2012. (accepted) 78
  • 3. Utilizing CHE technology to more evenly distribute responsibility and activities among people and machinesThere are almost 7 billion people on earth, and only ~10-15 million doctors (~700:1 - 467:1) 79
  • These doctors are severely overburdened, answering less than 60% of questions posed by patients regarding their health and well beingProviding people with the tools to monitor and manage their own health willdramatically reduce the burden on doctors, and improve the health of the people 80
  • The health-care ecosystem of the future includes machines, people, and social networks continuously, ubiquitously, and unobtrusively monitoring and managing our health 81
  • CHE approach to Health Care Continuous Monitoring Personal Assessment Medical Service 1 2 3Auxiliary Information – background knowledge, social/community support,personal context, personal medical history 82
  • Continuous Monitoring Phase Monitoring health metrics and vital signs utilizing unobtrusive body sensors Continuously collecting information, watching for worrisome symptoms 83
  • Personal Assessment Phase Assessment of symptoms from personal observation and/or health sensors available at home. Utilizing background knowledge, personal medical history, and current sensor data to formulate and ask specific questions of patient that will aid in explaining symptoms. 84
  • Medical Services Phase Assessment of symptoms gathered from continuous and personal phases, with additional sophisticated equipment, advanced treatment, and specialized medical knowledge not previously available. Utilizing background knowledge, personal medical history, and current sensor data to formulate a diagnosis. 85
  • CHE approach to Health Care Continuous Personal Medical Symptoms/ Symptoms/ Explanations Explanations access & access & access & update (PHR) update (PHR) update (EMR) Auxiliary InformationBackground Knowledge Social/Community Support Personal Context Personal Medical History (e.g., available sensors, (e.g., Electronic Medical Record,(e.g., Ontologies, (e.g., Patient Network, location, schedule) genomic sequence)Knowledgebases) crowd sourcing) 86
  • Continuous Monitoring Phase: Example Observed Symptoms Possible Explanations • Abnormal heart rate • Panic Disorder • Clammy skin • Hypoglycemia • Hyperthyroidism • Heart Attack • Septic Shock Electronic Medical Record Health Alert• Patient has history of Heart Disease • Check phone for instructions 87
  • Continuous Phase Technology Basis is a wrist-watch that also monitors pulse rate, movement, temperature, and galvanic skin response. 88
  • Continuous Phase Technology Fitbit Tracker uses a MEMS 3-axis accelerometer that measures your motion patterns to tell you your calories burned, steps taken, distance traveled, and sleep quality. 89
  • Personal Assessment Phase: Example Are you feeling lightheaded? yes Are you have trouble taking deep breaths? Observed Symptoms Possible Explanations • Abnormal heart rate • Panic Disorder yes • • Clammy skin Hypoglycemia • Lightheaded • Hyperthyroidism Do you have low blood pressure when • Trouble breathing • Heart Attack standing? • • Low blood pressure Septic Shock yes Have you taken your Methimazole medication? no Electronic Medical Record Health Alert • Patient has history of Hyperthyroidism 1. Take medication: Methimazole • Patient has prescription for Methimazole 2. See doctor: how about Tues. @ 11am? 90
  • Personal Phase Technology Lark is a sleep sensor that monitors circadian rhythms and functions as an "un- alarm," vibrating to wake you at a point in your sleep cycle when you feel alert, not groggy. 91
  • Personal Phase Technology Instant Heart Rate takes your pulse when you place your finger over your phone’s camera lens. The app uses light from the camera flash to detect color changes caused by blood moving through your finger. 92
  • Personal Phase Technology Telcare makes a blood glucose meter (right) for diabetics that broadcasts readings to a mobile-phone app (center) where patients can see results and set goals. 93
  • Personal Phase Technology iBGStar is a plug-in glucose meter for the iPhone, developed by Sanofi- Aventis, providing the ability for patients to monitor and manage Diabetes. 94
  • Personal Phase Technology Withings Blood Pressure Monitor provides easy and convenient blood pressure readings in the convenience of home. 95
  • Personal Phase Technology WebMD provides a wealth of health information and an application to diagnose symptoms. 96
  • Medical Services Phase: Example Are your blood sugar levels low? Observed Symptoms Possible Explanations • Abnormal heart rate • Hypoglycemia • Clammy skin • Hyperthyroidism • Lightheaded • Trouble breathing • Low blood pressure Electronic Medical Record• Patient has history of Hyperthyroidism• Patient has prescription for Methimazole 97
  • Medical Phase Technology Doctor. 98
  • Medical Phase Technology Health Guard provides a secure way to store and analyze health records for casual browsing or emergency use (i.e., MS Health Vault records). 99
  • Medical Phase Technology Mobile MIM gives physicians a sophisticated, hands-on mobile system for viewing and annotating radiology images, such as CT scans. 100
  • Medical Phase Technology Dr. Watson is a health and medical question and answering system developed by IBM, utilizing supercomputer intelligence for medical diagnostics. 101
  • CHE approach to Health Care Continuous Personal Medical Symptoms/ Symptoms/ Explanations Explanations access & access & access & update (PHR) update (PHR) update (EMR) Auxiliary InformationBackground Knowledge Social/Community Support Personal Context Personal Medical History (e.g., available sensors, (e.g., Electronic Medical Record,(e.g., Ontologies, (e.g., Patient location, schedule) genomic sequence)Knowledgebases) Network, crowd sourcing) 102
  • CHE holds the potential to revolutionize the practiceof health-care by embracing the relationship betweenourselves, our machines, and our health Improving the experience of health-care improves all other experiences 103
  • Physical-Cyber-Social Abstraction “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” – M. Weiser Integration Scalability 104
  • Computing for Human Experience thank you, and please visit us at http://knoesis.org Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing Wright State University, Dayton, Ohio, USA More: Vision Paper: Computing for HumanExperience:http://wiki.knoesis.org/index.php/Computing_For_Human_Experience 105
  • Image credits:Page 2:http://blogs.ebrandz.com/tag/social-networks/http://news.cnet.com/8301-13579_3-10141672-37.htmlhttp://www.thehindu.com/opinion/columns/sainath/article123884.ecehttp://www.samplestuff.com/2011/06/11/frugal-family-fun-nightsPage 3:http://www.naturewalls.org/category/stream/Page 4:http://www.cbi.umn.edu/about/babbage.htmlhttp://news.cnet.com/8301-13579_3-10141672-37.htmlhttps://fpm-www3.fpm.wisc.edu/safety/occupationalHealth/Ergonomics/LaptopErgonomics/LaptopErgonomicsQuickReference/tabid/106/Default.aspxPage 5:http://news.cnet.com/8301-13579_3-10141672-37.htmlhttps://fpm-www3.fpm.wisc.edu/safety/occupationalHealth/Ergonomics/LaptopErgonomics/LaptopErgonomicsQuickReference/tabid/106/Default.aspxhttp://jimilocker.info/?p=165Page 22:http://www.healthsciencestrategy.com/2011/04/will-mhealth-apps-and-devices-empower-epatients-for-wellness-and-disease-management-a-case-study-2/Page 27:http://www.fitbit.com/http://www.relativitycorp.com/socialnetworkmarketing/http://heinrich.house.gov/index.cfm?sectionid=11&parentid=2&sectiontree=2,11&itemid=131Page 46:http://depositphotos.com/4946293/stock-photo-Abstract-earth-puzzle.htmlPage 47:http://massthink.wordpress.com/2007/06/10/husserl-in-indubitable-response-to-descartes-and-kant/Page 61:http://www.ida.liu.se/~eriho/COCOM_M.htmhttp://www.idemployee.id.tue.nl/g.w.m.rauterberg/lecturenotes/ucd%20lecture-3/sld019.htmPage 79:http://thewvsr.com/doctors.htm