May 2021 snapshot of some of the Research and Collaborations in dHealth/personalized health, public health, epidemiology, biomedicine at the AI Institute of the University of South Carolina [AIISC]
Augmented Personalized Health: an explicit knowledge enhanced neurosymbolic d...Amit Sheth
Keynote at the SWAT4HCLS (Semantic Web Applications and Tools for Healthcare and Life Sciences), 12 Jan 2022. Event info:
http://www.swat4ls.org/workshops/leiden2022/keynotes/
Video: https://youtu.be/nwGAv9q2wsY
Healthcare as we know it is in the process of going through a massive change – from episodic to continuous, from disease-focused to wellness and quality of life focused, from clinic centric to anywhere a patient is, from clinician controlled to patient empowered, and from being driven by limited data to 360-degree, multimodal personal-public-population physical-cyber-social big data-driven. While the ability to create and capture data is already here, the upcoming innovations will be in converting this big data into smart data through contextual and personalized processing such that patients and clinicians can make better decisions and take timely actions. The exploitation of all relevant data, relevant medical knowledge, and explainable AI techniques will also support better communications between patients, clinicians, and virtual health assistants with higher-level abstractions (rather than low-level data) representing health choices, decisions and actions.
Augmented Personalized Healthcare (APH) strategy we are developing empowers patients with self-monitoring (collecting relevant data), self-appraisal (interpreting data in the patient’s context), self-management (assisting the patient in following personalized care plan to maintain health), to intervention (when the clinical help is needed) and disease progression tracking and prediction (http://bit.ly/AI-APH, http://bit.ly/APH-TED). We currently apply APH using mobile Apps and virtual health assistants for patients managing pediatric asthma (http://bit.ly/kAsthma), mental health, carbohydrate management for type 1 diabetes, hypertension, etc. In this talk, I will describe some of the technical components that incorporate context, personalization, and abstraction for supporting advanced capabilities such as patient engagement through meaningful question generation, chatbot safety, and explainable decision-making using knowledge-infused learning, a neurosymbolic AI strategy that utilizes many types and levels of explicit knowledge.
Augmented Personalized Health: dHealth approach to patient empowerment for ma...Amit Sheth
Web site: https://aihealth.ischool.utexas.edu/AIHealthWWW2021/index.html
Amit Sheth, Keynote at the International Workshop on AI in Health: Transferring and Integrating Knowledge for Better Health at The Web Conference 2021, 16 April 2021.
Abstract:
Healthcare as we know it is in the process of going through a massive change - from episodic to continuous, from disease-focused to wellness and quality of life focused, from clinic centric to anywhere a patient is, from clinician controlled to patient empowered, and from being driven by limited data to 360-degree, multimodal personal-public-population physical-cyber-social big data-driven. While the ability to create and capture data is already here, the upcoming innovations will be in converting this big data into smart data through contextual and personalized processing such that patients and clinicians can make better decisions and take timely actions. The exploitation of all relevant data, relevant medical knowledge, and AI techniques will extend and enhance human health and well-being.
Augmented Personalized Healthcare (APH) strategy as we have defined involves empowering patients with self-monitoring (collecting relevant data), self-appraisal (interpreting data in the patient's context), self-management (assisting the patient in following personalized care plan to maintain health), to intervention (when the clinical help is needed) and disease progression tracking and prediction (http://bit.ly/AI-APH, http://bit.ly/APH-TED). While we have early investigations for several diseases, we will share some experience (such as developing a digital phenotype) from pediatric asthma that involved an evaluation with ~200 patients (http://bit.ly/kAsthma).
kHealth: Semantic Multi-sensory Mobile Approach to Personalized Asthma CareAmit Sheth
P7: A New Paradigm for Health Care in the 21st Century
Scientific Session at AAAS2019 Annual Meeting
https://aaas.confex.com/aaas/2019/meetingapp.cgi/Session/21133
https://cra.org/ccc/ccc-at-aaas/2019-sessions/
Asthma is a chronic multifactorial disease and traditional clinical practice requires patients to meet their clinician in a timely yet infrequently meetings scheduled once in 3-6 months depending on the patient’s condition. The clinical diagnosis relies on the patient’s description of their current health condition. The patient’s description need not be accurate at times and may lack some important aspects needed for accurate diagnosis. We at Kno.e.sis work with clinicians and their pediatric asthma patients at the Dayton Children's Hospital to evaluate an IoT/mobileApp enabled personalized digital health management. We built a kHealth system for continuous monitoring and improved tracking of 30 parameters including the child’s symptoms, activities, sleep, and treatment adherence. It can allow precise determination of asthma triggers and a reliable assessment of medication compliance and effectiveness.
More at: https://aaas.confex.com/aaas/2019/meetingapp.cgi/Paper/23000
The university-wide AI Institute of UofSC (AIISC) is administratively part of the CEC with Dean Hossein Haj-Harriri as its chief patron. This presentation give a quick overview of the CEC.
https://maison-workshop.com/prof-amit-sheth/
Video: https://youtu.be/pRUXTuxm3as
Keynote at the 7th International Workshop on Mining Actionable Insights from Social Networks (MAISoN 2021) – Special Edition on Responsible, August 21, 2021 and is co-located with the 15th International Joint Conference on Artificial Intelligence (IJCAI 2021). Also ASONAM 2021.
With the increasing legalization of medical and recreational use of substances, more research is needed to understand the association between mental health and user behavior related to drug consumption. Specifically, drug overdose and substance use-related mental health issues have become two major topics that have been widely discussed on social media platforms. Big social media data has the potential to provide deeper insights about these associations to public health analysts for making policy decisions. Multiple national population surveys have found that about half of those who experience a mental health illness during their lives will also experience a substance use disorder and vice versa. The communications related to addiction and mental health are complex to process and understand given their language and contextual characteristics. Surface-level data analysis alone is not sufficient to understand the complex nature of relationships among the addiction and mental health context. Moreover, dark web vendors have been using social media as a new marketplace for drugs. Social media users also discuss the novel drugs emerging in dark web marketplaces and associated side effects/health conditions. These communications get complex when researchers try to annotate them or link them to a specific mental health entity. Considering the significant sensitivity of such communications and to protect user privacy on social media, a potential solution requires reliable algorithms for modeling such communications. We demonstrate the value of incorporating domain-specific knowledge in natural language understanding to identify the relationship between mental health and drug addiction. We discuss end-to-end knowledge-infused deep learning frameworks that leverage the pre-trained language representation model and domain-specific declarative knowledge source to extract entities and their relationships jointly. Our model is further tailored to focus on the entities mentioned in the sentence where ontology is used to locate the target entity’s position. We also demonstrate the capabilities of inclusion of the knowledge-aware representation in association with language models that can extract the Drug-Mental health condition associations.
Acknowledgments: Usha Lokala, Raminta Daniulaityte, Francois Lamy, Manas Gaur, Jyotishman Pathak, and collaborators on NIDA/NIH and NSF funded projects on Addiction and Mental Health.
http://wiki.aiisc.ai/index.php/Public_Health_Addictions_Research_at_AIISC
http://wiki.aiisc.ai/index.php/Modeling_Social_Behavior_for_Healthcare_Utilization_in_Depression
The corresponding video is at https://youtu.be/ztNHKLTHBrA AIISC conducts foundational and translational research in AI. In this talk, we review part of the AIISC's research in Social Good, Social Harm, and Public Health.
This talk was given to the UofSC College on Information and Communication.
Additional project details at http://wiki.aiisc.ai
Augmented Personalized Health: an explicit knowledge enhanced neurosymbolic d...Amit Sheth
Keynote at the SWAT4HCLS (Semantic Web Applications and Tools for Healthcare and Life Sciences), 12 Jan 2022. Event info:
http://www.swat4ls.org/workshops/leiden2022/keynotes/
Video: https://youtu.be/nwGAv9q2wsY
Healthcare as we know it is in the process of going through a massive change – from episodic to continuous, from disease-focused to wellness and quality of life focused, from clinic centric to anywhere a patient is, from clinician controlled to patient empowered, and from being driven by limited data to 360-degree, multimodal personal-public-population physical-cyber-social big data-driven. While the ability to create and capture data is already here, the upcoming innovations will be in converting this big data into smart data through contextual and personalized processing such that patients and clinicians can make better decisions and take timely actions. The exploitation of all relevant data, relevant medical knowledge, and explainable AI techniques will also support better communications between patients, clinicians, and virtual health assistants with higher-level abstractions (rather than low-level data) representing health choices, decisions and actions.
Augmented Personalized Healthcare (APH) strategy we are developing empowers patients with self-monitoring (collecting relevant data), self-appraisal (interpreting data in the patient’s context), self-management (assisting the patient in following personalized care plan to maintain health), to intervention (when the clinical help is needed) and disease progression tracking and prediction (http://bit.ly/AI-APH, http://bit.ly/APH-TED). We currently apply APH using mobile Apps and virtual health assistants for patients managing pediatric asthma (http://bit.ly/kAsthma), mental health, carbohydrate management for type 1 diabetes, hypertension, etc. In this talk, I will describe some of the technical components that incorporate context, personalization, and abstraction for supporting advanced capabilities such as patient engagement through meaningful question generation, chatbot safety, and explainable decision-making using knowledge-infused learning, a neurosymbolic AI strategy that utilizes many types and levels of explicit knowledge.
Augmented Personalized Health: dHealth approach to patient empowerment for ma...Amit Sheth
Web site: https://aihealth.ischool.utexas.edu/AIHealthWWW2021/index.html
Amit Sheth, Keynote at the International Workshop on AI in Health: Transferring and Integrating Knowledge for Better Health at The Web Conference 2021, 16 April 2021.
Abstract:
Healthcare as we know it is in the process of going through a massive change - from episodic to continuous, from disease-focused to wellness and quality of life focused, from clinic centric to anywhere a patient is, from clinician controlled to patient empowered, and from being driven by limited data to 360-degree, multimodal personal-public-population physical-cyber-social big data-driven. While the ability to create and capture data is already here, the upcoming innovations will be in converting this big data into smart data through contextual and personalized processing such that patients and clinicians can make better decisions and take timely actions. The exploitation of all relevant data, relevant medical knowledge, and AI techniques will extend and enhance human health and well-being.
Augmented Personalized Healthcare (APH) strategy as we have defined involves empowering patients with self-monitoring (collecting relevant data), self-appraisal (interpreting data in the patient's context), self-management (assisting the patient in following personalized care plan to maintain health), to intervention (when the clinical help is needed) and disease progression tracking and prediction (http://bit.ly/AI-APH, http://bit.ly/APH-TED). While we have early investigations for several diseases, we will share some experience (such as developing a digital phenotype) from pediatric asthma that involved an evaluation with ~200 patients (http://bit.ly/kAsthma).
kHealth: Semantic Multi-sensory Mobile Approach to Personalized Asthma CareAmit Sheth
P7: A New Paradigm for Health Care in the 21st Century
Scientific Session at AAAS2019 Annual Meeting
https://aaas.confex.com/aaas/2019/meetingapp.cgi/Session/21133
https://cra.org/ccc/ccc-at-aaas/2019-sessions/
Asthma is a chronic multifactorial disease and traditional clinical practice requires patients to meet their clinician in a timely yet infrequently meetings scheduled once in 3-6 months depending on the patient’s condition. The clinical diagnosis relies on the patient’s description of their current health condition. The patient’s description need not be accurate at times and may lack some important aspects needed for accurate diagnosis. We at Kno.e.sis work with clinicians and their pediatric asthma patients at the Dayton Children's Hospital to evaluate an IoT/mobileApp enabled personalized digital health management. We built a kHealth system for continuous monitoring and improved tracking of 30 parameters including the child’s symptoms, activities, sleep, and treatment adherence. It can allow precise determination of asthma triggers and a reliable assessment of medication compliance and effectiveness.
More at: https://aaas.confex.com/aaas/2019/meetingapp.cgi/Paper/23000
The university-wide AI Institute of UofSC (AIISC) is administratively part of the CEC with Dean Hossein Haj-Harriri as its chief patron. This presentation give a quick overview of the CEC.
https://maison-workshop.com/prof-amit-sheth/
Video: https://youtu.be/pRUXTuxm3as
Keynote at the 7th International Workshop on Mining Actionable Insights from Social Networks (MAISoN 2021) – Special Edition on Responsible, August 21, 2021 and is co-located with the 15th International Joint Conference on Artificial Intelligence (IJCAI 2021). Also ASONAM 2021.
With the increasing legalization of medical and recreational use of substances, more research is needed to understand the association between mental health and user behavior related to drug consumption. Specifically, drug overdose and substance use-related mental health issues have become two major topics that have been widely discussed on social media platforms. Big social media data has the potential to provide deeper insights about these associations to public health analysts for making policy decisions. Multiple national population surveys have found that about half of those who experience a mental health illness during their lives will also experience a substance use disorder and vice versa. The communications related to addiction and mental health are complex to process and understand given their language and contextual characteristics. Surface-level data analysis alone is not sufficient to understand the complex nature of relationships among the addiction and mental health context. Moreover, dark web vendors have been using social media as a new marketplace for drugs. Social media users also discuss the novel drugs emerging in dark web marketplaces and associated side effects/health conditions. These communications get complex when researchers try to annotate them or link them to a specific mental health entity. Considering the significant sensitivity of such communications and to protect user privacy on social media, a potential solution requires reliable algorithms for modeling such communications. We demonstrate the value of incorporating domain-specific knowledge in natural language understanding to identify the relationship between mental health and drug addiction. We discuss end-to-end knowledge-infused deep learning frameworks that leverage the pre-trained language representation model and domain-specific declarative knowledge source to extract entities and their relationships jointly. Our model is further tailored to focus on the entities mentioned in the sentence where ontology is used to locate the target entity’s position. We also demonstrate the capabilities of inclusion of the knowledge-aware representation in association with language models that can extract the Drug-Mental health condition associations.
Acknowledgments: Usha Lokala, Raminta Daniulaityte, Francois Lamy, Manas Gaur, Jyotishman Pathak, and collaborators on NIDA/NIH and NSF funded projects on Addiction and Mental Health.
http://wiki.aiisc.ai/index.php/Public_Health_Addictions_Research_at_AIISC
http://wiki.aiisc.ai/index.php/Modeling_Social_Behavior_for_Healthcare_Utilization_in_Depression
The corresponding video is at https://youtu.be/ztNHKLTHBrA AIISC conducts foundational and translational research in AI. In this talk, we review part of the AIISC's research in Social Good, Social Harm, and Public Health.
This talk was given to the UofSC College on Information and Communication.
Additional project details at http://wiki.aiisc.ai
Towards Smart Chatbots for Enhanced Health: Using Multisensory Sensing & Sem...Amit Sheth
https://sites.google.com/view/deep-dial-2019/keynotes
Understanding and managing health is complex. Throughout the last few decades of modern medicine, we have relied clinicians on most health-related decision making. New technologies have enabled a growing involvement of patients in their own health management, aided by increasing variety and amount of patient-generated health data. Augmented personalized health [http://bit.ly/k-APH, http://bit.ly/APH-HI] strategy has outlined a broad variety of patient and clinician engagement in devising an increasingly more sophisticated and powerful health management solutions - from self-monitoring, self-appraisal, self-management, intervention to the prediction of disease progression and planning. Chatbot could play a pivotal role throughout the unfolding data-driven, AI-supported ecosystem [http://bit.ly/H-Chatbot] that engages patients and clinicians in collecting data, in driving their actions, informing them of their choices, and even delivering part of the clinical care (e.g., Cognitive Behavioral Therapy (CBT) for mental health patients). Nevertheless, this will require quite a few advances in making a more intelligent technology. In this talk, we will share some experience and observations based on our ongoing collaborative projects that usually involve clinicians and patients targeting pediatric asthma management, pre-and-post bariatric surgery care regimen, depression and other mental health issues, and nutrition. Using use cases and prototypes, we will elucidate the need, support, and use of domain- and user-specific knowledge graphs, Natural Language Processing (NLP), machine learning, and conversational AI for:
- multimodal interactions including text, voice, and other media, along with the use of diverse devices and software platforms for “natural” communication
- context enabled by deep relevant medical/healthcare knowledge including clinical protocols
- personalization by collecting and using the history of the individual patient from IoT health devices, open data, and Electronic Medical Record (EMR)
- abstraction by aggregating and correlating diverse streams data to draw plausible explanation(s) based on public (cohort-level) data (for example percentage of asthmatic patient who gets symptom when exposed to certain triggers) and personal data
- smart dialogue (intent) management and response generations by causal relations and inference of association
Computing for Human Experience: Semantics empowered Cyber-Physical, Social an...Amit Sheth
Keynote at On the Move conference, October 2011, Greece.
Abstract:
Traditionally, we had to artificially simplify the complexity and richness of the real world to constrained computer models and languages for more efficient computation. Today, devices, sensors, human-in-the-loop participation and social interactions enable something more than a “human instructs machine” paradigm. Web as a system for information sharing is being replaced by pervasive computing with mobile, social, sensor and devices dominated interactions. Correspondingly, computing is moving from targeted tasks focused on improving efficiency and productivity to a vastly richer context that support events and situational awareness, and enrich human experiences encompassing recognition of rich sets of relationships, events and situational awareness with spatio-temporal-thematic elements, and socio-cultural-behavioral facets. Such progress positions us for what I call an emerging era of “computing for human experience” (CHE). Four of the key enablers of CHE are: (a) bridging the physical/digital (cyber) divide, (b) elevating levels of abstractions and utilizing vast background knowledge to enable integration of machine and human perception, (c) convert raw data and observations, ranging from sensors to social media, into understanding of events and situations that are meaningful to humans, and (d) doing all of the above at massive scale covering the Web and pervasive computing supported humanity. Semantic Web (conceptual models/ontologies and background knowledge, annotations, and reasoning) techniques and technologies play a central role in important tasks such as building context, integrating online and offline interactions, and help enhance human experience in their natural environment.
In this talk I will discuss early enablers of CHE including semantics-empowered social networking and sensor Web, and computation of higher level abstractions from raw and phenomenological data. An article in IEEE Internet Computing provides background information: http://bit.ly/HumanExperience
Keynote at: https://www.springer.com/us/book/9783642251054
Event Date: Oct 18, 2011
Wide adoption of smartphones and availability of low-cost sensors has resulted in seamless and continuous monitoring of physiology, environment, and public health notifications. However, personalized digital health and patient empowerment can become a reality only if the complex multisensory and multimodal data is processed within the patient context. Contextual processing of patient data along with personalized medical knowledge can lead to actionable information for better and timely decisions. We present a system called kHealth capable of aggregating multisensory and multimodal data from sensors (passive sensing) and answers to questionnaire (active sensing) from patients with asthma. We present our preliminary data analysis comprising data collected from real patients highlighting the challenges in deploying such an application. The results show strong promise to derive actionable information using a combination of physiological indicators from active and passive sensors that can help doctors determine more precisely the cause, severity, and control level of asthma. Information synthesized from kHealth can be used to alert patients and caregivers for seeking timely clinical assistance to better manage asthma and improve their quality of life.
Paper: http://www.knoesis.org/library/resource.php?id=2153
Citation:
Pramod Anantharam, Tanvi Banerjee, Amit Sheth, Krishnaprasad Thirunarayan, Surendra Marupudi, Vaikunth Sridharan, Shalini G. Forbis, Knowledge-driven Personalized Contextual mHealth Service for Asthma Management in Children , IEEE 4th International Conference on Mobile Services, June 27 - July 2, 2015, New York, USA.
kHealth Bariatrics is an effort to bout against weight recidivism post bariatric surgery. The computer scientists working at Kno.e.sis, an Ohio Center of Excellence in BioHealth Innovation, are collaborating with a bariatric surgeon and a behavioural specialist to bolster weight loss surgery patients for appropriate postsurgical progress.
Digital medicine comes of age - ISDM E-Newsletter Feb 2020David Wortley
Consumer digital technologies such as wearables and VR/AR are now being applied to diagnose, treat and manage clinical conditions. The ISDM Feb 2020 E-Newsletter shows some examples
Analyzing Consumer Reaction to the Fungal Meningitis Outbreak in Real-TimeEnspektos, LLC
This report provides an overview of initial research focusing on how digital health consumers responded to online content related to the ongoing meningitis outbreak sparked by contaminated injections developed by the New England Compounding Center.
Applied Artificial Intelligence & How it's Transforming Life SciencesKumaraguru Veerasamy
In this SlideShare, we cover an overview history of artificial intelligence (AI), before exploring its applications in healthcare, biotechnology & pharmaceuticals. The slides will also cover the market outlook of AI, and how big pharmaceutical companies are investing in the technology. In addition, there are a couple of case studies on applied AI, namely in genomics and liquid biopsy (glycoproteomics).
디지털 헬스케어 기반의 능동적, 선제적 보험
수동적, 사후적 대응에서 능동적, 선제적 관리로의 변화
- 디지털 헬스케어 기반의 가입자 데이터의 측정
- 데이터 분석을 통한 가입자 관리: 질병 위험군 분류, 계리
- 질병 관리 및 치료에 대한 능동적 개입: 관리 방안 및 인센티브
Augmented Personalized Health: using AI techniques on semantically integrated...Amit Sheth
Keynote @ 2018 AAAI Joint Workshop on Health Intelligence (W3PHIAI 2018), 2 February 2018, New Orleans, LA [Video: https://youtu.be/GujvoWRa0O8]
Related article: https://ieeexplore.ieee.org/document/8355891/
Abstract
Healthcare as we know it is in the process of going through a massive change - from episodic to continuous, from disease-focused to wellness and quality of life focused, from clinic centric to anywhere a patient is, from clinician controlled to patient empowered, and from being driven by limited data to 360-degree, multimodal personal-public-population physical-cyber-social big data-driven. While the ability to create and capture data is already here, the upcoming innovations will be in converting this big data into smart data through contextual and personalized processing such that patients and clinicians can make better decisions and take timely actions for augmented personalized health. In this talk, we will discuss how use of AI techniques on semantically integrated patient-generated health data (PGHD), environmental data, clinical data, and public social data is exploited to achieve a range of augmented health management strategies that include self-monitoring, self-appraisal, self-management, intervention, and Disease Progression Tracking and Prediction. We will review examples and outcomes from a number of applications, some involving patient evaluations, including asthma in children, bariatric surgery/obesity, mental health/depression, that are part of the Kno.e.sis kHealth personalized digital health initiative.
Background: Background: http://bit.ly/k-APH, http://bit.ly/kAsthma, http://j.mp/PARCtalk
Improving health care outcomes with responsible data scienceWessel Kraaij
Keynote presentation by Wessel Kraaij at the Dutch pattern recognition and impage processing society (NVPBV) 29/5/2018, Eindhoven.
This talk discusses
1. trends in health care and respondible data science and their intersection
2. Secure federated analytics on distributed data repositories
3. Generating clinically relevant hypotheses from patient forum discussions.
Towards Smart Chatbots for Enhanced Health: Using Multisensory Sensing & Sem...Amit Sheth
https://sites.google.com/view/deep-dial-2019/keynotes
Understanding and managing health is complex. Throughout the last few decades of modern medicine, we have relied clinicians on most health-related decision making. New technologies have enabled a growing involvement of patients in their own health management, aided by increasing variety and amount of patient-generated health data. Augmented personalized health [http://bit.ly/k-APH, http://bit.ly/APH-HI] strategy has outlined a broad variety of patient and clinician engagement in devising an increasingly more sophisticated and powerful health management solutions - from self-monitoring, self-appraisal, self-management, intervention to the prediction of disease progression and planning. Chatbot could play a pivotal role throughout the unfolding data-driven, AI-supported ecosystem [http://bit.ly/H-Chatbot] that engages patients and clinicians in collecting data, in driving their actions, informing them of their choices, and even delivering part of the clinical care (e.g., Cognitive Behavioral Therapy (CBT) for mental health patients). Nevertheless, this will require quite a few advances in making a more intelligent technology. In this talk, we will share some experience and observations based on our ongoing collaborative projects that usually involve clinicians and patients targeting pediatric asthma management, pre-and-post bariatric surgery care regimen, depression and other mental health issues, and nutrition. Using use cases and prototypes, we will elucidate the need, support, and use of domain- and user-specific knowledge graphs, Natural Language Processing (NLP), machine learning, and conversational AI for:
- multimodal interactions including text, voice, and other media, along with the use of diverse devices and software platforms for “natural” communication
- context enabled by deep relevant medical/healthcare knowledge including clinical protocols
- personalization by collecting and using the history of the individual patient from IoT health devices, open data, and Electronic Medical Record (EMR)
- abstraction by aggregating and correlating diverse streams data to draw plausible explanation(s) based on public (cohort-level) data (for example percentage of asthmatic patient who gets symptom when exposed to certain triggers) and personal data
- smart dialogue (intent) management and response generations by causal relations and inference of association
Computing for Human Experience: Semantics empowered Cyber-Physical, Social an...Amit Sheth
Keynote at On the Move conference, October 2011, Greece.
Abstract:
Traditionally, we had to artificially simplify the complexity and richness of the real world to constrained computer models and languages for more efficient computation. Today, devices, sensors, human-in-the-loop participation and social interactions enable something more than a “human instructs machine” paradigm. Web as a system for information sharing is being replaced by pervasive computing with mobile, social, sensor and devices dominated interactions. Correspondingly, computing is moving from targeted tasks focused on improving efficiency and productivity to a vastly richer context that support events and situational awareness, and enrich human experiences encompassing recognition of rich sets of relationships, events and situational awareness with spatio-temporal-thematic elements, and socio-cultural-behavioral facets. Such progress positions us for what I call an emerging era of “computing for human experience” (CHE). Four of the key enablers of CHE are: (a) bridging the physical/digital (cyber) divide, (b) elevating levels of abstractions and utilizing vast background knowledge to enable integration of machine and human perception, (c) convert raw data and observations, ranging from sensors to social media, into understanding of events and situations that are meaningful to humans, and (d) doing all of the above at massive scale covering the Web and pervasive computing supported humanity. Semantic Web (conceptual models/ontologies and background knowledge, annotations, and reasoning) techniques and technologies play a central role in important tasks such as building context, integrating online and offline interactions, and help enhance human experience in their natural environment.
In this talk I will discuss early enablers of CHE including semantics-empowered social networking and sensor Web, and computation of higher level abstractions from raw and phenomenological data. An article in IEEE Internet Computing provides background information: http://bit.ly/HumanExperience
Keynote at: https://www.springer.com/us/book/9783642251054
Event Date: Oct 18, 2011
Wide adoption of smartphones and availability of low-cost sensors has resulted in seamless and continuous monitoring of physiology, environment, and public health notifications. However, personalized digital health and patient empowerment can become a reality only if the complex multisensory and multimodal data is processed within the patient context. Contextual processing of patient data along with personalized medical knowledge can lead to actionable information for better and timely decisions. We present a system called kHealth capable of aggregating multisensory and multimodal data from sensors (passive sensing) and answers to questionnaire (active sensing) from patients with asthma. We present our preliminary data analysis comprising data collected from real patients highlighting the challenges in deploying such an application. The results show strong promise to derive actionable information using a combination of physiological indicators from active and passive sensors that can help doctors determine more precisely the cause, severity, and control level of asthma. Information synthesized from kHealth can be used to alert patients and caregivers for seeking timely clinical assistance to better manage asthma and improve their quality of life.
Paper: http://www.knoesis.org/library/resource.php?id=2153
Citation:
Pramod Anantharam, Tanvi Banerjee, Amit Sheth, Krishnaprasad Thirunarayan, Surendra Marupudi, Vaikunth Sridharan, Shalini G. Forbis, Knowledge-driven Personalized Contextual mHealth Service for Asthma Management in Children , IEEE 4th International Conference on Mobile Services, June 27 - July 2, 2015, New York, USA.
kHealth Bariatrics is an effort to bout against weight recidivism post bariatric surgery. The computer scientists working at Kno.e.sis, an Ohio Center of Excellence in BioHealth Innovation, are collaborating with a bariatric surgeon and a behavioural specialist to bolster weight loss surgery patients for appropriate postsurgical progress.
Digital medicine comes of age - ISDM E-Newsletter Feb 2020David Wortley
Consumer digital technologies such as wearables and VR/AR are now being applied to diagnose, treat and manage clinical conditions. The ISDM Feb 2020 E-Newsletter shows some examples
Analyzing Consumer Reaction to the Fungal Meningitis Outbreak in Real-TimeEnspektos, LLC
This report provides an overview of initial research focusing on how digital health consumers responded to online content related to the ongoing meningitis outbreak sparked by contaminated injections developed by the New England Compounding Center.
Applied Artificial Intelligence & How it's Transforming Life SciencesKumaraguru Veerasamy
In this SlideShare, we cover an overview history of artificial intelligence (AI), before exploring its applications in healthcare, biotechnology & pharmaceuticals. The slides will also cover the market outlook of AI, and how big pharmaceutical companies are investing in the technology. In addition, there are a couple of case studies on applied AI, namely in genomics and liquid biopsy (glycoproteomics).
디지털 헬스케어 기반의 능동적, 선제적 보험
수동적, 사후적 대응에서 능동적, 선제적 관리로의 변화
- 디지털 헬스케어 기반의 가입자 데이터의 측정
- 데이터 분석을 통한 가입자 관리: 질병 위험군 분류, 계리
- 질병 관리 및 치료에 대한 능동적 개입: 관리 방안 및 인센티브
Augmented Personalized Health: using AI techniques on semantically integrated...Amit Sheth
Keynote @ 2018 AAAI Joint Workshop on Health Intelligence (W3PHIAI 2018), 2 February 2018, New Orleans, LA [Video: https://youtu.be/GujvoWRa0O8]
Related article: https://ieeexplore.ieee.org/document/8355891/
Abstract
Healthcare as we know it is in the process of going through a massive change - from episodic to continuous, from disease-focused to wellness and quality of life focused, from clinic centric to anywhere a patient is, from clinician controlled to patient empowered, and from being driven by limited data to 360-degree, multimodal personal-public-population physical-cyber-social big data-driven. While the ability to create and capture data is already here, the upcoming innovations will be in converting this big data into smart data through contextual and personalized processing such that patients and clinicians can make better decisions and take timely actions for augmented personalized health. In this talk, we will discuss how use of AI techniques on semantically integrated patient-generated health data (PGHD), environmental data, clinical data, and public social data is exploited to achieve a range of augmented health management strategies that include self-monitoring, self-appraisal, self-management, intervention, and Disease Progression Tracking and Prediction. We will review examples and outcomes from a number of applications, some involving patient evaluations, including asthma in children, bariatric surgery/obesity, mental health/depression, that are part of the Kno.e.sis kHealth personalized digital health initiative.
Background: Background: http://bit.ly/k-APH, http://bit.ly/kAsthma, http://j.mp/PARCtalk
Improving health care outcomes with responsible data scienceWessel Kraaij
Keynote presentation by Wessel Kraaij at the Dutch pattern recognition and impage processing society (NVPBV) 29/5/2018, Eindhoven.
This talk discusses
1. trends in health care and respondible data science and their intersection
2. Secure federated analytics on distributed data repositories
3. Generating clinically relevant hypotheses from patient forum discussions.
Kno.e.sis Approach to Impactful Research & Training for Exceptional CareersAmit Sheth
Abstract
Kno.e.sis (http://knoesis.org) is a world-class research center that uses semantic, cognitive, and perceptual computing for gathering insights from physical/IoT, cyber/Web, and social and enterprise (e.g., clinical) big data. We innovate and employ semantic web, machine learning, NLP/IR, data mining, network science and highly scalable computing techniques. Our highly interdisciplinary research impacts health and clinical applications, biomedical and translational research, epidemiology, cognitive science, social good, policy, development, etc. A majority of our $12+ million in active funds come from the NSF and NIH. In this talk, I will provide an overview of some of our major research projects.
Kno.e.sis is highly successful in its primary mission of exceptional student outcomes: our students have exceptional publication and real-world impact and our PhDs compete with their counterparts from top 10 schools for initial jobs in research universities, top industry research labs, and highly competitive companies. A key reason for Kno.e.sis' success is its unique work culture involving teamwork to solve complex problems. Practically all our work involves real-world challenges, real-world data, interdisciplinary collaborators, path-breaking research to solve challenges, real-world deployments, real-world use, and measurable real-world impact.
In this talk, I will also seek to discuss our choice of research topics and our unique ecosystem that prepares our students for exceptional careers.
Trusted! Quest for data-driven and fair health solutions Sitra / Hyvinvointi
An inspiring online event on 3 February 2021. We are discussing the future of data-driven health solutions that focus on fairness for all stakeholders: people, business and the public sector. We are asking questions such as: What is fairness in health? What role does trust play in data-driven health services? What needs to change and who needs to act? Most of all, we are launching “The Fair Health Data Challenge“.
Event speakers:
- Jaana Sinipuro, Project Director, IHAN – Human-driven data economy, Sitra
- Dipak Kalra, President, The European Institute for Innovation through Health Data (i~HD)
- Pekka Kahri, Technology Officer, HUS Helsinki University Hospital
- Markus Kalliola, Project Director, Health data 2030, Sitra
- Tiina Härkönen, Leading Specialist, Sitra
Precision and Participatory Medicine - Medinfo 2015 Panel on big data. Includes the proposal to use the term Expotype to characterise the Exposome of an individual. Electronic expo typing would refer to the automatic construction of individual expo types from electronic clinical records and other sources of environmental risk factor and exposure data.
Karen Day, University of Auckland
Koray Atalag, University of Auckland
Denise Irvine, e3health
Bryan Houliston, Auckland University of Technology
(4/11/10, Illott, 1.45)
Overview of Health Informatics: survey of fundamentals of health information technology, Identify the forces behind health informatics, educational and career opportunities in health informatics.
http://iwma.lnmiit.ac.in/speakers.html
Third International Workshop on Multimedia Applications ( IWMA ), March 02-06, 2021.
The Holy Grail of machine intelligence is the ability to mimic the human brain. In computing, we have created silos in dealing with each modality (text/language processing, speech processing, image processing, video processing, etc.). However, the human brain’s cognitive and perceptual capability to seamlessly consume (listen and see) and communicate (writing/typing, voice, gesture) multimodal (text, image, video, etc.) information challenges machine intelligence research. Emerging chatbots for demanding health applications present the requirements for these capabilities. To support the corresponding data analysis and reasoning needs, we have explored a pedagogical framework consisting of semantic computing, cognitive computing, and perceptual computing. In particular, we have been motivated by the brain’s amazing perceptive power that abstracts massive amounts of multimodal data by filtering and processing them into a few concepts (representable by a few bits) to act upon. From the information processing perspective, this requires moving from syntactic and semantic big data processing to actionable information that can be weaved naturally into human activities and experience.
Exploration of the above research agenda, including powerful use cases, is afforded in a growing number of emerging technologies and their applications - such as chatbots and robotics for healthcare. In this talk, I will provide these examples and share the early progress we have made towards building health chatbots that consume contextually relevant multimodal data and support different forms/modalities of interactions to achieve various alternatives for digital health. I will also demonstrate the strong role of domain knowledge and personalization using domain and personalized knowledge graphs as part of various reasoning and learning techniques.
From personal health data to a personalized adviceWessel Kraaij
Invited talk at the health track of ICT.OPEN 2018, 20-3-2018
1. Related Data science challenges to Digital Health trends
2. Designing an infrastructure to support secure learning from distributed health data repositories, for personalized health advice
3. Supporting patients with rare diseases with patient driven research and the generation of new hypotheses based on patient experiences.
Data Science in Healthcare" by authors Sergio Consoli, Diego Reforgiato Recupero, and Milan Petkovic is an insightful guide that delves into the intersection of data science and healthcare. As a first-year student in Pharmaceutical Management, I found this book to be a valuable resource for understanding how data-driven approaches are transforming the healthcare industry, offering fresh perspectives and practical insights for future professionals like myself.
The Digital Health Society (by Julien Venne) @ICT2018 Vienna 6th Dec 2018Julien VENNE
The Digital Health Society is a movement involving all stakeholders innovating for a better health and wellbeing of citizens. Presentation done by Julien Venne at the ICT2018 organised by the European Commission in Vienna in December 2018. Learn about and join the movement on www.thedigitalhealthsociety.com
HEC 2016 Panel: Putting User-Generated Data in Action: Improving Interpretabi...Pei-Yun Sabrina Hsueh
Chair/Moderator: Pei-Yun Sabrina HSUEH, PhD (IBM T.J. Watson Research Center)
Panelists: XinXin ZHU, Bian YANG, Ying-Kuen CHEUNG , Thomas WETTER, and Sanjoy DEY
a IBM T.J. Watson Research Center, USA
b Norwegian University of Science and Technology, Norway
c Mailman School of Public health, Columbia University, USA
d, Department of Biomedical Informatics, University of Washington, USA
e Department of Medical Informatics, University of Heidelberg, Germany
The rise of consumer health awareness and the recent advent of personal health management tools (including mobile and health wearable devices) have contributed to another shift transforming the healthcare landscape. Despite the rise of health consumers, the impact of user-generated health data remains to be validated. In fact, many applications are hinged on the interpretability issues of this sort of data. The aim of this panel is two-fold. First, this panel aims to review the key dimensions in the interpretability, spanning from quality and reliability to information security and trust management. Secondly, since similar issues and methodologies have been proposed in different application areas ranging from clinical decision support to behavioral interventions and clinical trials, the panelists will also discuss both the success stories and the areas that fall short. The opportunities and barriers identified can then serve as guidelines or action items individuals can bring to their organizations to further improve the interpretability of user-generated data.
Trauma Outpatient Center is a comprehensive facility dedicated to addressing mental health challenges and providing medication-assisted treatment. We offer a diverse range of services aimed at assisting individuals in overcoming addiction, mental health disorders, and related obstacles. Our team consists of seasoned professionals who are both experienced and compassionate, committed to delivering the highest standard of care to our clients. By utilizing evidence-based treatment methods, we strive to help our clients achieve their goals and lead healthier, more fulfilling lives.
Our mission is to provide a safe and supportive environment where our clients can receive the highest quality of care. We are dedicated to assisting our clients in reaching their objectives and improving their overall well-being. We prioritize our clients' needs and individualize treatment plans to ensure they receive tailored care. Our approach is rooted in evidence-based practices proven effective in treating addiction and mental health disorders.
Dr. David Greene R3 stem cell Breakthroughs: Stem Cell Therapy in CardiologyR3 Stem Cell
Dr. David Greene, founder and CEO of R3 Stem Cell, is at the forefront of groundbreaking research in the field of cardiology, focusing on the transformative potential of stem cell therapy. His latest work emphasizes innovative approaches to treating heart disease, aiming to repair damaged heart tissue and improve heart function through the use of advanced stem cell techniques. This research promises not only to enhance the quality of life for patients with chronic heart conditions but also to pave the way for new, more effective treatments. Dr. Greene's work is notable for its focus on safety, efficacy, and the potential to significantly reduce the need for invasive surgeries and long-term medication, positioning stem cell therapy as a key player in the future of cardiac care.
We are one of the top Massage Spa Ajman Our highly skilled, experienced, and certified massage therapists from different corners of the world are committed to serving you with a soothing and relaxing experience. Luxuriate yourself at our spas in Sharjah and Ajman, which are indeed enriched with an ambiance of relaxation and tranquility. We could confidently claim that we are one of the most affordable Spa Ajman and Sharjah as well, where you can book the massage session of your choice for just 99 AED at any time as we are open 24 hours a day, 7 days a week.
Visit : https://massagespaajman.com/
Call : 052 987 1315
Stem Cell Solutions: Dr. David Greene's Path to Non-Surgical Cardiac CareDr. David Greene Arizona
Explore the groundbreaking work of Dr. David Greene, a pioneer in regenerative medicine, who is revolutionizing the field of cardiology through stem cell therapy in Arizona. This ppt delves into how Dr. Greene's innovative approach is providing non-surgical, effective treatments for heart disease, using the body's own cells to repair heart damage and improve patient outcomes. Learn about the science behind stem cell therapy, its benefits over traditional cardiac surgeries, and the promising future it holds for modern medicine. Join us as we uncover how Dr. Greene's commitment to stem cell research and therapy is setting new standards in healthcare and offering new hope to cardiac patients.
Rate Controlled Drug Delivery Systems, Activation Modulated Drug Delivery Systems, Mechanically activated, pH activated, Enzyme activated, Osmotic activated Drug Delivery Systems, Feedback regulated Drug Delivery Systems systems are discussed here.
The dimensions of healthcare quality refer to various attributes or aspects that define the standard of healthcare services. These dimensions are used to evaluate, measure, and improve the quality of care provided to patients. A comprehensive understanding of these dimensions ensures that healthcare systems can address various aspects of patient care effectively and holistically. Dimensions of Healthcare Quality and Performance of care include the following; Appropriateness, Availability, Competence, Continuity, Effectiveness, Efficiency, Efficacy, Prevention, Respect and Care, Safety as well as Timeliness.
Under Pressure : Kenneth Kruk's StrategyKenneth Kruk
Kenneth Kruk's story of transforming challenges into opportunities by leading successful medical record transitions and bridging scientific knowledge gaps during COVID-19.
Navigating Challenges: Mental Health, Legislation, and the Prison System in B...Guillermo Rivera
This conference will delve into the intricate intersections between mental health, legal frameworks, and the prison system in Bolivia. It aims to provide a comprehensive overview of the current challenges faced by mental health professionals working within the legislative and correctional landscapes. Topics of discussion will include the prevalence and impact of mental health issues among the incarcerated population, the effectiveness of existing mental health policies and legislation, and potential reforms to enhance the mental health support system within prisons.
ALKAMAGIC PLAN 1350.pdf plan based of door to door delivery of alkaline water...rowala30
Alka magic plan 1350 -we deliver alkaline water at your door step and you can make handsome money by referral programme
we also help and provide systematic guideline to setup 1000 lph alkaline water plant
ALKAMAGIC PLAN 1350.pdf plan based of door to door delivery of alkaline water...
AI & Healthcare @ AIISC: May 2021 Snapshot
1. Healthcare Research @ AI Institute:
dHealth, public health, epidemiology, biomedicine
Overview Presentation to the MUSC’s AI Hub and others
May 2021
Amit Sheth, Founding Director http://aiisc.ai
3. “
3
AIISC in core AI areas, and
interdisciplinary AI/AI applications
>> 25 researchers
including 4 faculty
(6 in Fall 2021), 2-3
postdocs, ~20 PhD
students, >10
MS/BS and several
interns/associates
5. D-Health, health informatics, public health, epidemiology –
sample collaborations (Pending & PLANNED Submissions only)
•College of Medicine/Prisma/Prisma-Upstate
•Mental health [M Natarajan], Addiction [A. Litwin], Asthma [R. Dawson], Diabetes & Obesity
[L. Knight], Hypertension - Diet & Nutrition [S. Donevant], Neutropenia [S. Craemer]
•College of Pharmacy: EOCRC [P. Backhaults, L. Hofseth, et al]
•College of Nursing: COVID-19 mobile app [R. Hughes, S. Donevant], mental health
chatbot [R. Hughes, S. Donevant, P. Raynor]
•Arnold School of Public Health: Mental Health [S. Qiao], Healthcare Big Data -
Education & Training [X. Li, et. al.]
•USCAND, Inst of Mind & Brain: Neuroscience [R. Desai] and neurodevelopmental
diseases [J. Bradshaw, J. Roberts]
•CEC- IIT, BME, CSE: Health IT/Smart Health [E. Regan], UI/UX [D. Wu]; Health
mApp [N. Boltin], (several in CSE).
6. Projects
➢ KHealth: Semantic Multisensory Mobile Approach to Personalized Asthma Care (NICHD)
➢ mHealth to Improve Carbohydrate Counting Accuracy in Pediatric Type 1 Diabetes
➢ Improving mental health of COVID-19 patients with an Artificial Intelligence-based chatbot
➢ Personalized Virtual Health Assistant Enabled by Knowledge-infused Reinforcement Learning for Adaptive
Mental Health Self-management
➢ Characterizing and supporting help seekers on social media using expert-in-the-loop learning
➢ Modeling Social Behavior for Healthcare Utilization in Depression (NIMH)
➢ eDarkTrends : Monitoring Cryptomarkets to Identify Emerging Trends of Illicit Synthetic Opioids Use
(NIDA)
➢ Trending: Social media analysis to monitor cannabis and synthetic cannabinoid use (NIDA)
➢ Innovative NIDA National Early Warning System Network (iN3) (NIDA)
➢ PREDOSE: PREscription Drug abuse Online Surveillance and Epidemiology (NIDA)
➢ Community-Driven Data Engineering for Substance Abuse Prevention in the Rural Midwest (NSF)
➢ Discrepancies in Diagnosis and Treatment of Cardiovascular Disease Based on Sex and Gender to Improve
Women’s Health (NHLBI)
➢ Early Onset of Colorectal Cancer
➢ Digestive Inflation Index
➢ more ...
7. Types of Healthcare Data
◎ EMR
◎ Social Media (Reddit,
Twitter,Web Forums)
◎ Conversations: Patient-
Clinician, Virtual Health
Assistant-Patient
◎ Patient Generated:
Wearable/sensor data,
mApp data
◎ Images: food, fMRI
AI Techniques and Technologies
◎ Knowledge Graphs/Ontologies
(contextualization, personalization,
abstraction)
◎ NLP/NLU
◎ Machine Learning/Deep Learning
(RL, GAN, CNN, LSTM,....)
◎ Conversational AI, Q/A
◎ mApp, Virtual Health Assistants
(Chatbots)
◎ Health sensors/IoTs/mobile
devices
8. Medical Conditions/Healthcare Challenges addressed
◎ Asthma
◎ Mental Health
◎ Addiction
◎ COVID-19
◎ Cardiovascular Disease
◎ Type 1 Diabetes in Children
◎ Adult Diabetes - Hypertension
◎ Neutropenia
◎ Sleep Disorders
◎ Gender and Race Disparity
◎ Demographics
◎ SDOH
◎ Drug Design
Partners:
Weill Cornell, UCSF Medical,
Prisma-Health, Addiction
Research Center, UofSC
Medical/Pharma/Public
Health/Nursing; Wright State
Physicians, ….
9. Unique Value Propositions and Strengths [for Health Apps]
◎ Development of Knowledge Graphs
◎ Knowledge-infused (Deep) Learning and Knowledge-
infused NLP: Explainable AI
◎ Conversational AI/collaborative agents
◎ Augmented Personalized Health
10. 10
Health Knowledge Graph
Drug Abuse Ontology
Lokala U, Daniulaityte R, Lamy F, Gaur M, Thirunarayan K, Kursuncu U, Sheth. A. (2020).
DAO: An ontology for substance use epidemiology on social media and dark web. JMIR.
https://doi.org/10.2196/preprints.24938
https://scholarcommons.sc.edu/aii_fac_pub/356/ [Shah and Sheth US patent 2015]
11. Knowledge-Infused
Learning with
Semantic,
Cognitive,
Perceptual
Computing
Framework
11
Overarching Theory
Knowledge
Domain (Ontology)
Personalized KG
Multisensory
Sensing &
Multimodal
Data Interactions
Images
Text Speech Videos
IoTs
Natural Language
Processing,
Machine with
Deep Learning
AUGMENTED PERSONALIZED
HEALTH (APH)
Modeling broader disease context, and
personalized user behavior
Reasoning & decision-
making framework
Minimize data overload, assist in making
choices, appraisal, recommendations
TEDx talk: Augmented Health with Personalized Data and AI
12. 12
Contextualization and
Personalization
kBOT initiates greeting
conversation.
Understands the patient’s health
condition (allergic reaction to high
ragweed pollen level) via the
personalized patient’s knowledge
graph generated from EMR,
PGHD, and prior interactions with
the kBot.
Generates predictions or
recommended course of actions.
Inference based on patient’s
historical records and
background health knowledge
graph containing contextualized
(domain-specific) knowledge.
Figure: Example kBot conversation which
utilizes background health knowledge graph
and patient’s knowledge graph to infer and
generate recommendation to patients.
★ Conversing only information relevant
to the patient
Context enabled by relevant
healthcare knowledge including
clinical protocols.
13. Why Knowledge
Infused Learning
(K-IL)?
By changing the inputs, it can
enrich the representation (E.g.
Radicalization on Social Media)
By changing parameters, we can
control the learned
patterns/correlations to adhere to
the knowledge.
Deep Infusion would allow us finer
grained control over learned
patterns to ensure adherence to
knowledge at every step of the
hierarchy
Explanations easy to derive from
the KG used 13
Contextual Modeling to
analyze Radicalization on
Social Media
(Hate)
16. What? Comprehensive, Customizable, Adaptive
● App/chatbot for individuals
○ Cohort 1: College of Nursing (testing, research)
○ Cohort 2: Students, Staff, Faculty
● Dashboards for different uses: general administration, health services, research
○ Support for custom protocol
Gamecock look-n-feel, easy to use and engaging, customizable, secure, privacy
disclosure/management, HIPAA compliance, accessibility (ADA compliance), scalable, well
tested, extensible
17. Why? Proactively keep Campus healthy
● Informing and Educating the Community, Regularly Check Health Status:
○ Stay informed and educated, make better individual decisions, feel safer, reduce
adverse outcomes
○ In case of concern, connect with Student Health Services, manage isolation protocols
● Providing Campus-wide View: tools needed to make campus-wide decisions
○ Insurance against possible adverse outcome from low-risk approach (CDC’s
Considerations for Institutes of Higher Education (May 21), proactive
implementation of protocols
● Research: COVID-19, Mental Health
18. How?
● Comprehensive campus-wide involvement and coordination: requirements,
development, testing, operations
○ College of Nursing: lead evaluation
○ Student Health Services
○ School of Public Health
○ Division of Information Technology
○ CEC; of course, the AI Institute
● Development team with extensive experience
● Much more functional, forward looking, and cheaper than vendors
19.
20. Demo: Health-e Gamecock COVID-19 App (WebApp, IOS, Android)
Note: development is now complete, app is evaluated and we plan to use it for a major study.
24. 24
Use Case: kHealth Asthma
Many Sources of Highly Diverse Data
(& collection methods: Active + Passive):
Up to 1852 data points/ patient /day
http://bit.ly/kHealth-Asthma
kBot with screen
interface for conversation
Images
Text
Speech
★ Episodic to Continuous Monitoring
★ Clinician-centric to Patient-centric
★ Clinician controlled to Patient-empowered
★ Disease Focused to Wellness-focused
★ Sparse data to Multimodal Big Data
*(Asthma-Obesity)
25. Self Monitoring with kHealthDash:
Knowledge enabled personalized DASHboard for Asthma Management
Video link - https://youtu.be/yUgXCPwc55M
26. Digital Phenotype Score vs Asthma Control Test Score
Digital Phenotype Score = Symptom Score + Rescue Score + Activity Score + Awakening Score
27. 27
Utkarshani Jaimini, Krishnaprasad
Thirunarayan, Maninder Kalra, Revathy
Venkataramanan, Dipesh Kadariya, Amit
Sheth, “How Is My Child’s Asthma?”
Digital Phenotype and Actionable Insights
for Pediatric Asthma”, JMIR Pediatr Parent
2018;1(2):e11988, DOI: 10.2196/11988.
28. Self Appraisal with Digital Phenotype Score
Jaimini U, Thirunarayan K, Kalra M, Venkataraman R, Kadariya D, Sheth A, “How Is My Child’s Asthma?” Digital Phenotype and Actionable Insights for Pediatric Asthma, JMIR Pediatric Parent 2018;1(2):e11988
https://medium.com/leoilab/digital-phenotyping-turning-our-smartphones-inward-141a75b2f2a3
● Digital Phenotype Score (DPS) is defined as the score
to quantify the digital phenotypes collected from the
social media, smartphones, wearables, and sensors
streams.
● DPS acts as a cumulative measure for the abstraction
of knowledge and information from the raw digital
phenotypic data.
● The integration of the DPS can enable personalized
interventions in real time which are directly responsive
to the healthcare need of a patient.
29. Using Knowledge Graphs to construct a contextualized and personalized profile for each patient
that can drive insights and personalized care strategies
30. ● Published in ISWC 2018 Contextualized Knowledge Graph Workshop, 2018. Amelie
Gyrard, Manas Gaur, Saeedeh Shekarpour, Krishnaprasad Thirunarayan, Amit Sheth
2018, ISWC.
● Sheth, A., Jaimini, U., & Yip, H. Y. (2018). How Will the Internet of Things Enable Augmented
Personalized Health? IEEE Intelligent Systems, 33(1), 89–97.
https://doi.org/10.1109/MIS.2018.012001556
31. Determining Personalized Asthma Triggers: Seasonal Dependency
Venkataramanan R, Thirunarayan K, Jaimini U, Kadariya D, Yip HY, Kalra M, Sheth A. Determination of Personalized Asthma Triggers From Multimodal Sensing and a Mobile App: Observational Study. JMIR Pediatr Parent 2019;2(1):e14300. doi: 10.2196/14300
PMID: 31518318 PMCID: 6716491
33. Evidence based Path to Personalization
Patient-A was monitored for 13 weeks encompassing winter to spring 2018. Type: Severe, low medication compliance.
Absence of Pollen
First 6 weeks
Presence of Pollen
Rest of the 7 weeks
Pre (observe)
4 weeks
Post (validate)
2 weeks
Pre
4 weeks
Post
3 weeks
Pollen 0 Pollen 0 days Pollen 17 days Pollen 3 days
PM2.5 20 days PM2.5 5 days PM2.5 14 days PM2.5 2 days
Ozone 1 day Ozone 0 Ozone 0 Ozone 1 day
Asthma
Episodes*
21 days Asthma
Episodes
5 days Asthma
Episodes
17 days Asthma
Episodes
3 days
● Absence of Pollen - PM2.5 is the trigger
● Presence of Pollen - Pollen and PM2.5. Severe symptoms occurred in this period. Presence of both PM2.5 and
Pollen increased the intensity of asthma episodes.
Venkataramanan R, Thirunarayan K, Jaimini U, Kadariya D, Yip HY, Kalra M, Sheth A. Determination of Personalized Asthma Triggers From Multimodal Sensing and a Mobile App: Observational Study. JMIR Pediatr Parent 2019;2(1):e14300. doi: 10.2196/14300
PMID: 31518318 PMCID: 6716491
36. 36
Q1: Do you feel restless in sleep?
A1: Two-three times a week
Q2: Do you wake early? How often does it happen?
A2: Too early, happens two times a week.
Q3: Do you persistently feel sad?
A3: Yes. A sense of loneliness
Q4: How often you stay alone?
A4: I am divorced.
Q5: Have you been diagnosed with PTSD?
A5: No
Q6: Have you seen an MHP for your anxiety disorder?
A6: hmm recently.
Participant was asked do they feel restless in sleep,
then participant said hmm,two three times week
Participant was asked, do they persistently feel sad,
participant said divorce loneliness
Participant was asked, have they been diagnosed with
PTSD, participant said hmm recently.
Diagnostic Interview Summaries
Contextualized Diagnosis of
patient conversations
Correct diagnosis: Post Traumatic Stress
Disorder
42. Interest
We are interested in
Matching Support Seekers
-SSs (left) with Support
Providers - SPs (right)
Current State
Currently, moderators
(center) do this matching
42
Proposal
Our AI system will replace/assist the moderators that use
medical knowledge, information about the user, extracted
from the posts to perform this matching
48. Knowledge Graph for better Information Extraction: Application
in Epidemiology
48
Cameron, Delroy, Gary A. Smith, Raminta Daniulaityte, Amit P. Sheth, Drashti Dave, Lu Chen, Gaurish Anand, Robert Carlson, Kera Z. Watkins, and Russel Falck.
"PREDOSE: a semantic web platform for drug abuse epidemiology using social media." Journal of biomedical informatics 46, no. 6 (2013): 985-997.
50. K-IL: Shallow Infusion with DAO in DL model to detect trends in
cryptomarkets
Table: Sample properties derived from cryptomarket with DAO
51. Motivation
● The opioid epidemic entrenched in
Ohio and the Midwest of the US.
● The prevalence of opioid and its
impact on the well-being of
individuals and the society in Ohio.
○ Mental Health & Suicide Risk
Questions
1. How can we use social media to measure
mental health impact of opioid
prevalence?
1. Are there association between opioid and
mental health/suicide risk based on social
media data?
Approach
Monitoring the prevalence of opioid and its impact on mental health and suicide in Ohio,
utilizing a scalable knowledge and data driven BIGDATA (BD) approach via social media.
BD Spoke: Opioid and Substance Use in Ohio
52. Score
Calculation
Opioid
Mental Health
Depression
Addiction
Suicide Risk
Ideation, Behavior
Attempt
Correlations
● Sheth, Amit, and Pavan Kapanipathi. "Semantic filtering for social data." IEEE Internet Computing 20, no. 4 (2016): 74-78.
● Hussein S. Al-Olimat, Krishnaprasad Thirunarayan, Valerie Shalin, and Amit Sheth. 2018. Location Name Extraction from Targeted Text Streams using Gazetteer-based Statistical
Language Models. In Proceedings of the 27th International Conference on Computational Linguistics (COLING2018), pages 1986-1997. Association for Computational Linguistics
● Gaur, M., Kursuncu, U., Alambo, A., Sheth, A., Daniulaityte, R., Thirunarayan, K., & Pathak, J. (2018, October). " Let Me Tell You About Your Mental Health!" Contextualized
Classification of Reddit Posts to DSM-5 for Web-based Intervention. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (pp. 753-
762).
● Gaur, M., Alambo, A., Sain, J. P., Kursuncu, U., Thirunarayan, K., Kavuluru, R., ... & Pathak, J. (2019, May). Knowledge-aware assessment of severity of suicide risk for early
intervention. In The World Wide Web Conference (pp. 514-525).
● Yazdavar, A. H., Al-Olimat, H. S., Ebrahimi, M., Bajaj, G., Banerjee, T., Thirunarayan, K., ... & Sheth, A. (2017, July). Semi-supervised approach to monitoring clinical depressive
symptoms in social media. In Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017 (pp. 1191-1198).
● Daniulaityte, R., Nahhas, R. W., Wijeratne, S., Carlson, R. G., Lamy, F. R., Martins, S. S., ... & Sheth, A. (2015). “Time for dabs”: Analyzing Twitter data on marijuana concentrates across
News
Articles
Twitter
Data
Domain
Knowledg
e
Content
Enrichment
DAO
DSM-5
Location Extraction
Keyphrase Extraction
Age-based
Clustering
Semantic Filtering
Entity
Extraction
NLM Training
f(.)
Knowledge Infused
Natural Language
Processing (Ki-NLP)
Semantic
Mapping
Semantic
Proximity
Topic Model
Language Model
DAO
DSM-5
Dashboard
Visualizations
(Online)
Offline
Analysis
&
Visualizations
BD Spoke: Opioid and Substance Use in Ohio
53. ● Substance use addictive disorder linked to
opioid with higher correlation.
● Gender dysphoria, Dissociative and OCD
disorders are correlating moderately.
Opioid Prevalence in Ohio vs. Mental Health & Suicide
● Suicide ideation (initial stage) with highest correlation.
● Mild severity level of suicide risk linked to higher
correlation.
● Weak correlation for suicide indication (before initial)
p
N counties
p
N counties
54. with a Social Quality Index
Insights from semantic analysis of Social Media Big Data
Psychidemic: Measuring the Spatio-Temporal
Psychological Impact of Novel Coronavirus
with a Social Quality Index
Insights from semantic analysis of Social Media Big Data
○ Mental Health: Depression, Anxiety
○ Addiction: Substance use/abuse
○ Gender-based/Domestic Violence
○ Mental Health: Depression, Anxiety
○ Addiction: Substance use/abuse
○ Gender-based/Domestic Violence
55. Capability Demonstration 2
● Spatio-temporal analysis of big data (1billion tweets,
700,000 articles)
● Use of domain knowledge graphs (mental health,
addiction)
● Complex language understanding (slang, specialized
domain terms)
○ Can do people/demographic, network, sentiment, emotion, intent
analysis
● Scenario: Understand the implication of policy
choices (e.g., school/business closing) and real-
world events during COVID-19
55
56. A calculated Social Quality Index (SQI)
aggregates mental health components
(Depression, Anxiety), Addiction and
Substance Use Disorders.
Social Quality Index (SQI)
vecteezy.com
● Change in SQI informs comparisons between
states.
● Raw transformed SQI into relative state rankings
changing over time.
57. e.g., IN, NH, OH,
OR, WA, WY
are worsening.
Results: Relative State Rankings Reveal Patterns
SQI Ranking April 4 - 10
SQI Ranking March 14 - 20 SQI Ranking March 21 - 27
SQI Ranking March 23-April 3
Darker: Better
Social Quality
58. Results: Cluster --Improving SQI Ranking
SQI bad SQI better SQI better
SQI better
Frequency
Depression: 125037
Addiction: 92897
Anxiety: 81891
Total: 299825
Frequency
Depression: 113830
Addiction: 81810
Anxiety: 74080
Total: 269720
Frequency
Depression: 81463
Addiction: 60166
Anxiety: 45998
Total: 187627
Frequency
Depression: 59088
Addiction: 49086
Anxiety: 46887
Total: 155061
IL, NY, MD,
AZ, NM, MA.
March 14-20 March 21-27 March 28-April 3
April 4-10
59. External Events
● Stay at home order
● Extension non-essential closure
● Closing parks
● State of emergency
● School Closure
● Mental Health Alarm
● Extension of small business closure
● Bill payment deadline extended
● 41k new job openings
● Child-care assistance for essential
workers
● Spike in number of
cases
● Stay at home order
extended
● Extension School
Closure
● State of emergency
Extension
● Unemployment
Increase(>800%)
● Tax deadline extended
● SNAP Benefits
● Death Benefits
● Domestic Violence Alarm
● Spike in number of cases
● Stay at home order extended
● Extension School Closure
● State of emergency Extension
● Closure barber shops and related
businesses
● Number of deaths cross 50000
● Unemployment Increase(>2.5k%)
● Tax deadline extended
● Phases reopening
● Limited indoor seating or gathering
● CARES Act
Change
in
SQI
relative
to
first
week
64. Knowledge-infused Reinforcement Learning
● The input to the agent is sequential through many steps, it gets an input and a reward at every step and
learns the right output gradually through reinforcement.
66. NOURICH
A system to monitor diet,
recommend meals and promote
healthy eating habits:
Current application: Type 1 diabetes
in Children
Contact: Revathy Venkataramanan
Acknowledgement: Thanks to my collaborators Hong Yung Yip and Thilini Wijayasriwartane for the slides
NOURICH
Know What you Eat
67. Overweight Obesity Hospitalization
Prevent
overweight
moving to
obesity
Prevent obesity
leading to
hospitalization
Bridging the gap
“Focusing on reducing excess and impulsive calorie
consumption and making an informed decision about food
choices and physical activity can help one attain a healthier
weight and minimize the risk of chronic illness”
The Dietary Guidelines of Americans 2010
68. ● Real-time food recognition
● Tensorflow Mobile net model
20
Food Categories
700
Images/Category
Image Data Source
Image Source: https://commons.wikimedia.org/wiki/File:Google_%22G%22_Logo.svg, https://freepik.com
Nutrition Management System
NOURICH
GOAL: A system to monitor
diet, recommend meals and
promote healthy eating habits.
69. Architecture for application to Type 1 Diabetes in Children
Data sources
- User specific (food allergies,
comorbidities, lab reports
including genetic profiles and
etc)
Personalized
Knowledge Base
- Meal name
- Nutrition
- Ingredients
- Cooking style
Data collected
DATA STORE
Data collected are stored along with
domain knowledge
Image
Voice
Text
Inputs
Processing engine
Image, voice,text to
keyword
DASHBOARD
- Carbohydrate count
70. Trained on
Food Images
Cheesecake
NOURICH
Bitmap
Conversion
ByteArray
Conversion
3 frames per
second
Python
Script
Data Cleaning
1) Removing png
2) Removing non jpeg
Shell
Script
Image
Annotation
Training
Model: Mobile-net
model
Accuracy: 83%
Graph
protobuf file
Data preprocessing and training layer
Crawled Images
Conversion to
TFLite
Image Recognition layer
Recognized image is displayed to the user
APPLICATION ARCHITECTURE
User
Display nutrition
info for the food
Food Logs
Sources: 1) User Icon by Gregor Cresnar from the Noun Project, 2)Food Images from Google Images, 3)TensorflowLite,Nutritionix,Bash, Instagram, Google - logos are from original vendors
71. Matching Support Seeker (SS) with Support Provider (SP)
Online: Current application - matching SS and SP on mental
health related subReddit
https://scholarcommons.sc.edu/aii_fac_pub/516/
73. 73
Modeling Exogenous Information into Epidemiological Models
(Exo-SIR)
Curve of Exogenous Information in Tamil Nadu
Infected Curve (yellow) shift left because of
Exogenous Information (Exo-SIR)
Architecture of Exo-SIR model
Simple SIR
Model
Infected
Curve for
Tamil Nadu
State
● The curve representing time to infection shifts left (28%
early) when we introduce exogenous events such as large
gatherings (eg. Tablighi Jamaat religious gathering) or labor
migration due to lockdown.
● Evaluated/validated for the impact of exogenous events on
three states in India (Rajasthan, Tamil Nadu, Kerala).
(Accepted at AI for COVID track in ACM KDD 2020)
74. There is a lot more:
http://aiisc.ai, http://wiki.aiisc.ai
https://scholarcommons.sc.edu/aii_fac_pub/
Many thanks to our sponsors, esp. ~10 NIH
grants (four R01s, three R21s, R56, etc) from
NIMH, NIDA, NICHD, and other institutes.