#ChatGPT #LLM #DistributionalSemantics are hot topics--both for their successes and failures/shortcomings. In Prof. Amit Sheth's keynote he delivered yesterday at #knowledgeNLP2023 -"From #NLP to #NLU: Why we need varied, comprehensive, and #StratifiedKnowledge, and how to use it for neuro-symbolic AI", he discusses several categories of deficiencies, and more importantly, how to address them using Knowledge-infused #neurosymbolicAI. #WorldModel #RealWorldSemantics Slides: http://bit.ly/kNLP2023 Abstract: https://lnkd.in/grzi5UyJ.
Photos: https://lnkd.in/gS_KRFvQ
https://asonam.cpsc.ucalgary.ca/2021/speakers.php
With the increasing legalization of medical and recreational use of substances, more research is needed to understand the association between mental health and user behavior related to drug consumption. Specifically, drug overdose and substance use- related mental health issues have become two major topics that have been widely discussed on social media platforms. Big social media data has the potential to provide deeper insights about these associations to public health analysts for making policy decisions. Multiple national population surveys have found that about half of those who experience a mental health illness during their lives will also experience a substance use disorder and vice versa. The communications related to addiction and mental health are complex to process and understand given their language and contextual characteristics. Surface-level data analysis alone is not sufficient to understand the complex nature of relationships among the addiction and mental health context. Moreover, dark web vendors have been using social media as a new marketplace for drugs. Social media users also discuss the novel drugs emerging in dark web marketplaces and associated side effects/health conditions. These communications get complex when researchers try to annotate them or link them to a specific mental health entity. Considering the significant sensitivity of such communications and to protect user privacy on social media, a potential solution requires reliable algorithms for modeling such communications. We demonstrate the value of incorporating domain-specific knowledge in natural language understanding to identify the relationship between mental health and drug addiction. We discuss end-to-end knowledge-infused deep learning frameworks that leverage the pre-trained language representation model and domain-specific declarative knowledge source to extract entities and their relationships jointly. Our model is further tailored to focus on the entities mentioned in the sentence where ontology is used to locate the target entity, position. We also demonstrate the capabilities of inclusion of the knowledge-aware representation in association with language models that can extract the Drug and Mental health condition associations.
Semantics of the Black-Box: Using knowledge-infused learning approach to make...Amit Sheth
Keynote at the IEEE ICSC Workshop on Semantic Machine Learning (#SML21: https://ist.gmu.edu/~hpurohit/events/sml21/#keynote):
Video of SML21: https://www.youtube.com/watch?v=cx-l0XDk9Tw
The recent series of deep learning innovations have shown enormous potential to impact individuals and society, both positively and negatively. The deep learning models utilizing massive computing power and enormous datasets have significantly outperformed prior historical benchmarks on increasingly difficult, well-defined research tasks across technology domains such as computer vision, natural language processing, signal processing, and human-computer interactions. However, the Black-Box nature of deep learning models and their over-reliance on massive amounts of data condensed into labels and dense representations pose challenges for the system’s interpretability and explainability. Furthermore, deep learning methods have not yet been proven in their ability to effectively utilize relevant domain knowledge and experience critical to human understanding. This aspect is missing in early data-focused approaches and necessitated knowledge-infused learning and other strategies to incorporate computational knowledge. Rapid advances in our ability to create and reuse structured knowledge as knowledge graphs make this task viable. In this talk, we will outline how knowledge, provided as a knowledge graph, is incorporated into the deep learning methods using knowledge-infused learning. We then discuss how this makes a fundamental difference in the interpretability and explainability of current approaches and illustrate it with examples relevant to a few domains.
The recent series of innovations in deep learning have shown enormous potential to impact individuals and society, both positively and negatively. The deep learning models utilizing massive computing power and enormous datasets have significantly outperformed prior historical benchmarks on increasingly difficult, well-defined research tasks across technology domains such as computer vision, natural language processing, signal processing, and human-computer interactions. However, the Black-Box nature of deep learning models and their over-reliance on massive amounts of data condensed into labels and dense representations pose challenges for the system’s interpretability and explainability. Furthermore, deep learning methods have not yet been proven in their ability to effectively utilize relevant domain knowledge and experience critical to human understanding. This aspect is missing in early data-focused approaches and necessitated knowledge-infused learning and other strategies to incorporate computational knowledge. Rapid advances in our ability to create and reuse structured knowledge as knowledge graphs make this task viable. In this talk, we will outline how knowledge, provided as a knowledge graph, is incorporated into the deep learning methods using knowledge-infused learning. We then discuss how this makes a fundamental difference in the interpretability and explainability of current approaches and illustrate it with examples relevant to a few domains.
We hosted a fantastic tutorial on Knowledge-infused Deep Learning at the 31st ACM Hypertext Conference on July 14. Broadly, the tutorial covered many exciting applications of Broad- and Community-based Knowledge Graph in Education, Clinical and Social-Media Healthcare, Pandemic, and Cryptomarkets.
We theorized the concept of Knowledge-infusion and showed its importance in gaining explainability and spectacular performance gains. We extended the idea of "Knowledge-infused Deep Learning" to Autonomous Driving, Cyber Social Harms, and DarkWeb.
The tutorial presentation with relevant resources and references are made online at http://kidl2020.aiisc.ai.
The Foundations of Artificial Intelligence, The History of
Artificial Intelligence, and the State of the Art. Intelligent Agents: Introduction, How Agents
should Act, Structure of Intelligent Agents, Environments. Solving Problems by Searching:
problem-solving Agents, Formulating problems, Example problems, and searching for Solutions,
Search Strategies, Avoiding Repeated States, and Constraint Satisfaction Search. Informed
Search Methods: Best-First Search, Heuristic Functions, Memory Bounded Search, and Iterative
Improvement Algorithms.
On the problems of interface: explainability, conceptual spaces, relevanceGiovanni Sileno
Summary talk of the research conducted at Télécom ParisTech and Paris Dauphine University during my postdoc project (2016-2018), in collaboration with Isabelle Bloch, Jamal Atif and Jean-Louis Dessalles.
https://asonam.cpsc.ucalgary.ca/2021/speakers.php
With the increasing legalization of medical and recreational use of substances, more research is needed to understand the association between mental health and user behavior related to drug consumption. Specifically, drug overdose and substance use- related mental health issues have become two major topics that have been widely discussed on social media platforms. Big social media data has the potential to provide deeper insights about these associations to public health analysts for making policy decisions. Multiple national population surveys have found that about half of those who experience a mental health illness during their lives will also experience a substance use disorder and vice versa. The communications related to addiction and mental health are complex to process and understand given their language and contextual characteristics. Surface-level data analysis alone is not sufficient to understand the complex nature of relationships among the addiction and mental health context. Moreover, dark web vendors have been using social media as a new marketplace for drugs. Social media users also discuss the novel drugs emerging in dark web marketplaces and associated side effects/health conditions. These communications get complex when researchers try to annotate them or link them to a specific mental health entity. Considering the significant sensitivity of such communications and to protect user privacy on social media, a potential solution requires reliable algorithms for modeling such communications. We demonstrate the value of incorporating domain-specific knowledge in natural language understanding to identify the relationship between mental health and drug addiction. We discuss end-to-end knowledge-infused deep learning frameworks that leverage the pre-trained language representation model and domain-specific declarative knowledge source to extract entities and their relationships jointly. Our model is further tailored to focus on the entities mentioned in the sentence where ontology is used to locate the target entity, position. We also demonstrate the capabilities of inclusion of the knowledge-aware representation in association with language models that can extract the Drug and Mental health condition associations.
Semantics of the Black-Box: Using knowledge-infused learning approach to make...Amit Sheth
Keynote at the IEEE ICSC Workshop on Semantic Machine Learning (#SML21: https://ist.gmu.edu/~hpurohit/events/sml21/#keynote):
Video of SML21: https://www.youtube.com/watch?v=cx-l0XDk9Tw
The recent series of deep learning innovations have shown enormous potential to impact individuals and society, both positively and negatively. The deep learning models utilizing massive computing power and enormous datasets have significantly outperformed prior historical benchmarks on increasingly difficult, well-defined research tasks across technology domains such as computer vision, natural language processing, signal processing, and human-computer interactions. However, the Black-Box nature of deep learning models and their over-reliance on massive amounts of data condensed into labels and dense representations pose challenges for the system’s interpretability and explainability. Furthermore, deep learning methods have not yet been proven in their ability to effectively utilize relevant domain knowledge and experience critical to human understanding. This aspect is missing in early data-focused approaches and necessitated knowledge-infused learning and other strategies to incorporate computational knowledge. Rapid advances in our ability to create and reuse structured knowledge as knowledge graphs make this task viable. In this talk, we will outline how knowledge, provided as a knowledge graph, is incorporated into the deep learning methods using knowledge-infused learning. We then discuss how this makes a fundamental difference in the interpretability and explainability of current approaches and illustrate it with examples relevant to a few domains.
The recent series of innovations in deep learning have shown enormous potential to impact individuals and society, both positively and negatively. The deep learning models utilizing massive computing power and enormous datasets have significantly outperformed prior historical benchmarks on increasingly difficult, well-defined research tasks across technology domains such as computer vision, natural language processing, signal processing, and human-computer interactions. However, the Black-Box nature of deep learning models and their over-reliance on massive amounts of data condensed into labels and dense representations pose challenges for the system’s interpretability and explainability. Furthermore, deep learning methods have not yet been proven in their ability to effectively utilize relevant domain knowledge and experience critical to human understanding. This aspect is missing in early data-focused approaches and necessitated knowledge-infused learning and other strategies to incorporate computational knowledge. Rapid advances in our ability to create and reuse structured knowledge as knowledge graphs make this task viable. In this talk, we will outline how knowledge, provided as a knowledge graph, is incorporated into the deep learning methods using knowledge-infused learning. We then discuss how this makes a fundamental difference in the interpretability and explainability of current approaches and illustrate it with examples relevant to a few domains.
We hosted a fantastic tutorial on Knowledge-infused Deep Learning at the 31st ACM Hypertext Conference on July 14. Broadly, the tutorial covered many exciting applications of Broad- and Community-based Knowledge Graph in Education, Clinical and Social-Media Healthcare, Pandemic, and Cryptomarkets.
We theorized the concept of Knowledge-infusion and showed its importance in gaining explainability and spectacular performance gains. We extended the idea of "Knowledge-infused Deep Learning" to Autonomous Driving, Cyber Social Harms, and DarkWeb.
The tutorial presentation with relevant resources and references are made online at http://kidl2020.aiisc.ai.
The Foundations of Artificial Intelligence, The History of
Artificial Intelligence, and the State of the Art. Intelligent Agents: Introduction, How Agents
should Act, Structure of Intelligent Agents, Environments. Solving Problems by Searching:
problem-solving Agents, Formulating problems, Example problems, and searching for Solutions,
Search Strategies, Avoiding Repeated States, and Constraint Satisfaction Search. Informed
Search Methods: Best-First Search, Heuristic Functions, Memory Bounded Search, and Iterative
Improvement Algorithms.
On the problems of interface: explainability, conceptual spaces, relevanceGiovanni Sileno
Summary talk of the research conducted at Télécom ParisTech and Paris Dauphine University during my postdoc project (2016-2018), in collaboration with Isabelle Bloch, Jamal Atif and Jean-Louis Dessalles.
Presentation (with Eamon Costello) from the Global Smart Education Conference (The 6th International Conference on Smart Learning Environments), Beijing National University, China.
The presentation explores issues in AI driven learning systems and implications of machine learning approaches for inclusion and access to education.
2019년 파이콘 한국에서 진행된 튜토리얼 자료입니다. 최재식 교수님께서 설명가능인공지능이란 무엇인가에 대해 발표해주신 Part 1 발표자료입니다. 아래 링크를 통해 행사 관련 정보를 확인하실 수 있습니다.
http://xai.unist.ac.kr/Tutorial/2018/
https://github.com/OpenXAIProject/PyConKorea2019-Tutorials
Part 1: https://www.slideshare.net/OpenXAI/2019-part-1
Part 2: https://www.slideshare.net/OpenXAI/2019-lrp-part-2
Part 3: https://www.slideshare.net/OpenXAI/2019-shap-part-3
Artificial Intelligence: The Promise, the Myth, and a Dose of RealityDagmar Monett
Keynote at the 33. Bremer Universitäts-Gespräche Data Science - Wunderwelt oder alter Wein in neuen Schläuchen (engl. Data science - Wonderworld or old wine in new bottles), October 7th, 2021, Universität Bremen, Germany.
The talk describes a paradigm of knowledge-infused learning in healthcare for explainability, interpretability, and traceability of outcome. Thus bridging the gap between AI and Clinical settings and developing architectures that are of clinical relevance.
PyData Salamanca knowledge infusion in healthcareManas Gaur
The talk describes a paradigm of knowledge-infused learning in healthcare for explainability, interpretability, and traceability of outcome. Thus bridging the gap between AI and Clinical settings and developing architectures that are of clinical relevance.
Week 12 neural basis of consciousness : frontiers in consciousness researchNao (Naotsugu) Tsuchiya
12-week lecture series on "the neural basis of consciousness" by Prof Nao Tsuchiya.
Given to 3rd year undergraduate level. No prerequisites.
Contents:
1) What does IIT propose about qualia?
2) How can we characterize qualia structures?
3) What are the possible empirical experiments that can be used to reveal the relationship between qualia and brain?
4) What are societal impacts of consciousness research?
Brain Research for Teachers & Other Curious Souls, 2013 updateCarolyn K.
After a background in critiquing research, learn about dozens of recent research studies of the brain, and what they show. Updated for all the great new research through 2013, don't miss this informative collection of research. by Wenda Sheard
Artificial intelligence (AI) is a commonly used term as a result of adopting an overly generalized representation.
The main problem is definitions of “intelligence,” which often misinterpret practical notions that the term indicates.
The word “artificial,” from medical and biological points of view, quite naturally designates a non-natural property.
Abstract
This paper provides an in-depth description of the Singularity Pyramid (SP) concept for an extensively organized online knowledge mapping system, also a futuristic version of Wikipedia wherein each unit of knowledge is associated with a probability distribution of probabilities to reflect the probabilistic nature of reality. Knowledge mapping wouldn't have any significance if not for four well-defined abstract dimensions (Conceptual, Purposeful, Scopic, and “Language”) that capture the collective body of knowledge in its entirety. With the Singularity placed at one end of each dimension as the ultimate goal, these dimensions give you a sense of direction in learning/creating knowledge towards that goal.
With knowledge mapping, the SP can provide Singularitarians with a bigger picture to realize the roadmap towards the Singularity by addressing numerous problems that impede their progress—lack of coherent knowledge structure for academic research, lack of strategic profiling, leader discord, unawareness of long-term danger, negligence of macro-vulnerabilities, and “language” barrier to pure knowledge. It is also a collaborative tool for planning, decision making, risk management, and resource allocation. The “theory of weaknesses” based on the SP has the potential to change your worldview forever.
Engineering the SP is an unprecedented and formidable task which I foresee rapid advances in technology in the next year may fulfill. Also, the intuitive (easy to memorize, visualize, and contribute) and functional (designed for practical use) structure of SP makes it a candidate for the brain of the Singularity Superintelligence (SS). Note: there is no technical detail in this paper.
In this unit, students will explore contemporary scientific media and art forms to understand what the advent of artificial intelligence (AI) might mean for the future of humankind.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
More Related Content
Similar to From NLP to NLU: Why we need varied, comprehensive, and stratified knowledge, and how to use it for Neuro-symbolic AI
Presentation (with Eamon Costello) from the Global Smart Education Conference (The 6th International Conference on Smart Learning Environments), Beijing National University, China.
The presentation explores issues in AI driven learning systems and implications of machine learning approaches for inclusion and access to education.
2019년 파이콘 한국에서 진행된 튜토리얼 자료입니다. 최재식 교수님께서 설명가능인공지능이란 무엇인가에 대해 발표해주신 Part 1 발표자료입니다. 아래 링크를 통해 행사 관련 정보를 확인하실 수 있습니다.
http://xai.unist.ac.kr/Tutorial/2018/
https://github.com/OpenXAIProject/PyConKorea2019-Tutorials
Part 1: https://www.slideshare.net/OpenXAI/2019-part-1
Part 2: https://www.slideshare.net/OpenXAI/2019-lrp-part-2
Part 3: https://www.slideshare.net/OpenXAI/2019-shap-part-3
Artificial Intelligence: The Promise, the Myth, and a Dose of RealityDagmar Monett
Keynote at the 33. Bremer Universitäts-Gespräche Data Science - Wunderwelt oder alter Wein in neuen Schläuchen (engl. Data science - Wonderworld or old wine in new bottles), October 7th, 2021, Universität Bremen, Germany.
The talk describes a paradigm of knowledge-infused learning in healthcare for explainability, interpretability, and traceability of outcome. Thus bridging the gap between AI and Clinical settings and developing architectures that are of clinical relevance.
PyData Salamanca knowledge infusion in healthcareManas Gaur
The talk describes a paradigm of knowledge-infused learning in healthcare for explainability, interpretability, and traceability of outcome. Thus bridging the gap between AI and Clinical settings and developing architectures that are of clinical relevance.
Week 12 neural basis of consciousness : frontiers in consciousness researchNao (Naotsugu) Tsuchiya
12-week lecture series on "the neural basis of consciousness" by Prof Nao Tsuchiya.
Given to 3rd year undergraduate level. No prerequisites.
Contents:
1) What does IIT propose about qualia?
2) How can we characterize qualia structures?
3) What are the possible empirical experiments that can be used to reveal the relationship between qualia and brain?
4) What are societal impacts of consciousness research?
Brain Research for Teachers & Other Curious Souls, 2013 updateCarolyn K.
After a background in critiquing research, learn about dozens of recent research studies of the brain, and what they show. Updated for all the great new research through 2013, don't miss this informative collection of research. by Wenda Sheard
Artificial intelligence (AI) is a commonly used term as a result of adopting an overly generalized representation.
The main problem is definitions of “intelligence,” which often misinterpret practical notions that the term indicates.
The word “artificial,” from medical and biological points of view, quite naturally designates a non-natural property.
Abstract
This paper provides an in-depth description of the Singularity Pyramid (SP) concept for an extensively organized online knowledge mapping system, also a futuristic version of Wikipedia wherein each unit of knowledge is associated with a probability distribution of probabilities to reflect the probabilistic nature of reality. Knowledge mapping wouldn't have any significance if not for four well-defined abstract dimensions (Conceptual, Purposeful, Scopic, and “Language”) that capture the collective body of knowledge in its entirety. With the Singularity placed at one end of each dimension as the ultimate goal, these dimensions give you a sense of direction in learning/creating knowledge towards that goal.
With knowledge mapping, the SP can provide Singularitarians with a bigger picture to realize the roadmap towards the Singularity by addressing numerous problems that impede their progress—lack of coherent knowledge structure for academic research, lack of strategic profiling, leader discord, unawareness of long-term danger, negligence of macro-vulnerabilities, and “language” barrier to pure knowledge. It is also a collaborative tool for planning, decision making, risk management, and resource allocation. The “theory of weaknesses” based on the SP has the potential to change your worldview forever.
Engineering the SP is an unprecedented and formidable task which I foresee rapid advances in technology in the next year may fulfill. Also, the intuitive (easy to memorize, visualize, and contribute) and functional (designed for practical use) structure of SP makes it a candidate for the brain of the Singularity Superintelligence (SS). Note: there is no technical detail in this paper.
In this unit, students will explore contemporary scientific media and art forms to understand what the advent of artificial intelligence (AI) might mean for the future of humankind.
Similar to From NLP to NLU: Why we need varied, comprehensive, and stratified knowledge, and how to use it for Neuro-symbolic AI (20)
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
The Metaverse and AI: how can decision-makers harness the Metaverse for their...Jen Stirrup
The Metaverse is popularized in science fiction, and now it is becoming closer to being a part of our daily lives through the use of social media and shopping companies. How can businesses survive in a world where Artificial Intelligence is becoming the present as well as the future of technology, and how does the Metaverse fit into business strategy when futurist ideas are developing into reality at accelerated rates? How do we do this when our data isn't up to scratch? How can we move towards success with our data so we are set up for the Metaverse when it arrives?
How can you help your company evolve, adapt, and succeed using Artificial Intelligence and the Metaverse to stay ahead of the competition? What are the potential issues, complications, and benefits that these technologies could bring to us and our organizations? In this session, Jen Stirrup will explain how to start thinking about these technologies as an organisation.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
From NLP to NLU: Why we need varied, comprehensive, and stratified knowledge, and how to use it for Neuro-symbolic AI
1. From NLP to NLU: Why we need
varied, comprehensive, and
stratified knowledge, and how to
use it for Neuro-symbolic AI
Keynote at KnowledgeNLP-AAAI’23
Amit Sheth
Director, AI Institute of South Carolina
University of South Carolina
amit@sc.edu
#AIISC, http://aiisc.ai
2. AIISC portfolio in Core AI & Translational AI
2
Knowledge-infused
Learning (Neuro-
symbolic
/Hybrid AI)
Knowledge Graph
Development
Deep Learning
Reinforcement Learning
Natural Language
Processing/
Understanding/Generation
Multimodal AI
(IoT/sensor, data
streams, images,
emoji)
Collaborative &
Personal Assistants
Multiagent
Systems
Interpretability/
Explainability/Safety/
Trust/Ethics in AI
Medicine/Healthcare/Nursing
(Nutrition, Neurodevelopmental Disease,
Asthma, Diabetes, Hypertension, Autism,
Aphasia, Cognitive Disorders,
Oncology,...)
Neuroscience
Brain Science
Epidemiology
Education
Social Good/Harm
(Disinformation,
Harassment,Toxic
Content, Deception,
Extremism,
Radicalization)
Public Health
(Mental Health,
Addiction, COVID-19,
Epidemics)
Smart Manufacturing
(Digital Twins, Factory
of Future)
Disaster Management
(Response, Resilience
Pharma: drug
discovery,
vigilance
Autonomous
Systems
(Vehicles)
Automated
Planning
Computer
Vision
Gaming
Cognitive
Science
Science & Engg:
Radiation,
Astrophysics, Civil
Infra & Transportation
Law
3. Contents
3
1. Challenges with Current LMs
2. Possible Fixes and Limitations
3. Knowledge Infused Neuro-Symbolic AI
4. Before Transformers, Historical Context [Neuro Symbolic AI]
5. After Transformers, Current Context [Neuro Symbolic AI]
6. Future Context [Neuro-Symbolic AI] with Process Knowledge
Infusion.
4. Challenges with Current LMs
4
Tycoon
Did you mean: tycoon
Did you mean: typhoon
Did you mean: typography
tycoon:
0.00567%
Distributional Semantics:
Language Models are trained to compute the
distributional plausibility of language tokens
from enormous amounts of training tokens.
World Model Semantics:
Relationships and effects among the objects
that the language tokens describe, grounded
in the physical or conceptual reality of the
world humans experience (ontological
commitment).
Do World Model Semantics Arise as an
Emergent Capability of Distributional
Semantics at Scale? [No- not directly, not
specifically, not adequately]
Sheth, A., Ramakrishnan, C., & Thomas, C. (2005). Semantics for the semantic web: The implicit, the formal and the powerful. International Journal on Semantic Web and
Information Systems (IJSWIS), 1(1), 1-18., 2005, link
World Model Semantics from Distributed Semantics at Scale?
1.
5. Challenges with Current LMs
5
Formally
∀i ⊆ [N], N = Vocabulary Size,
Model the probability:
P(tokeni, …. , tokenN | token1, … tokeni)
Example Usage
Once Trained on Enormous amounts of Data,
Predict:
1. y ~ P(S = “Obama just won the 2032
Election”)?
2. P(S) does not pass a threshold test
=> y = 0, or False.
Tycoon
Did you mean: tycoon
Did you mean: typhoon
Did you mean: typography
tycoon:
0.00567%
Recency : World Model Semantics from Distributed Semantics at Scale
1.
6. Challenges with Current LMs
6
Distributional Semantics:
Language Models are trained to compute the
distributional plausibility of language tokens
from enormous amounts of training tokens.
E.g., What was the color of the white horse of
Napolean?
It is not very distributionally plausible that a
person asks the answer to a question they
already know
The answer is abundantly clear in the question
But, (a) distribution semantics has problem
with the low probability, and (b) it does not
understand! More challenges arise due to lack
of context.
Common Sense : World Model Semantics from Distributed Semantics at Scale
1.
7. Challenges with Current LMs
7
Distributional Semantics:
Language Models are trained to compute the
distributional plausibility of language tokens from
enormous amounts of training tokens.
E.g., Mike’s mum had 4 kids; 3 of them are X,Y,Z. What
is the name of 4th kid?
- Missing information is not clear
Answer is obviously Mike
1. Mike is not a pet
2. More than one child cannot have the same name
3. Kid names cannot be any word
4. Impossible answers even after the hint:
Mike, Luis, Drake, and Matilda, and All other
tokens in the input!
Distribution semantics does not have commonsense
and probability does not help.
Challenge - Missing Information: World Model Semantics from Distributed Semantics at Scale
1.
8. Challenges with Current LMs
8
Distributional Semantics:
Language Models are trained to compute the
distributional plausibility of language tokens
from enormous amounts of training tokens.
E.g., Mike’s mum had 4 kids;3 of them are X,Y,Z.
What is the name of 4th kid?
Human’s first answers (We asked 10 fellow
humans)
1. Mike
2. I think this is a trick question
Distribution semantics does not know to make
human-like assumptions about missing
information.
Challenge - Missing Information: World Model Semantics from Distributed Semantics at Scale
1.
9. Challenges with Current LMs
9
Distributional Semantics are a Cloud of Probabilities
The
director, AIISC, of,
Amit Sheth
All other words
Prof
I kind of
understand (not
really though)
Dense Representational Spaces
Hallucinations - It makes up things !
1.
10. Recency
Insufficiency of data alone
Common-sense
Hallucinations
Challenges to be addressed for NLU
10
User-Explainability Application-level Safety
Characteristics of NLU Capable Systems
1.
12. Possible Fix: Instruct GPT
12
The Instruct GPT Framework
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., ... & Lowe, R. (2022). Training language models to
follow instructions with human feedback. arXiv preprint arXiv:2203.02155. link
No Problem I will make things larger and use
Instruction based Training
Hallucinations - It makes up things !
2.
13. 13
1. 40 humans to capture the breadth of knowledge corresponding to the
data in LLMs seems small
2. The richness of human knowledge is compressed into a mere label
2.
14. 14
Prof: Amit Sheth
Synonyms: Amit
Sheth
Degree:Ph.D.
Synonyms:
Doctorate
Company: AI
Institute,
Abbr: AIISC
Synonyms:
AI Institute,
South Carolina
employee_of
Let’s see what this model of
understanding can yield
Knowledge (Graphs) to the rescue
Fix: Addressing Hallucinations: Recency, Common-Sense, and Implicit Entity Mentions, etc.,
2.
15. 15
World as Concepts vs. World as Probabilities.
Explicit model of recency and common-sense.
Supply missing Knowledge (entities, relationships).
Semantics supported by Knowledge (Graphs)
User-level Explanations and Safety Constraints
2.
18. 18
3.
Shallow Infusion Semi-Deep Infusion
Shades of KiL - Shallow and Semi-Deep Infusion
Sheth, Gaur, Kursuncu, & Wickramarachchi, (2019). Shades of knowledge-infused learning for enhancing deep learning. IEEE Internet Computing, 23(6), 54-63., link
19. 3.
Shades of KiL - Deep Infusion
19
Deep Infusion
Sheth, Gaur, Kursuncu, & Wickramarachchi, (2019). Shades of knowledge-infused learning for enhancing deep learning. IEEE Internet Computing, 23(6), 54-63., link
20. Sheth, Gaur, Kursuncu, & Wickramarachchi, (2019). Shades of knowledge-infused learning for enhancing deep learning. IEEE Internet Computing, 23(6), 54-63., link
Characteristics/Method Distributed Semantics Shallow Infusion Semi-Deep Infusion Deep Infusion
Recency U-M- M M+ H
Filling in Missing
Information
U U M H
Hallucinations Unsatisfactory (U)
Get by but not really
solving the problem
(M)
Better but not fully
solve the problem (M+)
Broadly solve the
problem (H)
Characteristics of Knowledge Infusion
3.
20
23. 4.
KiL - SEDO (Shallow Infusion)
23
Gaur, Kursuncu, Alambo,, Sheth, Daniulaityte, Thirunarayan, & Pathak. (2018, October). " Let Me Tell You About Your Mental Health!" Contextualized Classification of
Reddit Posts to DSM-5 for Web-based Intervention. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (pp. 753-762).,
link
25. 5.
KiL - K-Adapter (Shallow Infusion)
25
Wang, R., Tang, D., Duan, N., Wei, Z., Huang, X. J., Ji, J., ... & Zhou, M. (2021, August). K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters. In Findings of the
Association for Computational Linguistics: ACL-IJCNLP 2021 (pp. 1405-1418), link
26. 5.
KiL - KALA (Semi-Deep Infusion)
26
Kang, M., Baek, J., & Hwang, S. J. (2022, July). KALA: Knowledge-Augmented Language Model Adaptation. In Proceedings of the 2022 Conference of the North American
Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 5144-5167)., link
27. 5.
KiL - TDLR (Semi-Deep Infusion)
27
Rawte, V., Chakraborty, M., Roy, K., Gaur, M., Faldu, K., Kikani, P., ... & Sheth, A. P. TDLR: Top Semantic-Down Syntactic Language Representation. In NeurIPS'22 Workshop on
All Things Attention: Bridging Different Perspectives on Attention., link
28. Knowledge Contexts Leads to Performance Gains With Smaller Models
Knowledge Contexts Leads to Performance Gains With Smaller Models and Smaller
Datasets
5.
TDLR - Results
28
Rawte, V., Chakraborty, M., Roy, K., Gaur, M., Faldu, K., Kikani, P., ... & Sheth, A. P. TDLR: Top Semantic-Down Syntactic Language Representation. In NeurIPS'22 Workshop on
All Things Attention: Bridging Different Perspectives on Attention., link
30. Neural Network Abstract / Contextualization
ACT DECIDE
reasoning
Planning
Inference
Apply Process
Knowledge: User has
Specific concerns due to
X, Y, Z Concepts
Action:
Further Interact with
System User on their
concerns
Explicit Knowledge
Data
6.
30
31. Really struggling with my bisexuality which
is causing chaos in my relationship with a
girl. I am equal to worthless for her. I’m
now starting to get drunk because I can’t
cope with the obsessive, intrusive thoughts,
and need to get out of my head.
288291000119102: High risk bisexual behavior
365949003: Health-related behavior finding 365949003: Health-related behavior finding
307077003: Feeling hopeless
365107007: level of mood
225445003: Intrusive thoughts
55956009: Disturbance in content of thought
26628009: Disturbance in thinking
1376001: Obsessive compulsive personality
disorder
Multi-hop
traversal on
medical
knowledge graphs
<is symptom>
Obsessive-compulsive disorder is a disorder
in which people have obsessive, intrusive
thoughts, ideas or sensations that make them
feel driven to do something repetitively
6.
Knowledge Verified Interpretable Prediction through
linking to KG and definitions
31
Gaur, M., Desai, A., Faldu, K., & Sheth, A. (2020). Explainable ai using knowledge graphs. In ACM CoDS-COMAD Conference. Link, slide.
32. Process Knowledge Structure in C-SSRS
C-SSRS: Columbia Suicide Severity Rating Scale
I wish I could give a shit about what
would make it to the front page. I have
been there and got nothing. Same as
my life. I do have a gun.’, ’I thought I
was talking about it. I am not on a
ledge or something, but I do have my
gun in my lap.’, ’No. I made sure she
got an education and she knows how
to get a job. I also have recently
bought her clothes to make her more
attractive. She has told me she only
loves me because I buy her things.
1. Wish to be dead - Yes
2. Non-specific Active Suicidal
Thoughts - Yes
3. Active Suicidal Ideation with Some
Intent to Act - Yes
4. Label: Suicide Behavior or Attempt
Interpretable for System Users
i.e., Clinicians and Patients
(1,2,3 verify adherence to the
clinical guideline on diagnosis
which a clinician understands)
47%
70%
LLMs Process Knowledge
(Ours)
Agreement with Experts
6.
Knowledge Verified Interpretable Prediction
through Process Knowledge Structures
32
Sheth, A., Gaur, M., Roy, K., Venkataraman, R., & Khandelwal, V. (2022). Process Knowledge-Infused AI: Toward User-Level Explainability, Interpretability, and Safety. IEEE
Internet Computing, 26(5), 76-84., link
33. Do you feel nervous?
More than half the days
Do you feel irritated or
self destructive?
Do you feel something
extreme might happen to
you?
Are you able to relax?
Do you feel nervous?
More than half the days
Do you feel Irritated?
Are you bothered by
becoming easily annoyed
or irritable?
Are you bothered by any
relaxation troubles?
Knowledge
Infusion using
Medical
Questionnaire
(MedQ)
These questions
are medically
valid and safe, in
right sequence..
Safety
Check
s
6.
Knowledge Verified Interpretable and Safe Text
Generation through Process Knowledge Structures
33
Roy, K., Gaur, M., Soltani, M., Rawte, V., Kalyan, A., & Sheth, A. (2023). ProKnow: Process knowledge for safety constrained and explainable question generation for mental
health diagnostic assistance. Frontiers in Big Data, 5., link
34. ● If the system were to give user-level explanation, it will need to incorporate/use
conceptual model (vocabularies, knowledge graph) used by the user. Purely data
driven system can at best give explanations that ML engineers (developers)
can use.
● Knowledge is multifaceted. I presented diverse knowledge to support different
levels of abstractions for NLU. There will be different knowledge for
abstractions involved in image understanding activity.
● Should you bring knowledge to the data level (embedding) or bring data to
the knowledge level (learn from data and align with knowledge)? For less
demanding intellectual activities (classification, prediction, recommendation) the
former will do; for more demanding intellectual activities (decision making and
taking actions with explanations), latter is most likely needed.
Takeaway
34
35. 35
Acknowledgement
Funding supported in part by NSF Award#:
2133842 EAGER:
Advancing Neuro-symbolic AI with Deep
Knowledge-infused Learning
and other projects (see http://wiki.aiisc.ai)
Learn more:
● Website - http://aiisc.ai (projects, people, opensource, demos, open
data/tools, tutorials, workshops, papers)
● Wiki Page - http://wiki.aiisc.ai
● LinkedIn - http://linkedin.com/company/aiisc
● YouTube - http://youtube.com/aiisc (demos, tutorials, dissertations,
keynotes, invited talks)
36. Sheth core group
AIISC
Artificial Intelligence Institute of South Carolina (#AIISC)
Kaushik Roy Vedant Khandelwal
Also, Megha Chakraborty, Vipula Rawte, Yuxin Zi
36
Contribution/special thanks (this talk):