SlideShare a Scribd company logo
From Jeopardy! To Cognitive Agents:
Effective Learning in the Wild
Eric Nyberg
Language Technologies Institute
School of Computer Science
Carnegie Mellon University
Language Technologies Institute
School of Computer Science
Carnegie Mellon University
History & Strengths:
Architecture for Info Systems
• Developed advanced service-oriented architectures for
information systems as part of IARPA AQUAINT [1]
• Contributed to the development of the Unstructured
Information Management Architecture (w/IBM) [2]
• Establish a framework for open advancement of Question
Answering systems (w/IBM) [3]
• Participated in the Jeopardy! Challenge (w/IBM) [4]
• Established OAQA Consortium at CMU for practical
applications of Question Answering (2012-)
– Sponsored by Boeing, Roche, Singapore DoD
• Joined IBM’s Cognitive Systems Institute in 2013 [5]
• Piloted Watson Challenge Course at CMU (F’14)
2
CMU’s Contributions to Watson & OAQA
Read more about CMU and Watson: http://www.cs.cmu.edu/~ehn/
• Modular architecture for QA systems
• Tools & process for error analysis
• Information retrieval for question answering
• Statistical machine learning for answer scoring
• How to find supporting evidence for answers
Dave Ferrucci and Watson visit CMU (3/11) Faculty & students receive Allan Newell
Award for Research Excellence (2/12)
IARPA AQUAINT Program
JAVELIN I JAVELIN II JAVELIN III
Book chapter
on advanced QA
architectures
CMU
adopts
UIMA
Roadmap
for QA R&D
(LREC 2002)
Ephyra I Ephyra II OpenEphyra
CMU joins Watson effort
(5 internships in 3 years)
OAQA defines common
framework, process, metrics
OAQA
Feb 2011: Watson
wins Jeopardy! Challenge
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
IBM Open Collaborative Research Awards
BlueJ / Watson
Research Sponsor
Key
Project @ Uni Karlsruhe
Project @ CMU
Project @ IBM
QA Research @ CMU:
The First 10 Years
(Oct. 2001 – Feb. 2011)
CMU QA Team: Core Collaborators (2001-2011)
Jamie Callan
Teruko Mitamura
Jaime Carbonell
Eric Nyberg
• Probabilistic Models for Answer Scoring
• Object type system / component architecture
• Source Expansion approach used by Watson
• Foundational work in machine learning for
answer extraction and answer scoring
• Tools for rapid development of QA apps
• Language-independent architecture
• Answer-scoring algorithms used by Watson
• Important extensions to the INDRI/Lemur
search engine used by Watson
What did we learn from Watson?
• QA systems can be fast, accurate, and confident enough to
perform in the real world
– Scalable, parallel architecture
– Plenty of training data available
– Agile, open advancement process
• Next big challenge: rapid domain adaptation
– Automatic configuration optimization: Given a labeled dataset
of inputs and expected outputs, automatically find the best
performing composition of existing analytics / agents to provide
a solution
– In-task learning : Cognitive agents improve performance
through proactive interaction with their users and other
external sources of knowledge (human/machine),
before/during/after performing a task
– Combine automatic configuration & optimization with in-task
learning to provide a set of personalized cognitive agents and
agent brokers to interact with end users
Automatic Optimization of QA
for TREC Genomics Questions
CSE Framework: Support automatic evaluation / optimization
of information systems using UIMA; part of the OAQA project [6]
Results of Automatic Optimization
CSE Framework found a significantly better configuration of
components compared to the prior published state of the art,
in 24 hours of clock time using a modest 30-node cluster. [7]
Other domains:QA4MRE
• Question Answering for Machine Reading
Evaluation
• Configuration space:
– 12 UIMA components were first developed
– Replace UIMA descriptors with ECD
• CSE
– 46 configurations
– 1,040 combinations
– 1,322 executions
The best trace identified by CSE
achieved 59.6% performance
gain over the original pipeline.
[Building Optimal Question Answering System Automatically using Configuration
Space Exploration (CSE) for QA4MRE 2013 Tasks Alkesh Patel, Zi Yang, Eric Nyberg
and Teruko Mitamura]
Leveraging Pre-Competitive, Open-Source
Development for Proprietary R&D
CMU
Student &
Advisor
Pre-Competitive
Requirements &
Data
Proprietary
Requirements
& Data
Open Source
Framework,
Modules &
Data
Proprietary
Modules &
Data
Industry
Sponsor
OA Consortium
Agreement
Non-Disclosure &
Employment
Agreements
proprietary extensions to
open-source software
Open Source Projects
• Repository Location: https://github.com/oaqa
• 18 public / 18 private project repositories
• 33 members (13 active committers)
QUADS: Question Answering
for Decision Support
Zi Yang1, Ying Li2, James Cai2, Eric Nyberg1
1) Carnegie Mellon University {ziy, ehn}@cs.cmu.edu
2) Roche Innovation Center {ying_l.li, james.cai}@roche.com
07/09/2014 at SIGIR 2014
Decision Making: Product
Recommendation from Review Text
Design and
usability
Brand
Functionality
Carrier
Operating
system
Weight
Thickness
Resolution
Keyboard
Decision decomposition Evidence gathering from Web
Synthesis
Brand Carrier Decision
aaa xxx Good
bbb yyy OK
ccc zzz Bad 13
Decision Making: Target Validation
Modulation
the activity
Expression in
tissues
Mutation
Clinical trials
Side effects
In vivo
In vitro
Normal
tissues
Disease
tissues
Decision decomposition Evidence gathering from
public/proprietary documents
Synthesis
In vivo Side effect Decision
Yes No Good
Yes Yes OK
No Yes Bad 14
Question Answering for Decision Support
• Decompose an end-user decision process into
weighted decision factors
• Values of atomic decision factors determined
by automatic QA system
• Overall decision value combines atomic
decision factors according to learned weights
• Significant improvement over baseline
methods for gene targeting, product rating [8]
10/02/2013: IBM Announces New
Collaboration with CMU
• Focus: “How systems should be architected to
support intelligent, natural interaction with all
kinds of information in support of complex
human tasks.” [5]
Vision
• Automatically learn and improve new analytics through
independent interaction with humans
• Examples:
1. Learn to code medical records for insurance payment
from a human expert
2. Learn to detect fraudulent transactions (e.g. insurance
claims) from a human expert
3. Automatically improve intelligent information systems
with proactive learning and machine reading
4. Learn and refine decision-making processes for accident
management & fault prediction that combine
information written in policy and procedure documents
will real-time sensor data, e.g. for mobile robot control
17
Conceptual Architecture
First phase
of framework
mostly complete
Perform
ReflectLearn
Automatically build and
execute analytic solutions
Proactively evaluate
task performance,
analyze errors, propose
learning tasks
Specification of required
analytic input/output types,
desired information sources,
example dataset.
1
23
Subject Matter
Experts (SMEs)
Analyst’s
Information
Need
Configure
Optimize
Measure
Train
Automatically execute
learning tasks, update
models, KBs, etc.
Machine Learning
Agents
• Targeted Machine
Reading
• E-R Extraction
• Set Extension
• Clarification Dialogs
• Type/instance
knowledge
• Concept learning
Crowd-Sourcing (e.g.
Amazon Mechanical Turk)
• Type instance
labeling
• Relevance
judgments
Proposed
work
Service Architecture
History and Strengths:
Proactive Machine Learning
• An approach that is more effective for learning independently
from multiple sources (“oracles”) (Carbonell et. al)
20
Traditional Active
Learning
Proactive Learning
Number of Oracles Individual (only one) Multiple, with different
capabilities, costs and areas of
expertise
Reliability Infallible (100% right) Variable across oracles and
queries, depending on difficulty,
expertise, …
Reluctance Indefatigable (always
answers)
Variable across oracles and
queries, depending on
workload, certainty, …
Cost per query Invariant (free or constant) Variable across oracles and
queries, depending on
workload, difficulty, …
Technical Challenges
• Extracting domain-specific entities, relations
– Which ones are important?
– How to interpret output of general NLP tools?
• Modeling inference
– How to represent e.g. complex biological processes
– How to leverage existing ontologies, inference rules to
build complex representations from text
• Incorporating direct user feedback
– How to present system data to the user
– What kinds / how to gather feedback
– How can the system learn effectively
Related Educational Programs @ CMU
• Language Technologies (MS, PhD)
• Master of Computational Data Science (MCDS)
• Biotechnology Innovation & Computing (MS)
• Intelligent Information Systems (MS)
References
• [1] Nyberg, E., Burger, J.D., Mardis, S., Ferrucci, D.A.: Software Architectures for Advanced
QA. ;In New Directions in Question Answering (2004) 19-30.
• [2] https://www.oasis-open.org/news/pr/oasis-members-approve-open-standard-for-
accessing-unstructured-information
• [3] https://www.research.ibm.com/deepqa/question_answering.shtml
• [4] http://www.prnewswire.com/news-releases/ibm-announces-eight-universities-
contributing-to-the-watson-computing-systems-development-115892914.html
• [5] http://www-03.ibm.com/press/us/en/pressrelease/42118.wss
• [6] http://oaqa.github.io/
• [7] Yang, Z., Garduno, E., Fang, Y., Maiberg, A., McCormack, C. and Nyberg, E. (2013).
“Building Optimal Information Systems Automatically: Configuration Space Exploration
for Biomedical Information Systems”, Proceedings of the ACM Conference on Information
and Knowledge Management
• [8] Zi Yang, Ying Li, James Cai, and Eric Nyberg. QUADS: Question Answering for Decision
Support. In Proceedings of SIGIR’2014: the Thirty-seventh Annual International ACM SIGIR
Conference on Research and Development in Information Retrieval, 2014.
Thank You!

More Related Content

What's hot

Mis
MisMis
Interpretive Structural Modeling of the Prospects of Ict Enabled Process Cont...
Interpretive Structural Modeling of the Prospects of Ict Enabled Process Cont...Interpretive Structural Modeling of the Prospects of Ict Enabled Process Cont...
Interpretive Structural Modeling of the Prospects of Ict Enabled Process Cont...
IOSR Journals
 
International Journal of Engineering (IJE) Volume (1) Issue (1)
International Journal of Engineering (IJE) Volume (1)  Issue (1)International Journal of Engineering (IJE) Volume (1)  Issue (1)
International Journal of Engineering (IJE) Volume (1) Issue (1)CSCJournals
 
10.1007%2 f978 0-387-34844-5-49
10.1007%2 f978 0-387-34844-5-4910.1007%2 f978 0-387-34844-5-49
10.1007%2 f978 0-387-34844-5-49
Ornrutai
 
Survey Based Reviewof Elicitation Problems
Survey Based Reviewof Elicitation ProblemsSurvey Based Reviewof Elicitation Problems
Survey Based Reviewof Elicitation Problems
IJERA Editor
 
A Ranking Model for Software Requirements Prioritization during Requirements ...
A Ranking Model for Software Requirements Prioritization during Requirements ...A Ranking Model for Software Requirements Prioritization during Requirements ...
A Ranking Model for Software Requirements Prioritization during Requirements ...
IJCSIS Research Publications
 
Ai project-report
Ai project-reportAi project-report
Ai project-report
zoniG
 
Taubenberger
TaubenbergerTaubenberger
Taubenberger
anesah
 
Senior ipt term4_casestudy
Senior ipt term4_casestudySenior ipt term4_casestudy
Senior ipt term4_casestudyhccit
 
Ai proposal
Ai proposalAi proposal
Ai proposal
zoniG
 
A Survey of Building Robust Business Models in Pervasive Computing
A Survey of Building Robust Business Models in Pervasive ComputingA Survey of Building Robust Business Models in Pervasive Computing
A Survey of Building Robust Business Models in Pervasive Computing
Osama M. Khaled
 
Artificial Inteligence: the begining
Artificial Inteligence: the beginingArtificial Inteligence: the begining
Artificial Inteligence: the begining
Dr. C.V. Suresh Babu
 
An Elite Model for COTS Component Selection Process
An Elite Model for COTS Component Selection ProcessAn Elite Model for COTS Component Selection Process
An Elite Model for COTS Component Selection Process
IJEACS
 
Upslis faculty tenure lecture presentation
Upslis faculty tenure lecture presentationUpslis faculty tenure lecture presentation
Upslis faculty tenure lecture presentation
Sonia Pascua
 
Affective Metacognitive Scaffolding and User Model Augmentation for Experient...
Affective Metacognitive Scaffolding and User Model Augmentation for Experient...Affective Metacognitive Scaffolding and User Model Augmentation for Experient...
Affective Metacognitive Scaffolding and User Model Augmentation for Experient...
Adam Moore
 
Iare ds lecture_notes_2
Iare ds lecture_notes_2Iare ds lecture_notes_2
Iare ds lecture_notes_2
RajSingh734307
 
Management of time uncertainty in agile
Management of time uncertainty in agileManagement of time uncertainty in agile
Management of time uncertainty in agile
ijseajournal
 
4 mohammad naderuzzaman_finalpaper--35-40
4 mohammad naderuzzaman_finalpaper--35-404 mohammad naderuzzaman_finalpaper--35-40
4 mohammad naderuzzaman_finalpaper--35-40
Alexander Decker
 
Education: Master IT (health)
Education: Master IT (health)Education: Master IT (health)

What's hot (20)

Mis
MisMis
Mis
 
Interpretive Structural Modeling of the Prospects of Ict Enabled Process Cont...
Interpretive Structural Modeling of the Prospects of Ict Enabled Process Cont...Interpretive Structural Modeling of the Prospects of Ict Enabled Process Cont...
Interpretive Structural Modeling of the Prospects of Ict Enabled Process Cont...
 
International Journal of Engineering (IJE) Volume (1) Issue (1)
International Journal of Engineering (IJE) Volume (1)  Issue (1)International Journal of Engineering (IJE) Volume (1)  Issue (1)
International Journal of Engineering (IJE) Volume (1) Issue (1)
 
10.1007%2 f978 0-387-34844-5-49
10.1007%2 f978 0-387-34844-5-4910.1007%2 f978 0-387-34844-5-49
10.1007%2 f978 0-387-34844-5-49
 
Survey Based Reviewof Elicitation Problems
Survey Based Reviewof Elicitation ProblemsSurvey Based Reviewof Elicitation Problems
Survey Based Reviewof Elicitation Problems
 
A Ranking Model for Software Requirements Prioritization during Requirements ...
A Ranking Model for Software Requirements Prioritization during Requirements ...A Ranking Model for Software Requirements Prioritization during Requirements ...
A Ranking Model for Software Requirements Prioritization during Requirements ...
 
Ai project-report
Ai project-reportAi project-report
Ai project-report
 
Taubenberger
TaubenbergerTaubenberger
Taubenberger
 
Senior ipt term4_casestudy
Senior ipt term4_casestudySenior ipt term4_casestudy
Senior ipt term4_casestudy
 
Ai proposal
Ai proposalAi proposal
Ai proposal
 
A Survey of Building Robust Business Models in Pervasive Computing
A Survey of Building Robust Business Models in Pervasive ComputingA Survey of Building Robust Business Models in Pervasive Computing
A Survey of Building Robust Business Models in Pervasive Computing
 
Artificial Inteligence: the begining
Artificial Inteligence: the beginingArtificial Inteligence: the begining
Artificial Inteligence: the begining
 
An Elite Model for COTS Component Selection Process
An Elite Model for COTS Component Selection ProcessAn Elite Model for COTS Component Selection Process
An Elite Model for COTS Component Selection Process
 
Multiview Methodology
Multiview MethodologyMultiview Methodology
Multiview Methodology
 
Upslis faculty tenure lecture presentation
Upslis faculty tenure lecture presentationUpslis faculty tenure lecture presentation
Upslis faculty tenure lecture presentation
 
Affective Metacognitive Scaffolding and User Model Augmentation for Experient...
Affective Metacognitive Scaffolding and User Model Augmentation for Experient...Affective Metacognitive Scaffolding and User Model Augmentation for Experient...
Affective Metacognitive Scaffolding and User Model Augmentation for Experient...
 
Iare ds lecture_notes_2
Iare ds lecture_notes_2Iare ds lecture_notes_2
Iare ds lecture_notes_2
 
Management of time uncertainty in agile
Management of time uncertainty in agileManagement of time uncertainty in agile
Management of time uncertainty in agile
 
4 mohammad naderuzzaman_finalpaper--35-40
4 mohammad naderuzzaman_finalpaper--35-404 mohammad naderuzzaman_finalpaper--35-40
4 mohammad naderuzzaman_finalpaper--35-40
 
Education: Master IT (health)
Education: Master IT (health)Education: Master IT (health)
Education: Master IT (health)
 

Viewers also liked

Bba diversity conference report 6.5.12
Bba diversity conference report 6.5.12Bba diversity conference report 6.5.12
Bba diversity conference report 6.5.12
SuffolkLALSA
 
Presentación sobre el Informe de Desarrollo Sostenible de Euskadi
Presentación sobre el Informe de Desarrollo Sostenible de EuskadiPresentación sobre el Informe de Desarrollo Sostenible de Euskadi
Presentación sobre el Informe de Desarrollo Sostenible de Euskadi
PEGIP2020
 
Mad Max Tribute
Mad Max TributeMad Max Tribute
Mad Max Tribute
Nova Media
 
Albania america community twining project 01 club twin
Albania america community twining project  01 club twinAlbania america community twining project  01 club twin
Albania america community twining project 01 club twin
Avi Dey
 
Gotham ΙΙΙ
Gotham ΙΙΙGotham ΙΙΙ
Gotham ΙΙΙ
Nova Media
 
Центр поддержки экологических проектов
Центр поддержки экологических проектовЦентр поддержки экологических проектов
Центр поддержки экологических проектовkulibin
 
СПГСГ,,Арх.Георги Козаров" гр.Сливен
СПГСГ,,Арх.Георги Козаров" гр.СливенСПГСГ,,Арх.Георги Козаров" гр.Сливен
СПГСГ,,Арх.Георги Козаров" гр.Сливен
Искра Косева
 
Birgit Plietzsch “RDM within research computing support” SALCTG June 2013
Birgit Plietzsch “RDM within research computing support” SALCTG June 2013Birgit Plietzsch “RDM within research computing support” SALCTG June 2013
Birgit Plietzsch “RDM within research computing support” SALCTG June 2013
SALCTG
 
5 Ways Your SMB Can Make More Money Using Social Media
5 Ways Your SMB Can Make More Money Using Social Media5 Ways Your SMB Can Make More Money Using Social Media
5 Ways Your SMB Can Make More Money Using Social Media
Dave Kerpen
 
Responsive design in sp seminar buildingi 20130918
Responsive design in sp seminar buildingi 20130918Responsive design in sp seminar buildingi 20130918
Responsive design in sp seminar buildingi 20130918
Bill England
 
Business innovation rick huijbregts 16oct2013
Business innovation   rick huijbregts 16oct2013Business innovation   rick huijbregts 16oct2013
Business innovation rick huijbregts 16oct2013
Rick Huijbregts
 
1 cv en wilson couto
1 cv en wilson couto1 cv en wilson couto
1 cv en wilson couto
Wilson de Almeida Couto, Jr.
 
Μαθαίνω τον ΗΥ
Μαθαίνω τον ΗΥΜαθαίνω τον ΗΥ
Μαθαίνω τον ΗΥ
Expertin
 
Debi banks digital footprint
Debi banks digital footprintDebi banks digital footprint
Debi banks digital footprintDebi Banks
 
5 Tips on Twitter: #Live Storytelling w/ Patrick Stewart at #CannesLions #Ogi...
5 Tips on Twitter: #Live Storytelling w/ Patrick Stewart at #CannesLions #Ogi...5 Tips on Twitter: #Live Storytelling w/ Patrick Stewart at #CannesLions #Ogi...
5 Tips on Twitter: #Live Storytelling w/ Patrick Stewart at #CannesLions #Ogi...
Ogilvy
 
New orleans
New orleansNew orleans
New orleans
György Vámos
 
Врезка_замка_в_межкомнатную_дверь
Врезка_замка_в_межкомнатную_дверьВрезка_замка_в_межкомнатную_дверь
Врезка_замка_в_межкомнатную_дверьspecdveri
 
Vimbai s Curriculum Vitae-1
Vimbai s Curriculum Vitae-1Vimbai s Curriculum Vitae-1
Vimbai s Curriculum Vitae-1Vimbai Vsingizi
 
задание 6 насекина
задание 6 насекиназадание 6 насекина
задание 6 насекина
Kriiiiis
 

Viewers also liked (20)

Bba diversity conference report 6.5.12
Bba diversity conference report 6.5.12Bba diversity conference report 6.5.12
Bba diversity conference report 6.5.12
 
Presentación sobre el Informe de Desarrollo Sostenible de Euskadi
Presentación sobre el Informe de Desarrollo Sostenible de EuskadiPresentación sobre el Informe de Desarrollo Sostenible de Euskadi
Presentación sobre el Informe de Desarrollo Sostenible de Euskadi
 
Mad Max Tribute
Mad Max TributeMad Max Tribute
Mad Max Tribute
 
Albania america community twining project 01 club twin
Albania america community twining project  01 club twinAlbania america community twining project  01 club twin
Albania america community twining project 01 club twin
 
Return vs output
Return vs outputReturn vs output
Return vs output
 
Gotham ΙΙΙ
Gotham ΙΙΙGotham ΙΙΙ
Gotham ΙΙΙ
 
Центр поддержки экологических проектов
Центр поддержки экологических проектовЦентр поддержки экологических проектов
Центр поддержки экологических проектов
 
СПГСГ,,Арх.Георги Козаров" гр.Сливен
СПГСГ,,Арх.Георги Козаров" гр.СливенСПГСГ,,Арх.Георги Козаров" гр.Сливен
СПГСГ,,Арх.Георги Козаров" гр.Сливен
 
Birgit Plietzsch “RDM within research computing support” SALCTG June 2013
Birgit Plietzsch “RDM within research computing support” SALCTG June 2013Birgit Plietzsch “RDM within research computing support” SALCTG June 2013
Birgit Plietzsch “RDM within research computing support” SALCTG June 2013
 
5 Ways Your SMB Can Make More Money Using Social Media
5 Ways Your SMB Can Make More Money Using Social Media5 Ways Your SMB Can Make More Money Using Social Media
5 Ways Your SMB Can Make More Money Using Social Media
 
Responsive design in sp seminar buildingi 20130918
Responsive design in sp seminar buildingi 20130918Responsive design in sp seminar buildingi 20130918
Responsive design in sp seminar buildingi 20130918
 
Business innovation rick huijbregts 16oct2013
Business innovation   rick huijbregts 16oct2013Business innovation   rick huijbregts 16oct2013
Business innovation rick huijbregts 16oct2013
 
1 cv en wilson couto
1 cv en wilson couto1 cv en wilson couto
1 cv en wilson couto
 
Μαθαίνω τον ΗΥ
Μαθαίνω τον ΗΥΜαθαίνω τον ΗΥ
Μαθαίνω τον ΗΥ
 
Debi banks digital footprint
Debi banks digital footprintDebi banks digital footprint
Debi banks digital footprint
 
5 Tips on Twitter: #Live Storytelling w/ Patrick Stewart at #CannesLions #Ogi...
5 Tips on Twitter: #Live Storytelling w/ Patrick Stewart at #CannesLions #Ogi...5 Tips on Twitter: #Live Storytelling w/ Patrick Stewart at #CannesLions #Ogi...
5 Tips on Twitter: #Live Storytelling w/ Patrick Stewart at #CannesLions #Ogi...
 
New orleans
New orleansNew orleans
New orleans
 
Врезка_замка_в_межкомнатную_дверь
Врезка_замка_в_межкомнатную_дверьВрезка_замка_в_межкомнатную_дверь
Врезка_замка_в_межкомнатную_дверь
 
Vimbai s Curriculum Vitae-1
Vimbai s Curriculum Vitae-1Vimbai s Curriculum Vitae-1
Vimbai s Curriculum Vitae-1
 
задание 6 насекина
задание 6 насекиназадание 6 насекина
задание 6 насекина
 

Similar to Ibm colloquium 070915_nyberg

Software Development Life Cycle
Software Development Life CycleSoftware Development Life Cycle
Software Development Life Cycle
university of education,Lahore
 
Computer Based Assessment.pptx
Computer Based Assessment.pptxComputer Based Assessment.pptx
Computer Based Assessment.pptx
salah327743
 
Preliminry report
 Preliminry report Preliminry report
Preliminry reportJiten Ahuja
 
C19013010 the tutorial to build shared ai services session 1
C19013010  the tutorial to build shared ai services session 1C19013010  the tutorial to build shared ai services session 1
C19013010 the tutorial to build shared ai services session 1
Bill Liu
 
Activity Monitoring Using Wearable Sensors and Smart Phone
Activity Monitoring Using Wearable Sensors and Smart PhoneActivity Monitoring Using Wearable Sensors and Smart Phone
Activity Monitoring Using Wearable Sensors and Smart Phone
DrAhmedZoha
 
CRITERION BASED AUTOMATIC GENERATION OF QUESTION PAPER
CRITERION BASED AUTOMATIC GENERATION OF QUESTION PAPERCRITERION BASED AUTOMATIC GENERATION OF QUESTION PAPER
CRITERION BASED AUTOMATIC GENERATION OF QUESTION PAPER
vivatechijri
 
Role of artificial intellligence in construction engg & management
Role of artificial intellligence in construction engg & managementRole of artificial intellligence in construction engg & management
Role of artificial intellligence in construction engg & management
Kundan Sanap
 
the application of machine lerning algorithm for SEE
the application of machine lerning algorithm for SEEthe application of machine lerning algorithm for SEE
the application of machine lerning algorithm for SEE
KiranKumar671235
 
Online examination management system..pdf
Online examination management system..pdfOnline examination management system..pdf
Online examination management system..pdf
Kamal Acharya
 
Machine Learning & Predictive Maintenance
Machine Learning &  Predictive MaintenanceMachine Learning &  Predictive Maintenance
Machine Learning & Predictive Maintenance
Arnab Biswas
 
AI for Software Engineering
AI for Software EngineeringAI for Software Engineering
AI for Software Engineering
Miroslaw Staron
 
Digital Security by Design: ISCF Digital Security by Design Research Projects...
Digital Security by Design: ISCF Digital Security by Design Research Projects...Digital Security by Design: ISCF Digital Security by Design Research Projects...
Digital Security by Design: ISCF Digital Security by Design Research Projects...
KTN
 
Web-Based Self- and Peer-Assessment of Teachers’ Educational Technology Compe...
Web-Based Self- and Peer-Assessment of Teachers’ Educational Technology Compe...Web-Based Self- and Peer-Assessment of Teachers’ Educational Technology Compe...
Web-Based Self- and Peer-Assessment of Teachers’ Educational Technology Compe...
Hans Põldoja
 
Software Estimation Techniques
Software Estimation TechniquesSoftware Estimation Techniques
Software Estimation Techniques
kamal
 
renita lobo-CV-Automation
renita lobo-CV-Automationrenita lobo-CV-Automation
renita lobo-CV-AutomationRenita Lobo
 
Intelligent Career Guidance System.pptx
Intelligent Career Guidance System.pptxIntelligent Career Guidance System.pptx
Intelligent Career Guidance System.pptx
Anonymous366406
 
Machine learning
Machine learningMachine learning
Machine learning
osman ansari
 

Similar to Ibm colloquium 070915_nyberg (20)

Sdlc 4
Sdlc 4Sdlc 4
Sdlc 4
 
Software Development Life Cycle
Software Development Life CycleSoftware Development Life Cycle
Software Development Life Cycle
 
Computer Based Assessment.pptx
Computer Based Assessment.pptxComputer Based Assessment.pptx
Computer Based Assessment.pptx
 
Preliminry report
 Preliminry report Preliminry report
Preliminry report
 
C19013010 the tutorial to build shared ai services session 1
C19013010  the tutorial to build shared ai services session 1C19013010  the tutorial to build shared ai services session 1
C19013010 the tutorial to build shared ai services session 1
 
Activity Monitoring Using Wearable Sensors and Smart Phone
Activity Monitoring Using Wearable Sensors and Smart PhoneActivity Monitoring Using Wearable Sensors and Smart Phone
Activity Monitoring Using Wearable Sensors and Smart Phone
 
CRITERION BASED AUTOMATIC GENERATION OF QUESTION PAPER
CRITERION BASED AUTOMATIC GENERATION OF QUESTION PAPERCRITERION BASED AUTOMATIC GENERATION OF QUESTION PAPER
CRITERION BASED AUTOMATIC GENERATION OF QUESTION PAPER
 
Naveen Prakash Resume
Naveen Prakash ResumeNaveen Prakash Resume
Naveen Prakash Resume
 
Role of artificial intellligence in construction engg & management
Role of artificial intellligence in construction engg & managementRole of artificial intellligence in construction engg & management
Role of artificial intellligence in construction engg & management
 
the application of machine lerning algorithm for SEE
the application of machine lerning algorithm for SEEthe application of machine lerning algorithm for SEE
the application of machine lerning algorithm for SEE
 
Online examination management system..pdf
Online examination management system..pdfOnline examination management system..pdf
Online examination management system..pdf
 
Machine Learning & Predictive Maintenance
Machine Learning &  Predictive MaintenanceMachine Learning &  Predictive Maintenance
Machine Learning & Predictive Maintenance
 
AI for Software Engineering
AI for Software EngineeringAI for Software Engineering
AI for Software Engineering
 
Student report
Student reportStudent report
Student report
 
Digital Security by Design: ISCF Digital Security by Design Research Projects...
Digital Security by Design: ISCF Digital Security by Design Research Projects...Digital Security by Design: ISCF Digital Security by Design Research Projects...
Digital Security by Design: ISCF Digital Security by Design Research Projects...
 
Web-Based Self- and Peer-Assessment of Teachers’ Educational Technology Compe...
Web-Based Self- and Peer-Assessment of Teachers’ Educational Technology Compe...Web-Based Self- and Peer-Assessment of Teachers’ Educational Technology Compe...
Web-Based Self- and Peer-Assessment of Teachers’ Educational Technology Compe...
 
Software Estimation Techniques
Software Estimation TechniquesSoftware Estimation Techniques
Software Estimation Techniques
 
renita lobo-CV-Automation
renita lobo-CV-Automationrenita lobo-CV-Automation
renita lobo-CV-Automation
 
Intelligent Career Guidance System.pptx
Intelligent Career Guidance System.pptxIntelligent Career Guidance System.pptx
Intelligent Career Guidance System.pptx
 
Machine learning
Machine learningMachine learning
Machine learning
 

More from diannepatricia

Teaching cognitive computing with ibm watson
Teaching cognitive computing with ibm watsonTeaching cognitive computing with ibm watson
Teaching cognitive computing with ibm watson
diannepatricia
 
Cognitive systems institute talk 8 june 2017 - v.1.0
Cognitive systems institute talk   8 june 2017 - v.1.0Cognitive systems institute talk   8 june 2017 - v.1.0
Cognitive systems institute talk 8 june 2017 - v.1.0
diannepatricia
 
Building Compassionate Conversational Systems
Building Compassionate Conversational SystemsBuilding Compassionate Conversational Systems
Building Compassionate Conversational Systems
diannepatricia
 
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”“Artificial Intelligence, Cognitive Computing and Innovating in Practice”
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”
diannepatricia
 
Cognitive Insights drive self-driving Accessibility
Cognitive Insights drive self-driving AccessibilityCognitive Insights drive self-driving Accessibility
Cognitive Insights drive self-driving Accessibility
diannepatricia
 
Artificial Intellingence in the Car
Artificial Intellingence in the CarArtificial Intellingence in the Car
Artificial Intellingence in the Car
diannepatricia
 
“Semantic PDF Processing & Document Representation”
“Semantic PDF Processing & Document Representation”“Semantic PDF Processing & Document Representation”
“Semantic PDF Processing & Document Representation”
diannepatricia
 
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...
diannepatricia
 
170330 cognitive systems institute speaker series mark sherman - watson pr...
170330 cognitive systems institute speaker series    mark sherman - watson pr...170330 cognitive systems institute speaker series    mark sherman - watson pr...
170330 cognitive systems institute speaker series mark sherman - watson pr...
diannepatricia
 
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”
diannepatricia
 
Cognitive Assistance for the Aging
Cognitive Assistance for the AgingCognitive Assistance for the Aging
Cognitive Assistance for the Aging
diannepatricia
 
From complex Systems to Networks: Discovering and Modeling the Correct Network"
From complex Systems to Networks: Discovering and Modeling the Correct Network"From complex Systems to Networks: Discovering and Modeling the Correct Network"
From complex Systems to Networks: Discovering and Modeling the Correct Network"
diannepatricia
 
The Role of Dialog in Augmented Intelligence
The Role of Dialog in Augmented IntelligenceThe Role of Dialog in Augmented Intelligence
The Role of Dialog in Augmented Intelligence
diannepatricia
 
Developing Cognitive Systems to Support Team Cognition
Developing Cognitive Systems to Support Team CognitionDeveloping Cognitive Systems to Support Team Cognition
Developing Cognitive Systems to Support Team Cognition
diannepatricia
 
Cyber-Social Learning Systems
Cyber-Social Learning SystemsCyber-Social Learning Systems
Cyber-Social Learning Systems
diannepatricia
 
“IT Technology Trends in 2017… and Beyond”
“IT Technology Trends in 2017… and Beyond”“IT Technology Trends in 2017… and Beyond”
“IT Technology Trends in 2017… and Beyond”
diannepatricia
 
"Curious Learning: using a mobile platform for early literacy education as a ...
"Curious Learning: using a mobile platform for early literacy education as a ..."Curious Learning: using a mobile platform for early literacy education as a ...
"Curious Learning: using a mobile platform for early literacy education as a ...
diannepatricia
 
Embodied Cognition - Booch HICSS50
Embodied Cognition - Booch HICSS50Embodied Cognition - Booch HICSS50
Embodied Cognition - Booch HICSS50
diannepatricia
 
KATE - a Platform for Machine Learning
KATE - a Platform for Machine LearningKATE - a Platform for Machine Learning
KATE - a Platform for Machine Learning
diannepatricia
 
Cognitive Computing for Aging Society
Cognitive Computing for Aging SocietyCognitive Computing for Aging Society
Cognitive Computing for Aging Society
diannepatricia
 

More from diannepatricia (20)

Teaching cognitive computing with ibm watson
Teaching cognitive computing with ibm watsonTeaching cognitive computing with ibm watson
Teaching cognitive computing with ibm watson
 
Cognitive systems institute talk 8 june 2017 - v.1.0
Cognitive systems institute talk   8 june 2017 - v.1.0Cognitive systems institute talk   8 june 2017 - v.1.0
Cognitive systems institute talk 8 june 2017 - v.1.0
 
Building Compassionate Conversational Systems
Building Compassionate Conversational SystemsBuilding Compassionate Conversational Systems
Building Compassionate Conversational Systems
 
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”“Artificial Intelligence, Cognitive Computing and Innovating in Practice”
“Artificial Intelligence, Cognitive Computing and Innovating in Practice”
 
Cognitive Insights drive self-driving Accessibility
Cognitive Insights drive self-driving AccessibilityCognitive Insights drive self-driving Accessibility
Cognitive Insights drive self-driving Accessibility
 
Artificial Intellingence in the Car
Artificial Intellingence in the CarArtificial Intellingence in the Car
Artificial Intellingence in the Car
 
“Semantic PDF Processing & Document Representation”
“Semantic PDF Processing & Document Representation”“Semantic PDF Processing & Document Representation”
“Semantic PDF Processing & Document Representation”
 
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...
Joining Industry and Students for Cognitive Solutions at Karlsruhe Services R...
 
170330 cognitive systems institute speaker series mark sherman - watson pr...
170330 cognitive systems institute speaker series    mark sherman - watson pr...170330 cognitive systems institute speaker series    mark sherman - watson pr...
170330 cognitive systems institute speaker series mark sherman - watson pr...
 
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”
“Fairness Cases as an Accelerant and Enabler for Cognitive Assistance Adoption”
 
Cognitive Assistance for the Aging
Cognitive Assistance for the AgingCognitive Assistance for the Aging
Cognitive Assistance for the Aging
 
From complex Systems to Networks: Discovering and Modeling the Correct Network"
From complex Systems to Networks: Discovering and Modeling the Correct Network"From complex Systems to Networks: Discovering and Modeling the Correct Network"
From complex Systems to Networks: Discovering and Modeling the Correct Network"
 
The Role of Dialog in Augmented Intelligence
The Role of Dialog in Augmented IntelligenceThe Role of Dialog in Augmented Intelligence
The Role of Dialog in Augmented Intelligence
 
Developing Cognitive Systems to Support Team Cognition
Developing Cognitive Systems to Support Team CognitionDeveloping Cognitive Systems to Support Team Cognition
Developing Cognitive Systems to Support Team Cognition
 
Cyber-Social Learning Systems
Cyber-Social Learning SystemsCyber-Social Learning Systems
Cyber-Social Learning Systems
 
“IT Technology Trends in 2017… and Beyond”
“IT Technology Trends in 2017… and Beyond”“IT Technology Trends in 2017… and Beyond”
“IT Technology Trends in 2017… and Beyond”
 
"Curious Learning: using a mobile platform for early literacy education as a ...
"Curious Learning: using a mobile platform for early literacy education as a ..."Curious Learning: using a mobile platform for early literacy education as a ...
"Curious Learning: using a mobile platform for early literacy education as a ...
 
Embodied Cognition - Booch HICSS50
Embodied Cognition - Booch HICSS50Embodied Cognition - Booch HICSS50
Embodied Cognition - Booch HICSS50
 
KATE - a Platform for Machine Learning
KATE - a Platform for Machine LearningKATE - a Platform for Machine Learning
KATE - a Platform for Machine Learning
 
Cognitive Computing for Aging Society
Cognitive Computing for Aging SocietyCognitive Computing for Aging Society
Cognitive Computing for Aging Society
 

Recently uploaded

Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
Cheryl Hung
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
Bhaskar Mitra
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Prayukth K V
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 

Recently uploaded (20)

Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 

Ibm colloquium 070915_nyberg

  • 1. From Jeopardy! To Cognitive Agents: Effective Learning in the Wild Eric Nyberg Language Technologies Institute School of Computer Science Carnegie Mellon University Language Technologies Institute School of Computer Science Carnegie Mellon University
  • 2. History & Strengths: Architecture for Info Systems • Developed advanced service-oriented architectures for information systems as part of IARPA AQUAINT [1] • Contributed to the development of the Unstructured Information Management Architecture (w/IBM) [2] • Establish a framework for open advancement of Question Answering systems (w/IBM) [3] • Participated in the Jeopardy! Challenge (w/IBM) [4] • Established OAQA Consortium at CMU for practical applications of Question Answering (2012-) – Sponsored by Boeing, Roche, Singapore DoD • Joined IBM’s Cognitive Systems Institute in 2013 [5] • Piloted Watson Challenge Course at CMU (F’14) 2
  • 3. CMU’s Contributions to Watson & OAQA Read more about CMU and Watson: http://www.cs.cmu.edu/~ehn/ • Modular architecture for QA systems • Tools & process for error analysis • Information retrieval for question answering • Statistical machine learning for answer scoring • How to find supporting evidence for answers Dave Ferrucci and Watson visit CMU (3/11) Faculty & students receive Allan Newell Award for Research Excellence (2/12)
  • 4. IARPA AQUAINT Program JAVELIN I JAVELIN II JAVELIN III Book chapter on advanced QA architectures CMU adopts UIMA Roadmap for QA R&D (LREC 2002) Ephyra I Ephyra II OpenEphyra CMU joins Watson effort (5 internships in 3 years) OAQA defines common framework, process, metrics OAQA Feb 2011: Watson wins Jeopardy! Challenge 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 IBM Open Collaborative Research Awards BlueJ / Watson Research Sponsor Key Project @ Uni Karlsruhe Project @ CMU Project @ IBM QA Research @ CMU: The First 10 Years (Oct. 2001 – Feb. 2011)
  • 5. CMU QA Team: Core Collaborators (2001-2011) Jamie Callan Teruko Mitamura Jaime Carbonell Eric Nyberg • Probabilistic Models for Answer Scoring • Object type system / component architecture • Source Expansion approach used by Watson • Foundational work in machine learning for answer extraction and answer scoring • Tools for rapid development of QA apps • Language-independent architecture • Answer-scoring algorithms used by Watson • Important extensions to the INDRI/Lemur search engine used by Watson
  • 6. What did we learn from Watson? • QA systems can be fast, accurate, and confident enough to perform in the real world – Scalable, parallel architecture – Plenty of training data available – Agile, open advancement process • Next big challenge: rapid domain adaptation – Automatic configuration optimization: Given a labeled dataset of inputs and expected outputs, automatically find the best performing composition of existing analytics / agents to provide a solution – In-task learning : Cognitive agents improve performance through proactive interaction with their users and other external sources of knowledge (human/machine), before/during/after performing a task – Combine automatic configuration & optimization with in-task learning to provide a set of personalized cognitive agents and agent brokers to interact with end users
  • 7. Automatic Optimization of QA for TREC Genomics Questions CSE Framework: Support automatic evaluation / optimization of information systems using UIMA; part of the OAQA project [6]
  • 8. Results of Automatic Optimization CSE Framework found a significantly better configuration of components compared to the prior published state of the art, in 24 hours of clock time using a modest 30-node cluster. [7]
  • 9. Other domains:QA4MRE • Question Answering for Machine Reading Evaluation • Configuration space: – 12 UIMA components were first developed – Replace UIMA descriptors with ECD • CSE – 46 configurations – 1,040 combinations – 1,322 executions The best trace identified by CSE achieved 59.6% performance gain over the original pipeline. [Building Optimal Question Answering System Automatically using Configuration Space Exploration (CSE) for QA4MRE 2013 Tasks Alkesh Patel, Zi Yang, Eric Nyberg and Teruko Mitamura]
  • 10. Leveraging Pre-Competitive, Open-Source Development for Proprietary R&D CMU Student & Advisor Pre-Competitive Requirements & Data Proprietary Requirements & Data Open Source Framework, Modules & Data Proprietary Modules & Data Industry Sponsor OA Consortium Agreement Non-Disclosure & Employment Agreements proprietary extensions to open-source software
  • 11. Open Source Projects • Repository Location: https://github.com/oaqa • 18 public / 18 private project repositories • 33 members (13 active committers)
  • 12. QUADS: Question Answering for Decision Support Zi Yang1, Ying Li2, James Cai2, Eric Nyberg1 1) Carnegie Mellon University {ziy, ehn}@cs.cmu.edu 2) Roche Innovation Center {ying_l.li, james.cai}@roche.com 07/09/2014 at SIGIR 2014
  • 13. Decision Making: Product Recommendation from Review Text Design and usability Brand Functionality Carrier Operating system Weight Thickness Resolution Keyboard Decision decomposition Evidence gathering from Web Synthesis Brand Carrier Decision aaa xxx Good bbb yyy OK ccc zzz Bad 13
  • 14. Decision Making: Target Validation Modulation the activity Expression in tissues Mutation Clinical trials Side effects In vivo In vitro Normal tissues Disease tissues Decision decomposition Evidence gathering from public/proprietary documents Synthesis In vivo Side effect Decision Yes No Good Yes Yes OK No Yes Bad 14
  • 15. Question Answering for Decision Support • Decompose an end-user decision process into weighted decision factors • Values of atomic decision factors determined by automatic QA system • Overall decision value combines atomic decision factors according to learned weights • Significant improvement over baseline methods for gene targeting, product rating [8]
  • 16. 10/02/2013: IBM Announces New Collaboration with CMU • Focus: “How systems should be architected to support intelligent, natural interaction with all kinds of information in support of complex human tasks.” [5]
  • 17. Vision • Automatically learn and improve new analytics through independent interaction with humans • Examples: 1. Learn to code medical records for insurance payment from a human expert 2. Learn to detect fraudulent transactions (e.g. insurance claims) from a human expert 3. Automatically improve intelligent information systems with proactive learning and machine reading 4. Learn and refine decision-making processes for accident management & fault prediction that combine information written in policy and procedure documents will real-time sensor data, e.g. for mobile robot control 17
  • 18. Conceptual Architecture First phase of framework mostly complete Perform ReflectLearn Automatically build and execute analytic solutions Proactively evaluate task performance, analyze errors, propose learning tasks Specification of required analytic input/output types, desired information sources, example dataset. 1 23 Subject Matter Experts (SMEs) Analyst’s Information Need Configure Optimize Measure Train Automatically execute learning tasks, update models, KBs, etc. Machine Learning Agents • Targeted Machine Reading • E-R Extraction • Set Extension • Clarification Dialogs • Type/instance knowledge • Concept learning Crowd-Sourcing (e.g. Amazon Mechanical Turk) • Type instance labeling • Relevance judgments Proposed work
  • 20. History and Strengths: Proactive Machine Learning • An approach that is more effective for learning independently from multiple sources (“oracles”) (Carbonell et. al) 20 Traditional Active Learning Proactive Learning Number of Oracles Individual (only one) Multiple, with different capabilities, costs and areas of expertise Reliability Infallible (100% right) Variable across oracles and queries, depending on difficulty, expertise, … Reluctance Indefatigable (always answers) Variable across oracles and queries, depending on workload, certainty, … Cost per query Invariant (free or constant) Variable across oracles and queries, depending on workload, difficulty, …
  • 21. Technical Challenges • Extracting domain-specific entities, relations – Which ones are important? – How to interpret output of general NLP tools? • Modeling inference – How to represent e.g. complex biological processes – How to leverage existing ontologies, inference rules to build complex representations from text • Incorporating direct user feedback – How to present system data to the user – What kinds / how to gather feedback – How can the system learn effectively
  • 22. Related Educational Programs @ CMU • Language Technologies (MS, PhD) • Master of Computational Data Science (MCDS) • Biotechnology Innovation & Computing (MS) • Intelligent Information Systems (MS)
  • 23. References • [1] Nyberg, E., Burger, J.D., Mardis, S., Ferrucci, D.A.: Software Architectures for Advanced QA. ;In New Directions in Question Answering (2004) 19-30. • [2] https://www.oasis-open.org/news/pr/oasis-members-approve-open-standard-for- accessing-unstructured-information • [3] https://www.research.ibm.com/deepqa/question_answering.shtml • [4] http://www.prnewswire.com/news-releases/ibm-announces-eight-universities- contributing-to-the-watson-computing-systems-development-115892914.html • [5] http://www-03.ibm.com/press/us/en/pressrelease/42118.wss • [6] http://oaqa.github.io/ • [7] Yang, Z., Garduno, E., Fang, Y., Maiberg, A., McCormack, C. and Nyberg, E. (2013). “Building Optimal Information Systems Automatically: Configuration Space Exploration for Biomedical Information Systems”, Proceedings of the ACM Conference on Information and Knowledge Management • [8] Zi Yang, Ying Li, James Cai, and Eric Nyberg. QUADS: Question Answering for Decision Support. In Proceedings of SIGIR’2014: the Thirty-seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2014.