© 2011 IBM Corporation© 2011 IBM Corporation
Building Watson
A Brief Overview of the Jeopardy! Challenge
Dr. Mark Sherman
...
© 2011 IBM Corporation
 Capture the imagination
– The Next Deep Blue
 Engage the scientific community
– Envision new way...
© 2011 IBM Corporation
Informed Decision Making: Search vs. Expert Q&A
Decision Maker
Search Engine
Finds Documents contai...
© 2011 IBM Corporation
Informed Decision Making: Search vs. Expert Q&A
Expert
Understands Question
Produces Possible Answe...
© 2011 IBM Corporation
Informed Decision Making: Search vs. Expert Q&A
Decision Maker
Search Engine
Finds Documents contai...
© 2011 IBM Corporation6
Broad Domain
Our Focus is on reusable NLP technology for analyzing vast volumes of as-is text.
Str...
© 2011 IBM Corporation7
What It Takes to compete against Top Human Jeopardy! Players
Our Analysis Reveals the Winner’s Clo...
© 2011 IBM Corporation8
What It Takes to compete against Top Human Jeopardy! Players
Our Analysis Reveals the Winner’s Clo...
© 2011 IBM Corporation
Baseline
v0.1 12/07
v0.3 08/08
v0.5 05/09
v0.6 10/09
v0.7 04/10
v0.4 12/08
DeepQA: Incremental Prog...
© 2011 IBM Corporation
One Jeopardy! question can take 2 hours on a single 2.6Ghz Core
Optimized & Scaled out on 2880-Core...
© 2011 IBM Corporation
Potential Business Applications
Tech Support: Help-desk, Contact Centers
Healthcare / Life Sciences...
© 2011 IBM Corporation
The Core Technical Team
Researchers and Engineers in NLP, ML, IR, KR&R and CL at
IBM Labs and a gro...
© 2011 IBM Corporation
THANK YOU
Upcoming SlideShare
Loading in …5
×

CMU 2011 Watson Event

340 views
231 views

Published on

Background information on Watson shown to the CMU community viewing of the final Jeopardy game.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
340
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

CMU 2011 Watson Event

  1. 1. © 2011 IBM Corporation© 2011 IBM Corporation Building Watson A Brief Overview of the Jeopardy! Challenge Dr. Mark Sherman IBM Software Group Strategy
  2. 2. © 2011 IBM Corporation  Capture the imagination – The Next Deep Blue  Engage the scientific community – Envision new ways for computers to impact society & science – Drive important and measurable scientific advances  Be Relevant to IBM Customers – Enable better, faster decision making – Business Intelligence, Knowledge Discovery and Management, Government, Compliance, Publishing, Legal, Healthcare, Business Integrity, Customer Relationship Management, Web Self-Service, Product Support, etc. A Grand Challenge Opportunity 2
  3. 3. © 2011 IBM Corporation Informed Decision Making: Search vs. Expert Q&A Decision Maker Search Engine Finds Documents containing Keywords Delivers Documents based on Popularity Has Question Distills to 2-3 Keywords Reads Documents, Finds Answers Finds & Analyzes Evidence
  4. 4. © 2011 IBM Corporation Informed Decision Making: Search vs. Expert Q&A Expert Understands Question Produces Possible Answers & Evidence Delivers Response, Evidence & Confidence Analyzes Evidence, Computes Confidence Asks NL Question Considers Answer & Evidence Decision Maker
  5. 5. © 2011 IBM Corporation Informed Decision Making: Search vs. Expert Q&A Decision Maker Search Engine Finds Documents containing Keywords Delivers Documents based on Popularity Has Question Distills to 2-3 Keywords Reads Documents, Finds Answers Finds & Analyzes Evidence Expert Understands Question Produces Possible Answers & Evidence Delivers Response, Evidence & Confidence Analyzes Evidence, Computes Confidence Asks NL Question Considers Answer & Evidence Decision Maker
  6. 6. © 2011 IBM Corporation6 Broad Domain Our Focus is on reusable NLP technology for analyzing vast volumes of as-is text. Structured sources (DBs and KBs) provide background knowledge for interpreting the text. We do NOT attempt to anticipate all questions and build databases. In a random sample of 20,000 questions we found 2,500 distinct types*. The most frequent occurring <3% of the time. The distribution has a very long tail. And for each these types 1000’s of different things may be asked. *13% are non-distinct (e.g, it, this, these or NA) Even going for the head of the tail will barely make a dent We do NOT try to build a formal model of the world
  7. 7. © 2011 IBM Corporation7 What It Takes to compete against Top Human Jeopardy! Players Our Analysis Reveals the Winner’s Cloud Winning Human Performance Winning Human Performance Grand Champion Human Performance Grand Champion Human Performance Each dot – actual historical human Jeopardy! games More ConfidentMore Confident Less ConfidentLess Confident
  8. 8. © 2011 IBM Corporation8 What It Takes to compete against Top Human Jeopardy! Players Our Analysis Reveals the Winner’s Cloud Winning Human Performance Winning Human Performance 2007 QA Computer System2007 QA Computer System Grand Champion Human Performance Grand Champion Human Performance Each dot – actual historical human Jeopardy! games More ConfidentMore Confident Less ConfidentLess Confident Computers? Not So Good.
  9. 9. © 2011 IBM Corporation Baseline v0.1 12/07 v0.3 08/08 v0.5 05/09 v0.6 10/09 v0.7 04/10 v0.4 12/08 DeepQA: Incremental Progress in Answering Precision: 6/2007-4/2010 v0.2 05/08
  10. 10. © 2011 IBM Corporation One Jeopardy! question can take 2 hours on a single 2.6Ghz Core Optimized & Scaled out on 2880-Core IBM HPC using UIMA-AS, Watson is answering in 2-6 seconds. Question 100s Possible Answers 1000’s of Pieces of Evidence Multiple Interpretations 100,000’s scores from many simultaneous Text Analysis Algorithms100s sources . . . Hypothesis Generation Hypothesis and Evidence Scoring Final Confidence Merging & Ranking Synthesis Question & Topic Analysis Question Decomposition Hypothesis Generation Hypothesis and Evidence Scoring Answer & Confidence
  11. 11. © 2011 IBM Corporation Potential Business Applications Tech Support: Help-desk, Contact Centers Healthcare / Life Sciences: Diagnostic Assistance, Evidenced- Based, Collaborative Medicine Enterprise Knowledge Management and Business Intelligence Government: Improved Information Sharing and Security
  12. 12. © 2011 IBM Corporation The Core Technical Team Researchers and Engineers in NLP, ML, IR, KR&R and CL at IBM Labs and a growing number of universities
  13. 13. © 2011 IBM Corporation THANK YOU

×