• Save
What Watson tell us about Cognitive Computing
Upcoming SlideShare
Loading in...5
×
 

What Watson tell us about Cognitive Computing

on

  • 4,098 views

http://thesocialweb2013.wordpress.com/2013/03/11/guest-lecture-by-chris-welty-ibm-research-what-watson-tell-us-about-cognitive-computing/ ...

http://thesocialweb2013.wordpress.com/2013/03/11/guest-lecture-by-chris-welty-ibm-research-what-watson-tell-us-about-cognitive-computing/

Watson is a computer system capable of answering rich natural language questions and estimating its confidence in those answers at a level rivaling the best humans at the task. On Feb 14-16, 2011, in an historic event, Watson triumphed over the best Jeopardy! players of all time. This success was important in numerous ways, one of which is as a prominent exemplar of a new generation of computing systems, that we now call Cognitive Computing Systems. Cognitive computing involves a new synthesis of classic AI problems such as language understanding, image and video processing, with big data, human computing, and massive processing. Social systems are an important driver of cognitive computing, as they provide data used by many systems (dbpedia, freebase, twitter) as well as numerous potential applications. In this talk I will discuss how Watson works, and cognitive computing with an eye towards social computing.

Statistics

Views

Total Views
4,098
Views on SlideShare
3,622
Embed Views
476

Actions

Likes
6
Downloads
0
Comments
0

9 Embeds 476

http://thesocialweb2014.wordpress.com 303
http://thesocialweb2013.wordpress.com 80
https://twitter.com 63
http://nsccdev.co.uk 20
http://informaticasergip.blogspot.com.es 5
http://www.pearltrees.com 2
http://informaticasergip.blogspot.com 1
http://www.scientias.nl 1
https://www.linkedin.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

What Watson tell us about Cognitive Computing What Watson tell us about Cognitive Computing Presentation Transcript

  • What Watson tells us aboutCognitive ComputingChris WeltyIBM Researchibmwatson.comDo Not Record. Do Not Distribute. © 2011 IBM Corporation
  • What is Watson? §  Open Domain Question-Answering Machine §  Given –  Rich Natural Language Questions –  Over a Broad Domain of Knowledge §  Delivers –  Precise Answers: Determine what is being asked & give precise response –  Accurate Confidences: Determine likelihood answer is correct –  Consumable Justifications: Explain why the answer is right –  Fast Response Time: Precision & Confidence in <3 seconds –  At the level of human experts – Proved its mettle in a televised match –  Won a 2-game Jeopardy match against the all-time winners –  viewed by over 50,000,0002 © 2011 IBM Corporation
  • What is Jeopardy?§ Jeopardy! is an American quiz show – 1964 – Today – Household name in U.S.§ answer-and-question format – contestants are presented with clues in the form of answers – must phrase their responses in question form. – Open domain trivia questions, speed is a big factor§  Example –  Category: General Science –  Clue: When hit by electrons, a phosphor gives off electromagnetic energy in this form –  Answer: What is light? © 2011 IBM Corporation
  • What is Cognitive Computing?§  Increasingly, machines are being asked to add their computational power to problems which are not inherently solvable§  Traditionally, these problems came from AI –  The hardest AI problems are the easiest for human intelligence: vision, speech, natural language – these are not actually associated with “being intelligent” –  Human intelligence provides solutions, but does not scale§  Cognitive Computing is founded on four principles Learn & improve. Cognitive computing systems Assist & augment human cognition. Cognitive focus on inexact solutions to unsolvable problems computing addresses problems that lie squarely in that utilize machine learning and improve over time. the province of human intelligence, but where we Often they combine multiple approaches and must cant handle the volume of information, penetrate the integrate them effectively. They must learn from complexity or otherwise extend our reach humans, in more and more seamless ways. (physically). Interact in a natural way. Cognitive computing Speed&Scale. Cognitive computing harnesses the provides technologies that support a higher level of clear advantage machines have over humans in human cognition by adapting to human approaches their ability to perform mundane tasks of arbitrary and interfaces...over the next several decades it will complexity repeatedly, whether it is the scale of the incorporate essentially all the ways humans sense data or the complexity of the task. and interact. © 2011 IBM Corporation
  • Examples of Cognitive Computing§ Web Search§ Image Search§ Event Search§ Social Computing§ Natural Language Processing © 2011 IBM Corporation
  • The Jeopardy! ChallengeHard for humans, hard for machines $200 $1000 Broad/Open If you are looking at The first person Domain the wainscoating,for different reasons.by name in But hard mentioned you are looking in ‘The Man in the Iron this direction. Mask’ is this hero of a Complex previous book by the Language Who is same author. What is down? D’Artagnan? High For people, the challenge is knowing the answer Precision For machines, the challenge is understanding the question Accurate Confidence $600 $800 In cell division, mitosis The conspirators against splits the nucleus & this man were wounded by High cytokinesis splits this each other while they Speed What is liquid cushioning the Who is Julius stabbed at him nucleus cytoplasm? Caesar?6 © 2011 IBM Corporation
  • The Winner’s CloudWhat It Takes to compete against Top Human Jeopardy! Players Each dot – actual historical human Jeopardy! gamesTop humanplayers areremarkably good. Winning Human Performance Grand Champion Human Performance 2007 QA Computer System More Confident Less Confident © 2011 IBM Corporation
  • The Winner’s Cloud What It Takes to compete against Top Human Jeopardy! Players Each dot – actual historical human Jeopardy! games Winning Human In 2007, we committed to Performance making a Huge Leap! Grand Champion Human PerformanceComputers? 2007 QA Computer SystemNot So Good. More Confident Less Confident © 2011 IBM Corporation
  • DeepQA: The Technology Behind Watson An example of a new software paradigm DeepQA generates and scores many hypotheses using an extensible collection of Natural Language Processing, Machine Learning and Reasoning Algorithms. These gather and weigh evidence over both unstructured and structured content to determine the answer with the best confidence. Learned Models help combine and weigh the Evidence Evidence Sources Answer Models Models Sources DeepQuestion Answer Evidence Models Models Evidence Scoring Retrieval Primary Candidate Scoring Answer Models Models Search GenerationQuestion & Final Confidence Question Hypothesis Hypothesis and Topic Synthesis Merging & Decomposition Generation Evidence Scoring Analysis Ranking Hypothesis Hypothesis and Evidence Generation Scoring Answer & Confidence ... © 2011 IBM Corporation
  • Example Question Keywords: 1894, C.W. Post, Related ContentIn 1894 C.W. Post created … (Structured & Unstructured)created his warm Lexical AnswerType: (Michingan city)cereal drink Postum in Date(1894)this Michigan city Primary Question Relations: Search Analysis Create(Post, cereal drink) … Candidate Answer Generation General Foods [0.58 0 -1.3 … 0.97] 1985 [0.71 1 13.4 … 0.72] Post Foods [0.12 0 2.0 … 0.40] 1)  Battle Creek (0.85) aramour Battle Creek [0.84 1 10.6 … 0.21] 2)  Post Foods ( 0.20) Grand Rapids [0.33 0 6.3 … 0.83] 3)  1985 (0.05) … [0.21 1 11.1 … 0.92] [0.91 0 -8.2 … 0.61] Merging & … Ranking … [0.91 0 -1.7 … 0.60] Evidence Retrieval Evidence © 2011 IBM Corporation Scoring
  • Broad Domain We do NOT attempt to anticipate all We do NOT try to build a formal questions and build databases. model of the world Our Focus is on reusable NLP technology for analyzing vast volumes of as-is text. Structured sources (DBs and KBs) provide background knowledge for interpreting the text. © 2011 IBM Corporation
  • Hypothesis Scoring Category: MICHIGAN MANIA Clue: In 1894 C.W. Post created his warm cereal drink Postum in this Michigan city Tycor Answer Scorers can be applied depending on different relations or constraints detected in the Temporal question. For example, this question focus with modifiers is “Michigan city.” Watson can detect this as a geospatial relation that indicates the correct answer must be a city spatially Spatial located within the sate of Michigan. Popularity … Candidate Answers Evidence Feature Scores (Answer Scoring + Passage Scoring) Doc Rank Pass Rank Ty Cor Geo General Foods 0 1 0.1 0 Post Foods 2 1 0.1 0 Battle Creek 1 2 0.8 1 Will Keith Kellogg 3 0.1 0 Grand Rapids 0.9 1 1895 0 0.0 0 © 2011 IBM Corporation
  • Passage Scoring Category: MICHIGAN MANIA Clue: In 1894 C.W. Post created his warm cereal drink Postum in this Michigan city In Deep Evidence Scoring, Watson retrieves evidence for each candidate answer, then evaluates the evidence using a large number of deep evidence scoring analytics. The evidence for a candidate answer may come from the original document or passage where the candidate answer was generated, or it may come from an evidence retrieval search performed by taking the keyword search query from Step 2, replacing the focus terms with the candidate answer, and retrieving the relevant passages that are found. The passages, or “context” in which the candidate answer occurs are evaluated as evidence to support or refute the candidate answer as the correct answer for the question. Battle Creek General Foods Post Foods 1895: In Battle Creek, Michigan, C.W. 1854 C. W. Post (Charles William) was Post made the first POSTUM , a cereal C.W. Post came to the Battle Creek born. He founded the Postum Cereal Co. beverage. Post created GRAPE-NUTS sanitarium to cure his upset stomach. in 1895 (renamed General Foods Corp.breakfast General Foods products go from cereal in 1897, and POST TOASTIES He later created Postum, a cereal- in 1922) to manufacture warm nightcaps (Postum, (Posts cereals) to Postum cereal corn flakes in 1908 based coffee substitute Sanka), also wash the pots and pans that its beverageThe company was incorporated in 1922, foods are cooked in (S.O.S. Scouring Pads Post Foods, LLC, also known as Post Cerealshaving developed from the earlier Postum (formerly Postum Cereals) was founded by C.W.Cereal Co. Ltd., founded by C.W. Post Post. It began in 1895 with the first Postum, a(1854-1914) in 1895 in Battle Creek, Mich. "cereal beverage", developed by Post in BattleAfter a number of experiments, Post Creek, Michigan. The first cereal, Grape-Nuts,marketed his first product-the cereal It was named after C. W. Post, the founder of was developed in 1897beverage called Postum-in 1895 the Postum Cereal Company that later became General Foods. The cereal company unit was later sold off and is now Post Foods © 2011 IBM Corporation
  • Merging and Confidence Category: MICHIGAN MANIA Clue: In 1894 C.W. Post created his warm cereal drink Postum in this … In the final processing step, Watson detects variants of the same answer and merges their feature scores together. Watson then computes the final confidence scores for the candidate answers by applying a series of Machine Learning models that weight all of the feature scores to produce the final confidence scores.Candidate Evidence Feature Scores CorrectAnswers Answer Doc Pass Ty Cor Geo LFACS Term Temp- Rank Rank Match oral Final Answers Confi-General Foods 0 1 0.1 0 0.2 22 1 dence Battle Creek 0.946Post Foods 2 1 0.1 0 0.4 41 1 Machine Learning Post Foods 0.152Battle Creek 1 2 0.8 1 0.5 30 0.9 Model 1895 0.040Will Keith Kellogg 3 0.1 0 0 23 0.5 Application Grand Rapids 0.033Grand Rapids 0.9 1 0 10 0.5 General Foods 0.0141895 0 0.0 0 0 21 0.6 © 2011 IBM Corporation
  • “Minimal” Deep QA Pipeline Category: MICHIGAN MANIA Clue: In 1894 C.W. Post created his warm cereal drink Postum in this Michigan cityQuestion Battle Creek Final Confidence Question Primary Hypothesis Hypothesis and Merging & Analysis Search Generation Evidence Scoring Ranking Document Search Results LAT Candidate Evidence Features R Title Answers Ty Cor Geo Final Answers Confi- Mitchigan 0 General General Foods dence City Foods 1 Battle 0.1 0 Battle Creek 0.946 Post Creek Foods 0.1 0 Post Foods 0.152 2 Post Foods Battle Creek 0.8 1 1895 0.040 3 Will Keith Kellogg © 2011 IBM Corporation
  • Cut to the chase…..Watson emerges victorious © 2011 IBM Corporation
  • Technology marches forward… © 2011 IBM Corporation
  • The arrival of Cognitive ComputingLearn & improve. The core of Watson is a group ofover 100 independent algorithms that approximate a Assist & augment human cognition. Watsonsolution to the “is this the right answer to the question” depended on primarily a set of backgroundproblem. Achieving winning (human expert) documents (the corpus). The value of having accessperformance, required two hallmarks of cognitive to this kind of fact-finding power over a large (andcomputing systems: a metric to measure improvements possibly changing) corpus provides a clearto the system (the winners cloud), and a significant augmentation to human abilities.ground truth (over 200K Q-A pairs). Interact in a natural way. Watson was a significantSpeed&Scale. Watson used big data, as well as a step forward in natural language understanding, the3000 node cluster for massive computation to get most basic interface for humans. Say goodbye toanswering speeds down into the 2s range. your mouse… © 2011 IBM Corporation
  • The arrival of Cognitive ComputingLearn & improve. The core of Watson is a group ofover 100 independent algorithms that approximate asolution to the “is this the right answer to the question”problem. Achieving winning (human expert) 100%performance, required two hallmarks of cognitivecomputing systems: a metric to measure improvements 90%to the system (the winners cloud), and a significantground truth (over 200K Q-A pairs). 80% 70% 60% 50% 40% 30% 20% 10% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % Answered © 2011 IBM Corporation
  • The arrival of Cognitive Computing Symptoms   Assist & augment human cognition. Watson depended on primarily a set of background documents (the corpus). The value of having access Family  History   to this kind of fact-finding power over a large (and Pa9ent  History   possibly changing) corpus provides a clear augmentation to human abilities. Medica9ons   Tests/Findings   Diagnosis  Models   Confidence   Renal failure Notes/Hypotheses   UTI Diabetes Influenza hypokalemia Huge  Volumes  of  Texts,   esophogitis Journals,  References,  DBs   etc.   Most  Confident  Diagnosis:  UTI     © 2011 IBM Corporation
  • The arrival of Cognitive ComputingSpeed&Scale. Watson used big data, as well as a3000 node cluster for massive computation to getanswering speeds down into the 2s range. © 2011 IBM Corporation
  • The arrival of Cognitive Computing Interact in a natural way. Watson was a significant step forward in natural language understanding, the most basic interface for humans. Say goodbye to your mouse… © 2011 IBM Corporation
  • The arrival of Cognitive ComputingLearn & improve. The core of Watson is a group ofover 100 independent algorithms that approximate a Assist & augment human cognition. Watsonsolution to the “is this the right answer to the question” depended on primarily a set of backgroundproblem. Achieving winning (human expert) documents (the corpus). The value of having accessperformance, required two hallmarks of cognitive to this kind of fact-finding power over a large (andcomputing systems: a metric to measure improvements possibly changing) corpus provides a clearto the system (the winners cloud), and a significant augmentation to human abilities.ground truth (over 200K Q-A pairs). Interact in a natural way. Watson was a significantSpeed&Scale. Watson used big data, as well as a step forward in natural language understanding, the3000 node cluster for massive computation to get most basic interface for humans. Say goodbye toanswering speeds down into the 2s range. your mouse… © 2011 IBM Corporation
  • …and for Social Web§  First and foremost, social web analytics (e.g. recommendations) and Social Computing in general lie clearly in the realm of Cognitive Computing –  Uncertainty, natural language, human intelligence –  Inexact solutions that can improve with time, training –  Problems & solutions need metrics to be solvable§  All cognitive computing systems require ground truth data –  This data is expensive to collect –  Crowdsourcing is a key new technology/approach§  The user interface moving closer to people –  Natural language, speech, gestures –  In addition, integrating the collection of training data seamlessly into the interface is a key development§  Cognitive computing systems require integration of multiple, disparate, data sources –  Structured, unstructured, semi-structured –  curated, crowdsourced © 2011 IBM Corporation