Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

IBM Watson Concept Insights

1,306 views

Published on

Using the IBM Watson API to build a cognitive application with the concept insights service.

Published in: Technology
  • Be the first to comment

IBM Watson Concept Insights

  1. 1. IBM WATSON CONCEPT INSIGHTS Building a Cognitive App Kory Becker 2016
  2. 2. WHAT IS WATSON?  2008  Able to compete with Jeopardy contestants.  2010  Capable of defeating human Jeopardy contestants on a regular basis  2011  First-place Jeopardy winner, defeating champion Ken Jennings  Present  2nd-year medical student equivalency  Preparing to take the U.S. Medical Board Exam  Watson API available to developers
  3. 3. WHAT IS WATSON, REALLY?  Natural language processing  Machine learning  Used for analyzing large amounts of unstructured data  Accessible via a collection of web APIs
  4. 4. WATSON SERVICES  Concept Expansion  Concept Insights  Dialog  Natural Language Classifier  Personality Insights  Relationship Extraction https://goo.gl/mNmiS3
  5. 5. NATURAL LANGUAGE PROCESSING  Convert text into a numerical representation  Find commonalities within data  Clustering  Make predictions from data  Classification  Category, Popularity, Sentiment, Relationships
  6. 6. BAG OF WORDS MODEL Cats like to chase mice. Dogs like to eat big bones. Corpus
  7. 7. CREATE A DICTIONARY Dictionary 0 - cats 1 - like 2 - chase 3 - mice 4 - dogs 5 - eat 6 - big 7 - bones Cats like to chase mice. Dogs like to eat big bones. Corpus
  8. 8. DIGITIZE TEXT Dictionary 0 - cats 1 - like 2 - chase 3 - mice 4 - dogs 5 - eat 6 - big 7 - bones Cats like to chase mice. 1 1 1 1 0 0 0 0 Dogs like to eat big bones. 0 1 0 0 1 1 1 1 Corpus Vector Length = 8
  9. 9. CLASSIFY DOCUMENTS (EATING) Dictionary 0 - cats 1 - like 2 - chase 3 - mice 4 - dogs 5 - eat 6 - big 7 - bones Cats like to chase mice. 1 1 1 1 0 0 0 0 Dogs like to eat big bones. 0 1 0 0 1 1 1 1 Corpus 0 1
  10. 10. PREDICT ON NEW DATA Dictionary 0 - cats 1 - like 2 - chase 3 - mice 4 - dogs 5 - eat 6 - big 7 - bones Cats like to chase mice. 1 1 1 1 0 0 0 0 Dogs like to eat big bones. 0 1 0 0 1 1 1 1 Bats eat bugs. 0 0 0 0 0 1 0 0 Corpus 0 1 ?
  11. 11. PREDICT ON NEW DATA Dictionary 0 - cats 1 - like 2 - chase 3 - mice 4 - dogs 5 - eat 6 - big 7 - bones Cats like to chase mice. 1 1 1 1 0 0 0 0 Dogs like to eat big bones. 0 1 0 0 1 1 1 1 Bats eat bugs. 0 0 0 0 0 1 0 0 Corpus 0 1 ?
  12. 12. PREDICT ON NEW DATA Dictionary 0 - cats 1 - like 2 - chase 3 - mice 4 - dogs 5 - eat 6 - big 7 - bones Cats like to chase mice. 1 1 1 1 0 0 0 0 Dogs like to eat big bones. 0 1 0 0 1 1 1 1 Bats eat bugs. 0 0 0 0 0 1 0 0 Corpus 0 1 1
  13. 13. DOES IT REALLY WORK? > data [1] "Cats like to chase mice." "Dogs like to eat big bones." > train big bone cat chase dog eat like mice y 1 0 0 1 1 0 0 1 1 0 2 1 1 0 0 1 1 1 0 1 > predict(fit, newdata = train) [1] 0 1 > data2 [1] "Bats eat bugs." > test big bone cat chase dog eat like mice 1 0 0 0 0 0 1 0 0 > predict(fit, newdata = test) [1] 1 Document Term Matrix 100% Accuracy Training Test Case Success! Source code: https://goo.gl/UxjPBs
  14. 14. DEMO 1 NATURAL LANGUAGE PROCESSING  Text analysis for:  Entity Extraction  Sentiment Analysis  Keywords and Concepts  Taxonomy  More http://www.alchemyapi.com/products/demo/alchemylanguage
  15. 15. DEMO 2 CONCEPT INSIGHTS  Discovering concept insights within AP content, which might not be found using traditional keyword search http://concept.herokuapp.com

×