Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
Download to read offline and view in fullscreen.


Frontiers of Computational Journalism week 9 - Knowledge representation

Download to read offline

Taught at Columbia Journalism School, Fall 2018
Full syllabus and lecture videos at

  • Be the first to like this

Frontiers of Computational Journalism week 9 - Knowledge representation

  1. 1. Frontiers of Computational Journalism Columbia Journalism School Week 9: Knowledge Representation November 14, 2018
  2. 2. This class • Structured Journalism • Ontologies and Graphs • Relations from Text
  3. 3. Structured Journalism
  4. 4. Unstructured data
  5. 5. Structured data
  6. 6. circa 2009
  7. 7. Connected China. Reuters, 2013
  8. 8. Article Metadata headline photo photo caption byline photo credit publication date dateline article body related articles
  9. 9. news markup Overall type of the object on this page, in HTML head Headline, dateline, date as additions to div/span properties Byline expressed as nested object (using itemscope) of type
  10. 10. Driving application: “rich snippets” covers not just news but music, restaurants, people, organizations, reviews, offers... Snippets, and better search-ability generally, are motivation for Google, Yahoo, Bing to push
  11. 11. Additional metadata from indexing team In database, but doesn't necessarily make it to HTML.
  12. 12. Application: content navigation Articles about “Syria” on NYT topic page More reliable than simple text search (because the relevance algorithm knows a story is "about" Syria.)
  13. 13. Wall Street is high on Molson Coors Brewing (TAP), expecting it to report earnings that are up 17.5% from a year ago when it reports its third quarter earnings on Wednesday, November 7, 2012. The consensus estimate is $1.34 per share, up from earnings of $1.14 per share a year ago. The consensus estimate has dipped over the past month, from $1.35, but it’s still up from the consensus estimate of $1.19 three months ago. For the fiscal year, analysts are expecting earnings of $3.89 per share. Revenue is projected to eclipse the year-earlier total of $954.4 million by 31%, finishing at $1.25 billion for the quarter. For the year, revenue is projected to roll in at $4.04 billion. The company’s net income has declined in the last two quarters. The company posted profit falling by 52.8% in the second quarter. This is after it reported a profit decline in the first quarter by 4.1%. Automatic story generation (AP/Narrative Science) Application: automatic stories
  14. 14. Ontologies and Graphs
  15. 15. What objects and relations are available? Often represented as class hierarchy. Arrows = “is_a” relation
  16. 16. (Part of) a real ontology, from Cyc
  17. 17. News as relations between entities “Alice attended the wedding” attended(alice, wedding) “IBM was founded in 1917.” founded(IBM, 1917) “Hurricane Sandy hit New York” hit(hurricane_sandy, New_York) Encode facts as relation(subject,object) also written (subject relation object)
  18. 18. Things we could do with this Question answering “The granddaughter of which actor starred in E.T.?” (?x acted-in “E.T.”)(?y is-a actor)(?x granddaughter-of ?y) Inference (bob brother-of alice) (alice mother-of lucy) => (bob uncle-of lucy) Answer questions using inference “how many executives of publicly-traded Canadian companies died in car crashes?
  19. 19. Every big news org has their own big ontology  topics, people, organizations, places...
  20. 20. Enter Linked Data Triples of (subject relation object), each a URL or literal <urn:x-states:New%20York> <> "NY” <> <> <> Abbreviations possible with many formats... <> rdf:type ns6:CollegeOrUniversity
  21. 21. NYT API can return linked data { "title": "Syria's Rebels Open Talks on Forging United Political Front" "body": "BEIRUT, Lebanon — Syria ’s fractious opposition groups began negotiations in Doha, Qatar, on Sunday to forge a more unified front to reshape the political landscape in a bloody conflict that claims more than 100 lives virtually every day. Given the scant prospects that any attempt to restructure the opposition will succeed — the", "dbpedia_resource_url": [ "", ""], "facet_terms": "CLINTON, HILLARY RODHAM ASSAD, BASHAR AL- SYRIA DOHA (QATAR) SYRIAN NATIONAL COUNCIL STATE DEPARTMENT WAR AND REVOLUTION DEFENSE AND MILITARY FORCES" }
  22. 22. Graph Databases in Journalism
  23. 23. Graph schema for the Panama Papers William Lyon, Neo4j blog
  24. 24. Property Graphs in the Panama Papers
  25. 25. Relations from Text
  26. 26. Objects and relations in text? names, dates, places, verbs.
  27. 27. Named Entity Recognition Extract subjects, objects, from text. Also, resolve pronouns if possible. "Gov. Andrew M. Cuomo on Wednesday gave a sea wall the nod. Because of the recent history of powerful storms hitting the area, he said, elected officials have a responsibility to consider new and innovative plans to prevent similar damage in the future."
  28. 28. Relations from sentence parsing “The water that made rivers of Avenues C and D receded on Tuesday, and the East Village was a mixture of disaster and nonchalance. A group of young men in pajama pants and shorts threw a football on East 12th Street, while workers pumped the basement of CHP Hardware on Avenue C and Eighth Street.” subject verb object
  29. 29. Stanford Open IE
  30. 30. Ontology explosions (water made rivers of Avenues C and D) (East Village was a mixture of disaster and nonchalance) (group of young men in pajama pants and shorts threw football) (workers pumped the basement of CHP Hardware ) Do we have all of these in the ontology?
  31. 31. “General Question Answering” Precision/recall tradeoff. State of the art is IBM’s DeepQA
  32. 32. DeepQA use of structured data “Watson can also use detected relations to query a triple store and directly generate candidate answers. Due to the breadth of relations in the Jeopardy domain and the variety of ways in which they are expressed, however, Watson’s current ability to effectively use curated databases to simply “look up” the answers is limited to fewer than 2 percent of the clues.” - Ferruci et. al. “Building Watson”

Taught at Columbia Journalism School, Fall 2018 Full syllabus and lecture videos at


Total views


On Slideshare


From embeds


Number of embeds