Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Reveal the talking points of every episode of Game of Thrones from fans' conversations

690 views

Published on

You may not be sure how Lord Varys collects information from his little birds, but in this talk you will hear how we can collect information from our little birds.

@kristw shares a behind-the-scenes view of his latest data visualization project, which shows how each #GameOfThrones episode was discussed on Twitter. Using data visualization, we can extract and reveal the stories of every episode from fans’ Tweets.
https://interactive.twitter.com/game-of-thrones

These slides are from a talk given at Bay Area d3 User Group meetup on June 9, 2016.
http://www.meetup.com/Bay-Area-d3-User-Group/events/231281298

Published in: Data & Analytics
  • Be the first to comment

Reveal the talking points of every episode of Game of Thrones from fans' conversations

  1. 1. from fans’ conversations Krist Wongsuphasawat / @kristw Reveal the talking points of every episode of
  2. 2. Computer Engineer Bangkok, Thailand PhD in Computer Science Information Visualization Univ. of Maryland IBM Microsoft Data Visualization Scientist Twitter Krist Wongsuphasawat / @kristw
  3. 3. interactive.twitter.com open-source visualization visual analytics tools
  4. 4. Problem is coming. CHAPTER I
  5. 5. “Problem first, not solution backward” — Brian Caffo (via Ron Brookmeyer)
  6. 6. “If all you have is a hammer, everything looks like a nail.” — Abraham Maslow
  7. 7. Problem Want to know what the audience talk about a TV show
  8. 8. Problem Want to know what the audience talk about a TV show from Tweets
  9. 9. HBO’s Game of Thrones Based on a book series “A Song of Ice and Fire” Medieval Fantasy. Knights, magic and dragons.
  10. 10. Brief Story
  11. 11. A King dies.  A lot of contenders wage a war to reclaim the throne.
  12. 12. Minor characters with no claim to the throne set their own plans in action to gain power when all the major characters end up killing each other.
  13. 13. Brave/Honest/Honorable characters die. Intelligent but shady characters and characters who know nothing continue to live.
  14. 14. On another continent, there is a princess who was supposed to be rightful heir to the throne. The dead king mentioned earlier, overthrew her father and killed her entire family. She escaped. Now she is finding her way back and she has dragons.
  15. 15. While humans are busy killing each other, ice zombies “White walkers” are invading from the North. The only group who seems to care about this is neutral group called the Night’s Watch.
  16. 16. HBO’s Game of Thrones Based on a book series “A Song of Ice and Fire” Medieval Fantasy. Knights, magic and dragons. Many characters. Anybody can die. 6 seasons (57 episodes) so far Multiple storylines in each episode
  17. 17. Problem Want to know what the audience talk about a TV show from Tweets
  18. 18. Ideas Common words Too much noise
  19. 19. Ideas Common words Too much noise Characters How o"en each character were mentioned?
  20. 20. I demand a trial by prototyping. CHAPTER II
  21. 21. Prototyping Pull sample data from Twitter API Entity recognition and counting naive approach
  22. 22. List of names Daenerys Targaryen,Khaleesi Jon Snow Sansa Stark Tyrion Lannister Arya Stark Cersei Lannister Khal Drogo Gregor Clegane,Mountain Margaery Tyrell Joffrey Baratheon Bran Stark Theon Greyjoy Jaime Lannister Brienne Eddard Stark,Ned Stark Ramsay Bolton Sandor Clegane,Hound Ygritte Stannis Baratheon Petyr Baelish,Little Finger Robb Stark Bronn Varys Catelyn Stark Oberyn Martell Daario Naharis Davos Seaworth Jorah Mormont Melisandre Myrcella Baratheon Tywin Lannister Tommen Baratheon Grey Worm Tyene Sand Rickon Stark Missandei Roose Bolton Robert Baratheon Jojen Reed Jeor Mormont Tormund Giantsbane Lysa Arryn Yara Greyjoy,Asha Greyjoy Samwell Tarly,Sam Hodor Victarion Greyjoy High Sparrow Dragon Winter Dothraki
  23. 23. Sample Tweet
  24. 24. Sample Tweet
  25. 25. Sample data Character Count Hodor 10000 Jon Snow 5000 Daenerys 4000 Bran Stark 3000 … … *These numbers are made up for presentation, not real data.
  26. 26. When you play the game of vis, you iterate or you die. CHAPTER III
  27. 27. Where to go from here?
  28. 28. + episodes The Guardian & Google Trends
 http://www.theguardian.com/news/datablog/ng-interactive/2016/apr/22/game-of-thrones-the-most-googled-characters-episode-by-episode
  29. 29. + emotion
  30. 30. + connections
  31. 31. + connections
  32. 32. Gain insights from a single episode emotion & connections
  33. 33. Better matching Regular Expression /gregor([ ]?clegane)?|(the mountain)/
  34. 34. Sample data Character Count Jon Snow+Sansa 1000 Tormund+Brienne 500 Bran Stark+Hodor 300 … … Character Count Hodor 10000 Jon Snow 5000 Daenerys 4000 … … INDIVIDUALS CONNECTIONS + top emojis + top emojis *These numbers are made up for presentation, not real data.
  35. 35. Graph NODES LINKS + top emojis + top emojis Character Count Jon Snow+Sansa 1000 Tormund+Brienne 500 Bran Stark+Hodor 300 … … Character Count Hodor 1000 Jon Snow 500 Daenerys 400 … … *These numbers are made up for presentation, not real data.
  36. 36. Network Visualization Node-link diagram Force-directed layout http://blockbuilder.org/kristw/762b680690e4b2b2666dfec15838a384
  37. 37. Issue: Hairball
  38. 38. Why? Too many nodes & edges nodes = nodes.filter(n => n.count > 100) links = links.filter(l => l.count > 100) The force is (too) strong. force .charge(…) .gravity(…) .linkDistance(…) .linkStrength(…)
  39. 39. Issue: Occlusions
  40. 40. Tried: Fixed positions
  41. 41. + Collision Detection http://blockbuilder.org/kristw/2850f65d6329c5fef6d5c9118f1de6e6
  42. 42. + Community Detection https://github.com/upphiminn/jLouvain
  43. 43. + Collision Detection (with clusters) https://bl.ocks.org/mbostock/7881887
  44. 44. Issue: Convex hull http://bl.ocks.org/mbostock/4341699 d3.geom.hull(vertices)
  45. 45. x & y only, no radius
  46. 46. Example
  47. 47. Fix it
  48. 48. Fix it
  49. 49. Let’s get other episodes.
  50. 50. Hadoop remembers. CHAPTER IV
  51. 51. More data Hadoop Rewrite the scripts in Scalding to get archived data
  52. 52. How much data do we need? Whole week? 5 days? 2 days? A day? etc.
  53. 53. How much data do we need?
  54. 54. Must work for every episode Too many nodes & edges nodes = nodes.filter(n => n.count > 100) links = links.filter(l => l.count > 100) Is 100 a good number for every episode?
  55. 55. Transitions
  56. 56. … Jumpy …
  57. 57. Too fast? Slow down force.friction(f) 0 = stop 
 0.1 = very slow
 1 = very fast (frictionless)
  58. 58. A$er switching episode 1. Store old positions for existing objects. 2. Assign new initial positions.*
  59. 59. Initial positions Default: random Better starting points Heuristics based on degree of nodes
  60. 60. A$er switching episode 1. Store old positions for existing objects. 2. Assign new initial positions.* 3. Run simulation without updating <svg> for n rounds 4. Animate objects from old to new positions. 5. Resume simulation and update <svg> every tick.
  61. 61. Animate Nodes & Links Remove delay Move & Change size/thickness Add new
  62. 62. const selection = svg.selectAll('g.node') .data(nodes, d => d.entity.id); selection.exit() .transition() .duration(1000) .style('opacity', 0) .remove(); const sEnter = selection.enter().append('g') .classed('node', true) .attr('transform', d => `translate(${d.x},${d.y})`) .style('opacity', 0) .call(force.drag); sEnter.append('circle') .attr('r', d=>d.r) .style('fill', d => options.colorScale(d.entity.group)); const sTrans = selection.transition() .delay(1000) .duration(2000) .attr('transform', d => `translate(${d.x},${d.y})`) .style('opacity', 1) sTrans.select('circle') .attr('r', d=>d.r) Add “enter” nodes with opacity 0 After 1s delay, use transition to move nodes and fade in new nodes Fade “exit” nodes to opacity 0 and remove Create selection
  63. 63. Animate Communities Remove delay Move & Change shape* Add new http://blockbuilder.org/kristw/f9ffe87dd8b4038b5867e853c27cebb7
  64. 64. Default t=0 t=1
  65. 65. Smoother t=0 t=1t=0.5 t=0.51
  66. 66. Code // original path.attr('d', hull); // with custom interpolation path.attrTween('d', (d,i,currentAttr) => interpolateHull(d, currentAttr) )
  67. 67. Colors Default: d3.category10() Distinct but nothing about the context Custom palette Colors related to the groups/houses. Black = Night’s Watch Blue = North Red = Daenerys Gold = Lannister …
  68. 68. Hold the vis. CHAPTER V
  69. 69. The vis is not enough.
  70. 70. Legend
  71. 71. Navigation
  72. 72. Top 3
  73. 73. Adjust threshold
  74. 74. Recap
  75. 75. Filtered Recap Tooltip
  76. 76. Demo https://interactive.twitter.com/game-of-thrones
  77. 77. Mobile Support
  78. 78. A visualizer always evaluates his work. CHAPTER VI
  79. 79. “Feedback is the breakfast of champion.” — Ken Blanchard
  80. 80. Self & Peer Does it solve the problem?
  81. 81. Google Analytics Pageviews Visitors Actions Referrals Sites/Social
  82. 82. Feedback
  83. 83. Feedback
  84. 84. You know nothing, algorithm. CHAPTER VII
  85. 85. Pros List of Characters + Tweets = Vis. No story or relationship as input
  86. 86. Cons Weak assumption about connections Can misclassify Noise
  87. 87. All talk must end. CHAPTER VIII
  88. 88. Summary Problem first, not solution backwards Prototype, Iterate, Scale & Adapt Identify characters from Tweets + Network visualization Vis is important, but there are other parts. Feedback Understand the pros & cons Rooms for improvement kristw.yellowpigz.com Krist Wongsuphasawat / @kristw
  89. 89. Robert Harris, Miguel Rios, Elaine Filadelfo and many colleagues at Twitter; Elijah Meeks for his network vis tutorial;
 Mike Bostock for D3 and examples Lastly, to my wife for taking care of our baby, so I had time to prepare these slides. Acknowledgement
  90. 90. Resources Quora: How would you explain the plot of Game of Thrones in Brief from the beginning? https://www.quora.com/How-would-you-explain-the-plot-of-Game-of-Thrones-in-brief-from-the-beginning Convex Hull https://en.wikipedia.org/wiki/Convex_hull Images Aemon - http://tinyurl.com/zqdzc2a Cersei - http://tinyurl.com/gumcs7g Crow - http://tinyurl.com/ju6up5g Eddard - http://tinyurl.com/jc23mel Feast - http://tinyurl.com/zcms4lq Hammer - http://tinyurl.com/hz8emrp Hodor - http://tinyurl.com/j748jfr House sigils - http://tinyurl.com/jywgcjx House Stark - http://tinyurl.com/jtdtrdy Jon Snow - http://tinyurl.com/h4hofe8 Tyrion - http://tinyurl.com/z7z2uow Tyrion - http://tinyurl.com/hvw4u89 Tyrion - http://tinyurl.com/jkrvqtb Watercolor Map by Stamen Design
  91. 91. Thank you
  92. 92. Questions?

×