HYDSPIN Dec14 visual story telling

1. MAKING SENSE OF BIG DATA: VISUAL STORY TELLING B GANES KESARI, VP, GRAMENER

2. A data visualisation and analytics company We handle terabyte-size data via non-traditional analytics and visualise it in real-time. Gramener visualises your data Gramener transforms your data into concise dashboards that make your business problem & solution visually obvious. We help you find insights quickly, based on cognitive research, and our visualisations guide you towards actionable decisions.

3. Generation Analysis Consumption Big data… Transaction data Increasing volumes of data being churned out by systems 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Social network data Consumers embracing Web 2.0 & social media lifestyle 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 M2M data Devices generating & logging data from every activity 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 creates opportunities Each industry is poised to take advantage of big data to varying degrees. Some factors that increase the relevance of big data to an industry are: Volume of data The larger the volume of data, the more likely it is that a firm will benefit from increasing use of data. Variability Greater fluctuations in performance offer more potential for a data-driven organisation to improve results. Customer intensity More customers (or stakeholders of any kind) offer greater potential for segmentation and tailored action. Transaction intensity This permits greater automation of decision making, allowing processing power to replace human judgement. Turbulence Frequency at which leaders and laggards change place in a sector indicates potential for disruption. McKinsey Global Institute, Big Data, June 2011 … but at a high cost Investments of various kinds are required to make the data actionable. It is not enough that the data just exists, or is collected. Some challenges are: Technology Collection, storage, analysis and visualisation of data – all require investments in modern technology. Talent The deeper the analysis & data expertise a firm has, the better it can leverage data. But such talent is rare. Organisational change A shift in mind-set from experience-driven decision making to data driven decision making is required. Data access Collecting relevant data, storing it, and making it available to analysts in an easy manner requires investment. Supplier ecosystem A mature vendor ecosystem providing end-to-end or piece-wise solutions to these is not yet a reality.

4. A DATA VISUALISATION CHALLENGE… You will see 3 questions. You have 30 seconds. Try it! Your timer starts now

5. HOW MANY NUMBERS ARE ABOVE 100? 1 23 32 71 72 58 87 11 77 70 16 17 21 56 44 68 51 84 20 60 40 37 8 107 14 12 41 69 14 18 71 62 55 59 64 33 55 71 58 103 92 101 56 45 34 43 15 73 78 6 93 39 53 22 26 26 94 60 82 99 74 11 12 36 67 70 71 97 59 73 99 75 74 69 69 51 48 2 66 92 98 15 10 41 58 104 94 92 84 74 82 12 52 10 57 33 77 88 81 81 91 15 56 25 30 21 7 66 66 78 87 29 23 5 34 11 96 74 99 99 88 37 10 43 15 50 71 65 60 101 98 46 34 19 102 57 70 95 84 63 91 3 34 39 37 60 81 65 63 9 71 48 46 25 50 22 64 91 76 71 79

6. HOW MANY NUMBERS ARE BELOW 10? 2 23 32 71 72 58 87 11 77 70 16 17 21 56 44 68 51 84 20 60 40 37 8 107 14 12 41 69 14 18 71 62 55 59 64 33 55 71 58 103 92 101 56 45 34 43 15 73 78 6 93 39 53 22 26 26 94 60 82 99 74 11 12 36 67 70 71 97 59 73 99 75 74 69 69 51 48 2 66 92 98 15 10 41 58 104 94 92 84 74 82 12 52 10 57 33 77 88 81 81 91 15 56 25 30 21 7 66 66 78 87 29 23 5 34 11 96 74 99 99 88 37 10 43 15 50 71 65 60 101 98 46 34 19 102 57 70 95 84 63 91 3 34 39 37 60 81 65 63 9 71 48 46 25 50 22 64 91 76 71 79

7. WHICH QUADRANT HAS THE HIGHEST TOTAL? 3 23 32 71 72 58 87 11 77 70 16 17 21 56 44 68 51 84 20 60 40 37 8 107 14 12 41 69 14 18 71 62 55 59 64 33 55 71 58 103 92 101 56 45 34 43 15 73 78 6 93 39 53 22 26 26 94 60 82 99 74 11 12 36 67 70 71 97 59 73 99 75 74 69 69 51 48 2 66 92 98 15 10 41 58 104 94 92 84 74 82 12 52 10 57 33 77 88 81 81 91 15 56 25 30 21 7 66 66 78 87 29 23 5 34 11 96 74 99 99 88 37 10 43 15 50 71 65 60 101 98 46 34 19 102 57 70 95 84 63 91 3 34 39 37 60 81 65 63 9 71 48 46 25 50 22 64 91 76 71 79

8. A DATA VISUALISATION CHALLENGE… We’ll answer the same questions again. But with simple visual cues. See how long it takes. Your timer starts now

9. HOW MANY NUMBERS ARE ABOVE 100? 1 23 32 71 72 58 87 11 77 70 16 17 21 56 44 68 51 84 20 60 40 37 8 107 14 12 41 69 14 18 71 62 55 59 64 33 55 71 58 103 92 101 56 45 34 43 15 73 78 6 93 39 53 22 26 26 94 60 82 99 74 11 12 36 67 70 71 97 59 73 99 75 74 69 69 51 48 2 66 92 98 15 10 41 58 104 94 92 84 74 82 12 52 10 57 33 77 88 81 81 91 15 56 25 30 21 7 66 66 78 87 29 23 5 34 11 96 74 99 99 88 37 10 43 15 50 71 65 60 101 98 46 34 19 102 57 70 95 84 63 91 3 34 39 37 60 81 65 63 9 71 48 46 25 50 22 64 91 76 71 79

10. HOW MANY NUMBERS ARE BELOW 10? 2 23 32 71 72 58 87 11 77 70 16 17 21 56 44 68 51 84 20 60 40 37 8 107 14 12 41 69 14 18 71 62 55 59 64 33 55 71 58 103 92 101 56 45 34 43 15 73 78 6 93 39 53 22 26 26 94 60 82 99 74 11 12 36 67 70 71 97 59 73 99 75 74 69 69 51 48 2 66 92 98 15 10 41 58 104 94 92 84 74 82 12 52 10 57 33 77 88 81 81 91 15 56 25 30 21 7 66 66 78 87 29 23 5 34 11 96 74 99 99 88 37 10 43 15 50 71 65 60 101 98 46 34 19 102 57 70 95 84 63 91 3 34 39 37 60 81 65 63 9 71 48 46 25 50 22 64 91 76 71 79

11. WHICH QUADRANT HAS THE HIGHEST TOTAL? 3 23 32 71 72 58 87 11 77 70 16 17 21 56 44 68 51 84 20 60 40 37 8 107 14 12 41 69 14 18 71 62 55 59 64 33 55 71 58 103 92 101 56 45 34 43 15 73 78 6 93 39 53 22 26 26 94 60 82 99 74 11 12 36 67 70 71 97 59 73 99 75 74 69 69 51 48 2 66 92 98 15 10 41 58 104 94 92 84 74 82 12 52 10 57 33 77 88 81 81 91 15 56 25 30 21 7 66 66 78 87 29 23 5 34 11 96 74 99 99 88 37 10 43 15 50 71 65 60 101 98 46 34 19 102 57 70 95 84 63 91 3 34 39 37 60 81 65 63 9 71 48 46 25 50 22 64 91 76 71 79

12. Humans are pattern-seeking story-telling animals.

13. Amit Kapoor, http://narrativeviz.com/playbook

16. VISUALIZING THE GENERAL ELECTIONS Can we understand the brief history of elections in India? How have the political fortunes changed over time? How did the biggest election of them all unfold in 2014? EXPLORATORY | INTERACTIVE

17. India’s General Elections landscape… ~300 Parties fielding 8000 candidates ~1 Mn booths served by 20 Mn people ~800 Mn Registered Voters Varied data on several parameters ~21,000 Votes/sec of live results A Big Data problem… in every sense

18. https://gramener.com/election/parliament

19. LIVE ELECTION ANALYSIS Our CNN-IBN Microsoft Election Analytics Canter, which you can see at www.bing.com/electio ns or election-results. ibnlive.in.com, served over 10 million requests on 16th May 2014 — the day of India election results. This is one of the largest real-time visualisations that we (and perhaps many others) have attempted http://ibn.gramener.com/live

20. <<Video recreating how the Election results unfolded>>

21. INDIA’S MOST PERSISTENT PARTY Does any party hold a consistent 100% failure rate? Which party holds record for being most persistent in adversity? Which party’s candidates have lost deposits for nearly a decade? EXPLANATORY| STATIC

22. https://gramener.com/election/parliament#story.ddp

23. VISUALIZING WEATHER How did weather change in India over the past century? What were the hottest and coldest places? Are there places that exhibit some interesting patterns? EXPLANATORY| VIDEO Image credit: https://www.flickr.com/photos/vesiaphotography/11627471004

24. 100 YEARS OF INDIA’S WEATHER 1901 1911 1921 1931 1941 1951 1961 1971 1981 1991 2001 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec http://www.youtube.com/watch?v=WT0Aq41BaOQ

25. STORIES FROM TEXT Can business impacting stories be mined from large bodies of text? Can investors better read companies by studying Investor earning calls? Can companies understand what analysts want & be better prepared? EXPLANATORY| INTERACTIVE Image credit: ttps://www.flickr.com/photos/a_mason/3009985823

26. HOW IS THE TEXT PROCESSED? Web Scraping Tokenization Part-of- Speech tagging Entity Transform detection Text Analytics Engine Analytics Engine Compute Visualization Engine Ticker Qtr #Qns AAPL 53% 3 AAPL 51% 7 GS 52% 6 MSFT 53% 4 ... ... ... MS 54% 9 JP 53% 6 ... ... ... Data Extraction Ticker Qtr %Gr AAPL 53% 23% AAPL 51% -35% GS 52% 95% MSFT 53% 101% ... ... ... MS 54% 14% JP 53% 20% ... ... ...

27. https://gramener.com/transcriptanalysis/

28. VISUALIZING MOVIES What are the popular, critically acclaimed ones? Where do my preferences figure? Which one should I watch next? EXPLORATORY| INTERACTIVE

29. The Shawshank Redepmption The Godfather The Dark Knight Titanic The Phantom Menace Twilight New Moon Wild Wild West Transformers The Good, The Bad, The Ugly 12 Angry Men 7 Samurai Rang De Basanti Taare Zameen Par Yojinbo MORE VOTES BETTER RATED Many unwatched movies Few unwatched movies Mix of watched & unwatched Few watched movies Many watched movies Movies on the IMDb 3 Idiots https://gramener.com/imdb/

30. http://demo.gramener.com:7056/twitteranalysis.html

31. BEST PLACES TO LIVE FINDING ‘BEST PLACES’ TO LIVE IN Can we plug into public data to better understand cities? Can we identify the best places to live? Can this be customized to an individual level? EXPLORATORY| INTERACTIVE Image credit: https://www.flickr.com/photos/dynamosquito/2431025077

32. http://indiatoday.intoday.in/best-cities-2014.jsp

33. WHAT DOES THE WORLD SEARCH FOR? What are some questions that interest people ? How does this vary across countries? Can we do ongoing ‘search-listening’? EXPLORATORY| INTERACTIVE Image credit: https://www.flickr.com/photos/uberculture/2561190022

34. https://gramener.com/search/#questions/how-to-

36. Session Slides available on Slideshare at: http://www.slideshare.net/gramener/hydspin-dec14-visual-story-telling Ganes Kesari Twitter: @kesaritweets Email: ganes.kesari@gramener.com  gramener.com  blog.gramener.com  http://slideshare.net/gramener