Trends in Visualising Machine Learning: An approach to humanising the intelligence
•Download as PPTX, PDF•
0 likes•113 views
This document discusses the need for humanizing machine learning models through visualization and storytelling. It provides examples of how Gramener has developed techniques to summarize and explain complex model outputs through abstraction, interactive visualization, and narrative frameworks. These techniques allow users to understand machine learning results at varying levels of detail and help address issues around adoption and distrust of "black box" algorithms.
Recommended
More Related Content
Similar to Trends in Visualising Machine Learning: An approach to humanising the intelligence
Similar to Trends in Visualising Machine Learning: An approach to humanising the intelligence (20)
More from Ganes Kesari
More from Ganes Kesari (20)
Recently uploaded
Recently uploaded (20)
Trends in Visualising Machine Learning: An approach to humanising the intelligence
- 2. 2 Trends in Visualizing Machine Learning: An approach to humanizing the Intelligence Ganes Kesari B Co-Founder, Head - Analytics & Desig Gramener
- 3. 3 AI… …but, is this an informed conversation yet? Salvation or Singularity?
- 4. 4 There’s a connection between the advances that are made in technology, and the sense of primitive fear people develop in response to it. – Don DeLillo “ 1877: NYT attacked Bell’s telephone for invasion of privacy 1680: Printing press was attacked for spreading ignorance http://lenwilson.us/11-examples-of-fear-and-suspicion-of-new-technology/ TECHNOLOGY & PRIMAL FEAR
- 5. 5 ADVANCED ANALYTICS WILL MAKE THIS DISCONNECT BIGGER We must address the issue with understanding of analytics Lets take a look at adoption in enterprises
- 6. 6 TELECOM CHURN “Churn of customers is a particularly severe problem in the telecom industry. The challenge is to identify the propensity of churn up to a month in advance, even before a customer moves out, so that proactive interventions can begin”
- 7. 7 Outgoing call (days ago)0 0 - 4 15+5-14 1 RECHARGE AMT > $20 0 1 Y N > 1 RECHARGE N Y 39% IMPROVEMENT Decision Tree MODELS 66% IMPROVEMENT SVM, Random Forest, Neuralnets MODELS 0 Y Y CHURN MODELS
- 8. INCREASINGLY, OUR BEST MODELS ARE BLACK-BOX
- 9. 9 Problem Approach Outcome A leading agricultural enterprise wanted price forecasts for their products in order to plan inventory release to optimise revenue. Incorrect timing was leading either to loss of revenue or unsold inventory. Gramener applied a suite of price forecasting models based on internal and external factors. The models were evaluated on multiple test datasets to select one that minimised median absolute deviation. The model was able to forecast the price to an accuracy of 88%. Within the first quarter of deploying the model, the revenue uplift attributable directly to pricing was +3.2%. PRICE FORECASTING FOR A GLOBAL AGRICULTURAL ENTERPRISE
- 10. 10 A COMPARISON OF PRICE FORECAST ACCURACY OF MODELS Product Moving Average Auto- regression Exponential Smoothing ARIMA Exponential Smoothing Over State Space Hybrid Model Neural Network Multi-Linear Regression Product 1 65.13 54.13 65.98 66.16 71.67 73.24 78.96 70.46 Product 2 66.89 56.66 66.74 68.12 74.41 74.65 89.15 73.87 Product 3 37.53 9.84 44.55 42.28 50.49 46.86 61.35 53.03 Product 4 37.16 4.92 50.22 43.50 52.19 53.40 68.63 53.15 Product 5 68.83 71.24 68.38 68.12 75.58 71.47 90.80 72.69 Product 6 69.41 69.60 69.24 70.16 77.55 75.75 80.41 75.09 Product 7 69.27 64.76 68.61 69.21 73.39 74.06 82.10 75.20 Product 8 64.54 52.50 63.93 64.41 68.31 70.82 79.70 70.78 Product 9 57.97 52.64 57.40 58.53 63.90 63.15 78.80 63.04 Product 10 53.61 55.90 54.54 56.47 59.78 58.63 90.28 61.96 Product 11 52.02 26.49 54.92 53.65 60.80 63.89 78.40 52.23 Product 12 45.83 28.50 53.59 49.43 56.09 53.63 85.34 48.33 Product 13 41.30 28.98 40.51 38.88 50.84 47.57 63.76 50.55 Product 14 41.14 17.41 41.51 38.05 45.95 48.69 71.55 44.10 Product 15 86.40 84.00 86.58 87.29 88.80 90.78 99.91 88.04 Product 16 85.76 83.83 85.66 85.59 85.30 88.43 91.76 78.59
- 11. WE NEED A WAY OF UNDERSTANDING BLACK-BOX MODELS
- 12. 12 INFORMATION DESIGN TO TELL STORIES WITH MODEL OUTPUTS 1 EVOLVING A VISUAL FRAMEWORK FOR MACHINE LEARNING
- 13. 13 BEHAVIORAL CLUSTERING “Delivering targeting media content to different regions of the country could improve reach. The challenge is to identify the right clustering of regions that are similar, but may not be geographically contiguous, so that targeted interventions can begin”
- 14. 14 SEGMENTING INDIA’S DISTRICTS BASED ON BEHAVIOR
- 15. 15 VISUALIZING THE BEHAVIOURALLY SEGMENTED DISTRICTS The 6 clusters were created using 3 composite indices : • Education (literacy, higher education) that leads to... • Skilled jobs (in mfg or services) that leads to... • Purchasing power (higher income, asset ownership) Districts were divided (at the average cut-off) by: Poor Rural, uneducated agri workers. Young population with low income and asset ownership. Mostly in Bihar, Jharkhand, UP, MP. Breakout Rural, educated agri workers poised for skilled labour. Higher asset ownership. Parts of UP, Bihar, MP. Aspirant Regions with skilled labour pools but low purchasing power. Cusp of economic development. Mostly WB, Odisha, parts of UP Owner Regions with unskilled labour but high economic prosperity (landlords, etc.) Mostly AP, TN, parts of Karnataka, Gujarat Business Lower education but working in skilled jobs, and prosperous. Typical of business communities. Parts of Gujarat, TN, Urban UP, Punjab, etc Rich Urban educated population working in skilled jobs. All metros, large cities, parts of Kerala, TN Skilled Poorer Richer Unskilled Skilled Uneducated Educated Uneducated Educated Unskilled Purchasing power Skilled jobs Education Poor Breakout Aspirant Owner Business RichThe 6 clusters are
- 16. 16 INFORMATION DESIGN MODEL UNPACKING TO DEMYSTIFY BLACK BOX ALGORITHMS 1 2 EVOLVING A VISUAL FRAMEWORK FOR MACHINE LEARNING
- 17. 17 NEURAL NETWORKS Inspired by biological networks, artificial neural networks are a network of interconnected nodes that make up a model, like humans & animals. Neural network processes information by passing it through layers: one input layer, 1 or more hidden layers, and an output layer.
- 18. 18https://distill.pub/2018/building-blocks/ THE BUILDING BLOCKS OF INTERPRETABILITY
- 19. 19 INFORMATION DESIGN MODEL UNPACKING TO DEMYSTIFY BLACK BOX ALGORITHMS ABSTRACTION TO PAINT STORY AT VARYING LEVELS OF INSIGHTS 1 2 3 EVOLVING A VISUAL FRAMEWORK FOR MACHINE LEARNING
- 20. 20 ABSTRACTION IN DESIGNING A TOY NAVIGATION SYSTEM http://worrydream.com/LadderOfAbstraction/ Abstracting over time Abstracting over data Abstracting over models Abstracting across dimensions
- 21. 21 FINDING PATTERNS “ Which securities move together? How should I diversify? What should I sell to reduce risk? What’s a reliable predictor of a security? SECURITIES
- 22. 22 LET’S EXAMINE CURRENCY FORECASTS Starting with security prices.. …then examine pairs of securities.. …lets abstract over time.. ..and, abstract across all currencies..
- 23. 23 68% correlation between AUD & EUR Plot of 6 month daily AUD - EUR values Block of correlated currencies … clustered hierarchically
- 24. 24 INFORMATION DESIGN MODEL UNPACKING TO DEMYSTIFY BLACK BOX ALGORITHMS ABSTRACTION TO PAINT STORY AT VARYING LEVELS OF INSIGHTS INTERACTION DESIGN FOR AN IMMERSIVE USER EXPERIENCE 1 2 3 4 EVOLVING A VISUAL FRAMEWORK FOR MACHINE LEARNING
- 25. 25 CARGO DELAY SIMULATION “A global cargo carrier is struggling to improve operations by better handling cargo at the airports. The challenge is to identify a combination of the most important factors that cause delays, and being able to simulate turnaround times for potential interventions”
- 27. 27 Information Design • User Centric • Representations • Visual design Raw Machine Learning Outcomes Visual Machine Learning FrameworkAdaptive Abstraction • Move up & down • Contextual • Fluid navigation Interaction Design • Storytelling UI • Consistency • Perceivable Model Unpacking • Unravel internals • Traceability • Simplify keywords Visualized and Humanized Intelligence THE VISUAL FRAMEWORK FOR MACHINE LEARNING
- 28. 28 BLACK-BOX MODELS ARE INCREASINGLY ACCURATE ANALYTICAL MODELS NEED INTERPRETATION (EVEN MORE) AS PRACTITIONERS, OUR RESPONSIBILITY IS TO SIMPLIFY AS CONSUMERS, SELF- EDUCATE & DEMAND EXPLANATIONS ..AND TOOLS ARE LESS IMPORTANT THAN TECHNIQUE IN SUMMARY…
- 30. 30 @kesaritweets @kesari INSIGHTS Extract meaning using automated patterns AI & MACHINE LEARNING SERVICES VISUAL NARRATIVES STORYTELLING Creative ThinkingCritical Reasoning SOFTWARE THROUGH SERVWARE: augmenting human intelligence with technology Binding visuals together into a logical story GRAMENER IS A DATA SCIENCE COMPANY THAT SIMPLIFIES DATA CONSUMPTION THANK YOU!
Editor's Notes
- We were working with the wealth management team of a European bank. They said, “We have a problem. When telling our customers what transactions to make, we base our advice on two very simple principles. First, if you have two securities that behave similarly, you should consolidate. For example, there is no benefit in holding shares of two oil companies. When the price of one rises, the other invariably rises too. So it’s practically like holding the same company’s stock.” “On the other hand, having consolidated, make sure you have a good hedge. For example, if you hold oil companies, buy a bit of gold. When oil companies drop, gold typically rises. Gold is a reasonably good hedge against oil companies.” He said, “This is the basis of the bulk of the advice we give clients. But in order to arrive at this advice, our analysts have to go through 150 reports, which is humanly impossible. We know they don’t actually do that. We sometimes pass these reports on to our clients. They clearly never read these. As a result, our transaction volumes are not as high as we would like to be, mainly because people do not understand why they need to make a trade.”
- So, what we did was put a variant of this visual together. On the right, you have a series of currencies like the Australian dollar, the Euro, the British pound, etc; some commodities like silver and gold; and some stock indices like Sensex, FTSE, and S&P. The cells here have a number inside that indicates the pairwise correlation between a pair of securities. For example, the number 68 on the top left indicates a 68% correlation between the Australian dollar and the Euro. To the left of the Euro and just below the dollar (diagonally opposite to the 68), there’s a scatter plot that shows the daily prices of both these currencies. Each dot is one day’s data. The x-axis shows the Australian dollar value. The y-axis shows the Euro value. This helps identify what the pattern of movements of any two currencies is. From this, you can easily see visually that the Australian dollar and the Euro both tend to move together. Or, where there are strong correlations like the FTSE & S&P, the pattern is almost a straight line. In some cases there are negative correlations. For instance, if you take the Sensex against the Japanese Yen, the correlation is -79%. The cells are coloured based on their correlation values. Greens indicate strong positive correlation. Reds indicate strong negative correlation. These are also grouped hierarchically. On the left, we have a series of lines indicating clusters. The most similar securities are grouped together. So FTSE and S&P with a 98% correlation are very close. The ones that are less correlated are kept further away based on a tree-structure. This leads to clustering of securities. For example, there is a green block in the center which has SGD, JPY, XAU, CHF and CNY. All of these are fairly well correlated. When any one currency in this block goes up, all the others go up as well. When any one goes down, all others go down as well. Similarly, you have another block to its top left: S&P, FTSE, Sensex and to a certain extent, the Pakistani Rupee. These move together as a block as well. But when this block goes up, all the currencies in the other block go down, as indicated by the red negative correlations between these two blocks. This can be used very easily for decision making. For example, one client who was trading with Singapore and Japan looked at the strong correlation and decided to consolidate their holdings in Japanese Yen. They then moved up and down this column to find a good hedge. FTSE looked like a good hedge – it was the most negatively correlated with JPY at that time -- and they decided to place a third of their portfolio in FTSE. A sheet like this improves people’s understanding of relatively complex data, and results in significantly increased trade volumes.