Selling Text Analytics to your boss

1. SELLING TEXT ANALYTICS TO YOUR BOSS Nov 2014

2. Intro – speaker, the talk & quick intro of the audience. Quick summary of challenges with Text Analytics Role play (what mistakes are done today?) The elevator pitch Proposed tactical approach References for reading materials CONTENTS Intended for Knowledge Sharing only 2

3. INTRO Intended for Knowledge Sharing only 3 About me: "As Director, Analytics for the Digital Developed Markets department at Visa Inc., I am responsible for helping the Leadership & Stakeholders with actionable insights derived from Analytics. The business questions span the whole spectrum across the Product, Marketing, Sales and Relationship. We leverage any of the various options, i.e., Strategic analysis, Advanced Analytics, Text Analytics or Mining depending on the problem being solved." The talk: Content will range start with why does Text Analytics need a special session on convincing boss, followed by a role play summarizing current mistakes, a sample elevator pitch for your boss and a proposed execution plan. The content is tailored for Mid to Senior Level Managers trying to convince Leaders/Executives/Heads. It doesn’t provide any technical details –methodologies, tools, vendors or hardware investments. *Disclaimer: Participation in this summit is purely on personal basis and not representing VISA in any form or matter. The talk is based on learnings from work across industries and firms. Care has been taken to ensure no proprietary or work related info of any of my the firms I have worked with is used in any materials.

5. TOUGHEST ELEVATOR PITCH TODAY Intended for Knowledge Sharing only 5 Don't you know I'm human too?

6. WHY DO THEY SAY NO? Intended for Knowledge Sharing only 6  PERCEPTION ISSUES – too niche, too specific problem solving.  COMPLEX & CHALLENGING – variety of data sources, structure, time consuming  RoI UNCLEAR  LACK OF TRAINED RESOURCES  MY CURRENT SUITE OF ANALYTICS IS ENOUGH

7. ARE ALL THOSE STATEMENTS TRUE? Intended for Knowledge Sharing only 7  PERCEPTION ISSUES – too niche, too specific problem solving.  COMPLEX & CHALLENGING – variety of data sources, structure, time consuming  RoI UNCLEAR  LACK OF TRAINED RESOURCES  MY CURRENT SUITE OF ANALYTICS IS ENOUGH MYTH TRUE TRUE TRUE FALSE

9. Intended for Knowledge Sharing only 9 How is it done today, more or less  https://www.youtube.com/watch?v=BUsiI47ol6g

11. NECESSARY COMPONENTS IN AN ELEVATOR PITCH Intended for Knowledge Sharing only 11  30 sec summary of the idea, customer & value proposition  What is it?  How does it help, aka, what will s/he get out of it?  How it works?  Target Customers  What do you need?  Why this and nothing else (USP & SWOT Analysis)  How will you grow this? Get your boss hooked, Once hooked, the detailed explanation…

12. THE ELEVATOR PITCH Intended for Knowledge Sharing only 12 Text Analytics is a “Mind Reading” Tool. Can tell us how Consumers perceive our products and what their needs are. It complements the other Analytical techniques to answer the why questions. We will use it to help Brad (a Product Manager) prioritize Password reset experience by showing him the magnitude of bad user reactions. It will be a quick survey followed by Text Analytics. We have identified an $1M+ Opportunity in Password reset experience via Text Analytics.

13. WHAT IS IT AND HOW IT HELPS? Intended for Knowledge Sharing only 13  Text mining/Analytics is the process of deriving high-quality information from text with the goal of turning text into data for analysis, via application of natural language processing (NLP) and analytical methods. Source: http://en.wikipedia.org/wiki/Text_mining http://en.wikipedia.org/wiki/Text_mining#Text_mining_and_text_analytics  Text mining usually involves the process of structuring the input text (usually parsing, along with the addition of some derived linguistic features and the removal of others, and subsequent insertion into a database), deriving patterns within the structured data, and finally evaluation and interpretation of the output.  e.g. frequency distributions, pattern recognition, tagging/annotation, information extraction, data mining techniques including link and association analysis, visualization, and predictive analytics.

14. A TYPICAL EXAMPLE OF FINDINGS OR INSIGHTS… Intended for Knowledge Sharing only 14  Overall NPS: 70% (Promoters: 80%; Detractors: 10%)  What are the Promoters happy about?*  60% of the users love the simplicity of use of Mobile App  40% feel the recommendations are relevant  30% like the two level authentication feature  What are the Detractors are unhappy about?*  10% hate the password reset experience  8% feel that password reset link takes too long to reach their email inbox  5% feel that the text updates don’t provide sufficient information  What new features did the users ask for?  Monthly reminders  Functionality for the receivers to confirm the payment  FX conversion change alerts *depends on the how the questions were framed in the surveys

15. A TYPICAL IMPACT SIZING… Intended for Knowledge Sharing only 15 KPIs Monthly unique users 20,000,000 #Page Views per user 40 #Total Page Views from Users 800,000,000 Quantifying the problem #users trying to reset Password 2,000,000 Users from the survey who hate the current experience (consistent and all detractors cited this as the reason) 100% Assuming everyone who tries to reset password hates the experience, so users at risk 2,000,000 We would be left with 720,000,000 Conservative Realistic Aggressive Impact Sizing ($) By changing the experience, we decrease detractors by 50% 80% 100% and save these many Page Views 40,000,000 64,000,000 80,000,000 At a CPM of $2 (Cost per 1000 impressions), we improved Ad Revenue by $80,000 $128,000 $160,000 Annual Revenue impact $960,000 $1,536,000 $1,920,000 Resetting the current Password reset experience can save at least 1 MM annually…

16. HIGH LEVEL STEPS INVOLVED… Intended for Knowledge Sharing only 16 Objective Translation to Analytical Framework Data Collection and Preparation Analysis, Validation & Verification Actionable insights and impact sizing A/B Testing Rollouts • Understand the core needs, expected outcomes, generate hypotheses and know if it’s an Unstructured or Structured problem • Consult with User/Market Research team on the applicability of the text input to your problem statement and how/where to get the data from. • Decide on Analytical methodology, e.g., Word bubble, Link, SVM • Data Collection: Structured (Surveys, Call Center logs) or Unstructured (Reviews, Social media comments, Blogs, Articles) • Data Preparation: Crowd sourcing for tagging, Cleaning (Stemming, Stop words, normalization, adjectives & entities, preposition removal, usage patterns, n-grams) • Data Transformations: TF, IDF, Medians, Max, transformations, Dim reductions, etc. • Marrying with internal data: Activity, Clickstream, Demo or Geo, Product. • Execution of Analysis on one sample and validation on another. • Verification of insights with other established processes, checking for reliability. • Generate recommendations and impact sizing (current problem and other areas) • Champion vs. Challenger testing of the impact. • Rollouts – New product, feature, changes, etc. 1 2 3 4 5 6 7 Hmmm, what’s this?

17. HIGH LEVEL STEPS INVOLVED… Intended for Knowledge Sharing only 17 Objective Translation to Analytical Framework Data Collection and Preparation Analysis, Validation & Verification Actionable insights and impact sizing A/B Testing Rollouts 1 2 3 4 5 6 7 Analyst & Stakeholder Analyst, Researcher, Data Instrumentation, & Data Manager, Developer, Data Scientist Analyst, Data Manager, Data Scientist Analyst, Data Scientist, Stakeholder and SME, Researcher Analyst, Stakeholder, Leader Analyst, A/B Testing, Stakeholder, Developer Stakeholder, Leadership & Executives Yeah, but who does it? Knowing what is the key need Right framework, right data, right questions Most challenging, time consuming, and tricky phase (smarts, patience & determination needed) Next most challenging – iterations on methodology/data to get to a acceptable lift Realistic sizing and allied applications of insights Buy-ins Corporate Strategy shifts Easy?

18. POSSIBLE SOLUTIONS SET BY STAKEHOLDERS … Intended for Knowledge Sharing only 18 MARKETING PRODUCT CUSTOMER SUPPORT • SOV, Brand awareness and Perception • Sentiment in Media coverage • Competitive and industry benchmarking FUNCTIONS USE CASES STRATEGY RISK • Consumer Pain points – Experience, Product stability, use cases, etc. • Machine logs analysis for user interactions studies • Informs Analytics of Consumer Sentiment, reactions, expectations, pain-points and helps in tailoring better actions • Market and User Research • Competitive and industry benchmarking • Gaming behavior and hacker communities for new use cases Others (Legal, HR, Finance) • Contract verifications, Backgrounds, etc. • Resume parsing, Career Planning, Talent Management, etc.

19. WHAT DO YOU NEED? Intended for Knowledge Sharing only 19  Executive support must.  Stakeholder buy-in and strong advocacy and support.  Clearly defined and communicated objectives mapped to company initiatives  Clearly defined and accepted success criteria for Text Analytics Initiatives  Tools & Technologies: • Non-Coders: KNIME, Rapidminer • Semi-Coders: SAS, Angoss & Polyanalyst • Full on Coders (Advanced Users): R and Python  Type of Talent: Closely tied to business needs. However one advanced programmer (Python, R) and an analyst are needs. 50% time of a Researcher and 20% time of Stakeholder customer needed.  Partnership with User/Market researcher team needed to frame right questions and make right readings from the data.  Partnership with Data Instrumentation and Data Managers to collect, prepare, execute and ramp up this process.

20. USP & SWOT… Intended for Knowledge Sharing only 20 STRENGHTS • More aha and surprises • Gut>Emotions>Metrics • Consumers feel respected and connect better when you talk to them THREAT • Big data technologies need to plan for ever changing text data & privacy concerns (Volume, variety, velocity and veracity) • More devices, more challenges OPPORTUNITIES • Cross functional team to standardize the data collection and preparation • Buy-ins on success criteria • Train the analysts • Evolving NLP techniques, vendors and market buy-ins Closest to truth! Insights from metrics based analysis is Analyst’s reasoning and Insights from Text Analytics are Voice of Consumers! WEAKNESSES • Cost and complexity of execution* • RoI – various ways to look at it • Not create and forget (nuances during automation) • Lack of resources *Unique data collection changes (low response rates of surveys), internationalization expansions, etc.

21. HIGH LEVEL THUMBRULES OF WHEN TO DO VS. NOT… Intended for Knowledge Sharing only 21  Minimum and reliable sample size (>=5% or 10K+ users are leaving some active or passive feedback somewhere).  Share of Voice, Brand awareness and perceptions have stabilized and product has been live for atleast a year for users to get familiar with it.  High Level analysis reveals correlation between Portfolio KPIs and relational NPS/SOV/call center metrics/Brand metrics.  BI, User Research, Analytics, A/B Testing have all been properly invested and strengthened in that order. Text Analytics and Mining usually go together into helping ramped up business. These two are also dependent on the previously mentioned practices for success and show most value when all others feel or are not able to answer the questions.  100% accuracy isn’t required but a strong and reliable directional insight is the need of the hour.

23. THE THREE STAGES OF EXECUTION Intended for Knowledge Sharing only 23 PICK PROVE SELL • Stakeholder discussions to find out pressing questions • Prioritize – Requester; urgency; impact; efforts • Choose “highest PR potential” problem for POC • Create action plan – data collection, cleansing, methodology, timelines, expected outcome template, success criteria. • SWAT team – Stakeholder rep, Analyst & Dev or Data Scientist • Check-ins & documentation of what worked and did not, do’s/don’ts, challenges & nuances. • Insights communication & Impact estimation. • Champion vs. Challenger measurement. • Highlight victories – underdog story, winning against the odds, challenges faced, etc. • Ramp plans: hiring, cost, time, areas where it can be used • Branding – Internal, and if possible, external too, make it ‘cool’ and desirable. Best to go prepared to you boss Boss buy-in would be really good Boss & Stakeholders primary doers. Should go up to Execs.

24. STILL NOT CONVINCED? Intended for Knowledge Sharing only 24 Not surprising, every change agent has it tough…  Just do it (forgiveness is better than permission),  Unsure of ability to pull off or too much to do already? No time or people or tool!  freelancers/contractors, vendors, POCs  either yourself by learning it or  informal partnership with a data scientist or a coder in the company  Cost concerns:  innovation labs, done by university students under NDA  Run a company innovation contest  Your boss unsure if Stakeholders will like it?  Get it done and sell to Stakeholders boss  Brownbags or other team meetings  Your boss just doesn’t want to do it: Create a Watsapp! …satisfaction of getting it done and value add is huge enough reward.

25. Intended for Knowledge Sharing only 25 REFERENCES Intended for Knowledge Sharing only 25

26. SAMPLE EXECUTIVE SUMMARY Intended for Knowledge Sharing only 26 Objective • Set up Text Analytics infrastructure to transform unstructured consumer inputs into actionable insights Methodology • POC to analyze the utility for our business and recommend a go/no-go decision Key Findings • Things worked • That didn’t work • Who are the largest customers Recommendations • Go/No go – Yes, but with as a cross functional team and clear success criteria • How big an opportunity?

27. Intended for Knowledge Sharing only 27 Some interesting applications of Text Mining/Analytics, Project Dreamcatcher: http://www.slate.com/articles/news_and_politics/victory_lab/2012/01/project_dreamcatcher_how _cutting_edge_text_analytics_can_help_the_obama_campaign_determine_voters_hopes_and_fear s_.html J.K.Rowling caught  http://blogs.wsj.com/speakeasy/2013/07/16/the-science-that-uncovered-j-k-rowlings-literary- hocus-pocus/ Movers and Shakers I follow in this space: Seth Grimes http://altaplana.com/TAS08-TextAnalyticsForDummies.pdf http://www.slideshare.net/SethGrimes http://altaplana.com/grimes.html Junling Hu http://www.aboutdm.com/ Intended for Knowledge Sharing only 27 Intended for Knowledge Sharing only 27 SOME ADDITIONAL STUFF TO READ ABOUT… *No endorsements or claims of validity/reliability. Used it myself and wanted to share with you.

28. Intended for Knowledge Sharing only 28 Good foundation on Text Mining from Statistica http://loyaltysquare.com/text_mining.php http://www.statsoft.com/Textbook/Text-Mining/button/3 http://www.nltk.org/book/ KNIME use cases (Google search sufficient for Angoss, Rapidminer, SAS whitepapers yields): http://www.meetup.com/Bay-Area-KNIME- Users/events/199339222/comments/418637362/?itemTypeToken=COMMENT&a=uc1_rd&read=1& _af_eid=199339222&_af=event Text Mining Video tutorials by Rapidminer https://www.youtube.com/watch?v=hpvda_Rfg3s&list=PLeE4eVNDyo0VKTTiPg0zlX_OYiWII-sow Text Mining & Analytics on Courseera https://www.coursera.org/specialization/datamining/20 https://www.coursera.org/course/textanalytics On Udemy (Text Mining for Bloggers) https://www.udemy.com/text-mining-for-bloggers/ Intended for Knowledge Sharing only 28 Intended for Knowledge Sharing only 28 IF YOU WANT TO LEARN YOURSELF… *No endorsements or claims of validity/reliability. Used it myself and wanted to share with you.

29. Intended for Knowledge Sharing only 29 Here is my contact, reach out may be  On Linkedin https://www.linkedin.com/pub/ramkumar-ravichandran/10/545/67a *Linkedin has all the other contact details – Email and Phone, etc. Slideshare http://www.slideshare.net/RamkumarRavichandran/ Blog coming soon… Intended for Knowledge Sharing only 29 Intended for Knowledge Sharing only 29 QUESTIONS/COMMENTS

Selling Text Analytics to your boss

Ramkumar Ravichandran

Selling Text Analytics to your boss