Self-Service Big Data for CI - A Few Examples - SLA 2014 Conference - Vancouver, BC - CI Division


Published on

This 30 minute talk was delivered at the SLA's 2014 conference in Vancouver, BC.

During the talk I covered a few ways in which Competitive Intelligence and other professionals can leverage the Big Data that's all around them for Competitive Intelligence, Market Intelligence, and Market Research projects.

Published in: Internet, Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Self-Service Big Data for CI - A Few Examples - SLA 2014 Conference - Vancouver, BC - CI Division

  1. 1. Squeezing the Internet… “Data is so widely available and so strategically important that the scarce thing is the knowledge to extract wisdom from it…” HalVarian – Google’s Chief Economist
  2. 2. Examples of Big Data  Macy’s  Reduced the time to price items from 27 hours to 1 hour.  Using products likeTableau / Cloudera, etc.  Sears  Calculating price elasticity across 12.6b parameters  Morton’s  Customer Monitoring  Tuxedo Clad Waiter meeting a customer at the airport with their favorite steak.  Michigan School of Business  Analyzed 50,000 annual reports using the Gunning Fog index.  Companies that used more complicated language were more likely to fail / have lower earnings, etc.
  3. 3. Technology – Driving Big Data  Storage and ComputeTrends  Cloud Based Storage and Compute – Amazon, Microsoft, Google, etc.  Visualization Products  Tableau, Splunk,Gephi  Specialized Database Platforms  Hadoop and it’s various flavors (Cloudera / Hortonworks, Pivotal, etc.)
  4. 4. Big Data If you have “all” (or nearly all) the data it changes how you tackle a problem…
  5. 5. Here’s an example…
  6. 6. “Where is the competitor sales team – footprint, coverage, etc.?” Who makes up the competitor sales team – backgrounds, etc.? Example KIQ’s
  7. 7. Old School Approach…  Talk to Fred, Mary, Stan, and Joan…  Ask them what they know about the competing sales team  They worked there – 4 years ago or so.
  8. 8. Big Data Approach = LinkedIn  Ask yourself these four questions:  How many people lie about where they work on LinkedIn?  How many people lie about their job “role” on LinkedIn?  How many people lie about where they live on LinkedIn?  In your industry, how likely is it that a sales rep has a LinkedIn profile that is kept moderately updated?  In other words is updated when they change jobs?  If the answer to the first three is None, Extremely Limited,Very Limited.  And the answer to #4 is nearly 100% (or even 90%) – given these are sales folks.  Mining LinkedIn’s Big Data is going to be a much better approach than talking to Mike, Stan, or Mary.
  9. 9. LinkedIn - Stats  Number of LinkedIn users in the US  100m  Total number of LinkedIn Groups  2.1m  Number of Countries with a “localized” version of LinkedIn  22  Number of new LinkedIn members every second  2
  10. 10. The Rules…  Look at web properties – especially those with millions of users – as DATABASES first  Then see what type of search and visualization tools these sites provide  Test for Significance  Test for Noise  Focus on the portions of the DATABASE that are likely to yield statistically significant results  When the amount of data that you need to extract becomes too much to handle find yourself the:  API  And a Programmer
  11. 11. Here’s another example…
  12. 12. “Where is the uptake for my competitor’s product / solution.?” How is the trend (that my competitor is riding) evolving? Example KIQ’s
  13. 13. Question & Recipe Question  Where has our competitor sold X Product? Example Recipe  Find job posting aggregators that have API’s or which you can manually mine the data  Mine data from these aggregators  Focus is on product names, solution names, related certifications, etc.  Plot over time  Extract company names, regional growth, etc.  Compare with other data – Internal CRMWin-Loss Data, etc.
  14. 14. Aggregation  Indeed  Over ½ of all job searches start on Indeed  100m unique visitors a month  Localized in 50 countries / 26 languages – covers 94% of Global GDP  Career Builder  24m unique visitors a month  60 markets worldwide
  15. 15. Mining Job Postings via API  Indeed a fantastic job posting aggregator with an API
  16. 16. Question & Recipe Question:  What is the momentum around XYZ market trend? Recipe:  Find job posting aggregators that haveAPI’s,Trend Graphs, etc.  Mine data from these aggregators  Focus is on industry trends by name  Drill down into jobs that have these terms in descriptions, titles, etc.  Plot over time  Extract regional growth, etc.  Compare with other data – MR Studies, Analyst Reports, CI / MI research efforts
  17. 17. Indeed – JobTrends – “title:cloud”
  18. 18. Indeed – JobTrends – “DevOps”
  19. 19. Indeed – JobTrends – “Telemarketer”
  20. 20. Indeed – JobTrends – “SQL Server”
  21. 21. Intersection of technology “A” with technologies “B,” “C,” and “D”…
  22. 22. Mining Web Metrics… …another example
  23. 23. What’s happening out there?  Consumers  Consult an average of 10.4 sources before making a decision - Google ZMOT Study  Businesses  B2B buyers get 57% of the way through the sales process before talking to a sales rep – Corporate Executive Board
  24. 24. What does this mean? Customers don’t start here… They start here…
  25. 25. GoogleTrends - Windows
  26. 26. GoogleTrends – WearableTech
  27. 27. GoogleTrends - CRM
  28. 28. Terminology - Matters
  29. 29. 0 20 40 60 80 100 120 Chile Mexico Ukraine Indonesia Argentina Brazil Poland SaudiArabia Egypt Hungary Philippines China Spain Thailand Italy Portugal Greece Iran France Russia CzechRepublic Finland Malaysia India Taiwan Denmark Japan Belgium Sweden Israel Canada Netherlands Germany Austria UnitedArabEmirates Ireland Norway NewZealand UnitedStates Singapore UnitedKingdom Australia Switzerland SouthAfrica HongKong Veeam Backup Exec ARCserve Regional Interest – 3 Enterprise Backup Solutions Source: GoogleTrends
  30. 30. Regional – Windows Phone
  31. 31. It’s always been a database…now go use it like one…  5,922,000,000  Number of Google Searches Per Day (2013)  2,161,530,000,000  Annual Number of Google Searches Per Day (2013)
  32. 32. A few resources… Quantifying –Getting Quant from the Internet Qualitative – with Competitor Customers, Partners, Influencers
  33. 33. “Unless you try to do something beyond what you have already mastered, you will never grow.” Ralph Waldo Emerson
  34. 34. A few reminders   Resources  Sign up to download – Competitive Intel Podcasts, Presentations, Webinar Recordings, Inbox Education Series – short email based education series    Our Newsletter  Download –Voice of the Competitor (VOTC) – eBook, Going Beyond Google – 5th edition – eBook  Contact Info: 
  35. 35. CASCADE INSIGHTS About your Presenter and thisTalk Contact: / 503.898.0004 CI Tips & Best Practices: The “Competitive Intel” Podcast
  36. 36. Cascade Insights Insight that drives better decision making…  Cascade Insights provides competitive and market intelligence to B2B high-tech hardware, software, and services companies so they can:  Handle the Competition  Capitalize on New Opportunities  Demographics  Founded in 2006  Based in Portland, Oregon  CoreTeam of Analysts + a collection of “booster rockets”
  37. 37. Cascade Insights OurToolbox…  First, a focus on software and hardware companies to the exclusion of other sectors.  This allows us to have an in-depth understanding of industry trends before a project starts.  Secondly, a focus on B2B focused companies over B2C.  Hence when we’re conducting research, we go into a project already knowledgeable about typical B2B market dynamics, such as:  Ecosystems  Channel Structures (Resellers,VAR’s, Distributors, SI’s, etc.)  Segments (SMB, Mid-Market, Enterprise)  Buyers and Influencers (CIO’s, IT Managers, Business Leadership, Developers, Architects, etc.)  Sales Motion (Direct / Channel Sales, Licensing Structures, Product and Service Sales, Partners, etc.)
  38. 38. Cascade Insights OurToolbox…  Third, a unique set of methodologies for gathering Open Source Intelligence (OSINT).  In short, we wrote the book on mining modern data sources in a B2B context.  The book - Going Beyond Google: Gathering Intelligence is now in its 4th edition with a 5th edition due out this week.  Company leaders who have been invited to speak on five continents and have delivered hundreds of talks, seminars, and workshops on the subjects of competitive and market intelligence.  Company leaders who have been interviewed on the topic by Forbes,The NewYorkTimes,The Los AngelesTimes, ZDNet, ABC News, and Macleans.  Fourth, a proven capability for recruiting and interviewing in-depth competitor customers, partners, and industry influencers.  We leverage our extensive open source intelligence assets to identify and find interviewees.  Cascade Insights does not use traditional “moderators” to perform interviews.  Instead, individuals who have significant tech industry expertise perform interviews. This lets us easily follow technical and/or industry focused conversations and allows us take an investigative approach to each and every interview.
  39. 39. Some of our clients…
  40. 40. Thought Leadership…
  41. 41. Thought Leadership…Resources  The “Competitive Intel” Podcast  Available on iTunes,Stitcher Radio, etc.  Episodes have covered topics such as mining modern data sources and effective research methods, analysis topics, etc.  Going Beyond Google: Gathering Internet Intelligence – 5th Edition  Our book focused on mining modern data sources  Listed as a must read by the Strategic and Competitive Intelligence Professionals society as early as it’s 1st edition.  Past Conference Presentations  We’ve been fortunate to be asked to speak on 5 continents and have delivered a variety of competitive and market intelligence talks at conferences and events around the world.  In fact we deliver over 25+ talks a year to various industry associations and groups To learn more about any of the above go to: