• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
GOT DATA? how Hadoop Market Analysis helped the California Milk Processing Board
 

GOT DATA? how Hadoop Market Analysis helped the California Milk Processing Board

on

  • 234 views

 

Statistics

Views

Total Views
234
Views on SlideShare
234
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Talk about our luxury of observing patterns of adoption

GOT DATA? how Hadoop Market Analysis helped the California Milk Processing Board GOT DATA? how Hadoop Market Analysis helped the California Milk Processing Board Presentation Transcript

  • © 2014 Luminar is a fully owned Entravisions business unit GOT DATA? How Hadoop Market Analysis Helped the California Milk Processing Board Better Serve the California Latino Market Presented by: Oscar E. Padilla, VP Strategy for Luminar Justin Sears, Product Marketing Manager , Hortonworks June 4, 2014
  • 2 ● How a marketing business adopts Hadoop to solve real challenges ● Luminar’s Hadoop evolution from HDP 1.x to 2.x ● Lessons from the successful Luminar/Hortonworks partnership ● A bit about the data architecture and how it all works together Key topics we’re looking to cover today
  • Page3 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Open Leadership Drive innovation in the open exclusively via the Apache community-driven open source process Enterprise Rigor Engineer, test and certify Apache Hadoop with the enterprise in mind Ecosystem Endorsement Focus on deep integration with existing data center technologies and skills Enable your Modern Data Architecture by delivering Enterprise Apache Hadoop Hortonworks Mission: Reseller Partners: Headquartered in Palo Alto, CA; 300+ employees and growing
  • Page4 © Hortonworks Inc. 2011 – 2014. All Rights Reserved New data puts pressure on the architectureAPPLICATIONSDATASYSTEM REPOSITORIES SOURCES Existing Sources (CRM, ERP, Clickstream, Logs) RDBMS EDW MPP Business Analytics Custom Applications Packaged Applications Source: IDC 2.8 ZB in 2012 85% from New Data Types 15x Machine Data by 2020 40 ZB by 2020 Unstructured documents, emails Clickstream Server logs Sentiment, Web Data Sensor. Machine Data Geolocation
  • Page5 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Hadoop in a Modern Data ArchitectureAPPLICATIONS Business Analytics Custom Applications Packaged Applications SOURCES OLTP, ERP, CRM Systems Documents, Emails Web Logs, Click Streams Social Networks Machine Generated Sensor Data Geolocation Data OPERATIONS TOOLS Provision, Manage & Monitor DEV & DATA TOOLS Build & Test DATASYSTEM REPOSITORIES RDBMS EDW MPP Governance &Integration Security Operations Data Access Data Management
  • Page6 © Hortonworks Inc. 2011 – 2014. All Rights Reserved New analytic apps for new types of data $ • Supplier Consolidation • Supply Chain and Logistics • Assembly Line Quality Assurance • Proactive Maintenance • Crowdsourced Quality Assurance • New Account Risk Screens • Fraud Prevention • Trading Risk • Maximize Deposit Spread • Insurance Underwriting • Accelerate Loan Processing • Call Detail Records (CDRs) • Infrastructure Investment • Next Product to Buy (NPTB) • Real-time Bandwidth Allocation • New Product Development • 360° View of the Customer • Analyze Brand Sentiment • Localized, Personalized Promotions • Website Optimization • Optimal Store Layout Financial Services Retail Telecom ManufacturingHealthcare Utilities, Oil & Gas • Genomic data for medical trials • Monitor patient vitals • Reduce re-admittance rates • Store medical research data • Recruit cohorts for pharmaceutical trials • Smart meter stream analysis • Slow oil well decline curves • Optimize lease bidding • Compliance reporting • Proactive equipment repair • Seismic image processing
  • Page7 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Companies typically start Hadoop for new analytic applications…SCALE SCOPE New Analytic Apps New types of data LOB-driven
  • Page8 © Hortonworks Inc. 2011 – 2014. All Rights Reserved … and incrementally grow to a ‘Data Lake’SCALE SCOPE New Analytic Apps New types of data LOB-driven Data Lake An architectural shift in the data center that uses Hadoop to deliver deeper insight across a large, broad, diverse set of data at efficient scale A Modern Data Architecture/Data Lake RDBMS MPP EDW Governance &Integration Security Operations Data Access Data Management
  • Page9 © Hortonworks Inc. 2011 – 2014. All Rights Reserved HDP delivers enterprise Hadoop HDP 2.1 Hortonworks Data Platform Provision, Manage & Monitor Ambari Zookeeper Scheduling Oozie Data Workflow, Lifecycle & Governance Falcon Sqoop Flume NFS WebHDFS YARN : Data Operating System DATA MANAGEMENT SECURITYDATA ACCESS GOVERNANCE & INTEGRATION Authentication Authorization Accounting Data Protection Storage: HDFS Resources: YARN Access: Hive, … Pipeline: Falcon Cluster: Knox OPERATIONS Script Pig Search Solr SQL Hive/Tez, HCatalog NoSQL HBase Accumulo Stream Storm Others In-Memory Analytics, ISV engines 1 ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° N HDFS (Hadoop Distributed File System) Batch Map Reduce Deployment Choice Linux Windows On- Premise Cloud Comprehensive enterprise Hadoop delivered completely in the open Wholly Integrated for deep ecosystem interoperability
  • Page10 © Hortonworks Inc. 2011 – 2014. All Rights Reserved Luminar is a Hortonworks pioneer • Early adopter: began using Hortonworks Data Platform in September 2012 • First customer case study on Hortonworks.com • Featured on “Advertisers do Hadoop” industry solutions page • Quantum migration from HDP 1.1 to HDP 2.0 • Numerous quotes, interviews and speaking events • Excellent results
  • 11 Santa Monica CA Denver CO Dallas TX Washington DC Mexico City Mexico Data Scientist Resource Buenos Aires Argentina Data Scientist Resource ● Luminar is an analytics and modeling company focused on helping clients achieve growth and gain greater efficiencies ● We build the first cloud-based Big Data/ Hadoop analytics environment in the US serving the Latino market ● Key client segments include: Retail, CPG, Financial Services, Media & Entertainment, automotive, and Publishing, and services sector ● Luminar is an Entravision Communications (NYSE: EVC) business unit Miami FL Chicago IL © 2014 Luminar is a fully owned Entravision business unit
  • 12 Why The U.S. May Be The New 'Emerging' Market P&G, Coke, GM and Others Are Showing Renewed Interest in the U.S. as Its Growth Potential Rises “The U.S. population is growing – 27 million more inhabitants in 10 years…The percentage of Hispanics, Asians and African-Americans keeps growing. They contribute to winning elections but, more importantly, they're over-consumers…” AdAge Feb. 2013 Frederic Roze, President & CEO L'Oreal USA © 2014 Luminar is a fully owned Entravision business unit
  • 13 Today’s Challenge in Targeting Hispanics Brands are making marketing investment decisions on limited information Targeting assumptions based mostly on survey or sampled methods (i.e. “Latinos over-index on mobile usage”) Limited access to quantitative insights © 2014 Luminar is a fully owned Entravision business unit
  • 14 Sampling is like a Digital Photo…Insights Become Less Precise the Closer You Examine Your Data Business decisions are inherently weakened if you solely rely on sampling methods © 2014 Luminar is a fully owned Entravision business unit
  • To Address this Challenge, Luminar Set Out to Build the Largest Empirical Data Set of its Kind… 15 Consumer Habits | Traditional Sampled Approach | Luminar’s Latino Business Intelligence Transactional Data (POS, CRM, loyalty e-commerce) Digital Media Interactions Relevant Analytical Models Cultural References © 2014 Luminar is a fully owned Entravision business unit
  • 16 Ingesting Large Data Set to Derive Value 150 Million Unique Records 15 Million US Adult Latinos © 2014 Luminar is a fully owned Entravision business unit Representing 68% of all US Adult Latinos over 18
  • 17 How Luminar Defines Latino Consumers Consumer Interactions Points Luminar Cultural Filter and Scoring Consumption Behavior Consumer Characteristics Household-level Analysis Cultural sub- groupings Household characteristics Consumption Patterns Non-ethnic Comparison Language dominance Persona definitions Consumer © 2014 Luminar is a fully owned Entravision business unit
  • Marketers Are Seeking Precise Answers to Fuel Growth and Increase Efficiencies How much have we earned through diverse promotional channels? How acculturated is my market? Do I target them in Spanish, English or both? What is the Efficiency of our media activities? Which marketing drivers have had the greatest effects? What’s the size of the prize in my trading area? Are we optimally allocating our budget across all products? What’s my market share? 18 © 2014 Luminar is a fully owned Entravision business unit
  • 19 California Milk Processing Board © 2014 Luminar is a fully owned Entravision business unit
  • ● Milk consumption is experiencing consumption decline. Business dynamics driving this decline include: ─ an aging population, ─ consumption of milk alternative products, as well as ─ consumption of milk substitutes (i.e. energy drinks, juices, etc.) ● Without an empirical understanding of historical performance CMPB was left with an incomplete and often inaccurate read into the Hispanic market ● To identify potential areas of growth, CMPB needed a means to analyze consumption data over 2-3 year ● This would require building a robust tool that could monitor milk consumption across multiple DMAs and across variety of “filter options” Background and Business Challenge 20 © 2014 Luminar is a fully owned Entravision business unit
  • ● Luminar set out to aggregate the largest transactional data across the state of California ● We took a “total market” approach including four major population sub-groupings ─ Hispanics ─ Asian Americans ─ African American ● Data included transactional records for both Northern and Southern CA Luminar Solution – Aggregate the Largest Transactional Data set on Milk Products 11. 5 million households in CA Luminar captured transactional data for nearly 70% of total CA households 21 © 2014 Luminar is a fully owned Entravision business unit
  • Building CMPB’s Foundational Data Asset Luminar 150 million Transactional datastore - Zip + household data - Transactional UPC-level data - Social/demographics - Population subsector - Language of dominance California Milk Transactional Data - Item/UPC codes - Product category - Milk diary and milk alternatives Luminar Analytics Process: - Data enrichment - Customer segmentation 22 © 2014 Luminar is a fully owned Entravision business unit Luminar / Client Ready-Data Asset Product Consumption Segmentation Analysis Trend Analysis Analysis Report Luminar BI Portal
  • 23 ● Varied data: credit card transactions, set top box streams, voter records and social media ● Easy integration: Amazon Cloud, R, Talend and Tableau ● Better ingest: ─ 300 to 2,000 data sources ─ 2TB to 15TB, monthly data volume ● Speed to insight: from 3 days to 3 hours processing time Hortonworks Data Platform Powers Luminar’s Analytics Models We are going to be improving our ability to listen for what U.S. Latino consumers want and to communicate that voice to more clients through innovative applications running on Hadoop. Franklin Rios, President – Luminar ” “ © 2014 Luminar is a fully owned Entravision business unit
  • Page24 © Hortonworks Inc. 2011 – 2014. All Rights Reserved HDP: Varied, Granular & Persistent Data for Analysis OPERATIONS TOOLS DATASYSTEM EXISTING REPOSITORY SOURCES AWS OLTP, ERP, CRM Systems Documents, Emails Web Logs, Click Streams Social Networks Machine Generated Sensor Data Geolocation Data Governance &Integration Security Operations Data Access Data Management APPLICATIONS Luminar Insights Data Onboarding
  • Page25 © Hortonworks Inc. 2011 – 2014. All Rights Reserved A Look Inside the HDP Technology Stack 2
  • 26 ● We ingested transactional data into our Big Data environment for processing and developing the analysis to support client objectives ● We delivered a BI tool that provided access to custom-made relevant KPIs overtime (3-years of historical data) ● The granularity of the data provided monthly, quarterly and annual reporting across all product segments ● We then worked with the Agency of Records to provide greater insights into answer “the so what” questions that help identify market growth potential Our Data Approach © 2014 Luminar is a fully owned Entravision business unit
  • 27 Develop Custom BI Application Consisting of 12 Dashboards intersecting multiple data points © 2014 Luminar is a fully owned Entravision business unit
  • 28 Data can be queried along a wide range of variables and on different intersecting points © 2014 Luminar is a fully owned Entravision business unit
  • 29 Milk Consumption (Gallons per Household) © 2014 Luminar is a fully owned Entravision business unit
  • 30 Milk Consumption by Ethnic Segments © 2014 Luminar is a fully owned Entravision business unit
  • A “Single Source of Truth” based on Empirical Consumer Behavior Data 31 Source: 2012 Gartner ─ What data do I have that is relevant and available to make decisions? ─ What data do I need to gather or acquire? Data Assets ─ What analytics technique are most appropriate for the business problem and data available? ─ Analytic techniques might include: ─ Classification ─ Product Consumption ─ Segmentation Analysis ─ Trend Analysis Insights ─ How do I tie insights to operational decisions? ─ How do I close the feedback loop to test and learn? Actions ─ How can I grow revenue? ─ How can I reduce risk and be more efficient? ─ What do I need to know? ─ What are my alternatives? ─ What are my constraints? Business problem © 2014 Luminar is a fully owned Entravision business unit
  • 32 Deriving Insights: Gallons/HH for “with kids” have declined significantly more than HH “without kids” over past 12 months © 2014 Luminar is a fully owned Entravision business unit
  • 33 Deriving Insights: Middle Income Level is the Most Price Sensitive © 2014 Luminar is a fully owned Entravision business unit
  • 34 Deriving Insights: Bilingual/English and English/only Hispanic consumption is declining more rapidly, while Spanish-only is on the rise © 2014 Luminar is a fully owned Entravision business unit
  • 35 Three Key Closing Remarks… The low hanging fruit of Hispanic consumers has been picked…the combination our Hadoop data environment and advanced analytics help drive effective frontline actions It not just about focusing what we know about Latinos, the true opportunities come from seeing something you never seen before…understanding the unknowns Reaching Hispanics is not just about language, acculturation or relevancy; it's about having precise measurability that can prove efficiencies and ROI © 2014 Luminar is a fully owned Entravision business unit