1. The document discusses the rise of "Big Data" and how it is transforming market research by providing more data than ever before from sources like social media. 2. It explores how machine learning techniques like neural networks and genetic algorithms can be used to analyze large, complex datasets and discover patterns to predict future behavior. 3. Market research must embrace new predictive approaches using large datasets and fusion of different data sources to gain a "whole respondent view" and understand what will happen next rather than just why things occurred.

1. Big Data
Andrew Jeavons
President
Survey Analytics LLC

2. Overview
• Coming out of the data desert.
• What do we mean by Big Data ?
• Behavior, Big Data and Market Research.
• Get a model to get ahead.....

3. Data Desert
• Data was scarce..
• The hunt for completes.
• Sampling ?
• Limited data.
• Limited potential...

4. The Deluge
• The Data Tsunami.
• What we get..
• What we don’t get...

5. Forms of Data
• Metadata, Paradata, Data
• Big Data and Prediction
of behavior.
• The central dogma:
• Big Data is Marxist.
• Black Swans - Taleb’s
cautionary tale...

6. Machine Learning
• What is meant by this ?
• Learning and pattern recognition.

7. Nerves and Genes
• Neural Networks
• Genetic Algorithms
• “Agnostic” algorithms...

8. Neural Networks
Originally sought to reproduce the way
the nervous system works.
Had the ability to learn complex
relationships with any assumptions.
Can learn complex non linear
relationships.

9. Back Propogation

10. Genetic Algorithms

11. Biological metaphor: encode problems as chromosomes.
Fitness function.
Evolve different mapping functions via principles of evolution.
Cross over and mutation (note: mutation rates not as powerful as you may think...)
MR example: used GA’s to evolve panel sample balancing - rim balancing.

12. Social Media Data
• We have a lot of text...that we didn’t have
before...
• Text has its own set of problems..unlike
Gore Vidal we don’t speak in sentences.
• “#itsucks phat”
• Txt messages, social media messages.
• Parsing and tokenizing

13. Sentiment Analysis
• One of the frontiers for MR and Big Data
• Can we tell the sentiment behind the
comment ?
• Text is messy because language is messy.
• Grammatical vs Stochastic

14. Surveys, Market Research
and Big Data
• Fusion of data
• Survey and behavioral
• “Whole respondent” view
• Surveys can be “big data”

15. Prediction and Market Research
• Statisticians are the new “rock stars” (Eric
Schmidt)
• MR has to embrace prediction and realize
the limits of explanation. It’s great to know
why, better to know what’s next ?
• Theories of respondents and consumers...