The document discusses the potential and pitfalls of big data. It notes that while big data will change the world, there are challenges to address like viewability of ads, inaccurate attribution windows, and incentives for bad behaviors. Proper tools and human expertise are needed to guide machines to realize big data's full potential while avoiding these pitfalls. Big data will require constant innovation, as seen with the creation of Map Reduce, Hadoop, and now newer technologies like Quantcast File System.
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
The Potential & Pitfalls of Big Data by Jag Duggal of Quantcast - SIC2014
1. The Potential & Pitfalls of Big Data
Jag Duggal, SVP Product, Quantcast
Seattle, WA
October 15, 2014
Copyright 2014 Quantcast
2. Copyright 2014 Quantcast
Quantcast is an advertising technology
company but a big data company first
100M
Access browsing
behavior of +100M
destinations
600X
We see each U.S.
consumer over 600
times per month
A Top 5 data processing
organization alongside Google
and U.S. Dept. of Defense
3. A data revolution, a relevance revolution
Radio
Copyright 2014 Quantcast
Health
Education
Politics
Agriculture
6. Copyright 2014 Quantcast
Economy
GDP % growth, annual
Source: The World Bank
4.00
3.00
2.00
1.00
0.00
-1.00
-2.00
-3.00
-4.00
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Annual % Growth
7. Copyright 2014 Quantcast
Geo-Politics
“When you look at our differing views
of the Arab Spring, on how to deal with
Iran, on changing energy markets that
make gulf oil less central — these
things have altered the basis of U.S.-
Saudi relations.”
Source: NY Times
8. Copyright 2014 Quantcast
Climate Change
7,000
6,000
5,000
4,000
3,000
2,000
1,000
0
Emmisions (million metric tons)
U.S. CO2 Emissions, 1990-2012
Source: U.S. EPA Inventory of U.S. Greenhouse Gas and Sinks
9. GDP % growth, annual
Source: The World Bank
Copyright 2014 Quantcast
Big data will change the world as we know
it
“When you look at our differing views of
the Arab Spring, on how to deal with
Iran, on changing energy markets that
make gulf oil less central — these things
have altered the basis of U.S.-Saudi
relations.”
U.S. CO2 Emissions, 1990-2012
Source: U.S. EPA Inventory of U.S. Greenhouse Gas and Sinks
7000
6000
5000
4000
3000
2000
1000
0
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
Emmisions (million metric tons)
Geo-Politics Climate Change
Economy
4.00
3.00
2.00
1.00
0.00
-1.00
-2.00
-3.00
-4.00
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Annual % Growth
is changing
10. Copyright 2014 Quantcast
Epicenter: Online Media & Advertising
Fragmentation at a
whole new level.
Impressions delivered
individually, each
entirely distinct.
Segments of One.
11. Copyright 2014 Quantcast
Big data…necessity is the mother of
invention
• Map Reduce & Hadoop invented by Google
and Yahoo…because personalized media
required it
• Today, used in almost every industry
• We buried last vestige of Hadoop at Quantcast
two weeks ago…
• ….no longer up to the scale required
• We use instead Quantcast File System (QFS)
which we open-sourced
18. Split funnel attribution
Pay for what you want… new customers
Copyright 2014 Quantcast
Site Visit
Conversion Event
(e.g. Jag buys plane ticket on April 4th)
Last View
1
2
Split Credit for Each
Conversion
You Decide How to
Divide Credit
Last View
Last View before Site Visit Before Visit
19. Tools emerging to allow humans to guide the
machines
Viewability
Moat, DoubleVerify,
Integral…and MRC
Copyright 2014 Quantcast
Conversion Window
Awareness by
agency experts
Last touch
Attribution that
“splits the funnel”
pioneered by
Abakus, Ensighten,
Quantcast…and
perhaps others
For the better part of the last decade, we’ve been amassing an unparalleled perspective on the entirety of internet activity at an individual level through our Quantcast Measure product.
The Quantcast code is embedded in hundreds of millions of websites. By the time a consumer visits your site we’ve seen them 600Xs that month already.
This provides us with an unique real-time perspective on consumer behavior.
We’re able sense instantaneous shifts and see patterns that are invisible to everyone else
No one else has and sees this activity all in real time and subsequently an advertising solution that’s woven into the fabric of the internet.
Decline in growth of health care costs
Repair of the housing market
Energy (natural gas) revolution
Rebound in manufacturing – driven in part by cheap energy
Its hard enough for Harry Crane to keep track of a large number of media choices, but RTB pushes the traditional approaches to breaking point.
When you then consider that we’re not just choosing from amongst thousands of media destinations, but in fact choosing from billions of individual impressions, the task is even more daunting for a traditional approach.
And this is not some small side show…
AI beat Gary Kasparov in 1997.
McDonalds 2013 US Ad Spend by Media
Source: eMarketer ($ in thousands)
2013
Print $47,609
TV $737,783
Radio $76,230
Outdoor $84,941
Internet (display-only) $29,008
Total Measured $975,571
Total Unmeasured $458,669
Total: $1,434,240
>20 ads
in
<10 minutes
After the site visit,
Most not viewable