SlideShare a Scribd company logo
1 of 48
Download to read offline
Ben Porterfield
Founder, VP Engineering
Business Analytics:
Asking the Right
Questions
B U S I N E S S I N T E L L I G E N C E
Operational Control
How many sales did I do today?
Understand & Improve Experience
Are users engaging? Do they like the new features?
Make business decisions
Should we start delivering in a new city?
—ANDREW LEONARD
Salon
“Data indicated that the same subscribers who
loved the original BBC production also gobbled
down movies starring Kevin Spacey or directed
by David Fincher”
1
1 Tracking Data
2 Storing Data
3 Merging Data (ETL)
4 Retrieving Data
5 Analysis & Decision Making
The Analytical
Process
Tracking Data
What To Track?
Views
Clicks
In-app actions
Event
Users
Orders
Inventory
Transac tional
Embed in
product
process
Server-
side too
Taxonomy
matters
Tracking - Event Data
Every new feature
should come with events
Lots of non-
transactional events
happen on server
Big flat event space
becomes unwieldy
Storing Data
Go with
SQL
Store all
states
Keep it
clean
Storing - Transactional Data
NoSQL could be a
burden long-term
Even offline processes Messy schema =
complicated analytics
—MICHAEL ERASMUS
Back-end Engineer,
Buffer
“We were relying on MongoDB…while it was
easier for developers to play with the data,
it became a hurdle for other team
members.”
Own it
Use eco-
system
too
Store all
the IDs
Storing - Event Data
Or, at a minimum, be
able to get it
Lots of great SaaS event
platforms
Need to be able to
correlate events to
transactions
Merging Data
Other Data?
Transactional
Data
Event Data
Raw Queries Biz-User Tools
You should combine
transaction and event data,
+more
Use an analytical database
Redshift is current leader
Difficult - data is heavy
Application
WITH	
  user_order_activity	
  AS	
  (	
  	
  
	
  	
  SELECT	
  user_id,	
  age	
  
	
  	
  FROM	
  ORDERS	
  
	
  	
  GROUP	
  BY	
  user_id)	
  
SELECT	
  AVG(users.age)	
  as	
  
average_age_of_purchaser	
  
FROM	
  user_order_activity	
  
LEFT	
  JOIN	
  users	
  ON	
  
user_order_activity.user_id	
  =	
  
users.user_id	
  
	
  
SUMMARY
Traditional Approach
OLAP / Data Summaries
S I LO E D
Restricted Q&A
L I M I T E D
I
G
M
N
L
D
Q
B
A
P
R
S
Q
D I F F I CU LT & CO M B E R S O M E
ETL - Heavy Transformation
END USERBI TEAMETL TEAM EDW TEAM
W A N T TO A S K N E W Q U E ST I O N S ?
A B
?CF
X
EB
A
EVENT
DATA
TRANSACTIONALDATA
Modern Approach
3R D PARTY APP
API
ANY DEVICE
Transformation at Query
F L E X I B L E
Anywhere for Anyone
A CC E SS I B L ECO N S O L I D AT E D
Simple Extract & Load
I
G
M N
D
Q
A
P
R
S
Q
T
U
W
X
G Q
U
S
A
Z
Data Modeling Layer
A G I L E
D ATA T E A M E N D U S E R S
Data
Model
- name: first_purchasers
type: single_value
base_view: orders
measures:
[orders.first_purchase_count]
listen:
- name:
orders_by_day_and_category
title: "Orders by Day and
Category"
type: looker_area
base_view: order_items
I N N O VAT I O N
TRANSACTIONAL
DATA
EVENT
DATA
Z
B
QA A Z
M P P | R E D S H I F T | I M PA L A
Asana’s Data Infrastructure
Retrieving Data
—TODD LEHR
SVP Engineering,
Dollar Shave Club
“We have a developer
name Juan and any
reports we needed would
flow through him.”
—TODD LEHR
SVP Engineering,
Dollar Shave Club
“When he got backlogged,
our team didn’t have
access to the data
immediately.”
—ANNIE CORBETT
Business Intelligence Analyst,
Venmo
“Initially whenever we
were asked for data, we
would write a custom
script…”
—ANNIE CORBETT
Business Intelligence Analyst,
Venmo
“..and then repeat this
process whenever the
product team wanted to
extend the timeframe.”
What’s selling? What colors
and sizes is it selling in?
What’s getting returned? Is
there a particular size/color?
Is there a product people
buy first that increases their
likelihood of becoming a
repeat customer?
Questions from a retail
buyer at e-commerce
store:
Get them
the tool
Decisions
vs. data
science
Game-
changing
insights
Self-Service is Key
People with questions
are running the
businesses.
“Should we open a new
market in Maine?”
Don’t only come from
analyst group
Analysis and Decision Making
1
1
Clearly define success
metrics
2
Look for low-hanging
fruit
3 Go one level deeper
Analysis and
Decision Making
Analysis and Decision Making:
Success Metrics
Focus on desired outcome
What do you want users to experience?
Measure Engagement
In most cases this is first-line business
analytics
Measure Retention
Are people coming back?
S U C C E S S M E T R I C S
H O W T O T R A C K E N G A G E M E N T ?
Not with page views
Usually not even with time on page
Upworthy’s attention minutes
Lots of indicators (mouse, video, etc)
Looker’s approximate usage
Any event in 2 minute window
Deriving Approximate Usage
SELECT	
  
	
  	
  event.created_at	
  AS	
  created_date,	
  
	
  	
  event.user_id	
  as	
  user_id,	
  
	
  	
  COUNT(*)	
  AS	
  count,	
  
	
  	
  COUNT(DISTINCT	
  
	
  	
  	
  	
  CONCAT(	
  
	
  	
  	
  	
  	
  	
  CONCAT(event.user_id,'|',event.user_browser_id),	
  
	
  	
  	
  	
  	
  	
  FLOOR(UNIX_TIMESTAMP(event.created_at)/(60*2))	
  
	
  	
  	
  	
  )	
  
	
  	
  )*2	
  AS	
  approximate_usage_in_minutes	
  
FROM	
  event	
  
GROUP	
  BY	
  created_date,	
  user_id	
  
created_date user_id	
   count	
   approximate_usage	
  
1/10	
   1 123 100 minutes
1/10	
   2 228 50 minutes
1/10	
   3 45 80 minutes
Derived Tables
SELECT	
  
	
  	
  orders.user_id	
  as	
  user_id	
  
	
  	
  COUNT(*)	
  as	
  lifetime_orders	
  
	
  	
  MIN(orders.created_at)	
  as	
  first_order	
  
	
  	
  MAX(orders.created_at)	
  as	
  latest_order	
  
	
  	
  COUNT(DISTINCT	
  DATE_TRUNC('month’))	
  as	
  	
  	
  	
  
	
  
distinct_months_with_orders	
  
FROM	
  orders	
  
GROUP	
  BY	
  user_id	
  
Transactional
Event
Analytical
Derived Table
Insights
Start
simple
Most
useful at
row level
Great for
cohorts and
sessionization
Derived Tables
Subselects until slow,
SQL on cron works
surprisingly well
Don’t roll up data, pre-
compute facts
Tiered derived
dimension vs. some
other metric
Derived Table - User Order Facts
SELECT	
  
	
  	
  orders.user_id	
  as	
  user_id	
  
	
  	
  COUNT(*)	
  as	
  lifetime_orders	
  
	
  	
  MIN(orders.created_at)	
  as	
  first_order	
  
	
  	
  MAX(orders.created_at)	
  as	
  latest_order	
  
	
  	
  COUNT(DISTINCT	
  DATE_TRUNC('month’))	
  as	
  	
  	
  	
  
	
  distinct_months_with_orders	
  
FROM	
  orders	
  
GROUP	
  BY	
  user_id	
  
user_id lifetime_orders	
   first_order	
   latest_order	
   distinct_months_with_orders	
  
1	
   10 1/10/15 2/14/15 2
Derived Table + Sourcing
Derived Table + Sourcing
Churn
Users that will likely
never do X again
Usage
How likely to purchase
if they do X
Time to
transaction
How long till first X
Retention
Are users coming back
???
Invent a metric
Repeat
buyers
What’s different about
them
Pay/Charge Mistake.
It was clear some
users were
accidentally paying
instead of charging,
but it wasn't clear
how widespread the
problem was and
whether it was worth
prioritizing a fix
Inventing Metrics
Identify
behavior
Measure %
of
population
Experiment
Inventing Metrics
Can be good or bad –
just something possibly
significant
Who is doing this thing? Ability to play with
numbers is crucial
Analysis and Decision Making:
Low-hanging Fruit
This is the kind of
very visual, very
data‑driven piece
of analysis that
helps us think, "Is
opening the sale at
noon the right
decision?”
???
Low-hanging Fruit
Out of stocks are
huge detractors from
the customer
experience - it sucks
ordering something
and then not getting
it - as well as
revenue we failed to
capture
Low-hanging Fruit
Analysis and Decision Making:
One Level Deeper
While this immediate
insight might have
led us to focus on
small groups, this
didn’t match our
expectations of
people planning an
outing on a Friday
night, prompting us
to look further.
One Level Deeper
2 3 4
Time To Book
2 3 4
Group Size
We analyze all the
platform data
available - When
someone attempts
to sign, completes
the signup, pushes
an app, has spend,
etc
One Level Deeper
Even though it
looks like we were
having nice
incremental
growth, looking into
the details we see
some things to look
into further
One Level Deeper
Don’t confuse an increase
in a metric with success.
Put data in
analytical
database
Give
business
users tool
Define
success
metrics
Takeaways
Make sure it’s fast and
speaks SQL
Empower them to
answer their own
questions
Focus on engagement
and retention
Ben Porterfield
Founder, VP Engineering
ben@looker.com

More Related Content

Similar to Looker's Ben Porterfield - Asking The Right Questions

Data At Pollfish, Dec. 2015, Euangelos Linardos
Data At Pollfish, Dec. 2015, Euangelos LinardosData At Pollfish, Dec. 2015, Euangelos Linardos
Data At Pollfish, Dec. 2015, Euangelos LinardosEuangelos Linardos
 
Data at Pollfish
Data at PollfishData at Pollfish
Data at PollfishPollfish
 
Human Sensor Conference Opening- Future of Real Estate Tech
Human Sensor Conference Opening- Future of Real Estate TechHuman Sensor Conference Opening- Future of Real Estate Tech
Human Sensor Conference Opening- Future of Real Estate TechKevin Kononenko
 
The Data Science Process
The Data Science ProcessThe Data Science Process
The Data Science ProcessVishal Patel
 
Dashboards Driving Decision Making - ui and me
Dashboards Driving Decision Making - ui and meDashboards Driving Decision Making - ui and me
Dashboards Driving Decision Making - ui and meMary Chant
 
How to Design for (Digital) Success
How to Design for (Digital) SuccessHow to Design for (Digital) Success
How to Design for (Digital) SuccessSøren Engelbrecht
 
Being data driven - our data journey
Being data driven - our data journeyBeing data driven - our data journey
Being data driven - our data journeyIstván Rechner
 
Monitoring System
Monitoring SystemMonitoring System
Monitoring SystemAnn Garcia
 
FDSeminar Reporting & controlling
FDSeminar Reporting & controllingFDSeminar Reporting & controlling
FDSeminar Reporting & controllingFDMagazine
 
BDW16 London - Scott Krueger, skyscanner - Does More Data Mean Better Decisio...
BDW16 London - Scott Krueger, skyscanner - Does More Data Mean Better Decisio...BDW16 London - Scott Krueger, skyscanner - Does More Data Mean Better Decisio...
BDW16 London - Scott Krueger, skyscanner - Does More Data Mean Better Decisio...Big Data Week
 
OpenEdge Character UI - Where to go?
OpenEdge Character UI - Where to go?OpenEdge Character UI - Where to go?
OpenEdge Character UI - Where to go?Gabriel Lucaciu
 
Data capabilities and competitive advantage
Data capabilities and competitive advantageData capabilities and competitive advantage
Data capabilities and competitive advantageNUS-ISS
 
Managing Quality And Performance Study Guide
Managing Quality And Performance Study GuideManaging Quality And Performance Study Guide
Managing Quality And Performance Study GuideChristina Ramirez
 
Solving 21st Century App Performance Problems Without 21 People
Solving 21st Century App Performance Problems Without 21 PeopleSolving 21st Century App Performance Problems Without 21 People
Solving 21st Century App Performance Problems Without 21 PeopleDynatrace
 
Cross Device Tracking - Thomas Danniau
Cross Device Tracking - Thomas DanniauCross Device Tracking - Thomas Danniau
Cross Device Tracking - Thomas DanniauThe Reference
 
How to Apply Machine Learning with R, H20, Apache Spark MLlib or PMML to Real...
How to Apply Machine Learning with R, H20, Apache Spark MLlib or PMML to Real...How to Apply Machine Learning with R, H20, Apache Spark MLlib or PMML to Real...
How to Apply Machine Learning with R, H20, Apache Spark MLlib or PMML to Real...Kai Wähner
 
Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...
Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...
Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...Jin Young Kim
 
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance Data
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance DataDigital Disruption Asia - Pleasing the Unpleasable with Digital Performance Data
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance DataDynatrace
 

Similar to Looker's Ben Porterfield - Asking The Right Questions (20)

Data At Pollfish, Dec. 2015, Euangelos Linardos
Data At Pollfish, Dec. 2015, Euangelos LinardosData At Pollfish, Dec. 2015, Euangelos Linardos
Data At Pollfish, Dec. 2015, Euangelos Linardos
 
Data at Pollfish
Data at PollfishData at Pollfish
Data at Pollfish
 
Human Sensor Conference Opening- Future of Real Estate Tech
Human Sensor Conference Opening- Future of Real Estate TechHuman Sensor Conference Opening- Future of Real Estate Tech
Human Sensor Conference Opening- Future of Real Estate Tech
 
The Data Science Process
The Data Science ProcessThe Data Science Process
The Data Science Process
 
Dashboards Driving Decision Making - ui and me
Dashboards Driving Decision Making - ui and meDashboards Driving Decision Making - ui and me
Dashboards Driving Decision Making - ui and me
 
How to Design for (Digital) Success
How to Design for (Digital) SuccessHow to Design for (Digital) Success
How to Design for (Digital) Success
 
Being data driven - our data journey
Being data driven - our data journeyBeing data driven - our data journey
Being data driven - our data journey
 
Monitoring System
Monitoring SystemMonitoring System
Monitoring System
 
FDSeminar Reporting & controlling
FDSeminar Reporting & controllingFDSeminar Reporting & controlling
FDSeminar Reporting & controlling
 
BDW16 London - Scott Krueger, skyscanner - Does More Data Mean Better Decisio...
BDW16 London - Scott Krueger, skyscanner - Does More Data Mean Better Decisio...BDW16 London - Scott Krueger, skyscanner - Does More Data Mean Better Decisio...
BDW16 London - Scott Krueger, skyscanner - Does More Data Mean Better Decisio...
 
OpenEdge Character UI - Where to go?
OpenEdge Character UI - Where to go?OpenEdge Character UI - Where to go?
OpenEdge Character UI - Where to go?
 
Acc 340 Preview Full Course
Acc 340 Preview Full Course Acc 340 Preview Full Course
Acc 340 Preview Full Course
 
Acc 340 Preview Full Course
Acc 340 Preview Full CourseAcc 340 Preview Full Course
Acc 340 Preview Full Course
 
Data capabilities and competitive advantage
Data capabilities and competitive advantageData capabilities and competitive advantage
Data capabilities and competitive advantage
 
Managing Quality And Performance Study Guide
Managing Quality And Performance Study GuideManaging Quality And Performance Study Guide
Managing Quality And Performance Study Guide
 
Solving 21st Century App Performance Problems Without 21 People
Solving 21st Century App Performance Problems Without 21 PeopleSolving 21st Century App Performance Problems Without 21 People
Solving 21st Century App Performance Problems Without 21 People
 
Cross Device Tracking - Thomas Danniau
Cross Device Tracking - Thomas DanniauCross Device Tracking - Thomas Danniau
Cross Device Tracking - Thomas Danniau
 
How to Apply Machine Learning with R, H20, Apache Spark MLlib or PMML to Real...
How to Apply Machine Learning with R, H20, Apache Spark MLlib or PMML to Real...How to Apply Machine Learning with R, H20, Apache Spark MLlib or PMML to Real...
How to Apply Machine Learning with R, H20, Apache Spark MLlib or PMML to Real...
 
Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...
Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...
Data Science for Online Services: Problems & Frontiers (Changbal Conference 2...
 
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance Data
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance DataDigital Disruption Asia - Pleasing the Unpleasable with Digital Performance Data
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance Data
 

More from Heavybit

Brian Doll: Land and Expand Strategies at Github and New Relic
Brian Doll: Land and Expand Strategies at Github and New RelicBrian Doll: Land and Expand Strategies at Github and New Relic
Brian Doll: Land and Expand Strategies at Github and New RelicHeavybit
 
Brian Balfour: Building A Growth Machine
Brian Balfour: Building A Growth MachineBrian Balfour: Building A Growth Machine
Brian Balfour: Building A Growth MachineHeavybit
 
451 Group on Analyst Relations
451 Group on Analyst Relations451 Group on Analyst Relations
451 Group on Analyst RelationsHeavybit
 
Sean Byrnes: Business Analytics: Are We There Yet?
Sean Byrnes: Business Analytics: Are We There Yet?Sean Byrnes: Business Analytics: Are We There Yet?
Sean Byrnes: Business Analytics: Are We There Yet?Heavybit
 
Using OKRs to Drive Results AKA "Secrets To Crushing Your Goals"
Using OKRs to Drive Results AKA "Secrets To Crushing Your Goals"Using OKRs to Drive Results AKA "Secrets To Crushing Your Goals"
Using OKRs to Drive Results AKA "Secrets To Crushing Your Goals"Heavybit
 
Creating Killer Trend Stories with Redis Labs' Cameron Peron
Creating Killer Trend Stories with Redis Labs' Cameron PeronCreating Killer Trend Stories with Redis Labs' Cameron Peron
Creating Killer Trend Stories with Redis Labs' Cameron PeronHeavybit
 
Don MacLennan: Investing in Customer Success
Don MacLennan: Investing in Customer SuccessDon MacLennan: Investing in Customer Success
Don MacLennan: Investing in Customer SuccessHeavybit
 
Stripe's Krithika Muthukumar: Effective Launches
Stripe's Krithika Muthukumar: Effective LaunchesStripe's Krithika Muthukumar: Effective Launches
Stripe's Krithika Muthukumar: Effective LaunchesHeavybit
 
CoreOS' Melissa Smolensky: Minimum Viable Launch, What You Need For Marketing...
CoreOS' Melissa Smolensky: Minimum Viable Launch, What You Need For Marketing...CoreOS' Melissa Smolensky: Minimum Viable Launch, What You Need For Marketing...
CoreOS' Melissa Smolensky: Minimum Viable Launch, What You Need For Marketing...Heavybit
 
Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...
Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...
Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...Heavybit
 
Heavybit Event Manager
Heavybit Event ManagerHeavybit Event Manager
Heavybit Event ManagerHeavybit
 
Heavybit is Hiring an Events Manager
Heavybit is Hiring an Events ManagerHeavybit is Hiring an Events Manager
Heavybit is Hiring an Events ManagerHeavybit
 
Jeff Gothelf: Lean Product Design
Jeff Gothelf: Lean Product DesignJeff Gothelf: Lean Product Design
Jeff Gothelf: Lean Product DesignHeavybit
 
Jeff Atwood - How to Talk So Your Community Will Listen and Listen So Your Co...
Jeff Atwood - How to Talk So Your Community Will Listen and Listen So Your Co...Jeff Atwood - How to Talk So Your Community Will Listen and Listen So Your Co...
Jeff Atwood - How to Talk So Your Community Will Listen and Listen So Your Co...Heavybit
 
Charity Majors - Bootstrapping an Ops Team
Charity Majors - Bootstrapping an Ops TeamCharity Majors - Bootstrapping an Ops Team
Charity Majors - Bootstrapping an Ops TeamHeavybit
 
Heroku's Craig Kerstiens: Developer Marketing Channels
Heroku's Craig Kerstiens: Developer Marketing ChannelsHeroku's Craig Kerstiens: Developer Marketing Channels
Heroku's Craig Kerstiens: Developer Marketing ChannelsHeavybit
 
Harrison Metal's Michael Dearing on Pricing
Harrison Metal's Michael Dearing on PricingHarrison Metal's Michael Dearing on Pricing
Harrison Metal's Michael Dearing on PricingHeavybit
 
Slack's Ali Rayl on Scaling Support for User Growth
Slack's Ali Rayl on Scaling Support for User GrowthSlack's Ali Rayl on Scaling Support for User Growth
Slack's Ali Rayl on Scaling Support for User GrowthHeavybit
 
Stripe's Amber Feng on API Design
Stripe's Amber Feng on API DesignStripe's Amber Feng on API Design
Stripe's Amber Feng on API DesignHeavybit
 

More from Heavybit (19)

Brian Doll: Land and Expand Strategies at Github and New Relic
Brian Doll: Land and Expand Strategies at Github and New RelicBrian Doll: Land and Expand Strategies at Github and New Relic
Brian Doll: Land and Expand Strategies at Github and New Relic
 
Brian Balfour: Building A Growth Machine
Brian Balfour: Building A Growth MachineBrian Balfour: Building A Growth Machine
Brian Balfour: Building A Growth Machine
 
451 Group on Analyst Relations
451 Group on Analyst Relations451 Group on Analyst Relations
451 Group on Analyst Relations
 
Sean Byrnes: Business Analytics: Are We There Yet?
Sean Byrnes: Business Analytics: Are We There Yet?Sean Byrnes: Business Analytics: Are We There Yet?
Sean Byrnes: Business Analytics: Are We There Yet?
 
Using OKRs to Drive Results AKA "Secrets To Crushing Your Goals"
Using OKRs to Drive Results AKA "Secrets To Crushing Your Goals"Using OKRs to Drive Results AKA "Secrets To Crushing Your Goals"
Using OKRs to Drive Results AKA "Secrets To Crushing Your Goals"
 
Creating Killer Trend Stories with Redis Labs' Cameron Peron
Creating Killer Trend Stories with Redis Labs' Cameron PeronCreating Killer Trend Stories with Redis Labs' Cameron Peron
Creating Killer Trend Stories with Redis Labs' Cameron Peron
 
Don MacLennan: Investing in Customer Success
Don MacLennan: Investing in Customer SuccessDon MacLennan: Investing in Customer Success
Don MacLennan: Investing in Customer Success
 
Stripe's Krithika Muthukumar: Effective Launches
Stripe's Krithika Muthukumar: Effective LaunchesStripe's Krithika Muthukumar: Effective Launches
Stripe's Krithika Muthukumar: Effective Launches
 
CoreOS' Melissa Smolensky: Minimum Viable Launch, What You Need For Marketing...
CoreOS' Melissa Smolensky: Minimum Viable Launch, What You Need For Marketing...CoreOS' Melissa Smolensky: Minimum Viable Launch, What You Need For Marketing...
CoreOS' Melissa Smolensky: Minimum Viable Launch, What You Need For Marketing...
 
Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...
Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...
Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...
 
Heavybit Event Manager
Heavybit Event ManagerHeavybit Event Manager
Heavybit Event Manager
 
Heavybit is Hiring an Events Manager
Heavybit is Hiring an Events ManagerHeavybit is Hiring an Events Manager
Heavybit is Hiring an Events Manager
 
Jeff Gothelf: Lean Product Design
Jeff Gothelf: Lean Product DesignJeff Gothelf: Lean Product Design
Jeff Gothelf: Lean Product Design
 
Jeff Atwood - How to Talk So Your Community Will Listen and Listen So Your Co...
Jeff Atwood - How to Talk So Your Community Will Listen and Listen So Your Co...Jeff Atwood - How to Talk So Your Community Will Listen and Listen So Your Co...
Jeff Atwood - How to Talk So Your Community Will Listen and Listen So Your Co...
 
Charity Majors - Bootstrapping an Ops Team
Charity Majors - Bootstrapping an Ops TeamCharity Majors - Bootstrapping an Ops Team
Charity Majors - Bootstrapping an Ops Team
 
Heroku's Craig Kerstiens: Developer Marketing Channels
Heroku's Craig Kerstiens: Developer Marketing ChannelsHeroku's Craig Kerstiens: Developer Marketing Channels
Heroku's Craig Kerstiens: Developer Marketing Channels
 
Harrison Metal's Michael Dearing on Pricing
Harrison Metal's Michael Dearing on PricingHarrison Metal's Michael Dearing on Pricing
Harrison Metal's Michael Dearing on Pricing
 
Slack's Ali Rayl on Scaling Support for User Growth
Slack's Ali Rayl on Scaling Support for User GrowthSlack's Ali Rayl on Scaling Support for User Growth
Slack's Ali Rayl on Scaling Support for User Growth
 
Stripe's Amber Feng on API Design
Stripe's Amber Feng on API DesignStripe's Amber Feng on API Design
Stripe's Amber Feng on API Design
 

Recently uploaded

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 

Recently uploaded (20)

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 

Looker's Ben Porterfield - Asking The Right Questions

  • 1. Ben Porterfield Founder, VP Engineering Business Analytics: Asking the Right Questions
  • 2. B U S I N E S S I N T E L L I G E N C E Operational Control How many sales did I do today? Understand & Improve Experience Are users engaging? Do they like the new features? Make business decisions Should we start delivering in a new city?
  • 3. —ANDREW LEONARD Salon “Data indicated that the same subscribers who loved the original BBC production also gobbled down movies starring Kevin Spacey or directed by David Fincher”
  • 4. 1 1 Tracking Data 2 Storing Data 3 Merging Data (ETL) 4 Retrieving Data 5 Analysis & Decision Making The Analytical Process
  • 6. What To Track? Views Clicks In-app actions Event Users Orders Inventory Transac tional
  • 7. Embed in product process Server- side too Taxonomy matters Tracking - Event Data Every new feature should come with events Lots of non- transactional events happen on server Big flat event space becomes unwieldy
  • 9. Go with SQL Store all states Keep it clean Storing - Transactional Data NoSQL could be a burden long-term Even offline processes Messy schema = complicated analytics
  • 10. —MICHAEL ERASMUS Back-end Engineer, Buffer “We were relying on MongoDB…while it was easier for developers to play with the data, it became a hurdle for other team members.”
  • 11. Own it Use eco- system too Store all the IDs Storing - Event Data Or, at a minimum, be able to get it Lots of great SaaS event platforms Need to be able to correlate events to transactions
  • 13. Other Data? Transactional Data Event Data Raw Queries Biz-User Tools You should combine transaction and event data, +more Use an analytical database Redshift is current leader Difficult - data is heavy Application WITH  user_order_activity  AS  (        SELECT  user_id,  age      FROM  ORDERS      GROUP  BY  user_id)   SELECT  AVG(users.age)  as   average_age_of_purchaser   FROM  user_order_activity   LEFT  JOIN  users  ON   user_order_activity.user_id  =   users.user_id    
  • 14. SUMMARY Traditional Approach OLAP / Data Summaries S I LO E D Restricted Q&A L I M I T E D I G M N L D Q B A P R S Q D I F F I CU LT & CO M B E R S O M E ETL - Heavy Transformation END USERBI TEAMETL TEAM EDW TEAM W A N T TO A S K N E W Q U E ST I O N S ? A B ?CF X EB A EVENT DATA TRANSACTIONALDATA
  • 15. Modern Approach 3R D PARTY APP API ANY DEVICE Transformation at Query F L E X I B L E Anywhere for Anyone A CC E SS I B L ECO N S O L I D AT E D Simple Extract & Load I G M N D Q A P R S Q T U W X G Q U S A Z Data Modeling Layer A G I L E D ATA T E A M E N D U S E R S Data Model - name: first_purchasers type: single_value base_view: orders measures: [orders.first_purchase_count] listen: - name: orders_by_day_and_category title: "Orders by Day and Category" type: looker_area base_view: order_items I N N O VAT I O N TRANSACTIONAL DATA EVENT DATA Z B QA A Z M P P | R E D S H I F T | I M PA L A
  • 18. —TODD LEHR SVP Engineering, Dollar Shave Club “We have a developer name Juan and any reports we needed would flow through him.”
  • 19. —TODD LEHR SVP Engineering, Dollar Shave Club “When he got backlogged, our team didn’t have access to the data immediately.”
  • 20. —ANNIE CORBETT Business Intelligence Analyst, Venmo “Initially whenever we were asked for data, we would write a custom script…”
  • 21. —ANNIE CORBETT Business Intelligence Analyst, Venmo “..and then repeat this process whenever the product team wanted to extend the timeframe.”
  • 22. What’s selling? What colors and sizes is it selling in? What’s getting returned? Is there a particular size/color? Is there a product people buy first that increases their likelihood of becoming a repeat customer? Questions from a retail buyer at e-commerce store:
  • 23. Get them the tool Decisions vs. data science Game- changing insights Self-Service is Key People with questions are running the businesses. “Should we open a new market in Maine?” Don’t only come from analyst group
  • 25. 1 1 Clearly define success metrics 2 Look for low-hanging fruit 3 Go one level deeper Analysis and Decision Making
  • 26. Analysis and Decision Making: Success Metrics
  • 27. Focus on desired outcome What do you want users to experience? Measure Engagement In most cases this is first-line business analytics Measure Retention Are people coming back? S U C C E S S M E T R I C S
  • 28. H O W T O T R A C K E N G A G E M E N T ? Not with page views Usually not even with time on page Upworthy’s attention minutes Lots of indicators (mouse, video, etc) Looker’s approximate usage Any event in 2 minute window
  • 29. Deriving Approximate Usage SELECT      event.created_at  AS  created_date,      event.user_id  as  user_id,      COUNT(*)  AS  count,      COUNT(DISTINCT          CONCAT(              CONCAT(event.user_id,'|',event.user_browser_id),              FLOOR(UNIX_TIMESTAMP(event.created_at)/(60*2))          )      )*2  AS  approximate_usage_in_minutes   FROM  event   GROUP  BY  created_date,  user_id   created_date user_id   count   approximate_usage   1/10   1 123 100 minutes 1/10   2 228 50 minutes 1/10   3 45 80 minutes
  • 30. Derived Tables SELECT      orders.user_id  as  user_id      COUNT(*)  as  lifetime_orders      MIN(orders.created_at)  as  first_order      MAX(orders.created_at)  as  latest_order      COUNT(DISTINCT  DATE_TRUNC('month’))  as           distinct_months_with_orders   FROM  orders   GROUP  BY  user_id   Transactional Event Analytical Derived Table Insights
  • 31. Start simple Most useful at row level Great for cohorts and sessionization Derived Tables Subselects until slow, SQL on cron works surprisingly well Don’t roll up data, pre- compute facts Tiered derived dimension vs. some other metric
  • 32. Derived Table - User Order Facts SELECT      orders.user_id  as  user_id      COUNT(*)  as  lifetime_orders      MIN(orders.created_at)  as  first_order      MAX(orders.created_at)  as  latest_order      COUNT(DISTINCT  DATE_TRUNC('month’))  as          distinct_months_with_orders   FROM  orders   GROUP  BY  user_id   user_id lifetime_orders   first_order   latest_order   distinct_months_with_orders   1   10 1/10/15 2/14/15 2
  • 33. Derived Table + Sourcing
  • 34. Derived Table + Sourcing
  • 35. Churn Users that will likely never do X again Usage How likely to purchase if they do X Time to transaction How long till first X Retention Are users coming back ??? Invent a metric Repeat buyers What’s different about them
  • 37. It was clear some users were accidentally paying instead of charging, but it wasn't clear how widespread the problem was and whether it was worth prioritizing a fix Inventing Metrics
  • 38. Identify behavior Measure % of population Experiment Inventing Metrics Can be good or bad – just something possibly significant Who is doing this thing? Ability to play with numbers is crucial
  • 39. Analysis and Decision Making: Low-hanging Fruit
  • 40. This is the kind of very visual, very data‑driven piece of analysis that helps us think, "Is opening the sale at noon the right decision?” ??? Low-hanging Fruit
  • 41. Out of stocks are huge detractors from the customer experience - it sucks ordering something and then not getting it - as well as revenue we failed to capture Low-hanging Fruit
  • 42. Analysis and Decision Making: One Level Deeper
  • 43. While this immediate insight might have led us to focus on small groups, this didn’t match our expectations of people planning an outing on a Friday night, prompting us to look further. One Level Deeper 2 3 4 Time To Book 2 3 4 Group Size
  • 44. We analyze all the platform data available - When someone attempts to sign, completes the signup, pushes an app, has spend, etc One Level Deeper
  • 45. Even though it looks like we were having nice incremental growth, looking into the details we see some things to look into further One Level Deeper
  • 46. Don’t confuse an increase in a metric with success.
  • 47. Put data in analytical database Give business users tool Define success metrics Takeaways Make sure it’s fast and speaks SQL Empower them to answer their own questions Focus on engagement and retention
  • 48. Ben Porterfield Founder, VP Engineering ben@looker.com