Applied Data Science for monetization: pitfalls, common misconceptions, and n...DevGAMM Conference
This talk guides us through modern twists on classic user-oriented data science tasks, such as churn prediction, clusterization, calculating user metrics, and others. We will discuss unusual angles for solving these tasks; how and why they can be used to improve player experience and monetization; the intuition behind these methods, and insights into inner machinery; and why conventional methods work poorly. Finally, I'll show you how you can apply this knowledge to improve your users' playing experience, and streamline analytics; and we'll talk general situation of applied data science and analytics in the industry.
Presentation for students at Howard University studying Information Systems and Supply Chain Management and taking the Digital Business course. The goal was to tell some stories about how I used data over my career so far in business situations to illustrate how important it can be for a business today.
I talk about my domain name business, master's thesis doing box office prediction for movies and review signal (http://reviewsignal.com) my startup which does web hosting reviews.
Creating a culture that provokes failure and boosts improvementBen Dressler
Everyone fails - but not everyone uses failed attempts as a source of learning and improvement. This talk outlines a framework to turn failure into gaining knowledge by understanding IF, HOW and WHY something fails.
Applied Data Science for monetization: pitfalls, common misconceptions, and n...DevGAMM Conference
This talk guides us through modern twists on classic user-oriented data science tasks, such as churn prediction, clusterization, calculating user metrics, and others. We will discuss unusual angles for solving these tasks; how and why they can be used to improve player experience and monetization; the intuition behind these methods, and insights into inner machinery; and why conventional methods work poorly. Finally, I'll show you how you can apply this knowledge to improve your users' playing experience, and streamline analytics; and we'll talk general situation of applied data science and analytics in the industry.
Presentation for students at Howard University studying Information Systems and Supply Chain Management and taking the Digital Business course. The goal was to tell some stories about how I used data over my career so far in business situations to illustrate how important it can be for a business today.
I talk about my domain name business, master's thesis doing box office prediction for movies and review signal (http://reviewsignal.com) my startup which does web hosting reviews.
Creating a culture that provokes failure and boosts improvementBen Dressler
Everyone fails - but not everyone uses failed attempts as a source of learning and improvement. This talk outlines a framework to turn failure into gaining knowledge by understanding IF, HOW and WHY something fails.
Decision Tree Algorithm With Example | Decision Tree In Machine Learning | Da...Simplilearn
This Decision Tree Algorithm in Machine Learning Presentation will help you understand all the basics of Decision Tree along with what is Machine Learning, problems in Machine Learning, what is Decision Tree, advantages and disadvantages of Decision Tree, how Decision Tree algorithm works with solved examples and at the end we will implement a Decision Tree use case/ demo in Python on loan payment prediction. This Decision Tree tutorial is ideal for both beginners as well as professionals who want to learn Machine Learning Algorithms.
Below topics are covered in this Decision Tree Algorithm Presentation:
1. What is Machine Learning?
2. Types of Machine Learning?
3. Problems in Machine Learning
4. What is Decision Tree?
5. What are the problems a Decision Tree Solves?
6. Advantages of Decision Tree
7. How does Decision Tree Work?
8. Use Case - Loan Repayment Prediction
What is Machine Learning: Machine Learning is an application of Artificial Intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed.
- - - - - - - -
About Simplilearn Machine Learning course:
A form of artificial intelligence, Machine Learning is revolutionizing the world of computing as well as all people’s digital interactions. Machine Learning powers such innovative automated technologies as recommendation engines, facial recognition, fraud protection and even self-driving cars.This Machine Learning course prepares engineers, data scientists and other professionals with knowledge and hands-on skills required for certification and job competency in Machine Learning.
- - - - - - -
Why learn Machine Learning?
Machine Learning is taking over the world- and with that, there is a growing need among companies for professionals to know the ins and outs of Machine Learning
The Machine Learning market size is expected to grow from USD 1.03 Billion in 2016 to USD 8.81 Billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period.
- - - - - -
What skills will you learn from this Machine Learning course?
By the end of this Machine Learning course, you will be able to:
1. Master the concepts of supervised, unsupervised and reinforcement learning concepts and modeling.
2. Gain practical mastery over principles, algorithms, and applications of Machine Learning through a hands-on approach which includes working on 28 projects and one capstone project.
3. Acquire thorough knowledge of the mathematical and heuristic aspects of Machine Learning.
4. Understand the concepts and operation of support vector machines, kernel SVM, naive bayes, decision tree classifier, random forest classifier, logistic regression, K-nearest neighbors, K-means clustering and more.
5. Be able to model a wide variety of robust Machine Learning algorithms including deep learning, clustering, and recommendation systems
- - - - - - -
How to Correctly Use Experimentation in PM by Google PMProduct School
Main takeaways:
- Common misconceptions and pitfalls in using experimentation
- Best practices on using the scientific method for experimentation
- Evaluating how other experimentation techniques such as Multi-Armed Bandit and Multivariate Testing can help you solve different types of problems
Problem Solving Skill merupakan sebuah teknik untuk memecahkan masalah secara terstruktur, kompleks, dan utuh sehingga dapat ditemukan pilihan kebijakan atau kebijaksanaan yang memiliki efektifitas dan efisiensi tinggi serta minim resiko.
Economist and futurist Rebecca Ryan presents insights about the future workforce, next-gen technologies, new social norms, customer expectations, and more to help YOU build (or rebuild) a company that's future-ready. Presented at the Halifax State of the Economy Conference on May 22, 2013.
North Raleigh Rotarian Katie Turnbull gave a great presentation at our Friday morning extension meeting about data visualization. Katie is a consultant at research and advisory firm, Gartner, Inc.
Better Living Through Analytics - Strategies for Data DecisionsProduct School
Data is king! Get ready to understand how a successful analytics team can empower managers from product, marketing, and other areas to make effective, data-driven decisions.
Louis Cialdella, a data scientist at ZipRecruiter, shared some case studies and successful strategies that he has used at ZipRecruiter as well as previous experiences. The purpose of this data talk was to enlighten people on how to make sure that analysts can successfully partner with other departments and get them the information they need to do great things.
BUSI 331Marketing Research Report Part 3 InstructionsData .docxhumphrieskalyn
BUSI 331
Marketing Research Report Part 3 Instructions
Data Submission
Review the Basic Data Analysis section in the Zikmund & Babin text and the presentation from Module/Week 4, Presentation: Using Excel for Data Analysis. There will be 2 submissions in this assignment: the Excel document with the raw data that includes a code guide and the Marketing Research Report as a continuation of your Part 1 Word document.
1. Submit the raw data from your survey results in an Excel document. To do this, you will need to build an Excel spreadsheet to organize your data. You may find that Survey Monkey or other online survey tools will already do this for you.
2. In order to get the best results from your data analysis, you will need to code your responses that are not already numerical. For example, if your question asked if the respondent was male or female, male=0, and female=1. If it was yes or no question, yes=0, no=1. Please include a code guide with your raw data.
Your Excel document submission is your raw data with a code chart to clarify what the raw numbers stand for. Please note that raw data is numerical and the data has not been manipulated in any way. You can post the Code Guide in Sheet 2 of your Excel document if that is easier for you. As an example, your raw data and code guide will look like this:
Respondent
Gender
Age
Q1
1
1
1
1
2
1
2
4
3
2
3
3
4
2
2
5
5
1
3
2
Code Guide:
Gender: 1=male, 2=female
Age Range: 1=18-24, 2=25-30, etc.
Q1 (5 point likert scale): 1=very unlikely, 2=unlikely, 3=neutral, etc.
3. Submit 3 tables that were created in Excel from your data, inclusive of 1 frequency table and 2 cross-tabulation tables. This needs to be relevant information that will directly impact your research problem. Please write 1 comprehensive paragraph underneath each individual table that clearly describes what the table is showing and what the inferences are from this table and information in relation to the research problem. Turn at least 1 of your tables into a graph (either a bar or pie chart) to show the data from the table. Place this material (three charts/tables and three written discussions of each) as Appendix 2 in your research report, and submit this part as a compilation with your Parts 1 and 2.
This assignment is due by 11:59 p.m. (ET) on Monday of Module/Week 6.
Show all your work neatly for full credit.
1) Solve the differential equations:
2) Compute the solution of the given initial value problem.
3) For the equation
a) determine the frequency of the beats.
b) determine the frequency of the rapid oscillations.
c) Use the informayion from parts a) and b) to give a rough sketch of the graph of a typical solution.
4) Consider the equation
a) Compute the general solution.
a)
Solve the initial value problem
...
Should UI/UX be gut-feeling or data-driven? How to stand out from the tough competition by perfecting your owned asset?
A/B testing a long-grind road but it does not have to be tough! Demyth the 4 steps approach to optimization and what it can bring to you
Big Data, Little Devices: Mobile A/B TestingZac Aghion
A/B testing applies the scientific method and principles of randomized experimentation to digital marketing and product design. In this presentation, Zac Aghion from Splitforce will share his experience with A/B testing in the unique context of mobile applications, as well as two case studies featuring Marks & Spencer - among the UK's largest retailers, and Spot the Difference - a kids' game for iPhone and iPad.
We all know that an ounce of prevention is worth a pound of cure, but we don’t always take it to heart. We skip well doctor visits. We go too long between oil changes. And we let websites sprawl and morph and get out of date. As someone who’s been on both sides of website creation and maintenance — first as a marketer and content manager, now as a user experience designer — I know the struggle is real. Together we’ll explore how to conduct periodic checkups and adopt healthy practices to keep your website in shape between redesigns.
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18Mariia Bocheva
While working with data we usually face several problems: we don't have enough data, we have too much data, we don't know what to do with this data.
In this session, I'll show how to make sure you can rely on your data and share my favorite ideas on how you can use Google Analytics and other for A/B testing, optimization and analysis.
You’ll gain a better understanding on what to look at to answer your UX questions, how to run a test properly and evaluate the its results.
Decision Tree Algorithm With Example | Decision Tree In Machine Learning | Da...Simplilearn
This Decision Tree Algorithm in Machine Learning Presentation will help you understand all the basics of Decision Tree along with what is Machine Learning, problems in Machine Learning, what is Decision Tree, advantages and disadvantages of Decision Tree, how Decision Tree algorithm works with solved examples and at the end we will implement a Decision Tree use case/ demo in Python on loan payment prediction. This Decision Tree tutorial is ideal for both beginners as well as professionals who want to learn Machine Learning Algorithms.
Below topics are covered in this Decision Tree Algorithm Presentation:
1. What is Machine Learning?
2. Types of Machine Learning?
3. Problems in Machine Learning
4. What is Decision Tree?
5. What are the problems a Decision Tree Solves?
6. Advantages of Decision Tree
7. How does Decision Tree Work?
8. Use Case - Loan Repayment Prediction
What is Machine Learning: Machine Learning is an application of Artificial Intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed.
- - - - - - - -
About Simplilearn Machine Learning course:
A form of artificial intelligence, Machine Learning is revolutionizing the world of computing as well as all people’s digital interactions. Machine Learning powers such innovative automated technologies as recommendation engines, facial recognition, fraud protection and even self-driving cars.This Machine Learning course prepares engineers, data scientists and other professionals with knowledge and hands-on skills required for certification and job competency in Machine Learning.
- - - - - - -
Why learn Machine Learning?
Machine Learning is taking over the world- and with that, there is a growing need among companies for professionals to know the ins and outs of Machine Learning
The Machine Learning market size is expected to grow from USD 1.03 Billion in 2016 to USD 8.81 Billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period.
- - - - - -
What skills will you learn from this Machine Learning course?
By the end of this Machine Learning course, you will be able to:
1. Master the concepts of supervised, unsupervised and reinforcement learning concepts and modeling.
2. Gain practical mastery over principles, algorithms, and applications of Machine Learning through a hands-on approach which includes working on 28 projects and one capstone project.
3. Acquire thorough knowledge of the mathematical and heuristic aspects of Machine Learning.
4. Understand the concepts and operation of support vector machines, kernel SVM, naive bayes, decision tree classifier, random forest classifier, logistic regression, K-nearest neighbors, K-means clustering and more.
5. Be able to model a wide variety of robust Machine Learning algorithms including deep learning, clustering, and recommendation systems
- - - - - - -
How to Correctly Use Experimentation in PM by Google PMProduct School
Main takeaways:
- Common misconceptions and pitfalls in using experimentation
- Best practices on using the scientific method for experimentation
- Evaluating how other experimentation techniques such as Multi-Armed Bandit and Multivariate Testing can help you solve different types of problems
Problem Solving Skill merupakan sebuah teknik untuk memecahkan masalah secara terstruktur, kompleks, dan utuh sehingga dapat ditemukan pilihan kebijakan atau kebijaksanaan yang memiliki efektifitas dan efisiensi tinggi serta minim resiko.
Economist and futurist Rebecca Ryan presents insights about the future workforce, next-gen technologies, new social norms, customer expectations, and more to help YOU build (or rebuild) a company that's future-ready. Presented at the Halifax State of the Economy Conference on May 22, 2013.
North Raleigh Rotarian Katie Turnbull gave a great presentation at our Friday morning extension meeting about data visualization. Katie is a consultant at research and advisory firm, Gartner, Inc.
Better Living Through Analytics - Strategies for Data DecisionsProduct School
Data is king! Get ready to understand how a successful analytics team can empower managers from product, marketing, and other areas to make effective, data-driven decisions.
Louis Cialdella, a data scientist at ZipRecruiter, shared some case studies and successful strategies that he has used at ZipRecruiter as well as previous experiences. The purpose of this data talk was to enlighten people on how to make sure that analysts can successfully partner with other departments and get them the information they need to do great things.
BUSI 331Marketing Research Report Part 3 InstructionsData .docxhumphrieskalyn
BUSI 331
Marketing Research Report Part 3 Instructions
Data Submission
Review the Basic Data Analysis section in the Zikmund & Babin text and the presentation from Module/Week 4, Presentation: Using Excel for Data Analysis. There will be 2 submissions in this assignment: the Excel document with the raw data that includes a code guide and the Marketing Research Report as a continuation of your Part 1 Word document.
1. Submit the raw data from your survey results in an Excel document. To do this, you will need to build an Excel spreadsheet to organize your data. You may find that Survey Monkey or other online survey tools will already do this for you.
2. In order to get the best results from your data analysis, you will need to code your responses that are not already numerical. For example, if your question asked if the respondent was male or female, male=0, and female=1. If it was yes or no question, yes=0, no=1. Please include a code guide with your raw data.
Your Excel document submission is your raw data with a code chart to clarify what the raw numbers stand for. Please note that raw data is numerical and the data has not been manipulated in any way. You can post the Code Guide in Sheet 2 of your Excel document if that is easier for you. As an example, your raw data and code guide will look like this:
Respondent
Gender
Age
Q1
1
1
1
1
2
1
2
4
3
2
3
3
4
2
2
5
5
1
3
2
Code Guide:
Gender: 1=male, 2=female
Age Range: 1=18-24, 2=25-30, etc.
Q1 (5 point likert scale): 1=very unlikely, 2=unlikely, 3=neutral, etc.
3. Submit 3 tables that were created in Excel from your data, inclusive of 1 frequency table and 2 cross-tabulation tables. This needs to be relevant information that will directly impact your research problem. Please write 1 comprehensive paragraph underneath each individual table that clearly describes what the table is showing and what the inferences are from this table and information in relation to the research problem. Turn at least 1 of your tables into a graph (either a bar or pie chart) to show the data from the table. Place this material (three charts/tables and three written discussions of each) as Appendix 2 in your research report, and submit this part as a compilation with your Parts 1 and 2.
This assignment is due by 11:59 p.m. (ET) on Monday of Module/Week 6.
Show all your work neatly for full credit.
1) Solve the differential equations:
2) Compute the solution of the given initial value problem.
3) For the equation
a) determine the frequency of the beats.
b) determine the frequency of the rapid oscillations.
c) Use the informayion from parts a) and b) to give a rough sketch of the graph of a typical solution.
4) Consider the equation
a) Compute the general solution.
a)
Solve the initial value problem
...
Should UI/UX be gut-feeling or data-driven? How to stand out from the tough competition by perfecting your owned asset?
A/B testing a long-grind road but it does not have to be tough! Demyth the 4 steps approach to optimization and what it can bring to you
Big Data, Little Devices: Mobile A/B TestingZac Aghion
A/B testing applies the scientific method and principles of randomized experimentation to digital marketing and product design. In this presentation, Zac Aghion from Splitforce will share his experience with A/B testing in the unique context of mobile applications, as well as two case studies featuring Marks & Spencer - among the UK's largest retailers, and Spot the Difference - a kids' game for iPhone and iPad.
We all know that an ounce of prevention is worth a pound of cure, but we don’t always take it to heart. We skip well doctor visits. We go too long between oil changes. And we let websites sprawl and morph and get out of date. As someone who’s been on both sides of website creation and maintenance — first as a marketer and content manager, now as a user experience designer — I know the struggle is real. Together we’ll explore how to conduct periodic checkups and adopt healthy practices to keep your website in shape between redesigns.
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18Mariia Bocheva
While working with data we usually face several problems: we don't have enough data, we have too much data, we don't know what to do with this data.
In this session, I'll show how to make sure you can rely on your data and share my favorite ideas on how you can use Google Analytics and other for A/B testing, optimization and analysis.
You’ll gain a better understanding on what to look at to answer your UX questions, how to run a test properly and evaluate the its results.
1. Prepared for:
NYC UX + DATA Meetup
March 12, 2014
Pivotal Labs, NewYork
A/B and PairwiseTesting
How I Learned to Stop Worrying and Love
Data-Driven Decisions
Wednesday, March 12, 14
2. About Me
• Founded Splitforce in 2013 - Data is
power, and it should be easy to leverage
• Marketing for Chinese media company
in Shanghai
• Designed experiments and predictive
analytics for ILABS in Montreal
• Studied in economics and statistics at
McGill University in Montreal
Wednesday, March 12, 14
5. User Base
Publish two different versions
of your app...
50% sees version B50% sees version A
Wednesday, March 12, 14
6. User Base
...and see which one is driving
desirable user behavior.
Publish two different versions
of your app...
50% sees version B50% sees version A
Wednesday, March 12, 14
13. And the Winner is...
+40%
increase in conversion
rate
2.9 million
additional donators
$60 million
value of additional
donations
Wednesday, March 12, 14
16. Obamalytics
• Original Conversion Rate: 8.3%
• New Conversion Rate: 11.6%
• 10 million signups from NewVersion would have been
7.12 million signups with the OriginalVersion
Wednesday, March 12, 14
17. Obamalytics
• Original Conversion Rate: 8.3%
• New Conversion Rate: 11.6%
• 10 million signups from NewVersion would have been
7.12 million signups with the OriginalVersion
• +2.88 million additional signups
Wednesday, March 12, 14
18. Obamalytics
• Original Conversion Rate: 8.3%
• New Conversion Rate: 11.6%
• 10 million signups from NewVersion would have been
7.12 million signups with the OriginalVersion
• +2.88 million additional signups
• $21 average donation per signup
Wednesday, March 12, 14
19. Obamalytics
• Original Conversion Rate: 8.3%
• New Conversion Rate: 11.6%
• 10 million signups from NewVersion would have been
7.12 million signups with the OriginalVersion
• +2.88 million additional signups
• $21 average donation per signup
• Approximately $60 million in additional donations
Wednesday, March 12, 14
21. MultivariateTesting
• Every screen has X components (ex: Marilyn’s hair)
• For each, we can test Y variations (ex.: Green)
• In total, we have [Y1 x Y2 x Y3] combinations
Wednesday, March 12, 14
22. Costs ofTesting
• Risk of false positives (Type I error, saying something
is there when it’s not)
• Need for adequate sample size
• Testing presents an opportunity cost
Wednesday, March 12, 14
23. Design of Experiments
• Let’s say we have four variables:
• Header Banner (A, B, C)
• Main Copy (1, 2, 3)
• Button Color (Cyan, Magenta,Yellow)
• Call to Action (Buy!, Check Out)
Wednesday, March 12, 14
24. Design of Experiments
• Option 1: Full factorial design -
multiply out for all different
combinations
Wednesday, March 12, 14
25. Design of Experiments
• Option 1: Full factorial design -
multiply out for all different
combinations
• Example: (3 header banners) x (3 main
copy) x (3 button colors) x (2 CTAs) =
54 combinations
Wednesday, March 12, 14
26. Design of Experiments
• Option 1: Full factorial design -
multiply out for all different
combinations
• Example: (3 header banners) x (3 main
copy) x (3 button colors) x (2 CTAs) =
54 combinations
• Can we get similar information
with fewer tests?
Wednesday, March 12, 14
27. Design of Experiments
Option 2: Orthogonal arrays tests pairs
of combinations instead of all combinations
Wednesday, March 12, 14
28. Design of Experiments
Option 2: Orthogonal arrays tests pairs
of combinations instead of all combinations
• Risk: pairing will hide some combinations,
and the effects that paired variables have on
each other
Wednesday, March 12, 14
29. Design of Experiments
Option 2: Orthogonal arrays tests pairs
of combinations instead of all combinations
• Risk: pairing will hide some combinations,
and the effects that paired variables have on
each other
• Mitigation: pair variables that are unlikely
to influence each other
Wednesday, March 12, 14
30. L9 Array
Compare any pair of variables across all combinations
and you’ll see that they’re all represented!
Wednesday, March 12, 14
31. Design of Experiments
• Let’s say we have four variables:
• Header Banner (A, B, C)
• Main Copy (1, 2, 3)
• Button Color (Cyan, Magenta,Yellow)
• Call to Action (Buy!, Check Out)
Wednesday, March 12, 14
32. Design of Experiments
• Four variables:
• Header Banner (A, B, C)
• Main Copy (1, 2, 3)
• Button Color (Cyan, Magenta,Yellow)
• Call to Action
(Buy, Purchase)
Combo # HB MC BC CTA
1 A 1 Cyan Buy
2 A 2 Magenta Purchase
3 A 3 Yellow
4 B 1 Magenta
5 B 2 Yellow Buy
6 B 3 Cyan Purchase
7 C 1 Yellow Purchase
8 C 2 Cyan
9 C 3 Magenta Buy
Wednesday, March 12, 14
33. Design of Experiments
• Four variables:
• Header Banner (A, B, C)
• Main Copy (1, 2, 3)
• Button Color (Cyan, Magenta,Yellow)
• Call to Action
(Buy, Purchase)
Combo # HB MC BC CTA
1 A 1 Cyan Buy
2 A 2 Magenta Purchase
3 A 3 Yellow Buy
4 B 1 Magenta Purchase
5 B 2 Yellow Buy
6 B 3 Cyan Purchase
7 C 1 Yellow Purchase
8 C 2 Cyan Buy
9 C 3 Magenta Buy
Wednesday, March 12, 14
34. Design of Experiments
• Four variables:
• Header Banner (A, B, C)
• Main Copy (1, 2, 3)
• Button Color (Cyan, Magenta,Yellow)
• Call to Action
(Buy, Purchase)
Combo # HB MC BC CTA
1 A 1 Cyan Buy
2 A 2 Magenta Purchase
3 A 3 Yellow Buy
4 B 1 Magenta Purchase
5 B 2 Yellow Buy
6 B 3 Cyan Purchase
7 C 1 Yellow Purchase
8 C 2 Cyan Buy
9 C 3 Magenta Buy
We’ve reduced need to
collect data on 54
combinations to just 9
(6x efficiency increase)
Wednesday, March 12, 14
35. FROM 54 COMBINATIONS
A1CyanBuy,
A1CyanPurchase,
A1MagentaBuy,
A1MagentaPurchase,
A1YellowBuy,
A1YellowPurchase,
A2CyanBuy,
A2CyanPurchase,
A 2 M a g e n t a B uy,
A 2 M a g e n t a P u rc h a s e ,
A 2 Ye l l ow B uy,
A2YellowPurchase,
A3CyanBuy,
A3CyanPurchase,
A3MagentaBuy,
A3MagentaPurchase,
A3YellowBuy,
A3YellowPurchase,
B1CyanBuy,
B1CyanPurchase,
B1MagentaBuy,
B1MagentaPurchase,
B1YellowBuy,
B1YellowPurchase,
B2CyanBuy,
B2CyanPurchase,
B 2 M a g e n t a B uy,
B 2 M a g e n t a P u rc h a s e ,
B 2 Ye l l ow B uy,
B2YellowPurchase,
B3CyanBuy,
B3CyanPurchase,
B3MagentaBuy,
B3MagentaPurchase,
B3YellowBuy,
B3YellowPurchase,
C1CyanBuy,
C1CyanPurchase,
C1MagentaBuy,
C1MagentaPurchase,
C1YellowBuy,
C1YellowPurchase,
C2CyanBuy,
C2CyanPurchase,
C 2 M a g e n t a B uy,
C 2 M a g e n t a P u rc h a s e ,
C 2 Ye l l ow B uy,
C2YellowPurchase,
C3CyanBuy,
C3CyanPurchase,
C3MagentaBuy,
C3MagentaPurchase,
C3YellowBuy,
C3YellowPurchase
Wednesday, March 12, 14
36. TO JUST 9 (+6X EFFICIENCY)
A1CyanBuy,
A1CyanPurchase,
A1MagentaBuy,
A1MagentaPurchase,
A1YellowBuy,
A1YellowPurchase,
A2CyanBuy,
A2CyanPurchase,
A 2 M a g e n t a B uy,
A 2 M a g e n t a P u rc h a s e ,
A 2 Ye l l ow B uy,
A2YellowPurchase,
A3CyanBuy,
A3CyanPurchase,
A3MagentaBuy,
A3MagentaPurchase,
A3YellowBuy,
A3YellowPurchase,
B1CyanBuy,
B1CyanPurchase,
B1MagentaBuy,
B1MagentaPurchase,
B1YellowBuy,
B1YellowPurchase,
B2CyanBuy,
B2CyanPurchase,
B 2 M a g e n t a B uy,
B 2 M a g e n t a P u rc h a s e ,
B 2 Ye l l ow B uy,
B2YellowPurchase,
B3CyanBuy,
B3CyanPurchase,
B3MagentaBuy,
B3MagentaPurchase,
B3YellowBuy,
B3YellowPurchase,
C1CyanBuy,
C1CyanPurchase,
C1MagentaBuy,
C1MagentaPurchase,
C1YellowBuy,
C1YellowPurchase,
C2CyanBuy,
C2CyanPurchase,
C 2 M a g e n t a B uy,
C 2 M a g e n t a P u rc h a s e ,
C 2 Ye l l ow B uy,
C2YellowPurchase,
C3CyanBuy,
C3CyanPurchase,
C3MagentaBuy,
C3MagentaPurchase,
C3YellowBuy,
C3YellowPurchase
Wednesday, March 12, 14
37. Design of Experiments
• Where do orthogonal arrays come from?
• Derived by hand (like playing Sudoku!)
• Look them up (U Michigan, UYork, Hexawise.com)
Wednesday, March 12, 14
38. Design of Experiments
• Where do orthogonal arrays come from?
• Derived by hand (like playing Sudoku!)
• Look them up (U Michigan, UYork, Hexawise.com)
• How to choose a design?
• Number of variables
• Number of states for each variable
Wednesday, March 12, 14
39. Design of Experiments
• Where do orthogonal arrays come from?
• Derived by hand (like playing Sudoku!)
• Look them up (U Michigan, UYork, Hexawise.com)
• How to choose a design?
• Number of variables
• Number of states for each variable
• How to analyze results?
• Plot data,Analysis ofVariance (ANOVA), binning
Wednesday, March 12, 14
40. Analyzing Results
• Plot data and look at it
• Some things you don’t need statistics to tell you, it’s just there
• Your eye is a pretty good analysis tool
Wednesday, March 12, 14
41. Analyzing Results
• Plot data and look at it
• Some things you don’t need statistics to tell you, it’s just there
• Your eye is a pretty good analysis tool
• Analysis ofVariance (ANOVA)
• One-way ANOVAs to find influence of a one variable on the
result (assume that other variables have minimal influence)
• Two-way ANOVAs to find influence of two variables on
result at once
Wednesday, March 12, 14
42. Analyzing Results
• Plot data and look at it
• Some things you don’t need statistics to tell you, it’s just there
• Your eye is a pretty good analysis tool
• Analysis ofVariance (ANOVA)
• One-way ANOVAs to find influence of a one variable on the
result (assume that other variables have minimal influence)
• Two-way ANOVAs to find influence of two variables on
result at once
• Binning
• Group combinations based on results (high vs. low)
• How many Header Banner A’s have high result? low result?
Wednesday, March 12, 14
43. Analyzing Results
• Plot data and look at it
• Some things you don’t need statistics to tell you, it’s just there
• Your eye is a pretty good analysis tool
• Analysis ofVariance (ANOVA)
• One-way ANOVAs to find influence of a one variable on the
result (assume that other variables have minimal influence)
• Two-way ANOVAs to find influence of two variables on
result at once
• Binning
• Group combinations based on results (high vs. low)
• How many Header Banner A’s have high result? low result?
Takeaway: You can extrapolate data from a subset of combinations
to make a conclusion about a full factorial set
Wednesday, March 12, 14
44. Design of Experiments
• Can get pretty complex, but super efficient!
• L36 array - reducing ~94 million combinations to 36
Wednesday, March 12, 14
45. Comparison of
A/BTesting Platforms
Google Analytics Optimizely Splitforce
Platform
Web / mWeb X X
Platform
Native Mobile X
A/BTesting X X
Experiment
Design
Multivariate X X
Automation X X
Other
In-Browser Editor X X
Other
Consulting X X
Wednesday, March 12, 14
46. In-House vs.Agency
In-House Agency
Pros
Lower initial costs
More control over testing process
Better understanding of business
objectives
No need for internal resources
Faster results as agency provides specialized
expertise
Learn best practices and accelerate internal
competency
Cons
Long time to build expertise from
scratch
Longer time to start achieving great test
results
Higher initial costs
Less understanding of complexities /
nuances of your business
Less control over testing
Wednesday, March 12, 14
47. ThankYou!
For more information:
Zac Aghion, CEO & Co-Founder
zac@splitforce.com
China: (+86)1592-1631-924
USA: (+1)617-750-6684
www.splitforce.com
Wednesday, March 12, 14
