The document discusses optimisation practices for digital marketing strategies. It emphasizes establishing clear processes for testing, including idea mining, sanity checks, test planning, running tests, analyzing results, and sharing outcomes. Key aspects include setting expectations, removing ambiguity through accountability and documentation, and gaining leadership endorsement to develop testing as a regular business practice. The goal is to consistently improve user experience and campaign effectiveness through a strategic, data-driven approach.

2. #digiespresso

3. The Art and Science of Testing – Setting
Scalable Optimisation Practices for Speed
#digiespresso

4. Hi. We’re
Digital Balance.
Our mission is to deliver
valuable outcomes to our clients
through the use of data.

5. DIGITAL ESPRESSO | AUG 2017
We take a strategic approach to data driven decisions.
RINSE &
REPEAT
START
HERE
Digital
Measurement
Strategy
Implementation Integrations Training Governance
Support &
Reporting
Insight Sessions
Testing &
Personalisation
Align your
measurement
objectives to your
goals and
audiences.
An implementation
fit for your
business.
Join the dots by
integrating your
disparate data
silos.
Leverage your
investment long
term.
Consistency of
campaign
effectiveness.
Performance
based, business
focused.
Find the story
behind the
numbers.
Consistently
improve user
experience
• KBO & KPI
planning
• Measurement
framework
• Segmentation,
scoring and
attribution
planning
• Platform choice
and integration
• Data quality
audits
• Custom code
• Classifications
• Dashboards
• Data
visualisations
• X-channel/device
• 3rd party data
integrations
• Tag Management
• Unified Cookies
• Solution Design
Documentation
• Custom
integration
solutions
• API calls
• Data hosting
• Data
management
• Data migrations
• 1st party
integrations
• 3rd party
integrations
• Business training
on analytic
principles
• Platform training
• Trained using
your data
• One on one
mentoring
• C-Level coaching
• Tag governance
• Tracking
methodology
• Agency and team
training
• Tools
• Dashboards
• Dashboards
• Visualisation
platforms
• Segmentation
• Conversions
• Engagement
• Tag Management
• Data
democratisation
• Deep-dive
analysis
• Through the
funnel
• Segmentation
• Planning
workshop
• Customer insights
• Social analysis
• Sales analysis
• Customer
retention
• A/B and
multivariate
testing
• Behavioral
targeting
• Structured
optimisation
programs
• Personalisation
programs

6. 55% of respondents stated that conversion
rate optimisation has become crucial to their
digital marketing strategy. | Econsultancy 2016

7. An act, process, or methodology of making
something (as a design, system, or decision) as fully
perfect, functional, or effective as possible;
specifically : the mathematical procedures (as
finding the maximum of a function) involved in
this.
SOURCE: www.merriam-webster.com/dictionary/optimization
Optimisation:
[op-tuh-muh-zey-shuh n]

8. Optimisation
through Testing
vs.
1.
A
X
Y
Z
show
they did
Optimisation
through Personalisation
2.

9. Optimisation
through Testing
vs.
1.
Our focus

10. A/B testing (sometimes called split testing) is comparing two
versions of a web page to see which one performs better. You
compare two web pages by showing the two variants (let's call
them A and B) to similar visitors at the same time. The one
that gives a better conversion rate, wins.
OR the act of scientifically testing one thing against another.
SOURCE: VWO
a/b test:
[a.bee tɛst n]

11. A Multivariate Test or MVT is a test where we test individual
page elements, instead of whole pages. In Multivariate Testing,
you select different elements of the website page or landing
page on which you want to run the test.
OR scientifically testing different combinations of content
within the same page.
SOURCE: VWO
Multivariate test:
[mʌltɪˈvɛːrɪət/ a]

12. The ‘Utopian’
world of testing.

13. DIGITAL ESPRESSO | AUG 2017
SOURCE:
OPTIMIZELY

14. DIGITAL ESPRESSO | AUG 2017
SOURCE:
OPTIMIZELY

15. DIGITAL ESPRESSO | AUG 2017
SOURCE:
OPTIMIZELY

16. DIGITAL ESPRESSO | AUG 2017
SOURCE:
OPTIMIZELY

17. DIGITAL ESPRESSO | AUG 2017

18. A typical adoption
curve.

20. Ambiguity never leads
to success.
Teams need to set clear expectations to ensure
clarity.

21. Create a culture of accountability to remove ambiguity.
UX, CRO
Responsible Accountable Consulted Informed
✓ Test ideation
✓ Sanity check
✓ Plan
✓ Design
✓ Build
✓ Results
✓ Test ideation ✓ Validation
✓ Analysis
✓ Data
integrations
✓ Test ideation
✓ Input
✓ Approvals
PRODUCT OWNER ANALYST STAKEHOLDERS

22. DIGITAL ESPRESSO | AUG 2017
Test plans leave no room for misinterpretation.

23. Test plans include three core components:
The goal. Visual of what is
being tested.
A hypothesis.

24. Write a clear hypothesis for clarity.
SOURCE: Building your company’s Data DNA

25. Sanity check -
before it costs you.
Measure twice. Cut
once.

26. DIGITAL ESPRESSO | AUG 2017
Is it worth it?
Assess impact for
effort.

27. Statistical significance is the likelihood that if you
ran this test again, you’d get the same results.
It actually ensures mathematically that the change in relationship between two or
more variables is caused by something other than random chance.

28. DIGITAL ESPRESSO | AUG 2017
WINNER?
This looks pretty convincing?
LOSER?

29. DIGITAL ESPRESSO | AUG 2017
Why calling a test can cost you dearly.
WINNER?
LOSER?

30. A/B testing is not
always the best option.
Explore other types of data to prove or disprove a
hypothesis.

31. DIGITAL ESPRESSO | AUG 2017
First, set an A/B
testing criteria.

32. Then, set testing criteria for alternative scenarios.

33. Setting expectations
eases pressure.
No alarms, no surprises.

35. A test here.
Impacts here.

36. A test here.
Impacts here.
Not here.

37. Because there are
too many
influencing factors
in between.
A test here.
Impacts here.
Not here.

38. No leadership endorsement
equals certain death.
Endorsement provides weight to your practice.

39. Get endorsement on roles and responsibilities.
UX, CRO
Responsible Accountable Consulted Informed
✓ Test ideation
✓ Sanity check
✓ Plan
✓ Design
✓ Build
✓ Results
✓ Test ideation ✓ Validation
✓ Analysis
✓ Data
integrations
✓ Test ideation
✓ Input
✓ Approvals
PRODUCT
OWNER
ANALYST STAKEHOLDERS

41. Remove Ambiguity

42. Fundamentals to developing optimisation as a
business as usual practice:

43. Fundamentals to developing optimisation as a
business as usual practice:

44. IDEA MINING
SANITY
CHECK
TEST
PLAN
RUN
TEST
ANALYSE
SHARE
RESULTSBUILDSteps
Strategic framework for testing.
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
Tasks
✓ ___________
Output
❑ _________________
❑ _________________
❑ _________________
❑ _________________
❑ _________________
✓ ___________ ✓ ___________ ✓ ___________ ✓ ___________ ✓ ___________ ✓ ___________
_______________
_______________
Roles _______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________
_______________

45. Thank you.
www.digitalbalance.com.au
Digital Balance
Melbourne +61 3 9020 7340
Perth +61 8 9227 8073
hello@digitalbalance.com.au

46. #digiespresso