1. STAT PACKAGES:
THE SECRET RECIPE TO
GET MORE PROPOSALS
IMPLEMENTED
Harry Sumner | iCrossing
https://www.scribd.com/presentation/56858
6307/Brighton-SEO-Talk-Slideshare
@HarryOReally2
9. There are two stat packages I’d like to
introduce you to…
Prophet & Causal Impact
10. A forecast is only as good as the data
you give it
‘the accuracy of any machine learning model is dependent
on the input information it is given.’
- Oncrawl
11. What you need: What can help do this:
1.
2.
3.
More data in the timeseries (than
less)
Segmented meaningfully
Expectation that results are indicative
vs. certain
Big Query. If not, proxy GA
Categorisation & labelling
Caveats for the unexpected
12. Forecasts are not certain, but they are the best we have
So you have to be clear and caveat that they cannot anticipate the unexpected
1.
2.
3.
Look at the past
Not made for one site
Always variation/Market changes
15. Here is how they work…
Prophet Causal Impact
Additive model of time
Fits nonlinear trends into
daily, weekly, monthly
seasonality
Uses predictions on how
that could change
Bayesian normative model
Infers impact that an
intervention has had on an
outcome metric
Shows extent of incremental
gain
16. Two libraries with distinct benefits in SEO application
Prophet Causal Impact
Handles holiday
effects/seasonality well
Can exclude events to not
skew set
Robust to missing data
= Automatable, less prone to error and scalable
Demonstrates the value of
an intervention/release
Shared as cumulative
growth
Great when a A/B test is not
available
19. Why they work well…
Prophet Causal Impact
Forward facing macro view
Places into scenarios
Great with seasonality
Pinpoint major events to
exclude
Respective view of what
performance could be
Relatively accurate
Interpretable results
Easier than an A/B test
22. Example Business Case
SEO Proposal:
1. Body Copy
2. Footer FAQ
3. Internal Linking
4. Load Speed
Client Challenges:
1. Resource
2. Clunky UX
3. ROI
4. Commercials
34. What does using Prophet & CSI give you?
1 2 3 4
PLACES THIS IN A
MARKET CONTEXT
TO SHOW
COMPETITORS THAT
ARE IMP’ING SEO
UNDERPIN SEO
BUISNESS CASES
WITH A BEFORE
AND AFTER TO
GET FUTURE BUY
IN
SHOWS A SITES
GROWTH POTENTIAL
AVAILABLE BY
TAKING SEO
SERIOUSLY
CAN CLEARLY
SHOWS THE
IMPACT OF A
RELEASE TO BUILD
DEV TEAM BUY-IN
35. What we face:
1.
2.
3.
Lost in translation when presented
Tussle with the dev queues
ROI is always in question
36. What we face: How they can help:
1.
2.
3.
Lost in translation when presented
Tussle with the dev queues
ROI is always in question
Google/Facebook branded
Show what devs have driven
All quantified commercially
Emotion: Grateful. Get moving.
Hope you had lovely lunches
Handover: Excitement and welcome
-----------------
Tips:
- Your body and voice tell the story
If distracted look at the slide
Stop with water if needs be
- Stage = whole body tells the story | Screen: voice and facial expressions are the focus.
- Pauses and vocal effects = happen through the understanding of what you are saying; respecting the end of each thought, conclude one thought before stratign the next one; intention (voice follows your intention - body and mind connected)
- Stage presence = energy between you, audience and your subject. You are the medium between the audience and the subject you are presenting. You GUIDE them through your thinking process with your voice and body.
- It's all about the other person (audience) - always check with them, see if they are still with you, if they are following, if they have questions, etc.
- Be clear on what I'm saying = for the other person to follow you, YOU need to understand and think of what you are saying.
On each section think of:
1 HOW DO YOU FEEL ABOUT THE SUBJECT?
Its cool and will help everyone
2 HOW DO YOU WANT PEOPLE TO FEEL ABOUT IT?
that their hard work can be taken more seriously
4 THOUGHTS/EMOTIONS BEHIND THE WORDS - THINK of what you're saying 5 AND HOW YOUR BODY AND VOICE WILL SHOW/DEMONSTRATE THOSE EMOTIONS AND INTENTIONS?
Questions raised:
What's new to what you are saying?
Putting the two together
Use for all business cases
Do you believe in what you are saying?
That it can help yes
That I use it for all business, no
How are you feeling about your talk?
Confident but not sure on day
How confident are you in the subject you are presenting?
6/10
Who are the people in the audience
Inhouse
Agencies
Tech teams
Who is your target audience?
Everyone that hasn’t had work taken forward
Why is it important they listen to you? - What will you add to their lives? / What will they gain from your talk?
- Strategies: VERBS (towards the other person - to inspire them, to help them, to incentivize them etc.) - BEHAVIOUR (body language, posture, voice)
What's your objective (to you)?
To show I know my shit
What's your objective towards them?
To help everyone with something that may help
- What are your obstacles? What will you do to overcome them?
Emotion: Excited and Happy
Slow down on keys points: Thank you
BSEO Special place
Graduated here
Since a regular
Got me into this beautiful industry
Merges [tech, psychology and testing]
Thankful to be here, special experience
Publicis – UX/ Tech SEO team. Testing/Back end of sites
Dentsu – SEO specialist. Presenting proposals
Sky – Search lead. Getting internal buyin
iCrossing – getting into my data science, coding
Handover: Stopped n thought
Emotion: Thoughtful
Slow down on keys points: of s
Thought of similarities
Handover: Sadness, Frustration
Emotion: Frustration and we all have this in common
Slow down on keys points:
SEO struggles to get recommendations taken further n implemented
Handover: Question
Emotion: understanding aquiantetd
Slow down on keys points:
Proposal > Stakeholder = explain key concepts very clearly
Right?
Stakeholder > Developer = why it needs to be prioritised
Right?
Stakeholder > Snr Management = what going to get in return
Right?
Handover: Disheartened/sad – And I get it, it can be frustrating
Emotion: Dishearten
Slow down on keys points:
Best work doesn’t get taken further
We know it is set to drive performance benefit
Know it will work
According to one study: (86%) of SEOs estimate that organic revenue would grow by at least 50% if they were able to implement outstanding changes.
Handover: I've got something for you
Emotion: Hope
Slow down on keys points:
Hope that this can help
Forecast that can show ahead possible growth opportunity
Forecast that can look back and validate this as accurate
Handover: Appreciate you may be using them – if so great! If not, there are some advantages
Emotion: Advances, going forward
Slow down on keys points:
Either use these = hope in agreement in how they really help
If not (these vs other forecasts)
Big tech branding (Page speed – CWV)
Click curve modelling (can be quite inaccurate in practice)
Great at isolate value of a change
Handover: So here they are
Emotion: Excitement for something that is going to help. Amazing
Slow down on keys points:
2 I want to show you
Prophet – great at looking ahead.
Show what opportunity is possible in implementing SEO
Causal Impact – great at looking back
Give a quantified number of what has been drive by implementing SEO
Handover: But be aware that..
Emotion: Cautious
Slow down on keys points:
But a forecast is only as good as the data you give it
Accuracy based on input
Vice versa, if data not aligned then it’s likely to be inaccurate
Handover: Over to you audience
Emotion: Don’t think its all straightforward – ITS DOWN TO YOU!
Slow down on keys points:
More data than less (means looking beyond SC’s 16 months)
Data then sliced n diced into groupings
Manage expectation that its a forecast
Move data storage into BQ where you ca export these groupings. Sessions Ive also found more precise
Categ KW [b/gen, intent, theme groupings], LPs [User journey into conversion]
Caveat to the flaws of a forecast
Handover: Prepared
Emotion: Be careful
Slow down on keys points:
What these are
Forecasts do not have a crystal ball on the back end
Instead
Look at the past (overlook strat of taking SEO more seriously)
Made using timeseries data aggregated (vs your site)
Variation (cannot map in a competitor tanking)
Handover: Now that’s out the way
Emotion: Problem out the way. Get excited. MAKE IT FUN
Slow down on keys points:
Now caveats out the way, lets look at them in practice
Handover: These are good!
Emotion: Tust. Well established
Slow down on keys points:
P
Article published - 2017
Challenge : need to create reliable and high quality forecasts — especially when there are a variety of time series
Mission : help forecasting at scale
CSI
Research paper - 2015
Challenges: Need to understand the extent different campaigns have contributed to an uplift in performance
Mission : clearly show impact of change
Handover: If your interested
Emotion: Thorough understanding/keeping it simple
Slow down on keys points:
P
Whys this good : Great with seasonality, can exclude to avoid skew, works well even if bits missing
CSI
Hows it work: Monte carlo simulation to say to a 80% degree of certainty this is what the change drove
What this means: Can be very clear on what results could be without an intervention or a change, and what they are now as a result
Whys this good : Maps in the culmative growth, shows the incrementality of SEO change and great when no A/B
Handover: And back into the room
Emotion: Perfect for us
Slow down on keys points:
P
Hows it work: Additive model that uses projections
What this means: time-series modeling as a curve-fitting problem with respect to time
Whys this good : Great with seasonality, can exclude to avoid skew, works well even if bits missing
CSI
Hows it work: Monte carlo simulation to say to a 80% degree of certainty this is what the change drove
What this means: Can be very clear on what results could be without an intervention or a change, and what they are now as a result
Whys this good : Maps in the culmative growth, shows the incrementality of SEO change and great when no A/B
Handover: Shop around
Emotion: Be cautious
Slow down on keys points:
So naturally there are other packages
P:
Lots of alternatives
Populate off timeseries data
alternatives and similar packages in ML
CSI:
- 1 main alternative
Handover: They know their original ones
Emotion: New/impressed
Slow down on keys points:
Newer versions built in response to criticisms of their original versions
Leverage advances in machine learning
NeuralProphet bridges the gap between traditional time-series models and deep learning methods
Causal ML - extracting an understanding of cause and effect with deep learning
Handover: But lets bring it back
Emotion: Focus on the us
Slow down on keys points:
Why they work well…
Prophet =
- forward view of the growth opportunity
Places these in scenarios
Robust to seasonality
Can exclude events to not skew data
Causal Impact
Retro view after a release to clearly show whats been drivem
Accurate
Results are easy to use
Best bet in absence of a AB test
Handover: Let me explain it a bit clearer/look into it
Emotion: Laugh – connect the speech.
Slow down on keys points:
So, lets look at how to use these in practice…
3rd section created a dummy scenario to show how to use them
Handover: Here is something we have all done before – A business case
Emotion: Standard stuff – whats going to be best for them
Slow down on keys points:
Go to a client recommending enhancing their page template
Add more theme rich body copy
Build a FAQ section to target the PAAs
Convert some of this copy into live internal links
All load flagged as green in CWV
Handover: And here they go
Emotion: Prepared
Slow down on keys points:
Client then comes back with some push back
Updating the body copy will require resource
Right?
Adding the FAQs will make the UX clunky
Right?
What are we going to get for this
Right?
And I want this as a ££
Right?
Handover: Taadaa
Emotion: You’ve got this
Slow down on keys points:
Here is where these packages really help a) showing ahead what is possible and b) after how accurate this prediction was
Handover: Confident. Prepared
Emotion: Ready
Slow down on keys points:
Ahead of time, you use prophet to set what growth opportunities are available (placed into scenarios)
And here is how Im going to show you as to how to put in into practice. Using a process success as this.
Handover: Its easy
Do a lot more work on here. And what they get. Work more on voice.
Emotion: Its quite straight forward. POINT TO THE SLIDE. See they are on the same page
Slow down on keys points:
Here is our iCrossing forecasting sheet we use
Pick the metric your time series is based on (Imps, Clicks, Position)
Set how far back the data look
Say how far in advance you want to forecast
Input in the data
End results looks like this
Month on month upper, forecast and lower scenarios
Visualised like this as a plotly graph
Can see the seasonality mapped in here with yellow upper, red lower
Upper ultimately saying x
Handover: Its not finished there
Emotion: This really works. POINT TO SLIDE. Divide focus
Slow down on keys points:
Next step is to pull in a view of competition using a third party tool
Ahrefs is perfect for this populating off their organic traffic estimates
Step by step tutorial and accompanying collab sheet
Very similar to the previous you export this time series
But this time alongside 5 competitors
This then shows the growth rate
Handover: Taadaa
Emotion: Confident
Slow down on keys points:
When placed into your proposal this shows
a) Amount of anticipated additional clicks
(can then pass this through cvr and aov to get to commercials)
b) Forecasted amongst competitive set
(show what competition are getting as a result of implementing SEO changes)
Helps get sign off.
Handover: But we have more
Emotion: Remember that thing
Slow down on keys points:
Now following the release goes live this is where you use causal impact
Handover: Roll it out
Emotion: Yes, Excitement
Slow down on keys points:
Change goes live
Handover: Wait
Emotion: Not yet, but you have something
Slow down on keys points:
Initial impact update
Handover: Wait
Emotion: Getting stable. Hold on
Slow down on keys points:
Mid-way check-in
Handover: Wait
Emotion: Ready
Slow down on keys points:
NOWS THE TIME TO SEE WHAT IT HAS DRIVEN = CSI
Handover: Here it is
Emotion: Remember how it can be easy
Slow down on keys points:
Causal Impact dashboard
Set up (15s)
Start and end date of data you put in
Identifying when the event or release was put live
Pegs the results to this date
Show growth from this intervention
Getting your data as per columns (15s)
Format your data correctly
Make sure dates follow the naming convention
Place into header fields
Must be a csv
Numbers are absolute with no commas
OR IT will break
= RUN
Output (15s)
Show what the response was as a number
In the absence of the release youd have X
That this is significant
Show cases this as % growth
That its not due to any random effect
Perfectly summarised as culmative growth attained
Handover: Now lets have a summary
Emotion: They work
Slow down on keys points:
Growth potential – take it seriously
Market context – show that others are
Impact – get very granular in proving this
Underpin SEO – to make it a priority within your business or for your clients
Handover: But lets go back. Remember? Lightbulb
Emotion: Seen these lots of times
Slow down on keys points:
To go back to the challenges SEOs face in getting things implemented
How recommendations don’t get taken further as business case values are overlooked
SEO recommendations then struggle to get into releases as devs don’t get behind the huge value
Always Qs around what the ROI has been
Handover: Wheyyy
Emotion: BOOM
Slow down on keys points:
Packages help by
Drumming up support by them being Google and Facebook
Can get devs on board by showing them what they have achievd
All able to be calculated into incremental gain
So go check them out, hope these help your best work get implemented
You can find all resources at the end.
Handover: Taaadaa – Im exited for you
Emotion: Thank you
Slow down on keys points:
Thank you so much for listening
Hope this shows;
The challenges we face
How stat packages can be used to super charge your business cases
Prophet – show ahead whats possible
Causal impact – incremental value driven
How to use these end to end
TO GET MORE OF YOUR HARD WORK IMPLEMENTED
Handover: You’ve got this
Emotion: Here you go
Handover: Stay in touch
Slow down on keys points:
