Fail and Win: Why a Failed Test Isn’t a Bad Thing

•

0 likes•881 views

Caleb Whitmore, CEO, Analytics Pros Ryan Lillis, Strategic Optimization Consultant, Optimizely Here's something you don't expect to hear at a CRO conference: most A/B tests don't produce a variation that's better than what you already have. If all you're doing is running an A/B test, viewing select metrics, and giving a "thumbs up" or "thumbs down," you won't have a successful optimization program — even if you happen upon a few "winners." But you don't have to run your optimization program this way. A/B testing done right allows you to draw winning insights from "losing" tests that have the power to genuinely affect your business. Caleb Whitmore, Founder and CEO of Analytics Pros, shows that you can achieve a 360-degree view of data that leverages your analytics engine as well as your testing platform to drive deep and genuine insights about the effects of your tests. You'll learn a holistic approach to testing that goes way beyond "winners" and "losers."

Marketing Technology Business

Fail
and
Win:

Why
a
Failed
Test
Isn’

Caleb
Whitmore

Founder
&
CEO

Analy<cs
Pros

@CalebWhitmore

And
get
to
work
with
some

awesome
clients!

Why
Test?

TESTING…

•  Should
focus
on
improving
an
outcome

•  Is
the
shortest
distance
between
you
and

improvements

•  Is
done
best
with
rapid
itera<on
and
lots
of
data

Tes=ng
is
a
Success
When:

•  A
process
is
followed

•  An
outcome
is
measured

•  A
complete
picture
is
seen

Crea=ng
a
360
Degree
View

•  A
test
will
impact
more
than
what
you
intended

•  You
must
measure
the
impact
across
your

spectrum
of
user
interac<on

•  Measure
your
test
with
your
exis<ng
infrastructure

Crea=ng
a
360
Degree
View

What
happens
to…

•  The
“key”
conversion
point

•  Engagement
on
the
test
page

•  Length
of
the
en<re
visit

•  Total
pageviews/interac<ons
in
the
visit

•  Secondary
conversion
goals
–
view
product,
view

content,
start
checkout,
download
a
ﬁle,
etc…

Example
Experiment

ASSUMPTIONS

•  Auto-‐play
repels
visitors

– Audio
without
warning

•  Thus,
bounce
rates
are
high
and
more
people
leave

•  Turn
oﬀ
auto
play
&
more
people
will
stay
and

engage

Setup
a
360
Degree
View
of
Tests

•  Tip:
your
Digital
Analy<cs
plaXorm
has
this

view
already!

Capture
Experiment
Data

•  Tie
experiment
data
into

your
Tag
Management

and
Digital
Data

frameworks

Leverage
Built-‐in
Repor=ng

•  Create
lots
of
Goals
in
Op<mizely
–
pages,

clicks,
revenue,
etc…

QUESTIONS?

@CalebWhitmore
of

@Analy<csPros

Fail
and
Win:

Why
a
Failed
Test
Isn’t
a
Bad
Thing

Caleb

Whitmore

Founder
&
CEO

Analy<cs
Pros

@CalebWhitmore

The web is full of best practices and results of AB-tests. Can’t you just copy those, lie back and watch your conversion rates go up? Bad idea. Best practices don’t always work. So, what to do instead? Just test every idea that pops into your head? No again! That will result in bitter disappointment because your tests will fail over and over again. The key is to get to know your visitors before you start testing. Research their behavior on your website. Learn how to create AB-tests that really make a difference thanks to user research. So you can focus on what your visitors want: their needs, their priorities and their actions.

AME Conference - Best Practice L-3 Wescam - Quick Hits An Engagement StrategyErin Swindall, PMP, LSSBB

10 Tactics for Building an Optimization Culture

Optimizely

Optimizing the customer-journey - Website AB-testing cases

AGConsult

Lean Startup Experimentation, outputs to outcomes, short version.

Andrew Rusling

Outcomes such as “subscriptions increased by 20%” or “complaints regarding the upload feature reduced to zero” are what makes a real difference in our customers lives and hence to the company’s bottom line. When a team is delivering outcomes like that, there is no denying their performance and hence their value to the company. Delivering outcomes comes from understanding our customers, producing an output that may result in an outcome and then validating if we have achieved the desired outcome. At the very least one of these cycles produces knowledge. The Lean Start-up by Eric Ries, clearly explained this cycle unfortunately it did not explain clearly how we should design, set up, run or analyse our experiments. I have met many people who agree we should follow the Lean Start-up approach; however, there is rarely any consensus on the experimentation approach that will make it a reality. In 2017 Australia’s largest independent game studio, Halfbrick Studios, embarked upon a mission to better understand their customer and experiment their way to renewed success. Fruit Ninja Fight is one of the results of that approach. In 2018 Australia’s largest Telco, Telstra, focused on “co-creation” with their customers through a series of experiments; delivering improved customer satisfaction and faster results than ever before. This presentation shares with you my experiences of working with those two based organisations as they sought to improve their outcomes through Experimentation.

Ideas are great - no doubt - but what do you do once you have an entire backlog of ideas? Prioritization is a critical part when building an efficient, impactful testing program. Creating a framework to make smarter choices and thinking deeply about the key factors will make your program successful in the long term. In this session we will talk about how to create a strong prioritization process, how to keep it running, and how to constantly benchmark and optimize your process.

Building High-Growth Products with Jobs-to-be-Done

Product School

Jared Ranere, a Growth Partner at thrv, talked about how Jobs-to-be-Done answers questions like "what is a customer need?" "Who is your real competition?" and "What does it mean to be "satisfied?" in a way that aligns your development team around your customer. It also gives you precise, measurable customer needs, and helps you know if your feature idea will deliver customer satisfaction before your team writes a line of code.

Master the essentials of conversion optimization

Arnas Rackauskas

Agile Impact 2018: Feature Experimentation

Thomas Rothe

Experimentation, as the gold standard to measure new product initiatives, has become an indispensable component of product development cycles in the online world. The ability to automatically collect user interaction data online has given companies an unprecedented opportunity to run many experiments at the same time, allowing them to iterate rapidly, fail fast, and deliver the highest user value. In this talk, i spoke about the approach of experimentation and how it fits into the development process of products. http://agileimpact.id/ #AICON18

Conversion Rate Optimization: Beat Competitiors and Get Paid

Affiliate Summit

Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web A...

Online Dialogue

Knowledge is Power: Visualizing JIRA's Performance Data

Atlassian

Keith hopper-product-market-fit

hopperomatic

Everything You Need to Know for a Better Website in 2014 | Crystal Olig | Oxi...

Lessing-Flynn

Павло Камишов “Health check model: refined edition” Lviv Project Management Day

Lviv Startup Club

Opticon 2015-Personalization Using Optimizely Audience Integrations

Optimizely

Featuring speakers from Microsoft Store, Citrix, Access Intelligence Trevin Chow, Director of Product Management, Microsoft Store Ravi Venkatesan, Conversion Analyst, Citrix Stacy Hill, Analytics Manager, Access Intelligence Moderated by: Clarissa Bukhan, Technology Partner Manager, Optimizely This session will highlight how Optimizely's Audience integrations allow you to deliver personalized experiences to your visitors. We'll hear from a panel of Optimizely customers about how they've used Optimizely’s integrations with first and third-party data platforms, such as BlueKai, Demandbase, and Lytics, to create custom audiences for targeted testing and personalization.

Team health check new

Kateryna Sapnova

Павел рассказал о методологии Spotify Squad Health Check Model и как он использует ее в EPAM. ✔Сначала мы решили понять какие проблемы нас окружают и анализировать их. ✔Вам, как проектному менеджеру нужно оправдать доверие сотрудников, поэтому не обязательно проводить интервью с сотрудниками, можно просто за чаем или в курилке спросить их мнение. ✔Мы получили полную картину о том, что происходит на проекте, тренды, многие негативные тренды мы убрали и ключевые моменты улучшили.

Minimum Delightful Product

ProductCampMelbourne

From SCRUM to LEAN

Vlad Mysla

Experimentation through Clients' Eyes

Optimizely

Aatif Awan, Head of Growth LinkedIn - Growth Hacking is Dead. Long Live Growth.

Traction Conf

How we built Talentpioneer by Productsquads

Productsquads

Why good european companies fail in naTimothy Brown

How To Fail: 25 Secrets Learned through Failure

Taylor Davidson

What's hot

Turing Festival 2016 - Building Products, More Than Meets the Eye

John Peebles

Don't do anything on your website you wouldn't do on a first date

AGConsult

[CXL Live 16] You Can’t Make This Stuff Up by Alex Harris

CXL

[CXL Live 16] What to Test Next - Prioritizing Your Tests by Pauline Marol

CXL

Building High-Growth Products with Jobs-to-be-Done

Product School

Master the essentials of conversion optimization

Arnas Rackauskas

Agile Impact 2018: Feature Experimentation

Thomas Rothe

Conversion Rate Optimization: Beat Competitiors and Get Paid

Affiliate Summit

Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web A...

Online Dialogue

Knowledge is Power: Visualizing JIRA's Performance Data

Atlassian

Keith hopper-product-market-fit

hopperomatic

Everything You Need to Know for a Better Website in 2014 | Crystal Olig | Oxi...

Lessing-Flynn

Павло Камишов “Health check model: refined edition” Lviv Project Management Day

Lviv Startup Club

Opticon 2015-Personalization Using Optimizely Audience Integrations

Optimizely

Team health check new

Kateryna Sapnova

Minimum Delightful Product

ProductCampMelbourne

From SCRUM to LEAN

Vlad Mysla

Experimentation through Clients' Eyes

Optimizely

Aatif Awan, Head of Growth LinkedIn - Growth Hacking is Dead. Long Live Growth.

Traction Conf

How we built Talentpioneer by Productsquads

Productsquads

What's hot (20)

Turing Festival 2016 - Building Products, More Than Meets the Eye

Don't do anything on your website you wouldn't do on a first date

[CXL Live 16] You Can’t Make This Stuff Up by Alex Harris

[CXL Live 16] What to Test Next - Prioritizing Your Tests by Pauline Marol

Building High-Growth Products with Jobs-to-be-Done

Master the essentials of conversion optimization

Agile Impact 2018: Feature Experimentation

Conversion Rate Optimization: Beat Competitiors and Get Paid

Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web A...

Knowledge is Power: Visualizing JIRA's Performance Data

Keith hopper-product-market-fit

Everything You Need to Know for a Better Website in 2014 | Crystal Olig | Oxi...

Павло Камишов “Health check model: refined edition” Lviv Project Management Day

Opticon 2015-Personalization Using Optimizely Audience Integrations

Team health check new

Minimum Delightful Product

From SCRUM to LEAN

Experimentation through Clients' Eyes

Aatif Awan, Head of Growth LinkedIn - Growth Hacking is Dead. Long Live Growth.

How we built Talentpioneer by Productsquads

Viewers also liked

Why good european companies fail in naTimothy Brown

How To Fail: 25 Secrets Learned through Failure

Taylor Davidson

Startup Marketing: A Systems Approach

April Dunford

Why do great companies fail diagram

http://www.drawpack.com

Why Good Companies Go BadJoe Bracone

Why good companies go bad

Dayanzi Tamang

What is the Theory of Disruption?

Christian Schultz

This presentation is on the theory of disruption by Clayton Christensen. While it is easy to explain why bad companies fail it is hard to understand how good companies, who are supposedly do everything right, fail in the end. The theory of disruption offers different ways of thinking to capture this phenomenon. I explain the theoretical foundations and present a popular example of a new market disruption (the iphone). In the end the different key points of disruption are summarized.

Disruptive Technology - Why large companies often fail to innovate

Norman Hiob

Ecommerce final pptreemalmarri

Viewers also liked (9)

Why good european companies fail in na

How To Fail: 25 Secrets Learned through Failure

Startup Marketing: A Systems Approach

Why do great companies fail diagram

Why Good Companies Go Bad

Why good companies go bad

What is the Theory of Disruption?

Disruptive Technology - Why large companies often fail to innovate

Ecommerce final ppt

Similar to Fail and Win: Why a Failed Test Isn’t a Bad Thing

Myths, Lies and Illusions of AB and Split Testing

Craig Sullivan

Patrick McKenzie Opticon 2014: Advanced A/B Testing

Patrick McKenzie

iSG Webinar – AB Testing: The most important thing you’re NOT doing

Bridgeline Digital

AB Testing requires a considerable amount of time, work, and planning. But as with most ventures that involve those components, the valuable results and insight gained from AB Testing more than payoff the initial investment. Watch a recording of a webinar with JR Hopwood from our iAPPS Success Group (iSG) to learn about the process of AB Testing game and how to convince others in your organization to AB Test as well.

Master Conversion Optimization

Austin Walker

Triple Your Ecommerce Sales & Leads - CROday & CRO Case Study

Alex Harris

How Truly Effective CRO Requires Great UX

Giles Thomas

#Measurefest : 20 Simple Ways to Fuck Up your AB tests

Craig Sullivan

The Finishing Line

Oban International

Webinar: Common Mistakes in A/B Testing

Optimizely

This webinar aims to show you how to avoid the common pitfalls that can affect your conversion rate optimization strategy. At the end of this webinar you will be able to: 1) Use A/B Testing best practices to get the most out of it 2) Identify the common mistakes preemptively 3) Appreciate how A/B Testing fits your whole conversion rate optimization strategy (based on real cases, such as Swedoffice and Spotify) hosted by Simon Dahla from Conversionista Sweden and Martijn Janssen, Partner Manager Optimizely

How to Effectively Experiment in PM by LendingTree Sr PM

Product School

#Measurecamp : 18 Simple Ways to F*** up Your AB Testing

Craig Sullivan

"How to measure your website effectiveness and get on the right track with we...Markedu - Innovative Marketing Education

Losing is the New Winning

Optimizely

Presentations - Zarget CRO meetup 2017

ZargetHQ

Optimize Your Funnel By Getting Inside Your Buyer's Head

David Skok

Ten Powerful Tips To Get More From Your Analytics

Michele Kiss

Today's marketers are not only expected to be proficient in marketing and communication but, with increasing reliance on data, be fluent in analytics as well. And even with multiple tools designed to simplify the process, many still struggle to prove the success of their efforts. Frustrations range from not knowing where to start to a deluge of data that provides no real insight. In this presentation, Michele will provide you with 10 tips and actions you can implement immediately to do more with your data—no expensive tools required! Whether you have an analytics team or you have no support at all, we'll show you how to get more from your data and analysts so you can stop struggling and start gaining valuable insight.

Lean marketing master

Brett Noyes

Conversion Rate Optimisation: The Science Behind Turning Visitors Into Customers

Web Marketing ROI

How to Successfully Run Your First Website A/B Test

Kissmetrics on SlideShare

What is A/B testing? A SCIENTIFIC METHOD OF PROOF WHAT IS A/B TESTING FOR MARKETING Where do you start testing? THE DON’TS WHERE DO YOU START TESTING? DON’T JUST GUESS SUB OPTIMIZE • Button colors • Small copy changes • Small layout changes • Fonts, font size WATCH OUT FOR BUZZ • Don’t just test what the gurus say • What works for others may not work for you PREMATURE TESTING • Testing without traffic • Before you know what and why you are testing Where do you start testing? THE DOS DO YOUR HOMEWORK You have to put in the work to get great results • Learn as much as you can about A/B testing • Study the great people out there • What’s their process and insights? A framework for success The basis of a successful testing program Research Know what to test and see the greatest improvements DIVE INTO YOUR DATA FIND YOUR BIGGEST OPPORTUNITIES INVESTIGATE THE OPPORTUNITY TO UNDERSTAND IT FORM A HYPOTHESIS By doing A, B will happen, Because of C BUILD THE TEST LAUNCH THE TEST VS. Control Variant THE TEST VS. +24.01% increase in registrations!! 100% Significance DOCUMENT AND SHARE YOUR FINDINGS Documenting all your tests and results • Increases company transparency • Helps others know where tests are going on • Makes it easier to call tests • Makes learnings available for everyone in company REPEAT THE PROCESS AS OFTEN AS POSSIBLE Control vs. Variant “Only 1 out of 8 A/B tests have driven significant change.” Noah Kagan AppSumo* “The goal of a test is to get a learning, not a lift. With enough learnings, you can get the real lift,” Dr. Flint McGlaughlin MECLABS RULES OF THUMB FOR TESTING Things to remember when we are testing • Min. 2000 visitors • Min. a few hundred conversions • At least one full cycle > 2 week • >99% statistical significance • Keep a backlog of test hypothesis • Build next tests as the current one is running *More info at http://kiss.ly/1vF4wUc and http://kiss.ly/1Q4vTm0 Tooltips AND BURGERS How I like my testing burger Tooltips for your testing framework Tool Company Why do I like it? Analytics Kissmetrics Shows me my actual customers and their behavior Surveys Qualaroo and/or SurveyMonkey Easy to use/Deep qualitative insights Project Management Trello Manage my time + communicate with collaborators Tool Company Why do I like it? Company Wiki Confluence Document initial thoughts for everyone Testing platform* Optimizely, Pardot, Adwords Easily build tests and get data Analytics Kissmetrics Let’s me confirm and verify down my funnel Company Wiki Confluence Documentation and learnings for everyone *Includes actual testing platform and tools that have testing capabilities Not everyone likes their burger the same way How do you take yours? Trust the process Share your leanings Stack your results A/B TESTING FOR SUCCESS

It Worked for Ustream

Balázs Kereskényi

Similar to Fail and Win: Why a Failed Test Isn’t a Bad Thing (20)

Myths, Lies and Illusions of AB and Split Testing

Patrick McKenzie Opticon 2014: Advanced A/B Testing

iSG Webinar – AB Testing: The most important thing you’re NOT doing

Master Conversion Optimization

Triple Your Ecommerce Sales & Leads - CROday & CRO Case Study

How Truly Effective CRO Requires Great UX

#Measurefest : 20 Simple Ways to Fuck Up your AB tests

The Finishing Line

Webinar: Common Mistakes in A/B Testing

How to Effectively Experiment in PM by LendingTree Sr PM

#Measurecamp : 18 Simple Ways to F*** up Your AB Testing

"How to measure your website effectiveness and get on the right track with we...

Losing is the New Winning

Presentations - Zarget CRO meetup 2017

Optimize Your Funnel By Getting Inside Your Buyer's Head

Ten Powerful Tips To Get More From Your Analytics

Lean marketing master

Conversion Rate Optimisation: The Science Behind Turning Visitors Into Customers

How to Successfully Run Your First Website A/B Test

It Worked for Ustream

More from Optimizely

Clover Rings Up Digital Growth to Drive Experimentation

Optimizely

Clover's Digital Growth team is responsible for optimizing the merchant's digital experience and they rely on experimentation to guide digital decision-making. This enables them to quickly learn and measure what changes deliver the best outcomes for users. Join us with Lead Product Manager of Growth, Monil Shah, to learn how Clover: - Increased digital conversions amongst merchants with an investment in experimentation - Grew experiment velocity by 4x after replacing Adobe Target - Designed a framework to efficiently capture and prioritize test ideas, and roll out winners

Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...

Optimizely

The Science of Getting Testing Right

Optimizely

Atlassian's Mystique CLI, Minimizing the Experiment Development Cycle

Optimizely

Mystique CLI is an Atlassian developed CLI for Optimizely Web. It is a multi-phase project that is currently focusing on improving the development cycle for growth engineers. Currently, Mystique is the standard for developing web experiments at Atlassian, and is capable of a wide variety of operations utilizing Optimizely's REST API. This includes creating, updating, testing, and duplicating experiments/personalization campaigns, as well as "promoting" these entities between Optimizely projects for different environments (e.g. from QA => Prod). It has significantly reduced manual overhead and decreased development time by up to 95% for particular actions.

Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...

Optimizely

Autotrader's Product and Engineering teams were ahead of the curve many years ago when they built a home-grown solution for leveraging feature flags to support server-side testing. Over the years, the industry eventually caught up and surpassed this proprietary tooling and the team had a choice to make: Re-invest into the local solution or completely retool. In this case study, Scott Povlot, Principal Technical Architect, and Seth Stuck, Director of R&D Analytics, will discuss their journey in selecting and then migrating to their next generation of experimentation tooling. They will discuss selection criteria, pros and cons, and outline how they were able to make the migration to Optimizely successful and lessons learned along the way.

Zillow + Optimizely: Building the Bridge to $20 Billion Revenue

Optimizely

The Future of Optimizely for Technical Teams

Optimizely

Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...

Optimizely

Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...

Optimizely

Building an Experiment Pipeline for GitHub’s New Free Team Offering

Optimizely

AMC Networks Experiments Faster on the Server Side

Optimizely

Speeding up innovation only matters if it helps you drive positive outcomes. At AMC, experimentation enables the product and platform teams to challenge their assumptions, maximize impact, and evaluate ideas as painted door tests before investing in significant development. A commitment to test everything across 9 platforms fueled their search for the most scalable solution. In this session, you'll learn how to: Leverage server-side testing to experiment quickly Scale across web, mobile, and OTT applications Determine when client-side testing is more efficient

Evolving Experimentation from CRO to Product Development

Optimizely

An obsession with data, efficiency, and delivering incredible customer experiences are just a few things that the CNN Consumer Science and Software Engineering teams have in common. Simple A/B testing practices evolved into a culture of experimentation, sparking new development practices across the organization. Learn how they drive results across their entire platform from websites to mobile apps.

Overcoming the Challenges of Experimentation on a Service Oriented Architecture

Optimizely

Growing from an early stage startup to a national leader in financial literacy is no small feat, and there are a ton of lessons that we have learned at Greenlight as we have grown. Long gone are the days where we would ship something and cross our fingers hoping that it makes some kind of impact on our customers. Now we’re in a world where we can learn ahead of time how much impact a feature will have on the business, before we even launch! In today’s conversation, we’ll discuss how we use Optimizely’s feature flags in our microservice architecture using Optimizely Agent while keeping user IDs and context synchronized. This session will cover: How we set up Optimizely Agent and use it in a kubernetes deployment How we created a user-aliasing service How we access Optimizely both on the frontend and in the backend services. How to build a full stack feature How to manage the rollout using Optimizely’s feature flags

How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...

Optimizely

A/B testing is an essential element in any product managers playbook. However having the freedom and flexibility to customize testing based on what the data is saying often requires a lot of time and effort, particularly when it comes to engineering resources. Optimizely offers a flexible approach to experimentation through the use of feature testing, which provides more customization options without the additional development effort typically required to implement these feature optimizations. Megan Bubley, a Senior Product Manager at The Zebra, will share her experience working with Optimizely’s feature tests to create a results page where users can compare multiple auto insurance options driven by actual user needs, as well as her experience customizing the experience based on device platform.

Making Your Hypothesis Work Harder to Inform Future Product Strategy

Optimizely

At Treatwell, each experiment goes beyond improving a single business metric. Experimentation works to evolve their product while enriching customer insights in order to deliver the best digital experience to their users. Join Laura Howard, Lead Product Manager, and Dennis Meisner, Senior Product Analyst, to learn their secret to making their hypothesis work harder and how getting their hypothesis right has improved Treatwell’s funnel progression and order health, as well as helped them make critical decisions on their product experience.

Kick Your Assumptions: How Scholl's Test-Everything Culture Drives Revenue

Optimizely

Amy Vetter, Consumer Experience Manager, Direct To Consumer, Europe, will walk you through some of the tests that she and her team run across the Scholl brand. Amy will highlight surprise learnings and how to remove the fear of failing. The team is empowered to test everything possible that will allow the customer to get the best experience and also support the brand’s goal for more revenue and customer data.

Shipping to Learn and Accelerate Growth with GitHub

Optimizely

Will 2020 mark the shift to a remote-first world in the long run? For GitHub, a distributed workforce is nothing new. Join Sha Ma, VP of Engineering, and Gregory Ceccarelli, Director of Data Science, to learn how they built and scaled a successful experimentation program. They'll share their experience implementing Optimizely across timezones, a remote workforce, and a new business model. In this session, you'll learn how to: Optimize UX for a freemium business model Use data to deliver customer-centered products Scale experimentation and accelerate growth

Test Everything: TrustRadius Delivers Customer Value with Experimentation

Optimizely

When done right, experimentation can help you validate the product you’re building and create winning customer experiences. And it doesn’t take a big engineering team to make this happen. TrustRadius, the most trusted review site for business technology, uses experimentation to build an online community through website and server-side experimentation. The small but mighty TrustRadius team runs experiments throughout the buyer’s journey to engage different user personas and understand outcomes in real-time. Watch the webinar recording featuring Rilo Stark, product manager at TrustRadius, and Jack Peden, senior software engineer, to understand their data-driven experimentation strategy and how TrustRadius uses Optimizely Web and Full Stack products to tailor experiences to different customer segments and mitigate risk through A/B/N and painted door tests.

Optimizely Agent: Scaling Resilient Feature Delivery

Optimizely

The Future of Software Development

Optimizely

Newly appointed Optimizely CTO, Lawrence Bruhmuller, will kick off Developer Summit discussing the new normals in software development. After decades of leading and scaling engineering teams for high growth startups and large tech companies, Lawrence has seen the same problems crop up repeatedly for technical teams. There is a new way of delivering software that makes it possible to move fast and get it right. That new way is Progressive Delivery & Experimentation. When Progressive Delivery & Experimentation are used together, you have an efficient system for validating both quality and customer engagement across your development lifecycle. Lawrence will discuss the key principles driving software development innovation, how our engineering team puts this into practice, and the success he’s seen at other companies.

More from Optimizely (20)

Clover Rings Up Digital Growth to Drive Experimentation

Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...

The Science of Getting Testing Right

Atlassian's Mystique CLI, Minimizing the Experiment Development Cycle

Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...

Zillow + Optimizely: Building the Bridge to $20 Billion Revenue

The Future of Optimizely for Technical Teams

Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...

Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...

Building an Experiment Pipeline for GitHub’s New Free Team Offering

AMC Networks Experiments Faster on the Server Side

Evolving Experimentation from CRO to Product Development

Overcoming the Challenges of Experimentation on a Service Oriented Architecture

How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...

Making Your Hypothesis Work Harder to Inform Future Product Strategy

Kick Your Assumptions: How Scholl's Test-Everything Culture Drives Revenue

Shipping to Learn and Accelerate Growth with GitHub

Test Everything: TrustRadius Delivers Customer Value with Experimentation

Optimizely Agent: Scaling Resilient Feature Delivery

The Future of Software Development

Recently uploaded

Core Web Vitals SEO Workshop - improve your performance [pdf]

Peter Mead

Is AI-Generated Content the Future of Content Creation?

Cut-the-SaaS

Discover the transformative power of AI in content creation with our presentation, "Is AI-Generated Content the Future of Content Creation?" by Puran Parsani, CEO & Editor of Cut-The-SaaS. Learn how AI-generated content is revolutionizing marketing, publishing, education, healthcare, and finance by offering unprecedented efficiency, creativity, and scalability. Understanding AI-Generated Content: AI-generated content includes text, images, videos, and audio produced by AI without direct human involvement. This technology leverages large datasets to create contextually relevant and coherent material, streamlining content production. Key Benefits: Content Creation: Rapidly generate high-quality content for blogs, articles, and social media. Brainstorming: AI simulates conversations to inspire creative ideas. Research Assistance: Efficiently summarize and research information. Market Insights: The content marketing industry is projected to grow to $17.6 billion by 2032, with AI-generated content expected to dominate over 55% of the market. Case Study: CNET’s AI Content Controversy: CNET’s use of AI for news articles led to public scrutiny due to factual inaccuracies, highlighting the need for transparency and human oversight. Benefits Across Industries: Marketing: Personalize content at scale and optimize engagement with predictive analytics. Publishing: Automate content creation for faster publication cycles. Education: Efficiently generate educational materials. Healthcare: Create accurate content for patients and professionals. Finance: Produce timely financial content for decision-making. Challenges and Ethical Considerations: Transparency: Disclose AI use to maintain trust. Bias: Address potential AI biases with diverse datasets. SEO: Ensure AI content meets SEO standards. Quality: Maintain high standards to prevent misinformation. Conclusion: AI-generated content offers significant benefits in efficiency, personalization, and scalability. However, ethical considerations and quality assurance are crucial for responsible use. Explore the future of content creation with us and see how AI is transforming various industries. Connect with Us: Follow Cut-The-SaaS on LinkedIn, Instagram, YouTube, Twitter, and Medium. Visit cut-the-saas.com for more insights and resources.

Your Path to Profits - The Game-Changing Power of a Marketing - Daniel Bussius

DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions

Most small businesses struggle to see marketing results. In this session, we will eliminate any confusion about what to do next, solving your marketing problems so your business can thrive. You’ll learn how to create a foundational marketing OS (operating system) based on neuroscience and backed by real-world results. You’ll be taught how to develop deep customer connections, and how to have your CRM dynamically segment and sell at any stage in the customer’s journey. By the end of the session, you’ll remove confusion and chaos and replace it with clarity and confidence for long-term marketing success. Key Takeaways: • Uncover the power of a foundational marketing system that dynamically communicates with prospects and customers on autopilot. • Harness neuroscience and Tribal Alignment to transform your communication strategies, turning potential clients into fans and those fans into loyal customers. • Discover the art of automated segmentation, pinpointing your most lucrative customers and identifying the optimal moments for successful conversions. • Streamline your business with a content production plan that eliminates guesswork, wasted time, and money.

De-risk Your Digital Evolution - Hannah Grap

DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions

It's another new era of digital and marketers are faced with making big bets on their digital strategy. If you are looking at modernizing your tech stack to support your digital evolution, there are a few can't miss (often overlooked) areas that should be part of every conversation. We'll cover setting your vision, avoiding siloes, adding a democratized approach to data strategy, localization, creating critical governance requirements and more. Attendees will walk away with actions they can take into initiatives they are running today and consider for the future.

How to Run Landing Page Tests On and Off Paid Social Platforms

VWO

Join us for an exclusive webinar featuring Mariate, Alexandra and Nima where we will unveil a comprehensive blueprint for crafting a successful paid media strategy focused on landing page testing.With escalating costs in paid advertising, understanding how to maximize each visitor’s experience is crucial for retention and conversion. This session will dive into the methodologies for executing and analyzing landing page tests within paid social channels, offering a blend of theoretical knowledge and practical insights. The Pearmill team will guide you through the nuances of setting up and managing landing page experiments on paid social platforms. You will learn about the critical rules to follow, the structure of effective tests, optimal conversion duration and budget allocation. The session will also cover data analysis techniques and criteria for graduating landing pages. In the second part of the webinar, Pearmill will explore the use of A/B testing platforms. Discover common pitfalls to avoid in A/B testing and gain insights into analyzing A/B tests results effectively.

Metaverse Marketing in the Generation of the Internet - Eugene Capon

DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions

BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin

BalmerLawrie

Playlist and Paint Event with Sony Music U

SemajahParker

Traditional Store Audits are Outdated: A New Approach to Protecting Your Bran...

Auxis Consulting & Outsourcing

How to use Short Form Video To Grow Your Brand and Business - Keenya Kelly

DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions

May 2024 - VBOUT Partners Meeting Group Session

Vbout.com

Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin

DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions

When most people in the industry talk about online or digital reputation management, what they're really saying is Google search and PPC. And it's usually reactive, left dealing with the aftermath of negative information published somewhere online. That's outdated. It leaves executives, organizations and other high-profile individuals at a high risk of a digital reputation attack that spans channels and tactics. But the tools needed to safeguard against an attack are more cybersecurity-oriented than most marketing and communications professionals can manage. Business leaders Leaders grasp the importance; 83% of executives place reputation in their top five areas of risk, yet only 23% are confident in their ability to address it. To succeed in 2024 and beyond, you need to turn online reputation on its axis and think like an attacker. Key Takeaways: - New framework for examining and safeguarding an online reputation - Tools and techniques to keep you a step ahead - Practical examples that demonstrate when to act, how to act and how to recover

Digital Commerce Lecture for Advanced Digital & Social Media Strategy at UCLA...

Valters Lauzums

E-commerce in 2024 is characterized by a dynamic blend of opportunities and significant challenges. Supply chain disruptions and inventory shortages are critical issues, leading to increased shipping delays and rising costs, which impact timely delivery and squeeze profit margins. Efficient logistics management is essential, yet it is often hampered by these external factors. Payment processing, while needing to ensure security and user convenience, grapples with preventing fraud and integrating diverse payment methods, adding another layer of complexity. Furthermore, fulfillment operations require a streamlined approach to handle volume spikes and maintain accuracy in order picking, packing, and shipping, all while meeting customers' heightened expectations for faster delivery times. Amid these operational challenges, customer data has emerged as an important strategy. By focusing on personalization and enhancing customer experience from historical behavior, businesses can deliver improved website and brand experienced, better product recommendations, optimal promotions, and content to meet individual preferences. Better data analytics can also help in effectively creating marketing campaigns, improving customer retention, and driving product development and inventory management. Innovative formats such as social commerce and live shopping are beginning to impact the digital commerce landscape, offering new ways to engage with customers and drive sales, and may provide opportunity for brands that have been priced out or seen a downturn with post-pandemic shopping behavior. Social commerce integrates shopping experiences directly into social media platforms, tapping into the massive user bases of these networks to increase reach and engagement. Live shopping, on the other hand, combines entertainment and real-time interaction, providing a dynamic platform for showcasing products and encouraging immediate purchases. These innovations not only enhance customer engagement but also provide valuable data for businesses to refine their strategies and deliver superior shopping experiences. The e-commerce sector is evolving rapidly, and businesses that effectively manage operational challenges and implement innovative strategies are best positioned for long-term success.

Email Marketing Master Class - Chris Ferris

DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions

The Forgotten Secret Weapon of Digital Marketing: Email Digital marketing is a rapidly changing, ever evolving industry--Influencers, Threads, X, AI, etc. But one of the most effective digital marketing tools is also one of the oldest: Email. Find out from two Houston-based digital experts how to maximize your results from email. Key Takeaways: Email has the best ROI of any digital tactic It can be used at any stage of the customer journey It is increasingly important as the cookie-less future gets closer and closer

How to Use AI to Write a High-Quality Article that Ranks

minatamang0021

In the world of content creation, many AI bloggers have drifted away from their original vision, resulting in low-quality articles that search engines overlook. Don't let that happen to you! Join us to discover how to leverage AI tools effectively to craft high-quality content that not only captures your audience's attention but also ranks well on search engines. Disclaimer: Some of the prompts mentioned here are the examples of Matt Diggity. Please use it as reference and make your own custom prompts.

Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin

DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions

When most people in the industry talk about online or digital reputation management, what they're really saying is Google search and PPC. And it's usually reactive, left dealing with the aftermath of negative information published somewhere online. That's outdated. It leaves executives, organizations and other high-profile individuals at a high risk of a digital reputation attack that spans channels and tactics. But the tools needed to safeguard against an attack are more cybersecurity-oriented than most marketing and communications professionals can manage. Business leaders Leaders grasp the importance; 83% of executives place reputation in their top five areas of risk, yet only 23% are confident in their ability to address it. To succeed in 2024 and beyond, you need to turn online reputation on its axis and think like an attacker.\ Key Takeaways: - New framework for examining and safeguarding an online reputation - Tools and techniques to keep you a step ahead - Practical examples that demonstrate when to act, how to act and how to recover

SMM Cheap - No. 1 SMM panel in the world

smmpanel567

Boost your social media marketing with our SMM Panel services offering SMM Cheap services! Get cost-effective services for your business and increase followers, likes, and engagement across all social media platforms. Get affordable services perfect for businesses and influencers looking to increase their social proof. See how cheap SMM strategies can help improve your social media presence and be a pro at the social media game.

Winning local SEO in the Age of AI - Dennis Yu

DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions

Mastering Local SEO for Service Businesses in the AI Era is tailored specifically for local service providers like plumbers, dentists, and others seeking to dominate their local search landscape. This session delves into leveraging AI advancements to enhance your online visibility and search rankings through the Content Factory model, designed for creating high-impact, SEO-driven content. Discover the Dollar-a-Day advertising strategy, a cost-effective approach to boost your local SEO efforts and attract more customers with minimal investment. Gain practical insights on optimizing your online presence to meet the specific needs of local service seekers, ensuring your business not only appears but stands out in local searches. This concise, action-oriented workshop is your roadmap to navigating the complexities of digital marketing in the AI age, driving more leads, conversions, and ultimately, success for your local service business. Key Takeaways: Embrace AI for Local SEO: Learn to harness the power of AI technologies to optimize your website and content for local search. Understand the pivotal role AI plays in analyzing search trends and consumer behavior, enabling you to tailor your SEO strategies to meet the specific demands of your target local audience. Leverage the Content Factory Model: Discover the step-by-step process of creating SEO-optimized content at scale. This approach ensures a steady stream of high-quality content that engages local customers and boosts your search rankings. Get an action guide on implementing this model, complete with templates and scheduling strategies to maintain a consistent online presence. Maximize ROI with Dollar-a-Day Advertising: Dive into the cost-effective Dollar-a-Day advertising strategy that amplifies your visibility in local searches without breaking the bank. Learn how to strategically allocate your budget across platforms to target potential local customers effectively. The session includes an action guide on setting up, monitoring, and optimizing your ad campaigns to ensure maximum impact with minimal investment.

Monthly Social Media News Update May 2024

Andy Lambert

TL;DR. These are the three themes that stood out to us over the course of last month. 1️⃣ Social media is becoming increasingly significant for brand discovery. Marketers are now understanding the impact of social and budgets are shifting accordingly. 2️⃣ Instagram’s new algorithm and latest guidance will help us maintain organic growth. Instagram continues to evolve, but Reels remains the most crucial tool for growth. 3️⃣ Collaboration will help us unlock growth. Who we work with will define how fast we grow. Meta continues to evolve their Creator Marketplace and now TikTok are beginning to push ‘collabs’ more too.

DMF Portfolio Piece Smart Goals - Artist Management.docx

TravisMalana

Recently uploaded (20)

Core Web Vitals SEO Workshop - improve your performance [pdf]

Is AI-Generated Content the Future of Content Creation?

Your Path to Profits - The Game-Changing Power of a Marketing - Daniel Bussius

De-risk Your Digital Evolution - Hannah Grap

How to Run Landing Page Tests On and Off Paid Social Platforms

Metaverse Marketing in the Generation of the Internet - Eugene Capon

BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin

Playlist and Paint Event with Sony Music U

Traditional Store Audits are Outdated: A New Approach to Protecting Your Bran...

How to use Short Form Video To Grow Your Brand and Business - Keenya Kelly

May 2024 - VBOUT Partners Meeting Group Session

Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin

Digital Commerce Lecture for Advanced Digital & Social Media Strategy at UCLA...

Email Marketing Master Class - Chris Ferris

How to Use AI to Write a High-Quality Article that Ranks

Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin

SMM Cheap - No. 1 SMM panel in the world

Winning local SEO in the Age of AI - Dennis Yu

Monthly Social Media News Update May 2024

DMF Portfolio Piece Smart Goals - Artist Management.docx

Fail and Win: Why a Failed Test Isn’t a Bad Thing

1. Fail and Win: Why a Failed Test Isn’ Caleb Whitmore Founder & CEO Analy<cs Pros @CalebWhitmore

2. Hi, I’m @CalebWhitmore

3. I climbed a mountain… once.

4. I work @Analy=csPros

5. And get to work with some awesome clients!

6. Why Test?

7. Why Test? TESTING… •  Should focus on improving an outcome •  Is the shortest distance between you and improvements •  Is done best with rapid itera<on and lots of data

8. Some things may seem obvious to test…

9. Bad Form

10. Good Form

11. But you should s=ll test them!

12. Tes=ng depends on data

13. Op=mizely Data

14. Google Analy=cs Data

15. Not all tests are created equally

16. Some tests will win

17. Some tests will fail

18. Is winning a success?

19. Is failing a loss?

20. Tes=ng is a Success When: •  A process is followed •  An outcome is measured •  A complete picture is seen

21. When Things Shake, What Moves?

22. When Things Shake, What Moves?

23. Crea=ng a 360 Degree View •  A test will impact more than what you intended •  You must measure the impact across your spectrum of user interac<on •  Measure your test with your exis<ng infrastructure

24. Crea=ng a 360 Degree View What happens to… •  The “key” conversion point •  Engagement on the test page •  Length of the en<re visit •  Total pageviews/interac<ons in the visit •  Secondary conversion goals – view product, view content, start checkout, download a ﬁle, etc…

25. Here’s an example

26. Example Experiment ASSUMPTIONS •  Auto-‐play repels visitors – Audio without warning •  Thus, bounce rates are high and more people leave •  Turn oﬀ auto play & more people will stay and engage

27. Op=mizely Shows a Winner

28. Bounce Rate Improved

29. Purchase Rate Tanked!

30. DO THIS

31. Setup a 360 Degree View of Tests •  Tip: your Digital Analy<cs plaXorm has this view already!

32. Op=mizely Automated Integra=ons

33. Pipes data straight into GA

34. Is this a good outcome?

35. What about this?

36. See the full picture

37. Capture Experiment Data •  Tie experiment data into your Tag Management and Digital Data frameworks

38. Leverage Built-‐in Repor=ng •  Create lots of Goals in Op<mizely – pages, clicks, revenue, etc…