Cox Automotive, the world’s leader in automotive remarketing services, and parent company to such brands as Autotrader, Kelley Blue Book, Manheim, and Dealer.com, has more than 40,000 auto dealer clients across five continents.
Cox Auto focuses on continually improving its products to create faster vehicle transactions and enabling consumers to have a seamless online-to-offline experience. Testing has a natural space to play here - as Cox Automotive’s businesses have learned to scale experimentation to optimize the design of its digital experiences.
In this webinar, Frances Reyes, Seth Stuck, and Sabrina Ho will discuss how Cox Automotive is building a culture of experimentation and testing across their digital properties.
You’ll learn:
- The impetus of testing at Cox Automotive
- How they leverage and share information across their business units, creating shared goals despite different business priorities
- How they created a framework for data-driven decisions across the company
A strong hypothesis is the heart of data-driven product discovery & development. It helps you turn data and insights about your users’ behavior into focused proposals that you’ll take action on.
Check out this very exclusive presentation from Jason G'Sell – Lead Training Consultant – and get a framework to help you and your team form strong experiment hypotheses and come up with the right products and features for your customers.
You’ll learn:
- How and when to introduce experimentation into your product development process
- Identifying the differences between Optimization & Discovery
- Building successful experiments in your product development lifecycle
Mailchimp: Scaling Experimentation Across TeamsOptimizely
Mailchimp, an all-in-one marketing platform for small businesses, has built innovative, beautiful products that empower those businesses to scale. No longer just an email marketing tool--the shiny, new marketing platform’s mission is to empower the underdog to connect with their audiences and create the customer loyalty that’s needed to grow.
In this webinar, Anne Reints, Senior Manager, Growth Analytics & Experimentation at Mailchimp, will discuss how Mailchimp is continuing to build a culture of experimentation and has leveraged testing across both their product and marketing teams.
Join this webinar and learn:
The history of growth and experimentation at Mailchimp
How Mailchimp grew their experimentation program by 6,000%+ in 12 months, going from just 1 experiment to over 60
About the launch of their internal Experimentation Resource Group
How they created a data-driven decision-making framework across the company
UX Analytics and Experimentation for eCommerce GrowthVWO
The primary challenge for eCommerce businesses is to get people on their website. The next challenge is to get them to purchase products. Digital marketers work tirelessly to understand the entire customer journey from discovery to purchase and find ways to influence customer intent.
If the website does not offer a great experience to users (from the beginning), these efforts will not translate into a commensurate growth in revenue.
In this session, Narayan Keshavan from Dell Technologies will focus on the importance of UX Analytics and Experimentation as key enablers for eCommerce revenue growth. Based on his experience in this area, he will outline some key principles that are necessary to effectively leverage experimentation to offer a superior experience to users and enhance the business KPIs. He will also use specific A/B tests to articulate the significance of these principles.
How Clorox Experiments Across Brands to Turn Visitors into ConsumersOptimizely
As more brands focus on digital marketing and Direct-to-Consumer strategies, experimentation can help them efficiently increase consumer engagement. At Clorox, a data-driven experimentation strategy helps them leverage insights across multiple brands.
Watch the on-demand webinar to learn:
- How Clorox gathers insights from omnichannel experimentation to turn visitors into consumers
- Clorox’s experimentation strategy including how an experiment to remove price friction helped their conversion rate optimization
- The process behind creating “Ways of Working” for experimentation programs across Clorox’s Direct-to-Consumer (DTC) brands
Outperform Webinar Series: How to Capture Your Customers at the Top of the Fu...Optimizely
How to Capture Your Customers at the Top of the Funnel
Website metrics, like purchases and form submissions, are top of mind for marketers—but how can you impact those key metrics if users leave as soon as they land on your website?
Join Optimizely’s lead strategy consultant, Alek Toumert, to learn how to engage visitors the moment they arrive at your site through experimentation.
The Wall Street Journal - Optimizing MembershipOptimizely
For the past three years, The Wall Street Journal has used experimentation to help create their membership experience. Starting with acquisition and expanding to engagement, product, and retention testing, WSJ has built a data-driven culture that leverages these experiments to deliver a premium experience that grows their base and keeps members coming back for more.
In this webinar, Olivia Simon from The Wall Street Journal’s optimization team shares how WSJ created their robust and successful testing program.
You’ll learn:
How to think about testing in a membership ecosystem
How WSJ prioritizes tests and earns stakeholder buy-in
About WSJ’s emphasis on testing throughout the customer acquisition funnel
How WSJ expanded their testing funnel beyond acquisition, and the key testing moments they target for consumer engagement and retention
A strong hypothesis is the heart of data-driven product discovery & development. It helps you turn data and insights about your users’ behavior into focused proposals that you’ll take action on.
Check out this very exclusive presentation from Jason G'Sell – Lead Training Consultant – and get a framework to help you and your team form strong experiment hypotheses and come up with the right products and features for your customers.
You’ll learn:
- How and when to introduce experimentation into your product development process
- Identifying the differences between Optimization & Discovery
- Building successful experiments in your product development lifecycle
Mailchimp: Scaling Experimentation Across TeamsOptimizely
Mailchimp, an all-in-one marketing platform for small businesses, has built innovative, beautiful products that empower those businesses to scale. No longer just an email marketing tool--the shiny, new marketing platform’s mission is to empower the underdog to connect with their audiences and create the customer loyalty that’s needed to grow.
In this webinar, Anne Reints, Senior Manager, Growth Analytics & Experimentation at Mailchimp, will discuss how Mailchimp is continuing to build a culture of experimentation and has leveraged testing across both their product and marketing teams.
Join this webinar and learn:
The history of growth and experimentation at Mailchimp
How Mailchimp grew their experimentation program by 6,000%+ in 12 months, going from just 1 experiment to over 60
About the launch of their internal Experimentation Resource Group
How they created a data-driven decision-making framework across the company
UX Analytics and Experimentation for eCommerce GrowthVWO
The primary challenge for eCommerce businesses is to get people on their website. The next challenge is to get them to purchase products. Digital marketers work tirelessly to understand the entire customer journey from discovery to purchase and find ways to influence customer intent.
If the website does not offer a great experience to users (from the beginning), these efforts will not translate into a commensurate growth in revenue.
In this session, Narayan Keshavan from Dell Technologies will focus on the importance of UX Analytics and Experimentation as key enablers for eCommerce revenue growth. Based on his experience in this area, he will outline some key principles that are necessary to effectively leverage experimentation to offer a superior experience to users and enhance the business KPIs. He will also use specific A/B tests to articulate the significance of these principles.
How Clorox Experiments Across Brands to Turn Visitors into ConsumersOptimizely
As more brands focus on digital marketing and Direct-to-Consumer strategies, experimentation can help them efficiently increase consumer engagement. At Clorox, a data-driven experimentation strategy helps them leverage insights across multiple brands.
Watch the on-demand webinar to learn:
- How Clorox gathers insights from omnichannel experimentation to turn visitors into consumers
- Clorox’s experimentation strategy including how an experiment to remove price friction helped their conversion rate optimization
- The process behind creating “Ways of Working” for experimentation programs across Clorox’s Direct-to-Consumer (DTC) brands
Outperform Webinar Series: How to Capture Your Customers at the Top of the Fu...Optimizely
How to Capture Your Customers at the Top of the Funnel
Website metrics, like purchases and form submissions, are top of mind for marketers—but how can you impact those key metrics if users leave as soon as they land on your website?
Join Optimizely’s lead strategy consultant, Alek Toumert, to learn how to engage visitors the moment they arrive at your site through experimentation.
The Wall Street Journal - Optimizing MembershipOptimizely
For the past three years, The Wall Street Journal has used experimentation to help create their membership experience. Starting with acquisition and expanding to engagement, product, and retention testing, WSJ has built a data-driven culture that leverages these experiments to deliver a premium experience that grows their base and keeps members coming back for more.
In this webinar, Olivia Simon from The Wall Street Journal’s optimization team shares how WSJ created their robust and successful testing program.
You’ll learn:
How to think about testing in a membership ecosystem
How WSJ prioritizes tests and earns stakeholder buy-in
About WSJ’s emphasis on testing throughout the customer acquisition funnel
How WSJ expanded their testing funnel beyond acquisition, and the key testing moments they target for consumer engagement and retention
Optimizely Workshop 1: Prioritize your roadmapOptimizely
When your testing roadmap includes dozens of ideas (each with unique requirements) and each team member is vying for her idea to be run first, effective prioritization becomes paramount. This session will focus on the considerations, tools and frameworks you can use to make sure your roadmap is appropriately prioritized to meet your goals.
Web optimization is a vital part of the evolution of customer experience. As performance has become an increasingly more important consideration and experimentation has been driven deeper into the behavioral layer of web pages or apps, brands are turning to server-side optimization solutions to help solve the need for greater testing & optimization across every aspect of the consumer’s journey.
Watch Dan Ross, Managing Director for Optimizely ANZ in our latest webinar from the Experimentation Insights Tour -- "7 Habits of Highly Effective Personalisation Organisations”
Watch the webinar here: https://optimizely.wistia.com/medias/cun66mnkwt
Take Optimizely's Maturity Assessment here: https://www.optimizely.com/maturity-model/
DESCRIPTION: Create a data-driven culture and affect business decisions at the broader company level. When most people think of experimentation or testing, they think of sales and marketing.
However, to do real customer experience optimisation, you need to think about all the ways your customers are interacting with you.
The right mix to support building your programme into a centre of excellence is critical: you need a team that helps create a data-driven culture.
Watch this webinar so you can:
* Think more deeply about the future of your program and the makeup of your team
* Consider which hard and soft skill sets your testing organisation needs
* Build a well-rounded optimisation team that is visible, sustainable, and efficient
About Optimizely
Optimizely is the world's leading experimentation platform, enabling businesses to deliver continuous experimentation and personalisation across websites, mobile apps and connected devices. Optimizely enables businesses to experiment deeply into their technology stack and broadly across the entire customer experience.
The platform’s ease of use and speed of deployment empower organisations to create and run bold experiments that help them make data-driven decisions and grow faster.
To date, marketers, developers and product managers have delivered over 700 billion experiences tailored to the needs of their customers. Optimizely’s global client base includes Atlassian, eBay, Fox, IBM, The New York Times, LendingClub, Hotwire, Microsoft and many more leading businesses.
To learn more about customer experience optimisation, visit optimizely.com
Getting Started with Server-Side TestingOptimizely
One of the most difficult aspects of deep experimentation ― which requires a full stack solution and server-side testing ― is laying a solid foundation for success. Join Optimizely and WiderFunnel to learn best practices for going beyond client-side testing, and implementing a full stack experimentation strategy to drive results on the entire customer journey.
-How to identify your key success metrics, such as customer retention and lifetime value
-How to integrate experimentation into your product roadmap
-How to start testing on your full customer journey
Optimizely Workshop: Mobile Walkthrough Optimizely
Testing and optimizing your mobile apps can help with shorter development cycles, data-driven decision-making, and higher user conversion rates. In this highly interactive session, we encourage you to bring your app (or a sample app), and we’ll walk through the top-to-tail process for using Optimizely on your mobile app. This training is designed for iOS and Android developers who are looking to use Optimizely on their mobile apps.
An Experimentation Framework: How to Position for Triple Digit GrowthOptimizely
You’ve done the button color A/B test, you’ve optimized your landing pages for better conversion. What next? At B2B organizations large and small, there is still tremendous potential for experimentation to drive innovation and growth. Learn how Brion’s growth team enables rapid iteration across a variety of different domains, teams, and organizations within Cisco. With an organization of 70,000 employees and many distributed divisions, enabling experimentation can be a complex initiative. Learn the framework for upleveling from random testing to
explicit strategy to position your org for triple digit growth.
Cro webinar what you're doing wrong in your cro program (sharable version)VWO
In this session, Shiva shares insights from his experience of running conversion rate optimization programs for the past several years. He talks about collaboration, how you can navigate the politics of experimentation, testing to learn and not win, and much more.
Optimizing Your B2B Demand Generation MachineOptimizely
If generating demand for your product is a struggle, rest assured that you are in the majority. 63% of marketers say their top challenge is generating traffic and leads. So what's a marketer to do? We say: hypothesize and experiment.
Join us for this free seminar where SurveyMonkey will share how their marketing team experiments and optimizes different parts of their demand gen machine. You'll hear real stories, tactics, and outcomes that will inspire your own demand gen experimentation.
Learn the real best practices and pitfalls of experimentation based on scientific research and insights. Hazjier is co-author of three studies on experimentation with Harvard Business School and his work is covered in the book Experimentation Works. This talk will dive into the best practices of experiment design, the role of hierarchy in experimentation teams, and the value of experimentation.
Workshop 6: Build Your Organization's Optimization CultureOptimizely
The key output of an effective testing organization is data, but data insights cannot be achieved without the collaborative input of the people that makeup the testing organization. Join this session to learn how Optimizely's most successful customers socialize testing and structure their testing organizations.
uShip - Building a Culture Rooted in ExperimentationOptimizely
uShip is an online marketplace that matches consumer and business shippers with transporters. Over the past year plus, they have been working to build their experimentation program, both from a product and marketing perspective.
Through this focused program investment, uShip has increased their experiment velocity by over 450%, and has multiple product teams testing and using staged feature rollouts. All of this has minimized risk to product rollouts while ensuring customer adoption. It hasn’t always been easy, but they know that doubling down on experimentation is integral to their success.
In this webinar, Jamy Squillace & Brooks Lyford from uShip will share how they started an experimentation program from scratch, gained stakeholder buy-in, and are building a culture of experimentation, focused on testing everywhere.
Join us and learn:
How to build an experimentation program from the ground up
Best practices to balance product and client-side experimentation, leveraging Optimizely’s full platform
How to socialize experimentation throughout the organization and begin creating a culture of experimentation
[Webinar] Innovate Faster by Adopting The Modern Growth StackOptimizely
Adopting a growth mindset requires that product teams become experts in their customers’ behavior. But often legacy technologies simply count clicks and users, and fail to provide insights that product teams need to accelerate growth.
Attend this webinar and learn how the modern growth stack can help you:
-Overcome obstacles that often prevent enterprises from moving quickly
-Use experimentation and product analytics to reduce uncertainty and increase data-driven decisions
-Design a Modern Growth Stack built from best of breed solutions to accelerate product innovation
This webinar is part of our Change the Game series.
World Class Optimization: Benchmarking 1,000+ CompaniesOptimizely
Innovation is required for any growing organization, but nearly impossible without a robust experimentation strategy. This webinar presents you with discoveries on best practices gleaned from an expansive data set spanning over 100,000 experiments in 2016.
Watch this on-demand webinar to learn:
- What traits define the best testing organizations
- How can you drastically increase your testing performance
- What are the right team goals to measure your testing program
The Optimizely Experience Keynote by Matt Althauser - Optimizely Experience L...Optimizely
In this the keynote of the Optimizely Experience London, Matt Althauser (GM Optimizely Europe) shows where Optimizely has started in 2010 and how the product has evolved since.
During his talk fellow team members explained these additional features in more detail. Features include:
- Drag & Drop WYSIWYG editor
- Mobile
- API
- Audiences
- Balanced Content Delivery Network (CDN)
Cultivating a Culture of ExperimentationOptimizely
By harnessing insights from experimentation, people across your organization can contribute ideas and decisions that take the customer experience to new levels. To take advantage of this, forward-thinking organizations are getting everyone involved in experimentation. These slides will share how General Assembly is cultivating a culture of experimentation and the impact it’s making company-wide.
How to Reduce Customer Acquisition Costs and Optimize Advertising SpendOptimizely
Discover how you can optimize your advertising spend, boost your ROI, and reduce your CAC with Optimizely’s senior strategy consultant, Brad Yee.
What you will learn:
- How to run effective landing page campaigns that boost the ROI of your digital programs
- Best practices for unlocking the power of symmetric messaging to acquire more leads at no cost
- What strategies we’ve seen to be successful at reducing CAC and sales support costs
Improve your content: The What, Why, Where and How about A/B Testingintrotodigital
A/B testing, also known as split testing, is a user experience research methodology where users are randomly split into two or more groups to see different versions of the same element. This presentation explains what is A/B testing, why you need it, where you can apply it and how to conduct an A/B test.
To go from one-off testing to building an experimentation program, you need new tools to help you manage ideas, coordinate across teams, and share knowledge across your organization.
We built Optimizely Program Management to help companies increase the scale and velocity of their experimentation programs so that they can iterate and innovate faster than ever before.
Attend this webinar to learn how to:
-Effectively scale your experimentation with Program Management
-Collaborate more effectively across multiple teams and stakeholders
-Report on your experimentation program holistically and uncover new insights
Testing Across the Enterprise: How Cox Automotive Scales Experimentation to M...Optimizely
In today’s digital economy, consumers expect to buy everything online, even a purchase as significant as a car. Cox Automotive, parent company to brands including Autotrader and Kelley Blue Book, has more than 40,000 auto dealer clients across five continents, and is enabling them to keep up with consumer behavior and insights by implementing an enterprise wide experimentation program. Cox is bridging the gap between consumers, manufacturers, dealers, and lenders at every stage of the automotive experience.
In this session, you’ll hear how this multi-brand, international, matrixed organization has built an enterprise experimentation program that is flexible enough to adapt to various business models and modern-day demands for speed while maintaining testing best practices, r
[Webinar] Visa's Journey to a Culture of ExperimentationOptimizely
Join us as we hear Ramkumar Ravichandran, the Director of A/B Testing at Visa Checkout, explain how he created a high impact experimentation program. Ram will take us through the growth of Visa’s program: from selling the value, to laying down the vision, the roadmap and success criteria, to creating the right team and driving engagement with the program.
Attend this webinar to learn:
-How an experimentation program drives business impact.
-A model to drive continuous stakeholder engagement with the program.
-How to build a roadmap that goes above and beyond simple UX optimization.
Optimizely Workshop 1: Prioritize your roadmapOptimizely
When your testing roadmap includes dozens of ideas (each with unique requirements) and each team member is vying for her idea to be run first, effective prioritization becomes paramount. This session will focus on the considerations, tools and frameworks you can use to make sure your roadmap is appropriately prioritized to meet your goals.
Web optimization is a vital part of the evolution of customer experience. As performance has become an increasingly more important consideration and experimentation has been driven deeper into the behavioral layer of web pages or apps, brands are turning to server-side optimization solutions to help solve the need for greater testing & optimization across every aspect of the consumer’s journey.
Watch Dan Ross, Managing Director for Optimizely ANZ in our latest webinar from the Experimentation Insights Tour -- "7 Habits of Highly Effective Personalisation Organisations”
Watch the webinar here: https://optimizely.wistia.com/medias/cun66mnkwt
Take Optimizely's Maturity Assessment here: https://www.optimizely.com/maturity-model/
DESCRIPTION: Create a data-driven culture and affect business decisions at the broader company level. When most people think of experimentation or testing, they think of sales and marketing.
However, to do real customer experience optimisation, you need to think about all the ways your customers are interacting with you.
The right mix to support building your programme into a centre of excellence is critical: you need a team that helps create a data-driven culture.
Watch this webinar so you can:
* Think more deeply about the future of your program and the makeup of your team
* Consider which hard and soft skill sets your testing organisation needs
* Build a well-rounded optimisation team that is visible, sustainable, and efficient
About Optimizely
Optimizely is the world's leading experimentation platform, enabling businesses to deliver continuous experimentation and personalisation across websites, mobile apps and connected devices. Optimizely enables businesses to experiment deeply into their technology stack and broadly across the entire customer experience.
The platform’s ease of use and speed of deployment empower organisations to create and run bold experiments that help them make data-driven decisions and grow faster.
To date, marketers, developers and product managers have delivered over 700 billion experiences tailored to the needs of their customers. Optimizely’s global client base includes Atlassian, eBay, Fox, IBM, The New York Times, LendingClub, Hotwire, Microsoft and many more leading businesses.
To learn more about customer experience optimisation, visit optimizely.com
Getting Started with Server-Side TestingOptimizely
One of the most difficult aspects of deep experimentation ― which requires a full stack solution and server-side testing ― is laying a solid foundation for success. Join Optimizely and WiderFunnel to learn best practices for going beyond client-side testing, and implementing a full stack experimentation strategy to drive results on the entire customer journey.
-How to identify your key success metrics, such as customer retention and lifetime value
-How to integrate experimentation into your product roadmap
-How to start testing on your full customer journey
Optimizely Workshop: Mobile Walkthrough Optimizely
Testing and optimizing your mobile apps can help with shorter development cycles, data-driven decision-making, and higher user conversion rates. In this highly interactive session, we encourage you to bring your app (or a sample app), and we’ll walk through the top-to-tail process for using Optimizely on your mobile app. This training is designed for iOS and Android developers who are looking to use Optimizely on their mobile apps.
An Experimentation Framework: How to Position for Triple Digit GrowthOptimizely
You’ve done the button color A/B test, you’ve optimized your landing pages for better conversion. What next? At B2B organizations large and small, there is still tremendous potential for experimentation to drive innovation and growth. Learn how Brion’s growth team enables rapid iteration across a variety of different domains, teams, and organizations within Cisco. With an organization of 70,000 employees and many distributed divisions, enabling experimentation can be a complex initiative. Learn the framework for upleveling from random testing to
explicit strategy to position your org for triple digit growth.
Cro webinar what you're doing wrong in your cro program (sharable version)VWO
In this session, Shiva shares insights from his experience of running conversion rate optimization programs for the past several years. He talks about collaboration, how you can navigate the politics of experimentation, testing to learn and not win, and much more.
Optimizing Your B2B Demand Generation MachineOptimizely
If generating demand for your product is a struggle, rest assured that you are in the majority. 63% of marketers say their top challenge is generating traffic and leads. So what's a marketer to do? We say: hypothesize and experiment.
Join us for this free seminar where SurveyMonkey will share how their marketing team experiments and optimizes different parts of their demand gen machine. You'll hear real stories, tactics, and outcomes that will inspire your own demand gen experimentation.
Learn the real best practices and pitfalls of experimentation based on scientific research and insights. Hazjier is co-author of three studies on experimentation with Harvard Business School and his work is covered in the book Experimentation Works. This talk will dive into the best practices of experiment design, the role of hierarchy in experimentation teams, and the value of experimentation.
Workshop 6: Build Your Organization's Optimization CultureOptimizely
The key output of an effective testing organization is data, but data insights cannot be achieved without the collaborative input of the people that makeup the testing organization. Join this session to learn how Optimizely's most successful customers socialize testing and structure their testing organizations.
uShip - Building a Culture Rooted in ExperimentationOptimizely
uShip is an online marketplace that matches consumer and business shippers with transporters. Over the past year plus, they have been working to build their experimentation program, both from a product and marketing perspective.
Through this focused program investment, uShip has increased their experiment velocity by over 450%, and has multiple product teams testing and using staged feature rollouts. All of this has minimized risk to product rollouts while ensuring customer adoption. It hasn’t always been easy, but they know that doubling down on experimentation is integral to their success.
In this webinar, Jamy Squillace & Brooks Lyford from uShip will share how they started an experimentation program from scratch, gained stakeholder buy-in, and are building a culture of experimentation, focused on testing everywhere.
Join us and learn:
How to build an experimentation program from the ground up
Best practices to balance product and client-side experimentation, leveraging Optimizely’s full platform
How to socialize experimentation throughout the organization and begin creating a culture of experimentation
[Webinar] Innovate Faster by Adopting The Modern Growth StackOptimizely
Adopting a growth mindset requires that product teams become experts in their customers’ behavior. But often legacy technologies simply count clicks and users, and fail to provide insights that product teams need to accelerate growth.
Attend this webinar and learn how the modern growth stack can help you:
-Overcome obstacles that often prevent enterprises from moving quickly
-Use experimentation and product analytics to reduce uncertainty and increase data-driven decisions
-Design a Modern Growth Stack built from best of breed solutions to accelerate product innovation
This webinar is part of our Change the Game series.
World Class Optimization: Benchmarking 1,000+ CompaniesOptimizely
Innovation is required for any growing organization, but nearly impossible without a robust experimentation strategy. This webinar presents you with discoveries on best practices gleaned from an expansive data set spanning over 100,000 experiments in 2016.
Watch this on-demand webinar to learn:
- What traits define the best testing organizations
- How can you drastically increase your testing performance
- What are the right team goals to measure your testing program
The Optimizely Experience Keynote by Matt Althauser - Optimizely Experience L...Optimizely
In this the keynote of the Optimizely Experience London, Matt Althauser (GM Optimizely Europe) shows where Optimizely has started in 2010 and how the product has evolved since.
During his talk fellow team members explained these additional features in more detail. Features include:
- Drag & Drop WYSIWYG editor
- Mobile
- API
- Audiences
- Balanced Content Delivery Network (CDN)
Cultivating a Culture of ExperimentationOptimizely
By harnessing insights from experimentation, people across your organization can contribute ideas and decisions that take the customer experience to new levels. To take advantage of this, forward-thinking organizations are getting everyone involved in experimentation. These slides will share how General Assembly is cultivating a culture of experimentation and the impact it’s making company-wide.
How to Reduce Customer Acquisition Costs and Optimize Advertising SpendOptimizely
Discover how you can optimize your advertising spend, boost your ROI, and reduce your CAC with Optimizely’s senior strategy consultant, Brad Yee.
What you will learn:
- How to run effective landing page campaigns that boost the ROI of your digital programs
- Best practices for unlocking the power of symmetric messaging to acquire more leads at no cost
- What strategies we’ve seen to be successful at reducing CAC and sales support costs
Improve your content: The What, Why, Where and How about A/B Testingintrotodigital
A/B testing, also known as split testing, is a user experience research methodology where users are randomly split into two or more groups to see different versions of the same element. This presentation explains what is A/B testing, why you need it, where you can apply it and how to conduct an A/B test.
To go from one-off testing to building an experimentation program, you need new tools to help you manage ideas, coordinate across teams, and share knowledge across your organization.
We built Optimizely Program Management to help companies increase the scale and velocity of their experimentation programs so that they can iterate and innovate faster than ever before.
Attend this webinar to learn how to:
-Effectively scale your experimentation with Program Management
-Collaborate more effectively across multiple teams and stakeholders
-Report on your experimentation program holistically and uncover new insights
Testing Across the Enterprise: How Cox Automotive Scales Experimentation to M...Optimizely
In today’s digital economy, consumers expect to buy everything online, even a purchase as significant as a car. Cox Automotive, parent company to brands including Autotrader and Kelley Blue Book, has more than 40,000 auto dealer clients across five continents, and is enabling them to keep up with consumer behavior and insights by implementing an enterprise wide experimentation program. Cox is bridging the gap between consumers, manufacturers, dealers, and lenders at every stage of the automotive experience.
In this session, you’ll hear how this multi-brand, international, matrixed organization has built an enterprise experimentation program that is flexible enough to adapt to various business models and modern-day demands for speed while maintaining testing best practices, r
[Webinar] Visa's Journey to a Culture of ExperimentationOptimizely
Join us as we hear Ramkumar Ravichandran, the Director of A/B Testing at Visa Checkout, explain how he created a high impact experimentation program. Ram will take us through the growth of Visa’s program: from selling the value, to laying down the vision, the roadmap and success criteria, to creating the right team and driving engagement with the program.
Attend this webinar to learn:
-How an experimentation program drives business impact.
-A model to drive continuous stakeholder engagement with the program.
-How to build a roadmap that goes above and beyond simple UX optimization.
Tune Agile Test Strategies to Project and Product MaturityTechWell
For optimum results, you need to tune agile project's test strategies to fit the different stages of project and product maturity. Testing tasks and activities should be lean enough to avoid unnecessary bottlenecks and robust enough to meet your testing goals. Exploring what "quality" means for various stakeholder groups, Anna Royzman describes testing methods and styles that fit best along the maturity continuum. Anna shares her insights on strategic ways to use test automation, when and how to leverage exploratory testing as a team activity, ways to prepare for live pilots and demos of the real product, approaches to refine test coverage based on customer feedback, and techniques for designing a production "safety net" suite of automated tests. Leave with a better understanding of how to satisfy your stakeholders’ needs for quality-and a roadmap for tuning your agile test strategies.
This will be presented at the Optimizely's San Francisco User Group session on Oct 4th. As with any program, an A/B Testing Practice also follows a specific maturity curve. Since it is much more complex and spans across various domains and business units, it begins with a "Sell" phase focused on getting buy-in from various stakeholders but with a specific focus on Engineering & QA, followed by "Scale" phase with focus on building team, efficiency and program and then on to "Expand" phase focused on wider scope/complex tests and strengthen the platform, over to the "Deepen" phase where the focus is to ingrain testing within the company's DNA, i.e., within the backend/algorithms, cross pollinate learning and testing across various business units. The final phase is the "Sustain" phase where Algorithmic Test Management takes over Testing, and Testing is productized as a Value Add service for monetization and brand captial creation. We will walk the audience through our own journey so far along the maturity curve, the lessons learnt along the way, the challenges and what worked for us. The session will be rounded up with a working session with the audience on their own journey, lessons and advice for others.
How to Achieve Customer Satisfaction Through Beta TestingCentercode
As the core stage of Customer Validation, Beta Testing offers a unique approach to product feedback completely different than any other testing strategy because you're able to evaluate the customer satisfaction of a product with your target market before it launches. Here, we'll teach you how to run an effective Beta Test, discuss the different types of feedback you'll collect, and show you what to do with the data you've collected.
A test strategy is the set of ideas that guides your test design. It's what explains why you test this instead of that, and why you test this way instead of that way. Strategic thinking matters because testers must make quick decisions about what needs testing right now and what can be left alone. You must be able to work through major threads without being overwhelmed by tiny details. James Bach describes how test strategy is organized around risk but is not defined before testing begins. Rather, it evolves alongside testing as we learn more about the product. We start with a vague idea of our strategy, organize it quickly, and document as needed in a concise way. In the end, the strategy can be as formal and detailed as you want it to be. In the beginning, though, we start small. If you want to focus on testing and not paperwork, this approach is for you.
Test Improvement - Any place, anytime, any whereRuud Teunissen
Test Improvement is all about giving an organization or a team the “means they can use” to help achieve their goals. Means that are in line with their skills and they can use in their context. That’s why successful Test Improvement requires leadership and management. In this presentation I share experiences in Test Improvement in a wide variety of environments, using different models and approaches.
A/B Testing best practices from strategic vision to operational considerations to communication and finally expectations management. We need to adhere to fundamental project management, technology, statistical, experimental design, UX Design, Customer Relationship, business and data principles to ensure that the insights and hence the decision is as trustworthy as possible.
Despite the belief that a shared context and collaboration drives quality, too often, software testers and quality professionals struggle to find their place within today's integrated agile teams. This session is a practitioner’s view of testing and testing practices within an iterative/incremental development environment. We will begin with a discussion of some of the challenges of testing within an agile environment and delve into the guiding principles of Agile Testing and key enabling practices. Agile Testing necessitates a change in mindset, and it is as much, if not more, about behavior, as it is about skills and tooling, all of which will be explored.
Experiment Your Way to Product Success: How User Acceptance Testing Can Save ...Aggregage
Before a new product or feature goes into development, you have to gauge whether it will be a worthwhile investment. You don’t want to spend time and money building a product that no one will use. But what’s the best way to do so - and how can you get honest insights from your end users?
Join J.B. Siegel, VP of Client Services at Seamgen, as he explores how to use wireframes and clickable prototypes to validate your product. He’ll discuss how user testing allows you to really understand your users - and how to use the insights to inform your product strategy
Clover Rings Up Digital Growth to Drive ExperimentationOptimizely
Clover's Digital Growth team is responsible for optimizing the merchant's digital experience and they rely on experimentation to guide digital decision-making. This enables them to quickly learn and measure what changes deliver the best outcomes for users.
Join us with Lead Product Manager of Growth, Monil Shah, to learn how Clover:
- Increased digital conversions amongst merchants with an investment in experimentation
- Grew experiment velocity by 4x after replacing Adobe Target
- Designed a framework to efficiently capture and prioritize test ideas, and roll out winners
Atlassian's Mystique CLI, Minimizing the Experiment Development CycleOptimizely
Mystique CLI is an Atlassian developed CLI for Optimizely Web. It is a multi-phase project that is currently focusing on improving the development cycle for growth engineers. Currently, Mystique is the standard for developing web experiments at Atlassian, and is capable of a wide variety of operations utilizing Optimizely's REST API. This includes creating, updating, testing, and duplicating experiments/personalization campaigns, as well as "promoting" these entities between Optimizely projects for different environments (e.g. from QA => Prod). It has significantly reduced manual overhead and decreased development time by up to 95% for particular actions.
Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...Optimizely
Autotrader's Product and Engineering teams were ahead of the curve many years ago when they built a home-grown solution for leveraging feature flags to support server-side testing. Over the years, the industry eventually caught up and surpassed this proprietary tooling and the team had a choice to make: Re-invest into the local solution or completely retool. In this case study, Scott Povlot, Principal Technical Architect, and Seth Stuck, Director of R&D Analytics, will discuss their journey in selecting and then migrating to their next generation of experimentation tooling. They will discuss selection criteria, pros and cons, and outline how they were able to make the migration to Optimizely successful and lessons learned along the way.
Zillow + Optimizely: Building the Bridge to $20 Billion RevenueOptimizely
Join Jason Tabert, Senior CRO Marketing Specialist, and learn how Zillow is using Optimizely’s experimentation, personalization and integrations to help grow their revenue to $20 billion by helping their customers cross the real estate chasm from despair to delight.
The Future of Optimizely for Technical TeamsOptimizely
Optimizely has been reimagining the future of progressive delivery and experimentation, improving every part of the platform to empower technical teams to build, ship, and iterate faster. Learn about the latest enhancements to Optimizely Full Stack and the Optimizely Data Platform, and get a sneak peek at the upcoming roadmap.
Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...Optimizely
The coronavirus pandemic has pushed contact center leaders to accelerate technology adoption and empower their teams to work remotely. Join this session with State Farm, Salesforce, and Optimizely to learn how contact centers can adapt quickly and successfully in the time of COVID.
Our new normal has accelerated eCommerce trends by 4-6 years. The Optimizely team shares how experimentation can help retailers fast forward their online sales strategy with Microsoft Dynamics 365 Commerce.
Building an Experiment Pipeline for GitHub’s New Free Team OfferingOptimizely
In April 2020, GitHub announced a new Free for Teams plan. Behind the scenes, the engineering team was also setting up an experiment pipeline and an integration with Optimizely. In this session, we will take a peek at the process of setting up the integration, learning about the behavior of this new Free for Teams customer segment, and the next steps for this experiment pipeline.
AMC Networks Experiments Faster on the Server SideOptimizely
Speeding up innovation only matters if it helps you drive positive outcomes. At AMC, experimentation enables the product and platform teams to challenge their assumptions, maximize impact, and evaluate ideas as painted door tests before investing in significant development. A commitment to test everything across 9 platforms fueled their search for the most scalable solution.
In this session, you'll learn how to:
Leverage server-side testing to experiment quickly
Scale across web, mobile, and OTT applications
Determine when client-side testing is more efficient
Evolving Experimentation from CRO to Product DevelopmentOptimizely
An obsession with data, efficiency, and delivering incredible customer experiences are just a few things that the CNN Consumer Science and Software Engineering teams have in common. Simple A/B testing practices evolved into a culture of experimentation, sparking new development practices across the organization. Learn how they drive results across their entire platform from websites to mobile apps.
Overcoming the Challenges of Experimentation on a Service Oriented ArchitectureOptimizely
Growing from an early stage startup to a national leader in financial literacy is no small feat, and there are a ton of lessons that we have learned at Greenlight as we have grown. Long gone are the days where we would ship something and cross our fingers hoping that it makes some kind of impact on our customers. Now we’re in a world where we can learn ahead of time how much impact a feature will have on the business, before we even launch! In today’s conversation, we’ll discuss how we use Optimizely’s feature flags in our microservice architecture using Optimizely Agent while keeping user IDs and context synchronized.
This session will cover:
How we set up Optimizely Agent and use it in a kubernetes deployment
How we created a user-aliasing service
How we access Optimizely both on the frontend and in the backend services.
How to build a full stack feature
How to manage the rollout using Optimizely’s feature flags
How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...Optimizely
A/B testing is an essential element in any product managers playbook. However having the freedom and flexibility to customize testing based on what the data is saying often requires a lot of time and effort, particularly when it comes to engineering resources. Optimizely offers a flexible approach to experimentation through the use of feature testing, which provides more customization options without the additional development effort typically required to implement these feature optimizations. Megan Bubley, a Senior Product Manager at The Zebra, will share her experience working with Optimizely’s feature tests to create a results page where users can compare multiple auto insurance options driven by actual user needs, as well as her experience customizing the experience based on device platform.
Making Your Hypothesis Work Harder to Inform Future Product StrategyOptimizely
At Treatwell, each experiment goes beyond improving a single business metric. Experimentation works to evolve their product while enriching customer insights in order to deliver the best digital experience to their users. Join Laura Howard, Lead Product Manager, and Dennis Meisner, Senior Product Analyst, to learn their secret to making their hypothesis work harder and how getting their hypothesis right has improved Treatwell’s funnel progression and order health, as well as helped them make critical decisions on their product experience.
Kick Your Assumptions: How Scholl's Test-Everything Culture Drives RevenueOptimizely
Amy Vetter, Consumer Experience Manager, Direct To Consumer, Europe, will walk you through some of the tests that she and her team run across the Scholl brand. Amy will highlight surprise learnings and how to remove the fear of failing. The team is empowered to test everything possible that will allow the customer to get the best experience and also support the brand’s goal for more revenue and customer data.
At Charles Schwab, they have a mantra of viewing the world through their client’s eyes. When it comes to building digital experiences and running experiments, winning isn’t just about moving metrics, it’s also about improving customer experience. Sara Tresch, SVP of Digital Services at Schwab will be discussing how Schwab designs products and experiments with a client-first mindset.
Shipping to Learn and Accelerate Growth with GitHubOptimizely
Will 2020 mark the shift to a remote-first world in the long run? For GitHub, a distributed workforce is nothing new. Join Sha Ma, VP of Engineering, and Gregory Ceccarelli, Director of Data Science, to learn how they built and scaled a successful experimentation program. They'll share their experience implementing Optimizely across timezones, a remote workforce, and a new business model.
In this session, you'll learn how to:
Optimize UX for a freemium business model
Use data to deliver customer-centered products
Scale experimentation and accelerate growth
Test Everything: TrustRadius Delivers Customer Value with ExperimentationOptimizely
When done right, experimentation can help you validate the product you’re building and create winning customer experiences. And it doesn’t take a big engineering team to make this happen.
TrustRadius, the most trusted review site for business technology, uses experimentation to build an online community through website and server-side experimentation. The small but mighty TrustRadius team runs experiments throughout the buyer’s journey to engage different user personas and understand outcomes in real-time.
Watch the webinar recording featuring Rilo Stark, product manager at TrustRadius, and Jack Peden, senior software engineer, to understand their data-driven experimentation strategy and how TrustRadius uses Optimizely Web and Full Stack products to tailor experiences to different customer segments and mitigate risk through A/B/N and painted door tests.
In this session, you will learn: how to embed feature flagging sitewide to deliver safer, faster releases, best practices for implementing feature flags in a services-oriented architecture, and the latest enhancements you need to help your team recover faster when ship happens.
Newly appointed Optimizely CTO, Lawrence Bruhmuller, will kick off Developer Summit discussing the new normals in software development. After decades of leading and scaling engineering teams for high growth startups and large tech companies, Lawrence has seen the same problems crop up repeatedly for technical teams. There is a new way of delivering software that makes it possible to move fast and get it right. That new way is Progressive Delivery & Experimentation. When Progressive Delivery & Experimentation are used together, you have an efficient system for validating both quality and customer engagement across your development lifecycle. Lawrence will discuss the key principles driving software development innovation, how our engineering team puts this into practice, and the success he’s seen at other companies.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
A Comprehensive Look at Generative AI in Retail App Testing.pdfkalichargn70th171
Traditional software testing methods are being challenged in retail, where customer expectations and technological advancements continually shape the landscape. Enter generative AI—a transformative subset of artificial intelligence technologies poised to revolutionize software testing.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxrickgrimesss22
Discover the essential features to incorporate in your Winzo clone app to boost business growth, enhance user engagement, and drive revenue. Learn how to create a compelling gaming experience that stands out in the competitive market.
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteGoogle
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-pilot-review/
AI Pilot Review: Key Features
✅Deploy AI expert bots in Any Niche With Just A Click
✅With one keyword, generate complete funnels, websites, landing pages, and more.
✅More than 85 AI features are included in the AI pilot.
✅No setup or configuration; use your voice (like Siri) to do whatever you want.
✅You Can Use AI Pilot To Create your version of AI Pilot And Charge People For It…
✅ZERO Manual Work With AI Pilot. Never write, Design, Or Code Again.
✅ZERO Limits On Features Or Usages
✅Use Our AI-powered Traffic To Get Hundreds Of Customers
✅No Complicated Setup: Get Up And Running In 2 Minutes
✅99.99% Up-Time Guaranteed
✅30 Days Money-Back Guarantee
✅ZERO Upfront Cost
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have.
For more Tendenci AMS events, check out www.tendenci.com/events
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
6. Revenue, share of wallet, funnel conversion, risk
mitigation, ops efficiency
What
We Do Next gen “Test and Learn” platform for enterprise-
wide digital experience optimization
Over 1B impressions daily
Replaces Digital Guesswork
With Evidence-based Optimization
Digital Experimentation
SaaS Platform
Apply the scientific method to “at-scale”
business decision making.
20X increase in Yield
8. Frontend UI Backend Business
Logic & Data
The anatomy of an experience
i.e. navigation,
search location
& visual treatment
Copy
Images &
Colors
Layout Search algorithms
Personalize content
based on previous
behavior
Recommendations
Make your headlines
more personal
9. “Our success is a function of
how many experiments we do
per year, per month, per week,
per day.”
Jeff Bezos
“Our aim is to create the best
product for our customers, and
we do that through constant
innovation and testing.”
Gillan Tans, CEO
“Our company culture
encourages experimentation
and free flow of ideas.”
Larry Page
“We use experimentation and
testing to inform as much of the
business as we possibly can” -
Gregory Peters, CPO
Today’s Digital Leaders Win By
Using Experimentation At-Scale
10. Spent with Digital Media
19 hr
week
2018
Digital-Centric Customer
Spent with Digital Media
41
$396m
Mobile commerce Mobile commerce
$1 8t.
2008
Digital-Convenient Customer
hr
week
Your customer has fundamentally
changed how they engage
28. 28
Workflow and RACI Overview
Test Stages Description Responsible
Who will do the work for
this step?
Accountable
Who validates this work and
pushes the test into the next
stage of the workflow?
Consulted
Who will likely need to help
the Responsible person(s)
Informed
Who needs to be informed
about the progress of this
work?
Requirements
Idea has all relevant test details
to move forward and has been
scored by necessary
stakeholders (identified in
"Consulted" column).
Whomever is ideating
the test - can be anyone
RTEs (to accept for
scoring), Testing
Analytics (to validate test
details)
UX, Product, Engineering,
Product Analytics
RTEs, Leadership
Creative
Assets for experiment variants
are received and are attached to
the idea.
UX UX Product Analytics, UX,
Product, Engineering,
Testing Analytics
RTEs
Development
Variants for experiment have
been built
UX / UI / Front-end
Engineering,
Engineering, and/or
Architecture
UX UX / UI / Front-end
Engineering, Engineering,
and/or Architecture,
Testing Analytics
RTEs, Product Analytics
Setup and QA
Experiment has been configured
in Optimizely and QA’ed and
accepted by necessary
stakeholders.
UX, Engineering,
Testing Analytics,
Product Analytics
Testing Analytics UX / UI / Front-end
Engineering, Product,
Engineering, and/or
Architecture
Product, RTEs
Testing
Experiment has been deployed
and is actively running
Product Analytics,
Testing Analytics
Product Analytics UX, Product, RTEs,
Testing Analytics
Leadership
Analysis
Experiment has concluded and
success is being determined
(usually requires some off-site
validation)
Product Analytics,
Testing Analytics
Product Analytics UX, Product, RTEs,
Testing Analytics
Leadership, Stakeholders
Completed
Experiment learnings and next
steps are distributed to rest of
the organization.
Product, Product
Analytics, Testing
Analytics
RTEs UX, Product, RTEs,
Testing Analytics
Leadership
32. 32
Date Range: 11/30/18 through 4/30/19
Velocity (week) Conclusive Rate Win Rate
Cox Automotive 2.2 25% 18%
All Customers
(Web + Full Stack)
0.4 27% 17%
All Web 0.5 25% 15%
Top 90%
All Web
1.7 31% 24%
Top 90%
Retail Web
3.0 33% 20%
Top 90%
Media Web
2.4 36% 26%
Top 5
Marketplace Web
2.9 34% 21%
39. 39
Knowing When to Test
When the clarity of a pre/post just won’t be enough
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Page Conversion
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Page Conversion
Week 1 Week 2
Pre Post
Experiment w/ concurrent
control and test groups
42. 42
Knowing When to Test
Types of Tests
Discovery and Light-
Weight Prototyping
• Used to very quickly
answer whether an
idea has legs – or to
help size up the
opportunity
• Common types of
tests include: Fake
door, A/B, Usability,
Focus Group
Optimization
• Used to compare
multiple design
variants to
determine which
has optimal
performance
towards a given
goal
• Common types of
tests include: A/B
or A/B/n tests
De-risking or Validation
• Used to determine how
functionality that has
already been developed
will perform – generally as
a means for ensuring that
the new product or
design performs as
expected
• Also helpful for
retrospectives (looking
back over a given time
period to assess what has
had the most impact)
• Common types of tests
include: A/B or Multi-
variate tests
Blue Sky
• Used to answer
heavy business
questions –
often, the thing
being tested
would never
be deployed
• Intended to
help shape up
hypotheticals
and limit
speculation in
strategy
43. 43
Knowing When to Test
Types of Tests
Discovery and Light-
Weight Prototyping
• Used to very quickly
answer whether an
idea has legs – or to
help size up the
opportunity
• Common types of
tests include: Fake
door, A/B, Usability,
Focus Group
Optimization
• Used to compare
multiple design
variants to
determine which
has optimal
performance
towards a given
goal
• Common types of
tests include: A/B
or A/B/n tests
De-risking or Validation
• Used to determine how
functionality that has
already been developed
will perform – generally as
a means for ensuring that
the new product or
design performs as
expected
• Also helpful for
retrospectives (looking
back over a given time
period to assess what has
had the most impact)
• Common types of tests
include: A/B or Multi-
variate tests
Blue Sky
• Used to answer
heavy business
questions –
often, the thing
being tested
would never
be deployed
• Intended to
help shape up
hypotheticals
and limit
speculation in
strategy
44. 44
Knowing When to Test
Types of Tests
Discovery and Light-
Weight Prototyping
• Used to very quickly
answer whether an
idea has legs – or to
help size up the
opportunity
• Common types of
tests include: Fake
door, A/B, Usability,
Focus Group
Optimization
• Used to compare
multiple design
variants to
determine which
has optimal
performance
towards a given
goal
• Common types of
tests include: A/B
or A/B/n tests
De-risking or Validation
• Used to determine how
functionality that has
already been developed
will perform – generally as
a means for ensuring that
the new product or
design performs as
expected
• Also helpful for
retrospectives (looking
back over a given time
period to assess what has
had the most impact)
• Common types of tests
include: A/B or Multi-
variate tests
Blue Sky
• Used to answer
heavy business
questions –
often, the thing
being tested
would never
be deployed
• Intended to
help shape up
hypotheticals
and limit
speculation in
strategy
45. 45
Example of a Product Validation Test
Vehicles priced below
Fair Market Range saw a
7% increase in overall value
47. 47
De-risking to Blue Sky
A/B Test #1
HP Redesign:
Control Challenger A
A/B Test #2
HP Redesign:
Control Challenger A
Test creative based on User Testing
Live site results differed from User Testing
Redesigned Hero section Clicks to main KPI increased
+7%
48. 48
Knowing When to Test
Types of Tests
Discovery and Light-
Weight Prototyping
• Used to very quickly
answer whether an
idea has legs – or to
help size up the
opportunity
• Common types of
tests include: Fake
door, A/B, Usability,
Focus Group
Optimization
• Used to compare
multiple design
variants to
determine which
has optimal
performance
towards a given
goal
• Common types of
tests include: A/B
or A/B/n tests
De-risking or Validation
• Used to determine how
functionality that has
already been developed
will perform – generally as
a means for ensuring that
the new product or
design performs as
expected
• Also helpful for
retrospectives (looking
back over a given time
period to assess what has
had the most impact)
• Common types of tests
include: A/B or Multi-
variate tests
Blue Sky
• Used to answer
heavy business
questions –
often, the thing
being tested
would never
be deployed
• Intended to
help shape up
hypotheticals
and limit
speculation in
strategy
49. 49
Example of a Blue Sky
Sponsored Search Links
Sponsorships
Sponsored Hero
Image and Search Links
Advertisement
Removing ads from the site
created a 15.1% lift in both
value events and VDPs
50. 50
Knowing When to Test
When the (un)certainty of a pre/post just won’t be enough
Uncertainty Certainty
High Level
of Effort
Low Level
of Effort
“Just do” launch with
no Pre/Post Analysis
“Just do” launch with
Pre/Post Analysis
Optimization
A/B Tests
Blue Sky Testing with
Iterative Learning Plan
De-Risking
and Validation
Testing
Discovery and
Light-Weight
Prototyping
52. 52
Creating an Enterprise Testing Program
High-Level Learnings
What HAS Worked?
• Leveraging larger, more experienced testing
programs and personnel
• Funding
• Workflows and RACI
• Pro-bono Support
• Comparing depth and breadth of test results
and analysis to what’s possible in a pre/post
• Transparency of test plans and results
• Quarterly Summit for enterprise-wide insights
and partnership
• Partnership with Optimizely
What HAS NOT Worked?
• Federating access to Individual teams
who don’t embrace basic best practices
• Over-reliance on Analytics to support all
aspects of the test
• Jumping straight to cross-brand testing
• Skipping testing in favor of “just do”
mentality