Opticon 2017 How Developers Can Take Experimentation

•Download as PPTX, PDF•

0 likes•611 views

The document discusses how developers can take experimentation to new levels with full stack experimentation. It describes an architecture that allows experiments across platforms and flexibility to experiment deep within the technical stack. Examples are given of how full stack experimentation could be used for tasks like A/B tests, feature rollouts, and algorithm changes. The benefits of the SDK architecture are outlined as being flexible, stateless, and replaceable.

Technology

opticon2017
How Developers can take
Experimentation to New Levels
Michael Ng
Software Eng, Optimizely
Ali Rizvi
Software Eng, Optimizely
Josh Wang
Software Eng, Optimizely

opticon2017
A story about
experimentation

opticon2017
$100 million
$50 million
Mike Ng
Business Impact

opticon2017
How Developers can take
experimentation to new levels with Full
Stack
• Architecture that allows developer flexibility
• Experiment across platforms
• Experiment deep in the stack

opticon2017opticon2017
Full Stack
Experimentatio
n

opticon2017
How experimentation works with Full Stack
Your servers
SDKs
Results
Experiment
management
datafile
events
Flexible SDKs Optimizely Dashboard
A
B
Node
Java
Ruby
Python
PHP
C#
Javascript
iOS
Android
User devices

opticon2017
Search algorithms
Redesigns
Feature rollouts
Sorting algorithms
Price changes
Backend changes
Use cases for Full Stack
experimentation

opticon2017
SDK architecture:
Built for flexibility
• Stateless
• Consistent implementation across SDKs
• Non-blocking API calls
• Replaceable components

opticon2017
Creating an SMS
experiment with Full Stack

opticon2017
SMS experiment
“Are you interested in learning
more about Optimizely Full
Stack?”
Variation B
“Want to learn how to
experiment easily with
Optimizely Full Stack? Reply
"YES" for a guide to best
practices!”
Variation A
Hypothesis: Having a clear call to action will increase
positive response rate.

opticon2017
Text “Hello” to
(415)-980-8918
to participate in our live experiment

opticon2017opticon2017
Collaborator
Email Experiment

opticon2017
Problem
• Email is being sent, but customers are not changing their password.
• Potential risk because if email is compromised, then Optimizely account can be compromised.
Potential solutions
• Send a reminder email.
• Have a call to action in the email.
• Link to the right place to make it easier for the user to change the password.
Hypothesis
More people will change password.

opticon2017
Original email Variation email

opticon2017
Hypothesis
We can use performance as a way to determine which tool to use.
Problem
Pick best tool to do cross-browser testing.

opticon2017opticon2017
Feature
Management

opticon2017
• Need to do code deploy to rollout feature.
• Need to do code deploy to roll back feature.
• Not easy to do feature randomization
amongst your users.
• No easy way to figure out feature access
amongst users.
• Generally, no access control.
• No one source of truth.
How we do feature management today

opticon2017
Takeaways
• Full Stack enables experimentation across platforms
• The Full Stack architecture allows for greater developer flexibility
• Experimentation can happen in any part of the stack

Optimizely announced new features to support experimentation on dynamic web applications. These include indefinite element detection using mutation observers, parallel execution of changes, and smarter page activation/deactivation. This allows experiments to work seamlessly across single page applications and sites with dynamic content updates, without requiring custom code. It also improves performance by reducing polling and timeouts. The new features are currently in beta testing and will make Optimizely better suited for modern web development without extra work from developers.

Opticon 2017 Day in the Life of a Modern Experimenter

Optimizely

The document outlines challenges experimenters face in scaling experimentation programs and building experiments on dynamic content. It then summarizes upcoming innovations in Optimizely's product including tools to manage the experimentation process, run multiple experiments simultaneously without interaction effects, extend experiments through custom features, measure important business metrics, develop experiments more flexibly on desktop and single page apps, gain insights into program performance, and accelerate results with artificial intelligence.

Opticon 2017 Pushing the Boundaries of Experimentation

Optimizely

This document outlines how Optimizely's APIs and extensions can be used to push the boundaries of experimentation. It discusses enabling automation and collaboration with Optimizely's APIs, how Atlassian automates workflows, and Gap's journey to personalization. The APIs allow users to automate experiment creation and management, monitor results, generate audiences, and integrate with other systems. Extensions offer reusable templates to reduce development time and enable advanced experiments. Recommendations include upgrading to Optimizely X, using the event endpoint for performance, integrating the REST API, and leveraging extensions.

Opticon 2017 Advanced Program Management

Optimizely

A successful experimentation program involves assessing opportunities, prioritizing experiments, and measuring program results. It assesses potential experiments by calculating expected value based on win rate, average lift, and revenue impact. Experiments are prioritized using a scoring model addressing business strategy, customer impact, opportunity size, and implementation effort. The program is measured using metrics like testing velocity, win rate, and revenue impact to track progress and identify improvement opportunities. Regular assessment allows the program to optimize operations and maximize overall expected value.

Developer Night Opticon 2017

Optimizely

Optimizely NYC Developer Meetup - Experimentation at Blue Apron

Optimizely

For a company like Blue Apron that is radically transforming the way we buy, prepare and eat meals, experimentation is mission critical for delivering a great customer experience. Blue Apron doesn’t just think about experimenting to improve short term conversion, they focus on ways to impact longer term metrics like retention, referrals, and lifetime value. John will take us through the journey of how Blue Apron built their experimentation program on top of Optimizely’s Full Stack platform. Presented at Optimizely NYC Developer Meetup by John Cline, Engineering Lead, Growth at Blue Apron on November 7, 2017

Opticon 2017 Decisions at Scale

Optimizely

This document discusses how an experimentation program manager, Kristen, can empower teams within her organization to make data-driven decisions through experimentation at scale. It outlines three key challenges Kristen faces: teams cannot measure what matters to them, cannot leverage experiment results, and lack confidence in results. The document then presents solutions from Optimizely's data platform to address each challenge by allowing teams to: 1) measure custom metrics that are important to their work, 2) analyze results through their preferred tools and workflows, and 3) gain confidence in results by reducing biases, discrepancies and false discoveries. This will help Kristen grow and maintain adoption of her organization's experimentation program.

AMC Networks Experiments Faster on the Server Side

Optimizely

Speeding up innovation only matters if it helps you drive positive outcomes. At AMC, experimentation enables the product and platform teams to challenge their assumptions, maximize impact, and evaluate ideas as painted door tests before investing in significant development. A commitment to test everything across 9 platforms fueled their search for the most scalable solution. In this session, you'll learn how to: Leverage server-side testing to experiment quickly Scale across web, mobile, and OTT applications Determine when client-side testing is more efficient

This document summarizes a presentation about how DonorsChoose.org uses a lean experimentation approach with Optimizely to test different optimizations on their website. Some key points: - DonorsChoose.org tests different designs and features using Optimizely to improve conversion rates and increase donations, including testing default sort options and text vs button forms. - They test optimizations specifically for days with media-driven traffic, which typically sees lower conversion rates. Tests of different splash pages and form designs increased donations by 5-10%. - Using Optimizely allows them to quickly test changes and make data-driven decisions to better achieve their mission of supporting public school teachers and students.

7 Habits of Highly Effective Personalisation Organisations | Optimizely ANZ W...

Optimizely

Watch Dan Ross, Managing Director for Optimizely ANZ in our latest webinar from the Experimentation Insights Tour -- "7 Habits of Highly Effective Personalisation Organisations” Watch the webinar here: https://optimizely.wistia.com/medias/cun66mnkwt Take Optimizely's Maturity Assessment here: https://www.optimizely.com/maturity-model/ DESCRIPTION: Create a data-driven culture and affect business decisions at the broader company level. When most people think of experimentation or testing, they think of sales and marketing. However, to do real customer experience optimisation, you need to think about all the ways your customers are interacting with you. The right mix to support building your programme into a centre of excellence is critical: you need a team that helps create a data-driven culture. Watch this webinar so you can: * Think more deeply about the future of your program and the makeup of your team * Consider which hard and soft skill sets your testing organisation needs * Build a well-rounded optimisation team that is visible, sustainable, and efficient About Optimizely Optimizely is the world's leading experimentation platform, enabling businesses to deliver continuous experimentation and personalisation across websites, mobile apps and connected devices. Optimizely enables businesses to experiment deeply into their technology stack and broadly across the entire customer experience. The platform’s ease of use and speed of deployment empower organisations to create and run bold experiments that help them make data-driven decisions and grow faster. To date, marketers, developers and product managers have delivered over 700 billion experiences tailored to the needs of their customers. Optimizely’s global client base includes Atlassian, eBay, Fox, IBM, The New York Times, LendingClub, Hotwire, Microsoft and many more leading businesses. To learn more about customer experience optimisation, visit optimizely.com

The Future of Optimizely for Technical Teams

Optimizely

How Optimizely Uses Optimizely - Opticon 2017

Takeshi Young

Opticon 2017 Hooked: How to Succeed

Optimizely

This document summarizes two presentations from the opticon2017 conference. The first presentation discusses driving deeper engagement to retain subscribers for Showtime and increasing consumption of Showtime programming. It also presents three case studies on personalization techniques that led to increased subscription leads and order clicks. The second presentation discusses the speaker's experience over 24 months optimizing at WSJ, emphasizing the importance of testing, prioritization, and quantifying the value of their work. It also provides examples of low-hanging fruit tests around acquisition and engagement that yielded substantial lift.

Opticon 2017 Beyond CRO

Optimizely

The document discusses Blue Apron's product and experiments with expanding their customer base. It summarizes Blue Apron's operations including demand forecasting, ingredient sourcing, recipe creation, purchasing and shipping. It then discusses moving into new lifestyle fits by offering additional recipes, two recipe plans per order, and modifying recipes on each order. The last section previews a panel discussion on experimenting in large product updates at Blue Apron to test theories of increasing order rates and reducing churn while managing risks of complexity and revenue loss.

Opticon 2017 Do the Thing That Makes the Money

Optimizely

The document discusses how data-driven experimentation and testing is a core strategy used by successful companies like Amazon, Facebook, and others to understand customer wants and drive growth. It emphasizes making testing and experimentation a priority to identify the most important metrics and accelerate learning. Specific tips include prioritizing high impact tests, increasing experiment throughput, and developing competencies like driving a culture of experimentation to maximize returns from a strategic testing program. Examples show how A/B tests of small variations can have large impacts on important metrics like conversions and highlight how data often outperforms intuition for decision making.

Test Everything: TrustRadius Delivers Customer Value with Experimentation

Optimizely

When done right, experimentation can help you validate the product you’re building and create winning customer experiences. And it doesn’t take a big engineering team to make this happen. TrustRadius, the most trusted review site for business technology, uses experimentation to build an online community through website and server-side experimentation. The small but mighty TrustRadius team runs experiments throughout the buyer’s journey to engage different user personas and understand outcomes in real-time. Watch the webinar recording featuring Rilo Stark, product manager at TrustRadius, and Jack Peden, senior software engineer, to understand their data-driven experimentation strategy and how TrustRadius uses Optimizely Web and Full Stack products to tailor experiences to different customer segments and mitigate risk through A/B/N and painted door tests.

Opticon 2017 How Optimizely Uses Optimizely

Optimizely

Optimizely uses Optimizely to run experiments on various pages of their website like the homepage, glossary pages, resources, and login page. They experiment with content, layouts, and personalization to meet objectives like increasing engagement, conversions, and reducing costs. The future will see Optimizely running more experiments by making experimentation easier for all teams through tools that integrate with their content management system and enable experiments without developer involvement. They aim to get everyone in the organization involved in experimentation.

Opticon 2017 - Building a Culture of Experimentation at IBM

Optimizely

1) The document discusses building a culture of experimentation at IBM by transitioning from limited testing to widespread adoption of data-driven experimentation across its 5,000 marketers. 2) It details challenges faced early on when only a few tests were run and results not fully utilized, and how the program has since scaled to over 1,000 active tests across business units and countries. 3) Lessons learned include the need for clear roles and metrics to define success, and building a culture where experimentation is a normal part of the process and failure is accepted, in order to continuously learn and improve outcomes.

Optimizely Agent: Scaling Resilient Feature Delivery

Optimizely

Opticon 2017 Cracking the Code

Optimizely

The document appears to be from a presentation about optimizing marketing campaigns. It discusses testing different hypotheses to improve conversion rates, such as adding social proof elements or creating a sense of urgency. It also covers understanding customer motivations and mapping them to the appropriate marketing messages. Various optimization strategies are listed, suggesting they could be tested, such as changing button colors, headlines, or page layouts.

Apply A/B Testing with NGINX Routing Policy

Supachai Jaturaprom

A/B Testing for WordPress & Drupal

Optimizely

This document discusses experimentation and A/B testing. It provides an overview of what A/B testing is and how to effectively conduct experiments. Key points include: - A/B testing involves comparing two variations of a webpage to see which performs better. - To set up an experiment, you need analytics tools, a hypothesis to test, ways to create test variations, and metrics to analyze results. - Case studies show how experiments can significantly impact goals like reducing returns or increasing signups. - Platforms like Optimizely make experimentation easy for both technical and non-technical users.

Optimizely X Seminar Amsterdam Nov 10

Optimizely

This document introduces Optimizely X, an experimentation platform that has evolved from an A/B testing tool to a full-stack experimentation platform. It provides experimentation capabilities for marketers and developers, including Optimizely X Web for experimenting on websites, Optimizely X Full Stack for experimenting across technologies like Python, Java and Node.js, and Optimizely X Mobile for mobile apps. The document highlights key features and capabilities through case studies and demos.

Opticon 2017 Driving Bottom Line Impact

Optimizely

Manan Singh, VP of Optimization at LendingTree, discusses how the company drives bottom line impact through mass-scale experimentation. He explains that they prioritize the biggest opportunities for improved ROI, run quick iterative experiments to learn fast, and measure success both quantitatively and qualitatively to understand why changes work. This comprehensive approach has helped LendingTree optimize its conversion funnel as a strategic initiative, with the stock price increasing 160% this year reflecting results translated to the bottom line.

Ahead of the Curve: How 23andMe Improved UX with Performance Edge

Optimizely

This document summarizes a webinar presented by Courtney Ball, Antonio Contreras, and Whelan Boyd from 23andMe on how the company improved user experience through A/B testing and implemented Performance Edge to optimize page load times. The webinar discussed how website performance directly impacts business metrics like conversion rates, highlighted best practices for measuring performance, and explained how standard A/B testing can negatively affect page speeds. It then introduced Performance Edge as a solution that delivers only necessary code variations to visitors, significantly reducing file sizes and browser execution times. 23andMe's case study showed how implementing Performance Edge through Cloudflare Workers dramatically improved their page load performance metrics and allowed them to scale up testing without slowing down

Overcoming the Challenges of Experimentation on a Service Oriented Architecture

Optimizely

Growing from an early stage startup to a national leader in financial literacy is no small feat, and there are a ton of lessons that we have learned at Greenlight as we have grown. Long gone are the days where we would ship something and cross our fingers hoping that it makes some kind of impact on our customers. Now we’re in a world where we can learn ahead of time how much impact a feature will have on the business, before we even launch! In today’s conversation, we’ll discuss how we use Optimizely’s feature flags in our microservice architecture using Optimizely Agent while keeping user IDs and context synchronized. This session will cover: How we set up Optimizely Agent and use it in a kubernetes deployment How we created a user-aliasing service How we access Optimizely both on the frontend and in the backend services. How to build a full stack feature How to manage the rollout using Optimizely’s feature flags

Opticon 2017 From CRO to Experience

Optimizely

I. Clearhead was acquired by Accenture Interactive on July 12, 2017. The document discusses Matty Wishnow, CEO of Clearhead, and his experience optimizing and personalizing customer experiences over 300,000 hours from CRO to full-stack optimization. II. It discusses the challenges of experience optimization and why more experiments sometimes do not lead to better results. It also discusses how conversion optimization differs from experience optimization. III. The document presents frameworks and approaches for solving problems through experiments, including Problem Solution Mapping and mapping the full customer experience across stages and channels to increase optimization success.

August 2017 GetFiveStars Customer Webinar

Don Campbell

This webinar covered new features for customer experience including: 1. New feedback flows that allow businesses to focus on feedback, surveys, or reviews. 2. The ability to directly respond to Google reviews from within the Google Forms platform. 3. A new faster JavaScript-based testimonials widget that works across all devices. 4. Using Google as your business' home page to better manage local presence and feedback.

Java 8 - Gateway Drug or End of Line?

Garth Gilmour

The document is a presentation on functional coding in Java. It discusses various topics related to functional programming in Java including functional interfaces, streams, lambda expressions, and reactive programming. It provides examples of coding techniques and annotations in Java 8 and later and compares Java to other languages that support functional programming like Scala. It also discusses libraries like Akka HTTP that can be used to build reactive applications on the JVM. The presentation aims to demonstrate how functional programming concepts and patterns can be applied in Java projects.

Continuous delivery @wcap 5-09-2013

David Funaro

What's hot

Opticon 2017 Achieving Success with a Lean Experimentation Team

Optimizely

7 Habits of Highly Effective Personalisation Organisations | Optimizely ANZ W...

Optimizely

The Future of Optimizely for Technical Teams

Optimizely

How Optimizely Uses Optimizely - Opticon 2017

Takeshi Young

Opticon 2017 Hooked: How to Succeed

Optimizely

Opticon 2017 Beyond CRO

Optimizely

Opticon 2017 Do the Thing That Makes the Money

Optimizely

Test Everything: TrustRadius Delivers Customer Value with Experimentation

Optimizely

Opticon 2017 How Optimizely Uses Optimizely

Optimizely

Opticon 2017 - Building a Culture of Experimentation at IBM

Optimizely

Optimizely Agent: Scaling Resilient Feature Delivery

Optimizely

Opticon 2017 Cracking the Code

Optimizely

Apply A/B Testing with NGINX Routing Policy

Supachai Jaturaprom

A/B Testing for WordPress & Drupal

Optimizely

Optimizely X Seminar Amsterdam Nov 10

Optimizely

Opticon 2017 Driving Bottom Line Impact

Optimizely

Ahead of the Curve: How 23andMe Improved UX with Performance Edge

Optimizely

Overcoming the Challenges of Experimentation on a Service Oriented Architecture

Optimizely

Opticon 2017 From CRO to Experience

Optimizely

August 2017 GetFiveStars Customer Webinar

Don Campbell

What's hot (20)

Opticon 2017 Achieving Success with a Lean Experimentation Team

7 Habits of Highly Effective Personalisation Organisations | Optimizely ANZ W...

The Future of Optimizely for Technical Teams

How Optimizely Uses Optimizely - Opticon 2017

Opticon 2017 Hooked: How to Succeed

Opticon 2017 Beyond CRO

Opticon 2017 Do the Thing That Makes the Money

Test Everything: TrustRadius Delivers Customer Value with Experimentation

Opticon 2017 How Optimizely Uses Optimizely

Opticon 2017 - Building a Culture of Experimentation at IBM

Optimizely Agent: Scaling Resilient Feature Delivery

Opticon 2017 Cracking the Code

Apply A/B Testing with NGINX Routing Policy

A/B Testing for WordPress & Drupal

Optimizely X Seminar Amsterdam Nov 10

Opticon 2017 Driving Bottom Line Impact

Ahead of the Curve: How 23andMe Improved UX with Performance Edge

Overcoming the Challenges of Experimentation on a Service Oriented Architecture

Opticon 2017 From CRO to Experience

August 2017 GetFiveStars Customer Webinar

Similar to Opticon 2017 How Developers Can Take Experimentation

Java 8 - Gateway Drug or End of Line?

Garth Gilmour

Continuous delivery @wcap 5-09-2013

David Funaro

Full stack conference talk slides

Sameer Al-Sakran

How to Embed Codeless Test Automation Into DevOps

Perfecto by Perforce

selenium_course_content.pdf

aswinisowmiya

Quality Jam 2017: Elise Carmichael and Corey Pyle "Jumpstarting Your Test Aut...

QASymphony

October 2018 ODTUG Webinar - Getting Started with Groovy in EPBCS

Kyle Goodfriend

Ionic Advisory: Your partner at every stage of development

Ionic Framework

Optimizely for Developers

Optimizely

Yahel Carmon, Director of Insight Produce, Blue State Digital James Fox, Solutions Architect, Optimizely Optimizely has powerful features, both exposed and under the hood, that make it easier for developers to implement sophisticated tests without re-inventing the wheel with each test. Yahel Carmon, Director of Product Insights at Blue State Digital, and James Fox, Solutions Architect at Optimizely, discuss the ways Optimizely makes developers' lives easier, giving demonstrations of the technical strengths of the platform and real-world examples of how to leverage Optimizely to manage an online testing program. Learn how to implement Optimizely the way advanced users and developers do, and ensure you're not missing out on its hidden gems.

DevOps Fest 2020. Kohsuke Kawaguchi. GitOps, Jenkins X & the Future of CI/CD

DevOps_Fest

CI/CD process has been something your DevOps engineer purpose-built for your team. But with Kubernetes & cloud-native, that’s becoming “legacy.” The rising level of platform abstraction allows all the good practices that the industry has developed over time to be integrated, hidden, and simplified behind just one practice called “GitOps.” That simplified world is what Jenkins X enables. We will discuss GitOps, Jenkins X, and how that combination drastically simplifies cloud-native web app development. You’ll understand why traditional DevOps is not suitable in a Kubernetes and cloud-native world, explore GitOps principles and discover how they facilitate high-velocity app development. And finally, Kohsuke will make a fool of himself by talking about the future — now that Jenkins X simplifies the CD process, where is the next frontier?

Testing for Logic App Solutions | Integration Monday

BizTalk360

Usable Software Design

Alexandru Bolboaca

Innovate Better Through Machine data Analytics

Hal Rottenberg

5 Steps To Deliver The Fastest Mobile Shopping Experience This Holiday Season

G3 Communications

Xcode eXtreme Programming - #pragmamark 2014, Milan

Giulio Roggero

Speed up the development and increase the app quality are the keywords for success. Good points, not so simple to achieve…. eXtreme Programming (XP) is an agile discipline of software development based on values of simplicity, communication, feedback, courage, and respect. The software is built around the needs of the customer through a continuous release of working software and creating a learning loop that dramatically improves the quality of the final product. Some XP practices, like TDD and Continuous Integration, can benefit of the support of software tools and frameworks. In this session we will see how XCTest and Xcode Continuous Integration can streamline the process of the iOS XP team.

FEDSPUG April 2014: Visual Studio 2013 for Application Lifecycle Management &...

WSPDC & FEDSPUG

This document summarizes new features in Visual Studio 2013, including improved debugging tools, better ASP.NET support that unifies Web Forms and MVC, and new capabilities for building and deploying SharePoint apps. It also outlines updates to .NET like asynchronous programming improvements and cloud-focused features, as well as how Visual Studio and Azure integration enhances development workflows.

SigOpt at MLconf - Reducing Operational Barriers to Model Training

SigOpt

Alexandra johnson reducing operational barriers to model training

MLconf

This document discusses reducing operational barriers to machine learning model training through building machine learning infrastructure. It presents challenges faced by both machine learning experts and infrastructure engineers. It then describes SigOpt's solution of building SigOpt Orchestrate to address these challenges through containerization, Kubernetes for parallel training, and a command line interface for viewing progress and debugging. The final slides invite connecting with SigOpt and note they are hiring.

[AI] ML Operationalization with Microsoft Azure

Korkrid Akepanidtaworn

I am an instructor of the MLOps workshop for some anonymous startup incubation program where the objectives are (1) to orchestrate and deploy updates to the application and the deep learning model in a unified way. (2) To design a DevOps pipeline to coordinate retrieving the latest best model from the model registry, packaging the web application, deploying the web application and inferencing web service.

Using BladeRunnerJS to Build Front-End Apps that Scale - Fluent 2014

Phil Leggetter

Developing large apps is difficult. Ensuring that code is consistent, well structured, tested and has an architecture that encourages enhancement and maintainability is essential. When it comes to building large server-focused apps the solutions to this problem have been tried and tested. But, how do you achieve this when building HTML5 single page apps? BladeRunnerJS is an open source developer toolkit and lightweight front-end framework that has helped Caplin Systems ensure that a 200k LoC JavaScript codebase hasn’t become a tangled mess of unstable spaghetti code. This codebase is packaged and delivered to customers as an SDK. Additionally customers receive a getting started application of around 50k LoC for them to build upon, and they’re expected not to turn that into a tangled … you get the idea. In this talk you’ll learn the main concepts to apply when building a front-end app that scales and how BladeRunnerJS can support the development process.

Similar to Opticon 2017 How Developers Can Take Experimentation (20)

Java 8 - Gateway Drug or End of Line?

Continuous delivery @wcap 5-09-2013

Full stack conference talk slides

How to Embed Codeless Test Automation Into DevOps

selenium_course_content.pdf

Quality Jam 2017: Elise Carmichael and Corey Pyle "Jumpstarting Your Test Aut...

October 2018 ODTUG Webinar - Getting Started with Groovy in EPBCS

Ionic Advisory: Your partner at every stage of development

Optimizely for Developers

DevOps Fest 2020. Kohsuke Kawaguchi. GitOps, Jenkins X & the Future of CI/CD

Testing for Logic App Solutions | Integration Monday

Usable Software Design

Innovate Better Through Machine data Analytics

5 Steps To Deliver The Fastest Mobile Shopping Experience This Holiday Season

Xcode eXtreme Programming - #pragmamark 2014, Milan

FEDSPUG April 2014: Visual Studio 2013 for Application Lifecycle Management &...

SigOpt at MLconf - Reducing Operational Barriers to Model Training

Alexandra johnson reducing operational barriers to model training

[AI] ML Operationalization with Microsoft Azure

Using BladeRunnerJS to Build Front-End Apps that Scale - Fluent 2014

More from Optimizely

Clover Rings Up Digital Growth to Drive Experimentation

Optimizely

Monil Shah from Clover presented on how Clover uses experimentation to drive digital growth. Clover started with walk experiments to test small changes and validate hypotheses. They then increased their experiment velocity by prioritizing high impact experiments and defining success metrics upfront. Clover also developed processes to conclude experiments early if clearly winning or losing, and to iterate based on experiment learnings. Clover evangelized experimentation across the company by finding executive sponsors, involving multiple teams, and educating and incentivizing experimentation.

Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...

Optimizely

Chris Stein is a lead consultant who helps enterprises determine ROI forecasts, pricing strategies, timelines, and resource needs. He has experience as a product manager for fintech SaaS solutions and has managed core product development. Some of his past roles include leading BPO and change management projects, spearheading agile project management processes, and inspiring customers to build experimentation as a strategic part of their business at Optimizely. Chris provides strategic consulting in digital experience optimization, program structures, and experiment-driven product development for enterprise customers.

The Science of Getting Testing Right

Optimizely

Atlassian's Mystique CLI, Minimizing the Experiment Development Cycle

Optimizely

Mystique CLI is an Atlassian developed CLI for Optimizely Web. It is a multi-phase project that is currently focusing on improving the development cycle for growth engineers. Currently, Mystique is the standard for developing web experiments at Atlassian, and is capable of a wide variety of operations utilizing Optimizely's REST API. This includes creating, updating, testing, and duplicating experiments/personalization campaigns, as well as "promoting" these entities between Optimizely projects for different environments (e.g. from QA => Prod). It has significantly reduced manual overhead and decreased development time by up to 95% for particular actions.

Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...

Optimizely

Autotrader's Product and Engineering teams were ahead of the curve many years ago when they built a home-grown solution for leveraging feature flags to support server-side testing. Over the years, the industry eventually caught up and surpassed this proprietary tooling and the team had a choice to make: Re-invest into the local solution or completely retool. In this case study, Scott Povlot, Principal Technical Architect, and Seth Stuck, Director of R&D Analytics, will discuss their journey in selecting and then migrating to their next generation of experimentation tooling. They will discuss selection criteria, pros and cons, and outline how they were able to make the migration to Optimizely successful and lessons learned along the way.

Zillow + Optimizely: Building the Bridge to $20 Billion Revenue

Optimizely

Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...

Optimizely

Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...

Optimizely

Building an Experiment Pipeline for GitHub’s New Free Team Offering

Optimizely

Evolving Experimentation from CRO to Product Development

Optimizely

An obsession with data, efficiency, and delivering incredible customer experiences are just a few things that the CNN Consumer Science and Software Engineering teams have in common. Simple A/B testing practices evolved into a culture of experimentation, sparking new development practices across the organization. Learn how they drive results across their entire platform from websites to mobile apps.

How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...

Optimizely

A/B testing is an essential element in any product managers playbook. However having the freedom and flexibility to customize testing based on what the data is saying often requires a lot of time and effort, particularly when it comes to engineering resources. Optimizely offers a flexible approach to experimentation through the use of feature testing, which provides more customization options without the additional development effort typically required to implement these feature optimizations. Megan Bubley, a Senior Product Manager at The Zebra, will share her experience working with Optimizely’s feature tests to create a results page where users can compare multiple auto insurance options driven by actual user needs, as well as her experience customizing the experience based on device platform.

Making Your Hypothesis Work Harder to Inform Future Product Strategy

Optimizely

At Treatwell, each experiment goes beyond improving a single business metric. Experimentation works to evolve their product while enriching customer insights in order to deliver the best digital experience to their users. Join Laura Howard, Lead Product Manager, and Dennis Meisner, Senior Product Analyst, to learn their secret to making their hypothesis work harder and how getting their hypothesis right has improved Treatwell’s funnel progression and order health, as well as helped them make critical decisions on their product experience.

Kick Your Assumptions: How Scholl's Test-Everything Culture Drives Revenue

Optimizely

Amy Vetter, Consumer Experience Manager, Direct To Consumer, Europe, will walk you through some of the tests that she and her team run across the Scholl brand. Amy will highlight surprise learnings and how to remove the fear of failing. The team is empowered to test everything possible that will allow the customer to get the best experience and also support the brand’s goal for more revenue and customer data.

Experimentation through Clients' Eyes

Optimizely

Shipping to Learn and Accelerate Growth with GitHub

Optimizely

Will 2020 mark the shift to a remote-first world in the long run? For GitHub, a distributed workforce is nothing new. Join Sha Ma, VP of Engineering, and Gregory Ceccarelli, Director of Data Science, to learn how they built and scaled a successful experimentation program. They'll share their experience implementing Optimizely across timezones, a remote workforce, and a new business model. In this session, you'll learn how to: Optimize UX for a freemium business model Use data to deliver customer-centered products Scale experimentation and accelerate growth

The Future of Software Development

Optimizely

Newly appointed Optimizely CTO, Lawrence Bruhmuller, will kick off Developer Summit discussing the new normals in software development. After decades of leading and scaling engineering teams for high growth startups and large tech companies, Lawrence has seen the same problems crop up repeatedly for technical teams. There is a new way of delivering software that makes it possible to move fast and get it right. That new way is Progressive Delivery & Experimentation. When Progressive Delivery & Experimentation are used together, you have an efficient system for validating both quality and customer engagement across your development lifecycle. Lawrence will discuss the key principles driving software development innovation, how our engineering team puts this into practice, and the success he’s seen at other companies.

Practical Use Case: How Dosh Uses Feature Experiments To Accelerate Mobile De...

Optimizely

Engineering organizations at companies know to anticipate bugs when they are about to launch a new product but, what tools can they use to reduce the blast radius and mitigate potential risks? Now, companies are thinking about preventative methods and safeguards they can put in place to make sure they deliver frictionless experiences to their customers with measurable results. In this session, you'll learn how to use feature flags and experiments across your stack (including mobile apps) to safely release meaningful features to your customers.

Run High Impact Experimentation with High-quality Customer Discovery

Optimizely

Developing a robust A/B/n testing program is critical for every product organization. But there is not a one size fits all approach. Your program must be tailored to your business and audience to be successful. Join Oji Udezue, VP of Product at Calendly, who will share how to craft high-impact customer insights and product experiments, tailored for your company. Oji will draw insights from 20+ years of Product Management at companies like Microsoft and Atlassian and discuss the importance of using qualitative methods to help develop smarter hypotheses with the right tools and practices for your organization. In this session you will learn: - Why a strong experimentation culture is important for your product organization - Proven ways to execute high-impact A/B/n tests with high-quality customer discovery - The importance of conducting interviews with successful and unsuccessful customers and how to see through your customer’s eyes with live watching techniques

Using Empathy to Build Custom Solutions at Scale

Optimizely

Segment supports a huge ecosystem of partners and platforms, but there are always requests for more customizable options. What if, instead of answering each custom request, we empower customers and partners to build exactly what they’re looking for? Join Product Manager Andy Yeo as he discusses the ideas, conversations, testing, and validation that brought their latest product release—Functions—to life. When you attend this webinar, you will: - Learn how product teams go from customer feedback to shipping features - Walk away with tactics for testing and iterating to launch features that solve customer needs - Hear an exclusive preview of an upcoming product launch at Segment

How to find data insights that will drive a 10X impact

Optimizely

Your product is the most important influence on your user experience and lifetime value. The ability to continually learn from your product changes and maximize your outcomes is a critical indicator of driving revenue for your business. Don't be complacent with incremental changes, learn how to find insights that will drive a 10x impact for your business. Hear from Tanner McGrath, Head of Product Analytics, and Bilal Mahmood, Head of Product from Amplitude cover: - Using your product as the influencer for your user’s experience and lifetime value. - Learning from your product changes and maximize your outcomes to drive revenue for your business - Finding insights that will drive a 10x impact for your business

More from Optimizely (20)

Clover Rings Up Digital Growth to Drive Experimentation

Make Every Touchpoint Count: How to Drive Revenue in an Increasingly Online W...

The Science of Getting Testing Right

Atlassian's Mystique CLI, Minimizing the Experiment Development Cycle

Autotrader Case Study: Migrating from Home-Grown Testing to Best-in-Class Too...

Zillow + Optimizely: Building the Bridge to $20 Billion Revenue

Empowering Agents to Provide Service from Anywhere: Contact Centers in the Ti...

Experimentation Everywhere: Create Exceptional Online Shopping Experiences an...

Building an Experiment Pipeline for GitHub’s New Free Team Offering

Evolving Experimentation from CRO to Product Development

How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...

Making Your Hypothesis Work Harder to Inform Future Product Strategy

Kick Your Assumptions: How Scholl's Test-Everything Culture Drives Revenue

Experimentation through Clients' Eyes

Shipping to Learn and Accelerate Growth with GitHub

The Future of Software Development

Practical Use Case: How Dosh Uses Feature Experiments To Accelerate Mobile De...

Run High Impact Experimentation with High-quality Customer Discovery

Using Empathy to Build Custom Solutions at Scale

How to find data insights that will drive a 10X impact

Recently uploaded

What is an RPA CoE? Session 1 – CoE Vision

DianaGray10

AppSec PNW: Android and iOS Application Security with MobSF

Ajin Abraham

Mobile Security Framework - MobSF is a free and open source automated mobile application security testing environment designed to help security engineers, researchers, developers, and penetration testers to identify security vulnerabilities, malicious behaviours and privacy concerns in mobile applications using static and dynamic analysis. It supports all the popular mobile application binaries and source code formats built for Android and iOS devices. In addition to automated security assessment, it also offers an interactive testing environment to build and execute scenario based test/fuzz cases against the application. This talk covers: Using MobSF for static analysis of mobile applications. Interactive dynamic security assessment of Android and iOS applications. Solving Mobile app CTF challenges. Reverse engineering and runtime analysis of Mobile malware. How to shift left and integrate MobSF/mobsfscan SAST and DAST in your build pipeline.

Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels

Northern Engraving

"Scaling RAG Applications to serve millions of users", Kevin Goedecke

Fwdays

[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...

Jason Yip

The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.

"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk

Fwdays

At this talk we will discuss DDoS protection tools and best practices, discuss network architectures and what AWS has to offer. Also, we will look into one of the largest DDoS attacks on Ukrainian infrastructure that happened in February 2022. We'll see, what techniques helped to keep the web resources available for Ukrainians and how AWS improved DDoS protection for all customers based on Ukraine experience

AI in the Workplace Reskilling, Upskilling, and Future Work.pptx

Sunil Jagani

Christine's Supplier Sourcing Presentaion.pptx

christinelarrosa

PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx

christinelarrosa

Essentials of Automations: Exploring Attributes & Automation Parameters

Safe Software

Building automations in FME Flow can save time, money, and help businesses scale by eliminating data silos and providing data to stakeholders in real-time. One essential component to orchestrating complex automations is the use of attributes & automation parameters (both formerly known as “keys”). In fact, it’s unlikely you’ll ever build an Automation without using these components, but what exactly are they? Attributes & automation parameters enable the automation author to pass data values from one automation component to the next. During this webinar, our FME Flow Specialists will cover leveraging the three types of these output attributes & parameters in FME Flow: Event, Custom, and Automation. As a bonus, they’ll also be making use of the Split-Merge Block functionality. You’ll leave this webinar with a better understanding of how to maximize the potential of automations by making use of attributes & automation parameters, with the ultimate goal of setting your enterprise integration workflows up on autopilot.

Mutation Testing for Task-Oriented Chatbots

Pablo Gómez Abajo

Conversational agents, or chatbots, are increasingly used to access all sorts of services using natural language. While open-domain chatbots - like ChatGPT - can converse on any topic, task-oriented chatbots - the focus of this paper - are designed for specific tasks, like booking a flight, obtaining customer support, or setting an appointment. Like any other software, task-oriented chatbots need to be properly tested, usually by defining and executing test scenarios (i.e., sequences of user-chatbot interactions). However, there is currently a lack of methods to quantify the completeness and strength of such test scenarios, which can lead to low-quality tests, and hence to buggy chatbots. To fill this gap, we propose adapting mutation testing (MuT) for task-oriented chatbots. To this end, we introduce a set of mutation operators that emulate faults in chatbot designs, an architecture that enables MuT on chatbots built using heterogeneous technologies, and a practical realisation as an Eclipse plugin. Moreover, we evaluate the applicability, effectiveness and efficiency of our approach on open-source chatbots, with promising results.

"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba

Fwdays

This is a session that details how PostgreSQL's features and Azure AI Services can be effectively used to significantly enhance the search functionality in any application. In this session, we'll share insights on how we used PostgreSQL to facilitate precise searches across multiple fields in our mobile application. The techniques include using LIKE and ILIKE operators and integrating a trigram-based search to handle potential misspellings, thereby increasing the search accuracy. We'll also discuss how the azure_ai extension on PostgreSQL databases in Azure and Azure AI Services were utilized to create vectors from user input, a feature beneficial when users wish to find specific items based on text prompts. While our application's case study involves a drug search, the techniques and principles shared in this session can be adapted to improve search functionality in a wide range of applications. Join us to learn how PostgreSQL and Azure AI can be harnessed to enhance your application's search capability.

Christine's Product Research Presentation.pptx

christinelarrosa

Dandelion Hashtable: beyond billion requests per second on a commodity server

Antonios Katsarakis

This slide deck presents DLHT, a concurrent in-memory hashtable. Despite efforts to optimize hashtables, that go as far as sacrificing core functionality, state-of-the-art designs still incur multiple memory accesses per request and block request processing in three cases. First, most hashtables block while waiting for data to be retrieved from memory. Second, open-addressing designs, which represent the current state-of-the-art, either cannot free index slots on deletes or must block all requests to do so. Third, index resizes block every request until all objects are copied to the new index. Defying folklore wisdom, DLHT forgoes open-addressing and adopts a fully-featured and memory-aware closed-addressing design based on bounded cache-line-chaining. This design offers lock-free index operations and deletes that free slots instantly, (2) completes most requests with a single memory access, (3) utilizes software prefetching to hide memory latencies, and (4) employs a novel non-blocking and parallel resizing. In a commodity server and a memory-resident workload, DLHT surpasses 1.6B requests per second and provides 3.5x (12x) the throughput of the state-of-the-art closed-addressing (open-addressing) resizable hashtable on Gets (Deletes).

Northern Engraving | Nameplate Manufacturing Process - 2024

Northern Engraving

Manufacturing custom quality metal nameplates and badges involves several standard operations. Processes include sheet prep, lithography, screening, coating, punch press and inspection. All decoration is completed in the flat sheet with adhesive and tooling operations following. The possibilities for creating unique durable nameplates are endless. How will you create your brand identity? We can help!

Discover the Unseen: Tailored Recommendation of Unwatched Content

ScyllaDB

The session shares how JioCinema approaches ""watch discounting."" This capability ensures that if a user watched a certain amount of a show/movie, the platform no longer recommends that particular content to the user. Flawless operation of this feature promotes the discover of new content, improving the overall user experience. JioCinema is an Indian over-the-top media streaming service owned by Viacom18.

A Deep Dive into ScyllaDB's Architecture

ScyllaDB

Must Know Postgres Extension for DBA and Developer during Migration

Mydbops

Mydbops Opensource Database Meetup 16 Topic: Must-Know PostgreSQL Extensions for Developers and DBAs During Migration Speaker: Deepak Mahto, Founder of DataCloudGaze Consulting Date & Time: 8th June | 10 AM - 1 PM IST Venue: Bangalore International Centre, Bangalore Abstract: Discover how PostgreSQL extensions can be your secret weapon! This talk explores how key extensions enhance database capabilities and streamline the migration process for users moving from other relational databases like Oracle. Key Takeaways: * Learn about crucial extensions like oracle_fdw, pgtt, and pg_audit that ease migration complexities. * Gain valuable strategies for implementing these extensions in PostgreSQL to achieve license freedom. * Discover how these key extensions can empower both developers and DBAs during the migration process. * Don't miss this chance to gain practical knowledge from an industry expert and stay updated on the latest open-source database trends. Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL. Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability. Contact us: info@mydbops.com Visit: https://www.mydbops.com/ Follow us on LinkedIn: https://in.linkedin.com/company/mydbops For more details and updates, please follow up the below links. Meetup Page : https://www.meetup.com/mydbops-databa... Twitter: https://twitter.com/mydbopsofficial Blogs: https://www.mydbops.com/blog/ Facebook(Meta): https://www.facebook.com/mydbops/

"$10 thousand per minute of downtime: architecture, queues, streaming and fin...

Fwdays

Direct losses from downtime in 1 minute = $5-$10 thousand dollars. Reputation is priceless. As part of the talk, we will consider the architectural strategies necessary for the development of highly loaded fintech solutions. We will focus on using queues and streaming to efficiently work and manage large amounts of data in real-time and to minimize latency. We will focus special attention on the architectural patterns used in the design of the fintech system, microservices and event-driven architecture, which ensure scalability, fault tolerance, and consistency of the entire system.

Leveraging the Graph for Clinical Trials and Standards

Neo4j

Recently uploaded (20)

What is an RPA CoE? Session 1 – CoE Vision

AppSec PNW: Android and iOS Application Security with MobSF

Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels

"Scaling RAG Applications to serve millions of users", Kevin Goedecke

[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...

"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk

AI in the Workplace Reskilling, Upskilling, and Future Work.pptx

Christine's Supplier Sourcing Presentaion.pptx

PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx

Essentials of Automations: Exploring Attributes & Automation Parameters

Mutation Testing for Task-Oriented Chatbots

"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba

Christine's Product Research Presentation.pptx

Dandelion Hashtable: beyond billion requests per second on a commodity server

Northern Engraving | Nameplate Manufacturing Process - 2024

Discover the Unseen: Tailored Recommendation of Unwatched Content

A Deep Dive into ScyllaDB's Architecture

Must Know Postgres Extension for DBA and Developer during Migration

"$10 thousand per minute of downtime: architecture, queues, streaming and fin...

Leveraging the Graph for Clinical Trials and Standards

Opticon 2017 How Developers Can Take Experimentation

1. opticon2017 How Developers can take Experimentation to New Levels Michael Ng Software Eng, Optimizely Ali Rizvi Software Eng, Optimizely Josh Wang Software Eng, Optimizely

2. opticon2017 A story about experimentation

6. opticon2017 $100 million $50 million Mike Ng Business Impact

7. Technical Impact

8. opticon2017 How Developers can take experimentation to new levels with Full Stack • Architecture that allows developer flexibility • Experiment across platforms • Experiment deep in the stack

9. opticon2017opticon2017 Full Stack Experimentatio n

10. opticon2017 How experimentation works with Full Stack Your servers SDKs Results Experiment management datafile events Flexible SDKs Optimizely Dashboard A B Node Java Ruby Python PHP C# Javascript iOS Android User devices

11. opticon2017 Search algorithms Redesigns Feature rollouts Sorting algorithms Price changes Backend changes Use cases for Full Stack experimentation

12. opticon2017

13. opticon2017 SDK architecture: Built for flexibility • Stateless • Consistent implementation across SDKs • Non-blocking API calls • Replaceable components

14. opticon2017 Creating an SMS experiment with Full Stack

15. opticon2017 SMS experiment “Are you interested in learning more about Optimizely Full Stack?” Variation B “Want to learn how to experiment easily with Optimizely Full Stack? Reply "YES" for a guide to best practices!” Variation A Hypothesis: Having a clear call to action will increase positive response rate.

16. opticon2017 Text “Hello” to (415)-980-8918 to participate in our live experiment

17. opticon2017 Optimizely on Optimizely

18. opticon2017

19. opticon2017opticon2017 Collaborator Email Experiment

20. opticon2017

21. opticon2017 Problem • Email is being sent, but customers are not changing their password. • Potential risk because if email is compromised, then Optimizely account can be compromised. Potential solutions • Send a reminder email. • Have a call to action in the email. • Link to the right place to make it easier for the user to change the password. Hypothesis More people will change password.

22. opticon2017 Original email Variation email

23. opticon2017

24. opticon2017opticon2017 Tool Face-off

25.

26. opticon2017 VS Tools at play

27. opticon2017 Hypothesis We can use performance as a way to determine which tool to use. Problem Pick best tool to do cross-browser testing.

28.

29. opticon2017opticon2017 Feature Management

30. opticon2017 • Need to do code deploy to rollout feature. • Need to do code deploy to roll back feature. • Not easy to do feature randomization amongst your users. • No easy way to figure out feature access amongst users. • Generally, no access control. • No one source of truth. How we do feature management today

31.

32. opticon2017 Takeaways • Full Stack enables experimentation across platforms • The Full Stack architecture allows for greater developer flexibility • Experimentation can happen in any part of the stack

33. opticon2017 Q& A

34. opticon2017 THANK YOU

35. opticon2017