A summary of my 4 years of A/B and Split testing, with case studies of work, photography guidelines, and key advice on which elements of the page to test for quick wins.
I enjoyed giving this talk at the Online Retailer Conference in Sydney, which is a fine place to visit.
The presentation c.overs a really good 'pizza' analogy for explaining testing to senior management and budget holders. Then covering how you go about discovering good places to test on your site, and what tools will help you get that data.
Lastly, I explore what worked for me in testing, show some examples of how similar our winners are across the globe and then cover some cross channel testing. The last one here is a big growth area and involves optimising contact centres and channels, using the web as a tool. Some interesting work going on here and I show some of ours, as well as new things in the pipeline.
There are some great resources attached, including a list of remote user testing services and the best 'guides' I could find on 'Conversion Rate Optimisation'. Hope you enjoyed the talk and thank you Sydney.
Myths, Lies and Illusions of AB and Split TestingCraig Sullivan
What are the common assumptions about AB (split) testing that are wrong? What are the lies told by vendors, consultants and the stuff you have convinced yourself about. What is illusory - what can you trust - what's it really all about. 20 top myths debunked after asking fellow CRO professionals what is on THEIR top list.
#Measurecamp : 18 Simple Ways to F*** up Your AB TestingCraig Sullivan
An expanded deck of the top 18 blockers to getting successful AB or Multivariate test results. In this deck, you get a complete checklist of the stuff you need to prepare, watch, launch and monitor your testing, so it gets you the *right* conclusions.
The top reasons and solutions for not getting value out of your AB tests - some practical tips for designing insightful and correctly instrumented test
Myths and Illusions of Cross Device Testing - Elite Camp June 2015Craig Sullivan
A compendium of the most common mistakes and problems people encounter when trying to optimise or split test cross device experiences (mobile, tablet, desktop, app, tv etc.)
12 Things to do Before Your Company Dies : Conversion Conference London - Oct...Craig Sullivan
A roundup of all the things to help you maintain a competitive edge in experience design and conversion optimisation. With examples of companies putting this stuff together, the tools they are using and their project management approaches, this presentation delves deeper into the cultural aspects of CRO.
Myths, Lies and Illusions of AB and Split TestingCraig Sullivan
What are the common assumptions about AB (split) testing that are wrong? What are the lies told by vendors, consultants and the stuff you have convinced yourself about. What is illusory - what can you trust - what's it really all about. 20 top myths debunked after asking fellow CRO professionals what is on THEIR top list.
#Measurecamp : 18 Simple Ways to F*** up Your AB TestingCraig Sullivan
An expanded deck of the top 18 blockers to getting successful AB or Multivariate test results. In this deck, you get a complete checklist of the stuff you need to prepare, watch, launch and monitor your testing, so it gets you the *right* conclusions.
The top reasons and solutions for not getting value out of your AB tests - some practical tips for designing insightful and correctly instrumented test
Myths and Illusions of Cross Device Testing - Elite Camp June 2015Craig Sullivan
A compendium of the most common mistakes and problems people encounter when trying to optimise or split test cross device experiences (mobile, tablet, desktop, app, tv etc.)
12 Things to do Before Your Company Dies : Conversion Conference London - Oct...Craig Sullivan
A roundup of all the things to help you maintain a competitive edge in experience design and conversion optimisation. With examples of companies putting this stuff together, the tools they are using and their project management approaches, this presentation delves deeper into the cultural aspects of CRO.
Slides to go with a talk on rapid, lightweight research you can do before tackling a landing page, funnel step or lead-gen form. Comes complete with all the Google Analytics reports you'll need to mine useful data to share! Less bullshit, more truth in meetings!
Mobile presentation - Sydney Online Retailer - 26 Sep 2011Craig Sullivan
In this presentation, I use analytics data from our global mobile reach, to illustrate the trends that are driving growth, how to take opportunity from them and what to do with your own site. I present a case for device and user knowledge, to allow you to optimise conversion rates, revenue and delight for visitors.
Surviving the hype cycle Shortcuts to split testing successCraig Sullivan
In this talk, I show the key shortcuts to stop doing stupid testing and move towards innovative and transformative design & build methodologies, including innovation through split testing exploration
Surviving the AB Testing Hype Cycle - Reaktor Breakpoint 2015Craig Sullivan
My Slides from Reaktor Breakpoint 2015 - This is by far the best deck (and hopefully talk) I've done this year. Masses of info, reading, articles, useful reports and more.
Web Analytics Wednesday - Session Replay Tools are VitalCraig Sullivan
Session Replay or Screen Recording tools are now part of an arsenal of discovery toolkits that can drive optimisation, bug fixes, funnel and journey analysis - using qual and quant techniques. Without these tools, the analytics data misses emotion, frustration, friction and more - I've collated the best tips, tricks, tools and approaches to yield the most valuable insights for CRO / Growth Hacking.
The Neuromarketing Toolkit - Chinwag Psych - 4 Feb 2014Craig Sullivan
A practical toolkit for getting inside customers heads, in order to design and create persuasive psychological approaches to copy, pages, buttons, designs and your entire service. Craig shows you here how to mine what you already have - to design a better bank balance and continuously improving future for your company, staff and your customers.
Cross Device Optimisation - Google Analytics ShortcutsCraig Sullivan
In this session, we explain how to mine GA for broken device experiences, flows, funnel blocks and more... Using a new grid tool we've developed, you can pull multi-dimensional segmented funnel and metric data from Google Analytics - we explain how it works, why you need it and what problems it solves. Find where your site is leaking money through data
Brighton CRO Meetup #1 - Oh Boy These AB tests Sure Look Like Bullshit to MeCraig Sullivan
An updated deck of a short talk (30m) given at the first Brighton CRO meetup. Contains useful AB testing tools as well as full speaker notes for most of the slides.
Product design is Poo - And how to fix it!Craig Sullivan
A look at why product design is still so poor, even after 22 years of digital design work. Why do these problems exist and how can we remove them from the way we build products? Lean corporate and startup growth models are explored in the solutions to this horrendous problem!
20 Ways to Shaft your Split Tesring : Conversion ConferenceCraig Sullivan
This talk is the latest deck showing common problems that will easily break or skew your ab and multivariate testing results. Avoid these problems by following the simple advice in this deck!
Onboard like a juggernaut - Elite camp 2015Conversionista
Conversionista's presentation at Digital EliteCamp in Estonia 2105.
Find the critical conversion points in your SaaS user onboarding journey.
- Get Registered Prospects to use the service
- Get Active users to pay up
- Get Paying Customers to stay
- Get churned users to return
- Get all of them to refer more users
Do this - And rock!
SAMPLE SIZE – The indispensable A/B test calculation that you’re not makingZack Notes
If you’re a marketer it’s very likely that you’ve run an A/B test. It’s also likely that you’ve never calculated the sample size for your tests, and instead, you run tests until they reach statistical significance. If this is the case, your strategy is statistically flawed. Conforming to sample size requires marketers to wait longer for test results, but choosing to ignore it will bear false positives and lead to bad decisions.
This deck was created for an email audience for there are valuable lessons for anyone who runs A/B tests.
Product Design is Poo - And we're all going to dieCraig Sullivan
A humorous presentation about what is wrong with the current way of building digital products. Showing what is wrong, explaining the signs and giving you a checklist for reforming your company - are laid out with links, resources and further reading.
Why Does My Conversion Rate Suck? Craig Sullivan, Senior Optimisation Consult...PRWD
Craig Sullivan, Senior Optimisation Consultant covers the top 10 reasons why your conversion rate might suck. Packed with actionable tips and resources this presentation is for anyone wanting to improve their Conversion Optimisation. Craig covers common problems and topic areas such as issues with Google Analytics setup, inputs, tools, testing, testing cycles, product cycles, Photo UX, how to analyse statistics / Data, segmentation, multiple channel optimisation. The resource pack also include a maturity model, Crowd-sourced UX, collaborative tools, testing tools for CRO & QA, Belron Methodology example, and CRO and testing resources.
Talks@Coursera - A/B Testing @ Internet Scalecourseratalks
Talks@Coursera
This tech talk will describe how to build an experiment platform that can handle large-scale experiments. The talk will also discuss several best practices in designing and analyzing online experiments learned from companies like Coursera, Microsoft and LinkedIn.
About the Speakers
Ya Xu has been working in the domain of online A/B testing for over 4 years. She currently leads a team of engineers and data scientists building a world-class online A/B testing platform at LinkedIn. She also spearheads taking LinkedIn's A/B testing culture to the next level by evangelizing best practices and pushing for broad-based platform adoption. She holds a Ph.D. in Statistics from Stanford University.
Chuong (Tom) Do currently leads a team of data engineers and analysts in the Analytics team at Coursera, which is responsible for data infrastructure and quantitative analysis in support of the product and business. He completed his Ph.D. in Computer Science at Stanford University in 2009 and worked as a scientist in the personal genetics company 23andMe until 2012, where his research has collectively spanned the fields of machine learning, computational biology, and statistical genetics.
Hi everyone. Great to see you all and totally maxed out on all the conversion advice. Thank you and in return, here are the slides with extra resources added.
Cross channel testing insights and recommendations - jump - oct 12 2011 - w...Craig Sullivan
This presentation is about how to enable your website to optimise the *rest* of your business. There are recommendations here for how to instrument your web analytics, so you can see cross channel contact and touch points.
I've researched this thoroughly, and also give you a handy list of all the web analytics/call tracking companies of note. You'll find plenty of experiments to run with your call centre, and advice on how to track all of this crazy omni-channel and screen world.
If you do ONE thing with this information, promise me this please - I will implement tracking for some of my major 'other' channels that web drives, including phone, contact forms or chat.
Slides to go with a talk on rapid, lightweight research you can do before tackling a landing page, funnel step or lead-gen form. Comes complete with all the Google Analytics reports you'll need to mine useful data to share! Less bullshit, more truth in meetings!
Mobile presentation - Sydney Online Retailer - 26 Sep 2011Craig Sullivan
In this presentation, I use analytics data from our global mobile reach, to illustrate the trends that are driving growth, how to take opportunity from them and what to do with your own site. I present a case for device and user knowledge, to allow you to optimise conversion rates, revenue and delight for visitors.
Surviving the hype cycle Shortcuts to split testing successCraig Sullivan
In this talk, I show the key shortcuts to stop doing stupid testing and move towards innovative and transformative design & build methodologies, including innovation through split testing exploration
Surviving the AB Testing Hype Cycle - Reaktor Breakpoint 2015Craig Sullivan
My Slides from Reaktor Breakpoint 2015 - This is by far the best deck (and hopefully talk) I've done this year. Masses of info, reading, articles, useful reports and more.
Web Analytics Wednesday - Session Replay Tools are VitalCraig Sullivan
Session Replay or Screen Recording tools are now part of an arsenal of discovery toolkits that can drive optimisation, bug fixes, funnel and journey analysis - using qual and quant techniques. Without these tools, the analytics data misses emotion, frustration, friction and more - I've collated the best tips, tricks, tools and approaches to yield the most valuable insights for CRO / Growth Hacking.
The Neuromarketing Toolkit - Chinwag Psych - 4 Feb 2014Craig Sullivan
A practical toolkit for getting inside customers heads, in order to design and create persuasive psychological approaches to copy, pages, buttons, designs and your entire service. Craig shows you here how to mine what you already have - to design a better bank balance and continuously improving future for your company, staff and your customers.
Cross Device Optimisation - Google Analytics ShortcutsCraig Sullivan
In this session, we explain how to mine GA for broken device experiences, flows, funnel blocks and more... Using a new grid tool we've developed, you can pull multi-dimensional segmented funnel and metric data from Google Analytics - we explain how it works, why you need it and what problems it solves. Find where your site is leaking money through data
Brighton CRO Meetup #1 - Oh Boy These AB tests Sure Look Like Bullshit to MeCraig Sullivan
An updated deck of a short talk (30m) given at the first Brighton CRO meetup. Contains useful AB testing tools as well as full speaker notes for most of the slides.
Product design is Poo - And how to fix it!Craig Sullivan
A look at why product design is still so poor, even after 22 years of digital design work. Why do these problems exist and how can we remove them from the way we build products? Lean corporate and startup growth models are explored in the solutions to this horrendous problem!
20 Ways to Shaft your Split Tesring : Conversion ConferenceCraig Sullivan
This talk is the latest deck showing common problems that will easily break or skew your ab and multivariate testing results. Avoid these problems by following the simple advice in this deck!
Onboard like a juggernaut - Elite camp 2015Conversionista
Conversionista's presentation at Digital EliteCamp in Estonia 2105.
Find the critical conversion points in your SaaS user onboarding journey.
- Get Registered Prospects to use the service
- Get Active users to pay up
- Get Paying Customers to stay
- Get churned users to return
- Get all of them to refer more users
Do this - And rock!
SAMPLE SIZE – The indispensable A/B test calculation that you’re not makingZack Notes
If you’re a marketer it’s very likely that you’ve run an A/B test. It’s also likely that you’ve never calculated the sample size for your tests, and instead, you run tests until they reach statistical significance. If this is the case, your strategy is statistically flawed. Conforming to sample size requires marketers to wait longer for test results, but choosing to ignore it will bear false positives and lead to bad decisions.
This deck was created for an email audience for there are valuable lessons for anyone who runs A/B tests.
Product Design is Poo - And we're all going to dieCraig Sullivan
A humorous presentation about what is wrong with the current way of building digital products. Showing what is wrong, explaining the signs and giving you a checklist for reforming your company - are laid out with links, resources and further reading.
Why Does My Conversion Rate Suck? Craig Sullivan, Senior Optimisation Consult...PRWD
Craig Sullivan, Senior Optimisation Consultant covers the top 10 reasons why your conversion rate might suck. Packed with actionable tips and resources this presentation is for anyone wanting to improve their Conversion Optimisation. Craig covers common problems and topic areas such as issues with Google Analytics setup, inputs, tools, testing, testing cycles, product cycles, Photo UX, how to analyse statistics / Data, segmentation, multiple channel optimisation. The resource pack also include a maturity model, Crowd-sourced UX, collaborative tools, testing tools for CRO & QA, Belron Methodology example, and CRO and testing resources.
Talks@Coursera - A/B Testing @ Internet Scalecourseratalks
Talks@Coursera
This tech talk will describe how to build an experiment platform that can handle large-scale experiments. The talk will also discuss several best practices in designing and analyzing online experiments learned from companies like Coursera, Microsoft and LinkedIn.
About the Speakers
Ya Xu has been working in the domain of online A/B testing for over 4 years. She currently leads a team of engineers and data scientists building a world-class online A/B testing platform at LinkedIn. She also spearheads taking LinkedIn's A/B testing culture to the next level by evangelizing best practices and pushing for broad-based platform adoption. She holds a Ph.D. in Statistics from Stanford University.
Chuong (Tom) Do currently leads a team of data engineers and analysts in the Analytics team at Coursera, which is responsible for data infrastructure and quantitative analysis in support of the product and business. He completed his Ph.D. in Computer Science at Stanford University in 2009 and worked as a scientist in the personal genetics company 23andMe until 2012, where his research has collectively spanned the fields of machine learning, computational biology, and statistical genetics.
Hi everyone. Great to see you all and totally maxed out on all the conversion advice. Thank you and in return, here are the slides with extra resources added.
Cross channel testing insights and recommendations - jump - oct 12 2011 - w...Craig Sullivan
This presentation is about how to enable your website to optimise the *rest* of your business. There are recommendations here for how to instrument your web analytics, so you can see cross channel contact and touch points.
I've researched this thoroughly, and also give you a handy list of all the web analytics/call tracking companies of note. You'll find plenty of experiments to run with your call centre, and advice on how to track all of this crazy omni-channel and screen world.
If you do ONE thing with this information, promise me this please - I will implement tracking for some of my major 'other' channels that web drives, including phone, contact forms or chat.
Radically Improve Conversion Rates - eMSF 2009Kayden Kelly
See how to methodically and radically improve your conversion rate through quantitative and qualitative tools like Google Analytics, Website Optimizer, Crazy Egg, 4Q Survey and more.
User Research: The Superpower Behind Experimentation Programs | VWO WebinarsVWO
User research is often a side-lined activity within the experimentation space, but it’s crucial for keeping your testing velocity up and your win rates high.
Many experimentation and optimization teams fail to make this connection due to a lack of time, resources, knowledge or the siloed structure of their organization.
In this talk, Chris will take you through some practical examples of how user research can drive the quantity of ideas, their quality, and originality, which in turn leads to a much more successful overall experimentation program.
Designing Outcomes For Usability Nycupa Hurst FinalWIKOLO
MarkoHurst.com :: My topic of discussion at the Feb 17 2009 NYC UPA.
Even as the pace of society, business, and the Internet continue to increase, many budgets and time lines continue to decrease. To compound this issue, there is a serious disconnect between business goals, user goals, and what visitors actually do on your site. UX practitioners need a simple and efficient way to reconcile these diverse needs while taking action on their data. Join us to learn about a new method for incorporating quantitative data such as web analytics and business intelligence into your qualitative user experience deliverables: personas, wireframes, and more. This presentation will include discussions of online business models, feedback loops for ensuring cross-discipline collaboration, and ongoing revisions.
iLive2014 Presentation | Viljo Vabrit - A bulletproof roadmap to boost your o...iLive Conference
Viljo believes that to remain competitive you should constantly improve your website. In his presentation he will give an answer to the question “How to identify where your website is leaking money and how to eliminate those leaks”. Viljo is going to walk you through the process you can use to identify and plug the leaks. He will introduce practical tools and guidelines how to identify problematic areas on a website and strategies how to work out treatments and run A/B tests to improve online sales. Viljo will share examples from real online businesses and proven methodology to improve online conversions. The methodology includes many important details that help optimizers to form better test hypothesis and get better results faster.
Slides from the usability seminar delivered by Paul Rouke, Head of Usability at PRWD, and Chris Bush, UX Consultant at Sigma, looking at usability and user experience for web and mobile
How To Build a Winning Conversion Optimization StrategyVWO
[VWO Webinar]: Getting your prospects to pay attention to your website is essential whether you’re running an eCommerce brand or generating leads for your sales team. So it's imperative to have a killer conversion optimization strategy in place to drive growth.
Watch VWO and Ladder sharing conversion rate optimization insights on:
How to craft a CRO strategy with structured hypotheses
How to make use of both technical best practices and
data-driven marketing
Leveraging data to identify the weakest point in your funnel
Your ideal CRO Toolkit - tools you need for optimization
Building a framework of priorities - knowing what to test/implement first
Usability: whats the use? Presented by We are Sigma and PRWDNexer Digital
For websites, good usability is a matter of survival. If a website is difficult to use, people leave. If the homepage fails to clearly state what a company offers and what users can do on the site, people leave. If users get lost on a website, they leave. For intranets and applications the question is one of productivity. In many organisations employees waste inordinate amounts of time searching for and assimilating the information they need to do their jobs. This lost time has a real, tangible value so ROI for designing internal systems with User Experience in mind, and spending some time testing and improving the usability of the system, is pretty compelling.
As people with a strong User Experience focus we don’t need to be convinced of the value of good usability, but for many companies who are thinking of revamping their site, intranet or portal it isn’t quite so clear cut.
Presented by Chris Bush, www.wearesigma.com and
Paul Rouke, www.prwd.co.uk
Startup Metrics for Pirates (March 2009)Dave McClure
Slightly updated presentation from my talk at http://EntrepreneurTrek.org at Stanford University (March 2009). note: basically same as FOWA talk, minor update added slide 17.
Enhancing the user experience in our Web ApplicationsChrisCariglia
Summary of the UIE conference. I used this presentation to communicatet what changes we could bring to our own site / tools to enhance the user experience.
Conversion Optimization Framework to Build Sustainable and Repeat GrowthTushar Purohit
The goal of the this presentation on Conversion optimization Framework is to remove the guesswork from the conversion optimization process. It provides comprehensive analysis to anyone interested in optimization with a specific methodology to produce consistent results.
How to meet user expectations - WordCamp Praha 2015Igor Kolosov
A visitor came to your website. Does he understand what you are offering? Does he know where to click next to get what he came for? Are his expectations met? You have a couple of seconds to attract and keep the visitor on the website. Are you using this time in a smart way? Let me show you how to find out what your visitors expect from you and improve your WordPress website goal conversion rate.
3 TED style talks of 15-20 minutes, featuring:
(1) Conversion methodologies, Lean UX and Agile? What gives?
(2) #Measurecamp and my Top Analytics Tips of 2013
(3) Conversion tools of the CRO masters
The tools used by the CRO masters round the world to optimise analytics, UX, VOC,insight and testing - all to optimise your insight or conversion figures.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
16. Over 18 years of boring meetingsGroup CX Manager for Belron® (O’Brien ®) I get to do it with crowds
17. Outline What is A/B and Multi Variate testing? What is optimisation? How does it all work? What tools do I need? Where do I start testing? What results should I expect? Case studies
18. "If you go to the men's washrooms at the Schiphol airport in Amsterdam, you may notice there's a fly in the urinals. It’s screen printed . So what do you think most men do? That's right, they aim at the fly when they urinate. They don't even think about it, and they don't need to read a user's manual; it's just an instinctive reaction that means 85% less spillage! The interesting feature of these urinals is that they're deliberately designed to take advantage of this inherent human male tendency.“ This is my job.
29. What is optimisation? We’re a customer experience team, but focused on how we can influence customer behaviour in nice ways! So, our definition of Optimization: “A set of techniques, implemented in order to influence customer behaviour towards these outcomes:” Increased revenue or profit Removing cost from the business or contact center Increasing productivity or labor flexibility Increasing NPS (Customer Sat) scores Simply delighting customers All work is challenged on this basis!
30. Small team, big challenge Team of 3 – 2 x optimisers, 1 x part time designer Higher ROI without spending more (or lower costs) Stuff that doesn’t hinder SEO or PPC Measurable ROI on changes, not ego or opinion Tools with low or zero IT footprint, easy to deploy Low cost or fast return techniques Specialised tools for specific optimisation work Optimisation is a culture, not a tool though Stop guessing and start testing No IT department! MVT and A/B testing has the fastest ROI of *any* technique
34. Killer idea generation sources Analytics data Customer insight – contact, customer feedback, research Usability testing Diary studies Clicktale recordings Business owners – market data, customer mix, local knowledge Customer contact centre immersion (listening) Books, Websites, Tips Other optimisers Test data – variables plotted on axis Making lots of mistakes – Fail faster
39. Low cost of entry tools: Visual Website Optimizer www.visualwebsiteoptimizer.com Google Website Optimizer www.google.com/websiteoptimizer Optimizely www.optimizely.com If you want to move up from here, try Autonomy Optimost or Webtrends Optimize.
40. Is a testing tool all you need? Analytics Talent Session recording : Clicktale/Tealeaf Performance optimisation : Google/Strangeloop Feedback : 4Q/Kampyle Usability testing : See resources Remote testing : See resources Forms and search analytics : Custom or Bought Testing tools : Browsercam Deviceanywhere Google speed test : Cool stuff! Survey tools : See resources Multi-variate and A/B testing : Free, Paid or Both
42. How do you find your test areas? Analytics talent, tools and instrumentation Develop a leak model Identify key loss or influence pages Session recording tools Usability testing & UCD Contact centre feedback vital Traffic * opportunity is the key Grade targets by impact, time, resource, difficulty Free evaluation for 2 people – be the first to ask me!
43. Analytics / Insight – where to test? It depends on your site... High traffic entry pages High dropout or bail pages Key value pages (credit card) High cost pages (PPC) Conversion tipping points Persuasion points Anywhere there is traffic and CTA! USE A LEAK MODEL!
44. How a leak model helps How many people continue? How many leave? How many channel switch? Tip! Dynamic site numbering! Where do they go? Are there influence pages? Any error clusters? What are my pain points?
47. Leak surprises! Confirmation page loses 8% visitors One small change reduced this by 75% to 2% 1 page drives 60% traffic flow to phone Quote page has low switch – big test opportunity Bugs in address lookup, branch finder Forms errors contribute hugely to ‘leavers’ Losses aren’t always big – they’re just switching In most cases, over 50% of funnel ‘losses’ are switch Phone tracking!
48. Session recording Vital for optimizers Fills in a ‘missing link’ for insight Rich source of data Segment by attribute/source/behavior Can be used to optimise in real time! Tealeaf www.tealeaf.com Clicktale www.clicktale.com Clixpy www.clixpy.com
49. Usability Testing and UCD Lab based testing Remote testing tools (see resources) Eye tracking Prototype testing Low budget testing Not what they say, watch what they do Hurdles to conversion, comprehension, action Excellent for ‘black box’ forms/funnels Fills a serious gap in your knowledge
50. Performance? Huge impact on conversion and bailout Enable Google site speed – get customer KPIs Read anything by Steve Souders Read Joshua Bixby (ace!) : bit.ly/8YB3ET Use Keynote, Gomez for site monitoring Consider CDN acceleration (Akamai etc.) Who owns customer facing performance?
51. Forms and Search Analytics Ignore at your peril Forms are often seen as a ‘technical’ area What are the common errors? What errors precede abandonment? Validation improvements Form design – Luke Wroblewski Good example – Postcode Lookup CR0 1XA CRO lXA 2.5% loss due to strict validation Search analytics – Hurol Inan
52. Browser Compatibility “Anyone who slaps a ‘this page is best viewed with Browser X’ label on a Web page appears to be yearning for the bad old days, before the Web, when you had very little chance of reading a document written on another computer, another word processor, or another network.” Tim Berners-Lee My challenge to you! What browsers have problems? Who doorsteps you? The ‘Mac Users’ problem Measure entry point stats only Create a browser matrix Use a testing service, not simulators Test mobile devices too Try the ‘Checkout browser’ test
53. Where to test? High traffic, High impact areas High ‘bounce index’ landing pages Key conversion points or creatives Funnels / Forms Registration, Signup processes PPC/SEO high traffic landing Low hanging fruit first Micro testing
55. What about us? Easy wins? In priority order: Homepage Core funnel (common to all websites) High traffic landing pages (hero SEO and PPC terms) Local landing pages Influencing pages Lower traffic landing pages
71. Photo Guidelines Groups, vans, stores, corporate guff = NO Single person – not groups or even person + customer Open, Friendly, Natural, Smiling, Engaged = YES Uniform – has huge effect, even wearing branded ball cap improves conversion Look straight at viewer or slightly towards CTA Plain backgrounds only No folded arms, arms on hips, pointing, scowling, cheesy smiles, hands behind back, hanging useless by the side, clasped by groin, fiddling with hands Arms with prop instead – hold a clipboard, rag, tool – I’m working for *you* Female images work best in almost every country – they slay the guys Now we get slightly better images with the guidelines! We still got a genital framing one last week Photoshoots are now used to drive advertising strategy Video tests of potential ‘actors’ as a trial
72. Key learnings – on page elements Less is more – simplicity, comprehension, clarity in design Copy is king – not content, but key persuasive copy Large, prominent, visible, button like CTA CTA above fold, especially on landing page Do not split attention on CTA (Save, Cancel, Print, Back, Help) Use goal oriented keywords (Fix my glass!) Where is the core value proposition? Maintain a scent – PPC advert, Landing page, Funnel, Reflect the user CTA wording, headlines, copy, framing = 50%+ of uplift Think about emotional states and make use of them Use badging, testimonials, social proofing and trust symbols Run a betting game in the office – even UX experts little better than 60/40 Guidelines are good but every page and site will vary
73. Key learnings – test approach Stop testing rubbish subtleties (e.g. Button colour) Cultural factors are minimal Market and traffic composition are much larger drivers Techniques work well, regardless of market or language Huge similarities in testing, but not identical results Hub and spoke works, but only at a traffic level Faster cycles of smaller tests = better (fail faster) A fast test that fails is better than a slow test You must not test inside a bubble – keep challenging Use A/B to test radical approaches/disruptive change
74. Summary Don’t overcook tests with too many variables Don’t test things very similar (blue vs. green button?) Don’t be timid – fail faster, be bolder International tests show little cultural differences Market conditions & traffic mix are more important Biggest drivers on page = Copy, CTA and CTA copy, Images, Badging, Nav/Layout Biggest external factors = Page and site performance, Browser compatibility We are now doing up to 7M page tests per month
75. What’s the ROI like on this work? 12.4% from funnel work, 12.5% from MVT 3.5% increase in downstream conversion rate 30 seconds reduction in call time (1 sec = $20k p.a.) Faster booking time (< 8%) Greater customer delight – feedback. Less questions and worries 5.5% increase in NPS (Customer sat) score 4% increase from mobile optimised site Total increase in conversion (measured) = 32% Actual increase in conversion = 41% NO additional marketing spend needed Web Channel : ROI positive within 2 weeks
76. Cross channel optimisation Call centre optimisation Phone tracking Emergency phone switchoff Telephony MVT Chat testing Usability testing of internal apps Simple no training overspill systems Contact deflection systems and self servicing Scripting MVT Customer reminders Safelite – Technician profile and ‘on their way’ notice
77. Chat testing What happened? Ran a trial for 1 month A/B split test – very detailed One week – 17 calls avoided But swapped that for 139 chats As these were 8.25 minutes, much longer than phone No impact on conversion Assumption was big impact Customers happy, yes, but not in a productive way Like putting your customer service desk into the street You must measure phone volumes if testing chat Be very careful – prove your ROI on any pilot.
78. What’s next Segmentation, Segmentation, Segmentation E.g. - screen size work on Mobile site, iPads Better testing tools, self administered A/B More funnel optimisation, increased volumes of page tests Move from 8 tests -> 20+ tests per month International MVT – now onto landing pages #1, homepage #3 Telephony and Email MVT Hero term PPC, SEO + Local landing pages a priority Dynamic Landing Page generation – human and rules based The perfect clothes store journey
79. My advice You’re crazy if you’re not testing So easy to do A/B tests Invest in optimisation tools and talent Tools may be low cost – expertise is worth paying for Biggest ROI is high traffic landing page optimisation Funnel is good but page is quicker (weeks vs 6 mth) Work on images, copy, call-to-action, layout, hurry up messages, scarcity calls, badging, value proposition Stop thinking about testing and doing it instead Micro test, if you can’t do big stuff Start a gambling club at work – have fun!
84. Feedback tools Simplest example – 4q – bail from site Deeper sampling: Light touch – micro conversion (social, newsletter) Starts key process – registration/purchase Gets to tipping point – payment page Get feedback at different funnel points Review the user session if captured Like 4Q but think of an onion
85. Onsite feedback tools Onsite feedback, always available, immediate, contextual Feedback tools: Kampyle Opinionlab Foresee Polldaddy Crowdsourcing: Uservoice Ideascale Google moderator Kampyle has excellent international support – trial in 2012.
86. Testing tools Browser testing www.browsercam.com Mobile devices www.perfectomobile.com www.deviceanywhere.com Performance testing www.keynote.com www.gomez.com New on the block strangeloopnetworks.com End browser and device discrimination!
87. Survey tools Dust off those survey tools and ask stuff: My personal rating: Surveymonkey www.surveymonkey.com Zoomerang www.zoomerang.com Surveygizmo www.surveygizmo.com
88. Remote testing, usability, feedback Userzoom www.userzoom.com Usabilla www.usabilla.com Chalkmark www.chalkmark.com Userfly www.userfly.com Silverback www.silverbackapp.com Usertesting www.usertesting.com Loop11 www.loop11.com Pidoco www.pidoco.com Uservue www.techsmith.com Ethnio www.ethnio.com Feedback army www.feedbackarmy.com Five second test www.fivesecondtest.com Mechanical turk www.mechanicalturk.com Open Hallway www.openhallway.com