3 TED style talks of 15-20 minutes, featuring:
(1) Conversion methodologies, Lean UX and Agile? What gives?
(2) #Measurecamp and my Top Analytics Tips of 2013
(3) Conversion tools of the CRO masters
Some years ago, Eric Ries, Steve Blank and others initiated The Lean Startup movement. The Lean Startup is a movement, an inspiration, a set of principles and practices that any entrepreneur initiating a startup would be well advised to follow.
Projecting myself into it, I think that if I had read Ries' book before, or even better Blank's book, I would maybe own my own company today, around AirXCell or another product, instead of being disgusted and honestly not considering it for the near future.
In addition to giving a pretty important set of principles when it comes to creating and running a startup, The Lean Startup also implies an extended set of Engineering practices, especially software engineering practices.
Product-led Growth - Building the Ultimate SystemHarsh Jawharkar
Product-led Growth can and should feed your entire company’s progress and velocity. PLG may be product-led but it needs to be a whole company DNA effort. By applying a systems mindset, you can build a perpetual growth machine for years to come.
Lean Product Management: The Art of Known UnknownsNatalie Hollier
(This presentation was given at the Lean Strategy + Design Salon meetup in New York: http://www.meetup.com/LeanStrategyPlusDesign/events/200913392/)
"Innovate or die” is the mantra of successful companies. So how can we build innovation into our product development process? By combining design thinking, lean startup and agile we get a recipe for repeatable innovation: lean UX. Lean UX and lean startup methods are being used today by many startups and innovation labs to take a learning approach to discovering and building the best product for customers.
But what does repeatable innovation look like scaled across an enterprise? This talk will share how to apply lean product practices as a continuous process across multiple products and agile development teams in an organization. With real examples and artifacts you will learn how to manage - and thrive - in uncertainty to create awesome products.
[Webinar] Scaling experimentation: 5 key pillars of maturity by Nick SoChris Goward
Go behind the curtain with WiderFunnel's Director of Strategy, Nick So, as he shares exclusive learnings from his experience building successful experimentation programs for companies like HP, Asics, Magento, and IBM. During this webinar, you’ll get an insider look at the frameworks and strategies that WiderFunnel developed to help clients scale experimentation and accelerate through the phases of maturity.
My Slides from the "Implementing Lean Startup" workshop held at the inaugural Ideas to Profit conference at North Central College.
Original sources (Eric Ries & Steve Blank) for the lean startup are in the last slide.
Introduction to Lean Startup leading up to a 3-hour workshop. Presented by me at EFYI (European Forum for Young Innovators) 2016, conference organized by Poland Innovative (Polska Innowacyjna).
[DevDay2019] Growth Hacking - How to double the benefits of your startup with...DevDay.org
What is growth hacking? Why do all startup need it? Examples of Growth Hack with 10 Classic (Facebook, Dropbox, Airbnb, etc.). How to create robot to automatize your task. How to find clients automatically in 5 minutes. 6 SEO hacks to grow up super fast on Google.
Why Does My Conversion Rate Suck? Craig Sullivan, Senior Optimisation Consult...PRWD
Craig Sullivan, Senior Optimisation Consultant covers the top 10 reasons why your conversion rate might suck. Packed with actionable tips and resources this presentation is for anyone wanting to improve their Conversion Optimisation. Craig covers common problems and topic areas such as issues with Google Analytics setup, inputs, tools, testing, testing cycles, product cycles, Photo UX, how to analyse statistics / Data, segmentation, multiple channel optimisation. The resource pack also include a maturity model, Crowd-sourced UX, collaborative tools, testing tools for CRO & QA, Belron Methodology example, and CRO and testing resources.
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfVWO
You can utilize various forms of Generative Research to deepen your understanding of how people interact with your product or service.
Craig has amassed a vast toolkit of research methods, which he has employed to optimize websites and apps for over 500 companies. He'll share which methods yielded the highest return on investment, identified key customer pain points, and generated the best experiment ideas.
By sharing the top inspection methods essential for our work, Craig will provide advice for each technique. Anticipate insights on driving experiment hypotheses from research, a list of essential toolkit components for tomorrow, and additional resources for further reading.
Some years ago, Eric Ries, Steve Blank and others initiated The Lean Startup movement. The Lean Startup is a movement, an inspiration, a set of principles and practices that any entrepreneur initiating a startup would be well advised to follow.
Projecting myself into it, I think that if I had read Ries' book before, or even better Blank's book, I would maybe own my own company today, around AirXCell or another product, instead of being disgusted and honestly not considering it for the near future.
In addition to giving a pretty important set of principles when it comes to creating and running a startup, The Lean Startup also implies an extended set of Engineering practices, especially software engineering practices.
Product-led Growth - Building the Ultimate SystemHarsh Jawharkar
Product-led Growth can and should feed your entire company’s progress and velocity. PLG may be product-led but it needs to be a whole company DNA effort. By applying a systems mindset, you can build a perpetual growth machine for years to come.
Lean Product Management: The Art of Known UnknownsNatalie Hollier
(This presentation was given at the Lean Strategy + Design Salon meetup in New York: http://www.meetup.com/LeanStrategyPlusDesign/events/200913392/)
"Innovate or die” is the mantra of successful companies. So how can we build innovation into our product development process? By combining design thinking, lean startup and agile we get a recipe for repeatable innovation: lean UX. Lean UX and lean startup methods are being used today by many startups and innovation labs to take a learning approach to discovering and building the best product for customers.
But what does repeatable innovation look like scaled across an enterprise? This talk will share how to apply lean product practices as a continuous process across multiple products and agile development teams in an organization. With real examples and artifacts you will learn how to manage - and thrive - in uncertainty to create awesome products.
[Webinar] Scaling experimentation: 5 key pillars of maturity by Nick SoChris Goward
Go behind the curtain with WiderFunnel's Director of Strategy, Nick So, as he shares exclusive learnings from his experience building successful experimentation programs for companies like HP, Asics, Magento, and IBM. During this webinar, you’ll get an insider look at the frameworks and strategies that WiderFunnel developed to help clients scale experimentation and accelerate through the phases of maturity.
My Slides from the "Implementing Lean Startup" workshop held at the inaugural Ideas to Profit conference at North Central College.
Original sources (Eric Ries & Steve Blank) for the lean startup are in the last slide.
Introduction to Lean Startup leading up to a 3-hour workshop. Presented by me at EFYI (European Forum for Young Innovators) 2016, conference organized by Poland Innovative (Polska Innowacyjna).
[DevDay2019] Growth Hacking - How to double the benefits of your startup with...DevDay.org
What is growth hacking? Why do all startup need it? Examples of Growth Hack with 10 Classic (Facebook, Dropbox, Airbnb, etc.). How to create robot to automatize your task. How to find clients automatically in 5 minutes. 6 SEO hacks to grow up super fast on Google.
Why Does My Conversion Rate Suck? Craig Sullivan, Senior Optimisation Consult...PRWD
Craig Sullivan, Senior Optimisation Consultant covers the top 10 reasons why your conversion rate might suck. Packed with actionable tips and resources this presentation is for anyone wanting to improve their Conversion Optimisation. Craig covers common problems and topic areas such as issues with Google Analytics setup, inputs, tools, testing, testing cycles, product cycles, Photo UX, how to analyse statistics / Data, segmentation, multiple channel optimisation. The resource pack also include a maturity model, Crowd-sourced UX, collaborative tools, testing tools for CRO & QA, Belron Methodology example, and CRO and testing resources.
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfVWO
You can utilize various forms of Generative Research to deepen your understanding of how people interact with your product or service.
Craig has amassed a vast toolkit of research methods, which he has employed to optimize websites and apps for over 500 companies. He'll share which methods yielded the highest return on investment, identified key customer pain points, and generated the best experiment ideas.
By sharing the top inspection methods essential for our work, Craig will provide advice for each technique. Anticipate insights on driving experiment hypotheses from research, a list of essential toolkit components for tomorrow, and additional resources for further reading.
Are you ready for Data science? A 12 point testBertil Hatt
Presentation for the MancML on data readiness.
If you are considering starting to invest in Data science, this is a helpful guide to understand:
- what you need *before* you start looking for a Data scientist
- the skillset and experience that you should be looking for when you do.
If you don't know where you're going it doesn't matter how fast you get thereNicole Forsgren
The best-performing organizations have the highest quality, throughput, and reliability while also delivering value. They are able to achieve this by focusing on a few key measurement principles, which Nicole and Jez will outline in this talk. These include knowing your outcome measuring it, capturing metrics in tension, and collecting complementary measures… along with a few others. Nicole and Jez explain the importance of knowing how (and what) to measure—ensuring you catch successes and failures when they first show up, not just when they’re epic, so you can course correct rapidly. Measuring progress lets you focus on what’s important and helps you communicate this progress to peers, leaders, and stakeholders, and arms you for important conversations around targets such as SLOs. Great outcomes don’t realize themselves, after all, and having the right metrics gives us the data we need to be great SREs and move performance in the right direction.
Post-agile approaches - agile for the real world and how to avoid agile failureYuval Yeret
A session for an ILTAM forum in Israel - Agile is really great. Can it fail? Are failures due to mismatch of practices? principles? Only implementation details?
We will look at the strengths weaknesses opportunities threats related to the major agile frameworks as well as common failure modes and what to do about them
(the actual session includes case studies from audience and agilesparks experience)
Why And How to Transition into Product Management by Google PMProduct School
Nabil Shahid walks through their journey to Product Management in the world of tech, talking about how to market your skills and how to get into the industry. He also touches on balancing knowledge and personal experience with what's best for a wider user group.
I believe that our existing models of testing are not fit for purpose – they are inconsistent, controversial, partial, proprietary and stuck in the past. They are not going to support us in the rapidly emerging technologies and approaches. The certification schemes that should represent the interests and integrity of our profession don’t, and we are left with schemes that are popular, but have low value, lower esteem and attract harsh criticism. My goal in proposing the New Model is to stimulate new thinking in this area.
eurostarconferences.com
testhuddle.com
I believe that our existing models of testing are not fit for purpose – they are inconsistent, controversial, partial, proprietary and stuck in the past. They are not going to support us in the rapidly emerging technologies and approaches. The certification schemes that should represent the interests and integrity of our profession don’t, and we are left with schemes that are popular, but have low value, lower esteem and attract harsh criticism. My goal in proposing the New Model is to stimulate new thinking in this area.
eurostarconferences.com
testhuddle.com
Agile concepts for quality and process engineers for slideshareYuval Yeret
Excerpt from a session introducing agile concepts for a group of quality engineers in a big enterprise undergoing an agile transition.
The aim was to expose Quality/Process engineers to the concepts of agility and emphasize the impact on process/methodology development, the approach of evolution vs big design up front and its impact on their work. I used a lot of the classic agile examples (a lot of them by Henrik Kniberg) and adjusted for the development of methodology, to show that actually the agile approach should be discovered using an agile process.
Also covers some complexity thinking aspects.
And of course - this is not limited to methodology for IT/product development, but to many kinds of change management.
Conversion Rate Optimization for Business GrowthReapDigital
Conversion rate optimization is often limited to testing elements of a website. Button colors, button sizes, images and image placements are few common elements that are tested for optimization. Conversion Rate Optimization can be easily used to optimize beyond website elements to optimize business outcomes.
In this webinar, we will discuss how psychological theories, behavioral insights, research insights and insights through usability testing can be used to build hypothesis for testing. We will also discuss how these hypotheses can be effectively tested and implemented to gain optimum business outcomes.
Similar to 3 Optimisation Decks : WAW Copenhagen - 27 Feb 2013 (20)
Cross Device Optimisation - Google Analytics ShortcutsCraig Sullivan
In this session, we explain how to mine GA for broken device experiences, flows, funnel blocks and more... Using a new grid tool we've developed, you can pull multi-dimensional segmented funnel and metric data from Google Analytics - we explain how it works, why you need it and what problems it solves. Find where your site is leaking money through data
Product Design is Poo - And we're all going to dieCraig Sullivan
A humorous presentation about what is wrong with the current way of building digital products. Showing what is wrong, explaining the signs and giving you a checklist for reforming your company - are laid out with links, resources and further reading.
Product design is Poo - And how to fix it!Craig Sullivan
A look at why product design is still so poor, even after 22 years of digital design work. Why do these problems exist and how can we remove them from the way we build products? Lean corporate and startup growth models are explored in the solutions to this horrendous problem!
Slides to go with a talk on rapid, lightweight research you can do before tackling a landing page, funnel step or lead-gen form. Comes complete with all the Google Analytics reports you'll need to mine useful data to share! Less bullshit, more truth in meetings!
Web Analytics Wednesday - Session Replay Tools are VitalCraig Sullivan
Session Replay or Screen Recording tools are now part of an arsenal of discovery toolkits that can drive optimisation, bug fixes, funnel and journey analysis - using qual and quant techniques. Without these tools, the analytics data misses emotion, frustration, friction and more - I've collated the best tips, tricks, tools and approaches to yield the most valuable insights for CRO / Growth Hacking.
Surviving the AB Testing Hype Cycle - Reaktor Breakpoint 2015Craig Sullivan
My Slides from Reaktor Breakpoint 2015 - This is by far the best deck (and hopefully talk) I've done this year. Masses of info, reading, articles, useful reports and more.
Surviving the hype cycle Shortcuts to split testing successCraig Sullivan
In this talk, I show the key shortcuts to stop doing stupid testing and move towards innovative and transformative design & build methodologies, including innovation through split testing exploration
Myths and Illusions of Cross Device Testing - Elite Camp June 2015Craig Sullivan
A compendium of the most common mistakes and problems people encounter when trying to optimise or split test cross device experiences (mobile, tablet, desktop, app, tv etc.)
Myths, Lies and Illusions of AB and Split TestingCraig Sullivan
What are the common assumptions about AB (split) testing that are wrong? What are the lies told by vendors, consultants and the stuff you have convinced yourself about. What is illusory - what can you trust - what's it really all about. 20 top myths debunked after asking fellow CRO professionals what is on THEIR top list.
20 Ways to Shaft your Split Tesring : Conversion ConferenceCraig Sullivan
This talk is the latest deck showing common problems that will easily break or skew your ab and multivariate testing results. Avoid these problems by following the simple advice in this deck!
Brighton CRO Meetup #1 - Oh Boy These AB tests Sure Look Like Bullshit to MeCraig Sullivan
An updated deck of a short talk (30m) given at the first Brighton CRO meetup. Contains useful AB testing tools as well as full speaker notes for most of the slides.
#Measurecamp : 18 Simple Ways to F*** up Your AB TestingCraig Sullivan
An expanded deck of the top 18 blockers to getting successful AB or Multivariate test results. In this deck, you get a complete checklist of the stuff you need to prepare, watch, launch and monitor your testing, so it gets you the *right* conclusions.
The Neuromarketing Toolkit - Chinwag Psych - 4 Feb 2014Craig Sullivan
A practical toolkit for getting inside customers heads, in order to design and create persuasive psychological approaches to copy, pages, buttons, designs and your entire service. Craig shows you here how to mine what you already have - to design a better bank balance and continuously improving future for your company, staff and your customers.
12 Things to do Before Your Company Dies : Conversion Conference London - Oct...Craig Sullivan
A roundup of all the things to help you maintain a competitive edge in experience design and conversion optimisation. With examples of companies putting this stuff together, the tools they are using and their project management approaches, this presentation delves deeper into the cultural aspects of CRO.
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
3 Optimisation Decks : WAW Copenhagen - 27 Feb 2013
1. #WAWCPH
Meet @OptimiseOrDie
Reason for being : I Optimise Stuff
Past : John Lewis, Lovefilm, Belron, PwC
Now : Director of Optimisation, Rush
Email : sullivac@gmail.com
Twitter : @OptimiseOrDie
: linkd.in/pvrg14
Slideshare : slidesha.re/nlCDm6
2. "If you go to the men's washrooms at
the Schiphol airport in
Amsterdam, you may notice there's a
fly in the urinals. It’s screen printed .
So what do you think most men do?
That's right, they aim at the fly when
they urinate.
They don't even think about it, and
they don't need to read a user's
manual; it's just an instinctive
reaction that means 85% less spillage!
The interesting feature of these
urinals is that they're deliberately
designed to take advantage of this
inherent human male tendency.“
This is my job.
4. Three Presentations
#1 Conversion Methodologies
How does CRO fit in with Lean UX, Agile, User Centred, Waterfall?
#3 Analytics Genius Tips
Top tips from #Measurecamp
+ my 2013 roundup of cool stuff
#2 The Optimisers Toolkit
The best tools recommended by CRO practitioners
6. What’s the problem?
• #1 User Experience and Conversion
Optimisation are not a checkbox or a
step in the process – it needs to be
integrated in everything you do.
• #2 This work isn’t a one off exercise
either – it’s an ongoing continuous
improvement process – like Kaizen
• #3 Usability testing isn’t enough –
other UX factors like the visceral and
behavioural emotional responses we
have to products need tuning too.
• #4 It’s not just about the user!
• #5 It’s usually Self Centred Design
driven by Ego, Opinion, Assumption
7.
8. Also, the dial won’t turn anymore
PPC SEO
With thanks to @morys
9. Why is this happening?
• PPC changes
• Advertising models flattening – i.e. mobile Google costs
• Comprehensive SEO changes
• Competition increasing
• Fleetness of foot – Asos in Australia
• Entry costs are lower now
• New entrants compete without the cruft
• Startups are using better ‘build and optimise’ methodologies
than nearly all corporates
• The old way of doing things is going to die
• The new way of doing things is your only survival ticket
10. So what do people do?
• They throw tools at the problem
• They try usability testing and research
• They generate more data to look at
• They make changes without measuring or testing
• They hope to randomly create the optimal system
• They get an expensive agency to help them
• They push their team harder, like galley slaves
• They experiment with riskier advertising models
• They wonder why they’re burning rubber
• They try more random things
• Then they call a CRO person and say
“We’ve tried everything. It isn’t working. Help!”
11. Skinner’s Pigeon Experiment
• Participants invited into a room with objects
• Told to score 100 points within 30 minutes
• Participants moved objects around, made
noise, jumped around, tried anything to make a
counter increase the points score.
• They got horribly confused
• They then created convincing lies for
themselves, to explain what they thought was
working. They became superstitious and made
rituals.
• The points allocation was made randomly by a
goldfish, swimming back and forward in a tank.
• Know any marketing departments like this?
• I’ve seen this a lot – and it paralyses companies
• We need a better way. A methodology?
12.
13. Lean UX
“The application of UX design methods into product
development, tailored to fit Build-Measure-Learn cycles.”
Positive
– Lightweight and very fast methods
– Realtime or rapid improvements
– Documentation light, value high
– Low on wastage and frippery
– Fast time to market, then optimise
– Allows you to pivot into new areas
Negative
– Often needs user test feedback to
steer the development, as data not
enough
– Bosses distrust stuff where the
outcome isn’t known
14. Agile UX / UCD / Collaborative Design
“An integration of User Experience Design and Agile*
Software Development Methodologies”
*Sometimes
Research
Positive
– User centric
– Goals met substantially
– Rapid time to market (especially when
using Agile iterations) Wireframe
Negative
Concept Prototype
– Without quant data, user goals can
drive the show – missing the business
sweet spot
– Some people find it hard to integrate
with siloed teams Analyse Test
– Doesn’t’ work with waterfall IMHO
16. Lean Conversion Optimisation
“A blend of User Experience Design, Agile PM, Rapid Lean
UX Build-Measure-Learn cycles, triangulated data
sources, triage and prioritisation.”
Positive
– A blend of several techniques
– Multiple sources of Qual and Quant data aids triangulation
– CRO analytics focus drives unearned value inside all
products
Negative
– Needs a one team approach with a strong PM who is a
Polymath (Commercial, Analytics, UX, Technical)
– Only works if your teams can take the pace – you might be
surprised though!
18. Triage and Triangulation
“This is where the smarts of CRO are – in identifying the
easiest stuff to test or fix that will drive the largest uplift.”
• Starts with the analytics data
• Then UX and user journey walkthrough from SERPS -> key paths
• Then back to analytics data for a whole range of reports:
• Segmented reporting, Traffic sources, Device viewport and
browser, Platform (tablet, mobile, desktop) and many more
• We use other tools or insight sources to help form hypotheses
• We triangulate with other data where possible
• We estimate the potential uplift of fixing/improving something
as well as the difficulty (time/resource/complexity/risk)
• A simple quadrant shows the value clusters
• We then WORK the highest and easiest scores by…
• Turning every opportunity spotted into an OUTCOME
19. The Bucket Methodology
“Helps you to stream actions from the insights and prioritisation work.
Forces an action for every issue, a counter for every opportunity being lost.”
Test
If there is an obvious opportunity to shift behaviour, expose insight or
increase conversion – this bucket is where you place stuff for testing. If
you have traffic and leakage, this is the bucket for that issue.
If an issue is placed in this bucket, it means we need to beef up the
Instrument analytics reporting. This can involve fixing, adding or improving tag or
event handling on the analytics configuration. We instrument both
structurally and for insight in the pain points we’ve found.
Hypothesise This is where we’ve found a page, widget or process that’s just not working
well but we don’t see a clear single solution. Since we need to really shift
the behaviour at this crux point, we’ll brainstorm hypotheses. Driven by
evidence and data, we’ll create test plans to find the answers to the
questions and change the conversion or KPI figure in the desired direction.
Just Do It
JFDI (Just Do It) – is a bucket for issues where a fix is easy to identify or the
change is a no-brainer. Items marked with this flag can either be deployed
in a batch or as part of a controlled test. Stuff in here requires low effort
or are micro-opportunities to increase conversion and should be fixed.
Investigate You need to do some testing with particular devices or need more
information to triangulate a problem you spotted. If an item is in this
bucket, you need to ask questions or do further digging.
20. How is it working out?
• Methodologies are not Real Life ™
• It’s mainly about the mindset of the team and
managers, not the tools or methodologies they
play with
• Not all my clients have all the working parts
• You should not be a methodology slave
• Feel free to make your own or flexibly adapt
• Use some, any techniques instead of ‘guessing’
• Blending lean and agile with conversion
optimisation outcomes is my critical learning of
the last 5 years
• Doing rapid cycles of this outcome driven work
for Belron:
• World Conversion Rate Increase:
2009 +5%, 2010 +10%, 2011 +15%, 2012 +25%
• If you’d like to develop a good one for your
company, talk to me first!
• Don’t over complicate it.
23. Tips Roundup 2013
#1 Performance tune-ups
#2 Browser money
#3 Keyboard shortcuts
#4 Ranking data
#5 Content engagement
#6 Enhanced In page
#7 Duplicate transactions
#8 Event tracking
#9 Google API + more
24. #1 : Performance Tune-ups
• With thanks to @Danbarker
• Add “_gaq.push(['_setSiteSpeedSampleRate', 100]);”
• Amps up sampling for small & medium websites
• Use the distribution report - % of pages < 3 seconds
• DOM timings vital – let me explain
• Avg. Document Content Loaded Time (sec)
• This data is very accurate and helps conversion – pretty vital for
landing pages, where I find lots of stuff
• Make yourself a pageview*content loaded time report
• Work your way down from the top
• Mobile speed doesn’t count Safari – please be careful!
• Read more at:
http://p.barker.dj/sitespeedtips
25. #2 : Browser money
• Create a desktop only segment [Exclude mobile inc. tablet ]
• Create a mobile only segment [Include Mobile (including tablet) & Exclude Tablet
containing yes]. Apply 3 segments desktop, tablet, mobile ONLY.
• Now start segmenting like this:
Browser Segment Conv rate Quote the
Safari Mobile Traffic 0.79%
Internet Explorer Desktop only 1.34%
opportunity!
Chrome Desktop only 1.01% IE8 = 1.41% revenue
Safari Tablet Traffic 1.00%
Safari Desktop only 1.28% IE8 converts at 20%
Firefox Desktop only 1.20% of IE9 and IE10
Android Browser Mobile Traffic 0.31%
Some nasty bugs
Safari (in-app) Mobile Traffic 0.69%
cause the problem
Chrome Mobile Traffic 0.62%
Safari (in-app) Tablet Traffic 0.89% Worth fixing?
Chrome Tablet Traffic 0.84%
Opera Desktop only 0.10% That 1.41 is worth
Android Browser Tablet Traffic 0.71% nearly 6% more
Mozilla Compatible Agent Mobile Traffic 0.51% revenue
26. #2 : Browser money
• You should see something like this:
27. #3 : GA Keyboard Shortcuts
• Thanks to Farid Alhadi and @fastbloke
dt Set date range to TODAY
dy Set date range to YESTERDAY
dw Set date range to LAST WEEK
dm Set date range to LAST MONTH
dc
Toggle date comparison mode (to the previous period of whatever you are looking at.
Example, if you’re looking at 6 days, this will compare it to the 6 days before it)
dx Toggle date comparison mode (to the previous year of the period you are looking at)
? Open keyboard shortcut help
h Search help center
a open account panel
shift + a Go to account list
s/ Search reports
shift + d Go to the default dashboard of the current profile
28. #4 : Ranking data
• Someone searches on Google for ‘Term’
• Clicks on a link to your site
• What was the keyword rank for that term?
• Reverse engineer the actual rankings from users machines
• More accurate than some SEO tools (IMHO)
• Two articles to read:
http://bit.ly/Vaisno
http://bit.ly/13lmYF2
29. #5 : Content Engagement
• Get a better bounce rate metric
• See more detailed engagement metrics
• Measure scrolling and reading activity
• Came from @fastbloke but originally Justin Cutroni
• Measure your scrolling and exit points from long form
content
• Very nice technique – read more at:
http://bit.ly/13lmYF2
http://goo.gl/1AZZb
30. #6 : Enhanced Page Analytics
var _gaq = _gaq || [];
var pluginUrl =
'//www.google-analytics.com/plugins/ga/inpage_linkid.js';
_gaq.push(['_require', 'inpage_linkid', pluginUrl]);
_gaq.push(['_setAccount', 'UA-XXXXXX-Y']);
_gaq.push(['_trackPageview']);
• Correct link attribution for in-page Analytics
• Very nice
31. #7 : Duplicate transactions in GA
• Thanks to Matt Clarke and @timlb
• Stop skewing the data with duplicates/reloads
• Custom report to check if you’re affected
http://techpad.co.uk/content.php?sid=247
• I’ve seen this a few places so worth
checking, particularly if figures don’t tally!
• Read more at : http://bit.ly/13lmYF2
32. #8 : Event tracking
• Thanks to #Measurecamp – check the stream
• A beginners guide : http://bit.ly/13RFoJs
• Some great ideas here : http://bit.ly/UCcptx
• Don’t go for ‘Event Blizzard’
• Focus on specific areas where insight is needed
• Choose your naming structure carefully:
http://bit.ly/WJ4R4c
• Read this complete guide : http://bit.ly/VmFSJ4
33. #9 : Google API and more
• Use the Google API to get super custom reports
• You can fetch different data types (on the fly as
well as pre-calculated)
• Automate a lot of Excel work – more later…
• @timlb recorded all the tips for #Measurecamp
• We then had an unconference
• He recorded a special session for us
• Listen to great tips and more:
www.youtube.com/watch?v=JWXg1_4quwU
www.youtube.com/watch?v=z41Cdg9xbk0
• Roundup : www.measurecamp.org/aftermath/
35. #10 : Microdata – SERPS UX
• Reviews – huge increases in CTR and Conversion
• People
• Products
• Businesses and Organisations
• Recipes
• Events
• Music
• Local
• Video
Helps to:
• Dominate the page
• Push other stuff down
• Makes it more persuasive
• The conversion journey starts here!
36. #11 : Measure viewport size
• Thanks to @Beantin and others!
• Measure the viewport size, not the resolution
• Why?
• Toolbars, chrome and setup varies
• UK average is 2.2 toolbars installed
• Code example here : http://bit.ly/4xaNYK
• A common conversion issue
• Your desk vs. Users = different
• Turn off the wifi, reduce the viewport
• Or keep the inside of toilet rolls
40. #1 : Session Replay
• 3 kinds of tool :
Client side
• Normally Javascript based
• Pros : Rich mouse and click data,
errors, forms analytics, UI interactions.
• Cons : Dynamic content issue, Performance hit
Server side
• Black Box -> Proxy, Sniffer, Port copying device
• Pros : Gets all dynamic content, fast, legally tight
• Cons : No client side interactions, Ajax, HTML5 etc.
Hybrid
• Clientside and Sniffing with central data store
41. #1 : Session Replay
• Vital for optimisers & fills in a ‘missing link’ for insight
• Rich source of data on visitor experiences
• Segment by browser, visitor type, behaviour, errors
• Forms Analytics (when instrumented) are awesome
• Can be used to optimise in real time!
Session replay tools
• Clicktale (Client) www.clicktale.com
• Mouseflow (Client) www.mouseflow.com
• Ghostrec (Client) www.ghostrec.com
• Tealeaf (Hybrid) www.tealeaf.com
• Usabilla (Client) www.usabilla.com
• UserReplay (Server) www.userreplay.com
• SessionCam (Client) www.sessioncam.com
42.
43.
44.
45. #2 : Feedback / VOC tools
• Anything that allows immediate realtime onpage feedback
• Comments on elements, pages and overall site & service
• Can be used for behavioural triggered feedback
• Tip! : Take the Call Centre for beers
• Kampyle
www.kampyle.com
• Qualaroo
www.qualaroo.com
• 4Q
4q.iperceptions.com
• Usabilla
www.usabilla.com
Any more?
47. #2a : Survey Tools
I’ve used
• Surveymonkey www.surveymonkey.com (1/5)
• Zoomerang www.zoomerang.com (3/5)
• SurveyGizmo www.surveygizmo.com (5/5)
• Read anything by and follow @cjforms
• Any others?
48. #4 : Guerrilla Usability Testing
• All you need is a device, time and people!
• Use one of these tools for session recording:
Mediacam AV (cheap)
www.netu2.co.uk
Camtasia
www.techsmith.com
Silverback (Mac)
www.silverbackapp.com
Screenflow (Mac)
www.telestream.net
UX Recorder (iOS)
www.uxrecorder.com
52. #5 Google Docs
• Seriously wasting your time doing manual Excel?
• Fed up doing stuff that takes hours?
• Use the Google API to roll your own reports straight into Big G
• Lots of good articles but ask for advice from:
@danbarker
@timlb
#measurecamp
• Google Analytics + API + Google docs integration = A BETTER LIFE!
• Hack your way to having more productive weeks
• Learn how to do this and to have fun with GA custom reports
• Ask me about the importance of training
24 Jan 2012
60. #7 – UX and Crowd tools
Som, feedback
Remote UX tools (P=Panel, S=Site recruited, B=Both)
Usertesting (B) www.usertesting.com
Userlytics (B) www.userlytics.com
Userzoom (S) www.userzoom.com
Intuition HQ (S) www.intuitionhq.com
Mechanical turk (S) www.mechanicalturk.com
Loop11 (S) www.loop11.com
Open Hallway (S) www.openhallway.com
What Users Do (P) www.whatusersdo.com
Feedback army (P) www.feedbackarmy.com
User feel (P) www.userfeel.com
Ethnio (For Recruiting) www.ethnio.com
Feedback on Prototypes / Mockups
Pidoco www.pidoco.com
Verify from Zurb www.verifyapp.com
Five second test www.fivesecondtest.com
Conceptshare www.conceptshare.com
Usabilla www.usabilla.com
61. #8 : Web Analytics Love
• Properly instrumented analytics
• Investment of 5-10% of developer time
• Add more than you need
• Events insights
• Segmentation
• Call tracking love!
62. #8 : Tap 2 Call tracking
Step 1 : Add a unique phone number on ALL channels
(or insert your own dynamic number)
Step 2 : For phones, add “Tap to Call” or “Click to Call”
• Add event or PV trigger for any phone interaction
• Incredibly useful keyword level call data
• Very reliable data, easy & cheap to do
• What did they do before calling?
• Which page did they call you from?
• What PPC or SEO keyword did they use?
• What are you over or underbidding for?
• Will help you shave 10, 20%+ off PPC
• Which online marketing really sucks?
63. What about desktop?
Step 1 : Add ‘Click to reveal’
• Can be a link, button or a collapsed section
• Add to your analytics software
• This is a great budget option!
Step 2 : Invest in call analytics
• Unique visitor tracking for desktop
• Gives you that detailed marketing data
• Easy to implement
• Integrates with your web analytics
• Let me explain…
64. 10.0
15.0
20.0
25.0
0.0
5.0
safe lit
windshield chip repair
safelite windshield
autoglass
auto window replacement
safelight auto
auto glass repair
windshield replacement costs
safelite auto glass
safe auto glass
safelite repair
windshield
autoglass replacement
car glass
safelite locations
auto glass repair quotes
Phone to Booking Ratio
windshield repair
mobile windshield replacement
replace windshield
car windshield repair
new windshield cost
auto glass windshield replacement
car window repair cost
safe gl
auto glass repair houston tx
windshield crack
65.
66. What about desktop?
Onbound Call analytics:
Split testing
Priority routing
Wait time
VIP teams
Call centre
Abandon
A 222 1234 Call length music
On hold
0800 222 1234 0800 Operator outcome
Menu
Sales value
0800 222 1245
Messages
Ratings
Call recording
B 222 1263
0800
Offline promotions!
0800 222 1299 Profit line!
C
Web
Analytics Telephony Cloud
67. So what does phone tracking get you?
• You can do it for free on your online channels
• If you’ve got any phone sales or contact operation, this will
change the game for you
• For the first time, analytics for PHONE for web to claim
• Optimise your PPC spend
• Track and Test stuff on phones, using web technology
• The two best tests?
• Talk to Freespee today!
• Other countries, see the next slide
68. Company Website Coverage
Who?
Mongoose Metrics* www.mongoosemetrics.com UK, USA, Canada
Ifbyphone* www.ifbyphone.com USA
TheCallR* www.thecallr.com USA, Canada, UK, IT, FR, BE, ES, NL
Call tracking metrics www.calltrackingmetrics.com USA
Hosted Numbers www.hostednumbers.com USA
Callcap www.callcap.com USA
UK, SE, FI, NO, DK, LT, PL, IE, CZ,
Freespee* www.freespee.com
SI, AT, NL, DE
Adinsight* www.adinsight.co.uk UK
Infinity tracking* www.infinity-tracking.com UK
Optilead* www.optilead.co.uk UK
Switchboard free www.switchboardfree.co.uk UK
Freshegg www.freshegg.co.uk UK
Avanser www.avanser.com.au AUS
Jet Interactive* www.jetinteractive.com.au AUS
* I read up on these or talked to them. These are my picks .
69. #9 : Web Analytics Love
• UX of web analytics tools and reports
• People, Process, Human problems
• Make the UI force decisions!
• Playability and exploration
• Skunkworks project time (5-10%)
• Give it love, time, money and iteration
• Spend to automate, get more time
70. Evil CRO
• Optimisation can be used for evil!
• Either by accident (just stupid)
• Or by design, to trick people
wiki.darkpatterns.org/wiki/Home
71. So how do I become an Optimiser?
• It’s a state of mind, not a course!
• You need to read lots, test loads and watch customers even more
• Ask someone to mentor you
• Go to conferences, events, find people online
• Don’t say “I think” – say “I’ll find out” or “I’ll test that”
• Volunteer for shit jobs and do them well
• Be the glue between people – volunteer for cross team work
• Fix bad things so your team loves you
• Always connect others with outcomes
• Hack your job to a better life
• Make yourself more valuable, not less
• Challenge fucking everything
• Be annoying and never give up!
72. Don’t give up!
• Start with the customer at the heart of everything
• It’s liberating!
• Use low budget solutions, cheap or free tools to get business cases
made or gain insight
• Money talks so even micro testing will illustrate ROI
• Invest in people who care about customers, as well as conversion
• Invest in analytics talent and instrumentation
• If you can make a 10% shift in a key metric (CTR, conversion), just
from smart thinking, then what would a team of 5 get you?
• If you want to hire people, ask me for advice!
75. More reading. Slides and resources on slideshare.net
Email : sullivac@gmail.com
Twitter : @OptimiseOrDie
: linkd.in/pvrg14
Slideshare : slidesha.re/nlCDm6
Editor's Notes
“A piece of paper with your design mockup. A customer in a shop or bookstore. Their finger is their mouse, the paper their screen. Where would they click? Do they know what these labels mean? Do they see the major routes out of the page? Any barriers.Congratulations, you just got feedback on your design, before writing a single freaking line of code or asking your developers to keep changing stuff.”
“A piece of paper with your design mockup. A customer in a shop or bookstore. Their finger is their mouse, the paper their screen. Where would they click? Do they know what these labels mean? Do they see the major routes out of the page? Any barriers.Congratulations, you just got feedback on your design, before writing a single freaking line of code or asking your developers to keep changing stuff.”
“A piece of paper with your design mockup. A customer in a shop or bookstore. Their finger is their mouse, the paper their screen. Where would they click? Do they know what these labels mean? Do they see the major routes out of the page? Any barriers.Congratulations, you just got feedback on your design, before writing a single freaking line of code or asking your developers to keep changing stuff.”
Create a suck index = pageviews * load time.
Here I show you some examples of well known brands, some of whom should know better. The larger the size of the page, the longer it will take to download and render on the device, especially when you don’t have perfect data conditions. The numberof requests also makes a difference, as it’s inefficient on mobile to open lots of connections like this. In short, the smaller the pagesize and number of requests you can aim for, the better. I’m patient with bad data connections but do people have the tolerance for 10-15 seconds on mobile? No – it has to happen much faster.
These are the results of a live test on a site, where an artificial delay is introduced in the performance testing. I’ve done some testing like this myself on desktop and mobile sites and confirm this is true – you’re increasing bounce rate, decreasing conversion, site engagement…It doesn’t matter what metric you use, performance equals MONEY or if not measured, a HUGE LOSS.
Performance also harms the lifeblood of e-commerce and revenue generating websites – repeat visitors! The gap here in one second of delay is enormous over time. You’re basically sucking a huge portion of potential business out of your site, with every additional bit of waiting time you add.
Add unique phone numbers to all your mobile sites and apps. That’s for starters.Then configure your analytics to collect data when people Click or Tap to make a phone call.Make sure you add other events like ringbacks, email, chat – any web forms or lead gen activity too.
Add unique phone numbers to all your mobile sites and apps. That’s for starters.Then configure your analytics to collect data when people Click or Tap to make a phone call.Make sure you add other events like ringbacks, email, chat – any web forms or lead gen activity too.
So what does this graph say? That I have a long tail thing I want to talk to you about?No – this shows how much the ratio of phone to online conversion we have, by keyword.Some keywords generate nearly 25 times the call volume of others, which is a huge differential.This means that if you thought you got ‘roughly’ the same proportion of phone calls for different marketing activity, you are wrong.What this graph tells me is that the last 2 years of my stats are basically a big dog poo.
So what does this graph say? That I have a long tail thing I want to talk to you about?No – this shows how much the ratio of phone to online conversion we have, by keyword.Some keywords generate nearly 25 times the call volume of others, which is a huge differential.This means that if you thought you got ‘roughly’ the same proportion of phone calls for different marketing activity, you are wrong.What this graph tells me is that the last 2 years of my stats are basically a big dog poo.
Add unique phone numbers to all your mobile sites and apps. That’s for starters.Then configure your analytics to collect data when people Click or Tap to make a phone call.Make sure you add other events like ringbacks, email, chat – any web forms or lead gen activity too.
Phone tracking costs you nothing – you can add it in a few minutes to your app or mobile website, by changing your analytics tracking.Now you can see exactly which bits of inbound marketing are driving telephone and other contact channelsIf you have any sort of phone component in your service or support, the insight could be vitalYou can take traffic by keyword, source, campaign or advert creative and work out the TRUE mix of conversion activityAnd all this is also available on Desktop too – by using dynamic numbers, we can track exactly the same stuff.Talk to this company : www.infinity-tracking.com