SlideShare a Scribd company logo
Using data to
make informed
design decisions.
Mike Parker
User Experience Director
CA Technologies
?
The ask.
2
The problem.
3
?
Larger problem.

4
+ 20%
…
…
…
Organization
UX maturity.
Organization
UX maturity.
oh dear.
7
Gain a baseline
understanding of
your product’s
performance. 

8
How?



Usage analytics

Call support logs

Contextual interviews

Sales representative interviews

Usability testing

Jira tickets

Social media 

3rd party measurement

9
How?



Usage analytics

Call support logs

Contextual interviews

Sales representative interviews

Usability testing 

Jira tickets

Social media

3rd party measurement

10
Where we started.



11
Where we started.



12
Where we started.
13
$5,535
support cost
Look for KPIs that matter.
84
line setup18,297
customer calls
17 mins
avg. length
1.1
avg. calls per
customer
123
web portal
support tickets
$402,534
support cost
$45
per ticket
28
AR setup
11
HG setup
52%
web portal
$209,308
support cost
9,514
web portal
customer calls
$22
per call
$5,535
support cost
Focus on biggest impact.
84
line setup18,297
customer calls
17 mins
avg. length
1.1
avg. calls per
customer
123
web portal
support tickets
$402,534
support cost
$45
per ticket
28
AR setup
11
HG setup
52%
web portal
$209,308
support cost
9,514
web portal
customer calls
$22
per call
16
AR Issues Severity Effort Est. Mo. Cost Priority
Locating the AR feature H L $3,586 H
Special characters not allowed in Caller ID field M L $2,492 H
Incorrect setup of advanced features (i.e., Simultaneous Ring, Bridging, etc.) H M $3,827 H
Call Forwarding Busy feature set to on by default. M L $1,407 M
Active status for AR when not setup M L $178 M
Setting up AR call routing M H $1,157 M
Recording AR custom greetings M H $1,068 M
Setting up AR schedule M H $938 M
Build a case.
$5,535
support cost
Sell your case.
84
line setup18,297
customer calls
17 mins
avg. length
1.1
avg. calls per
customer
123
web portal
support tickets
$402,534
support cost
$45
per ticket
28
AR setup
11
HG setup
52%
web portal
$209,308
support cost
9,514
web portal
customer calls
$22
per call
Sell your case.
$5,535
support
cost
84
line setup
18,297
customer
calls
17 mins
avg. length
1.1
avg. calls
per
customer
123
web portal
support
tickets
$402,534
support cost
$45
per ticket
28
AR setup
11
HG setup
52%
web portal
$209,308
support cost
9,514
web portal
customer
calls
$22
per call
ROI
$59,846monthly savings
EFFORT
4 weeks
So who is this guy anyway?
19
“Not only does this
thing work great,
it’s actually fun to
use too.”

- Jeffrey W.
20
Fix this.
21
22
How will you know it’s fixed?
23
Back to baseline.
Test the before
(Baseline)
Test the after
(Benchmark)
24
Quantitative
Success / failure
Time on task
Number of clicks
Which KPIs to measure?
Qualitative
Ease of use
Confidence
Consistency
25
1. I think that I would like to use this system frequently.
2. I found the system to be simple.
3. I thought the system was easy to use.
4. I can use the system without the support of a technical person.
5. The various functions in the system were well integrated.
6. I thought there was a lot of consistency in the system.
7. I would imagine that most people would learn to use the system very quickly.
8. I found the system very intuitive.
9. I felt confident using the system.
10. I could use the system without having to learn anything new.
Don’t forget your SUS.
26
Online, Unmoderated.
UserZoom.
Clickable prototype.

8 target participants, non-customers (no biases).

5-7 tasks.

Qualitative questionnaire following each task.

SUS questionnaire at test completion.
Same design for the before and after test.
Decide on methodology.
Put it all together.
Task 1
Task 2
Test
Screen 1
Test
Screen 2
Task 1
Instruction
Question-
naire
Test
Screen 2
Test
Screen 3
Task 2
Instruction
Question-
naire
SUS
Each task should be
finite.
Participant attempts
task on screen 1.
When testing a flow, break the larger task
into bite size chunks that can be
completed by the user 1 screen at a time.
Screen 2 provides the participant with
closure so they can provide meaningful
answers to the questionnaire.
Start the next task at the
previous closure page for
continuity.
27
Quantitative (Automatic):
• Success / Fail
• Time to complete
• Number of clicks
Qualitative:
• Ease of Use
• Confidence
• Consistency
28
Do it.
1.
Usability test.
Establish baseline.
Understand problems.
3.
Usability test.
Benchmark progress.
Validate design decisions.
2.
Design solutions.
29
improvement in user confidence (that they’ve
completed the task correctly)
The result.
32%
20%
37%
improvement in flow efficiency (or time the user spent
completing the AR setup flow)
improvement in ease of use
3030
Study comparison.
3131
SUS comparison.
BenchmarkBaseline
90.2562.25
0 10 20 30 40 50 60 70 80 90 100
F D C B A
3231
UX ROI.
45%Easier to use
$40,096
Monthly cost savings
28Issues fixed
Total UX effort5 wks
33
AR Redesign
Benchmark usability study
Call center visit
w1 w2 w3 w4 w5
What it looked like.
Improvements usability study
Refine
34
Rock on!



There’s no right way to include your users.

Always start with a baseline.

Look for impact.
Sell it.
Track. Measure.
Sell it.



Mike Parker
User Experience Director,
Agile Management, CA Technologies
@parkahux
35
Thank you.

More Related Content

What's hot

9 things to consider when implementing a telematics solution
9 things to consider when implementing a telematics solution9 things to consider when implementing a telematics solution
9 things to consider when implementing a telematics solution
Travis Claeys
 
Enabling the Shift-left Service Strategy
Enabling the Shift-left Service StrategyEnabling the Shift-left Service Strategy
Enabling the Shift-left Service Strategy
McGarahan & Associates, Inc.
 
Prophit Forecast Manager - Supply Chain Planning - Prepare for future demand
Prophit Forecast Manager - Supply Chain Planning - Prepare for future demandProphit Forecast Manager - Supply Chain Planning - Prepare for future demand
Prophit Forecast Manager - Supply Chain Planning - Prepare for future demand
Tim Gray ♦ Supply Chain Solutions Expert
 
Stop using spreadsheets to monitor quality in your call centre!
Stop using spreadsheets to monitor quality in your call centre!Stop using spreadsheets to monitor quality in your call centre!
Stop using spreadsheets to monitor quality in your call centre!
scorebuddy
 
After sales in a product failure scenario
After sales in a product failure scenarioAfter sales in a product failure scenario
After sales in a product failure scenario
Hari Talupur
 
Lect8
Lect8Lect8
N4B SRL & SOLARWINDS MSP - 5 STEPS TO TAKE BACKUP FROM GOOD TO GREAT
N4B SRL & SOLARWINDS MSP - 5 STEPS TO TAKE BACKUP FROM GOOD TO GREATN4B SRL & SOLARWINDS MSP - 5 STEPS TO TAKE BACKUP FROM GOOD TO GREAT
N4B SRL & SOLARWINDS MSP - 5 STEPS TO TAKE BACKUP FROM GOOD TO GREAT
Alberto Ferretti
 
Keeping customer satisfaction high final
Keeping customer satisfaction high finalKeeping customer satisfaction high final
Keeping customer satisfaction high final
Ghaji
 
Akira brand strategy
Akira brand strategyAkira brand strategy
Akira brand strategy
Herminio Adorna
 
Microsigns Platform Benefits
Microsigns Platform BenefitsMicrosigns Platform Benefits
Microsigns Platform Benefits
Microsigns Inc.
 
Arun Dutta: Digital Performance Monitoring
Arun Dutta: Digital Performance MonitoringArun Dutta: Digital Performance Monitoring
Arun Dutta: Digital Performance Monitoring
Anna Royzman
 
Finance Presentation
Finance PresentationFinance Presentation
Finance Presentation
Tim Demorais
 
"Lean startups : what happens when you put hardware into the mix?" - Gal Shau...
"Lean startups : what happens when you put hardware into the mix?" - Gal Shau..."Lean startups : what happens when you put hardware into the mix?" - Gal Shau...
"Lean startups : what happens when you put hardware into the mix?" - Gal Shau...
Product of Things
 
Lean Six Sigma
Lean Six SigmaLean Six Sigma
Lean Six Sigma
Shankaran Rd
 
Business Process Optimization PowerPoint Presentation Slides
Business Process Optimization PowerPoint Presentation Slides Business Process Optimization PowerPoint Presentation Slides
Business Process Optimization PowerPoint Presentation Slides
SlideTeam
 
Design of experiments (doe) for process development and validation
Design of experiments (doe) for process development and validationDesign of experiments (doe) for process development and validation
Design of experiments (doe) for process development and validation
GlobalCompliancePanel
 

What's hot (16)

9 things to consider when implementing a telematics solution
9 things to consider when implementing a telematics solution9 things to consider when implementing a telematics solution
9 things to consider when implementing a telematics solution
 
Enabling the Shift-left Service Strategy
Enabling the Shift-left Service StrategyEnabling the Shift-left Service Strategy
Enabling the Shift-left Service Strategy
 
Prophit Forecast Manager - Supply Chain Planning - Prepare for future demand
Prophit Forecast Manager - Supply Chain Planning - Prepare for future demandProphit Forecast Manager - Supply Chain Planning - Prepare for future demand
Prophit Forecast Manager - Supply Chain Planning - Prepare for future demand
 
Stop using spreadsheets to monitor quality in your call centre!
Stop using spreadsheets to monitor quality in your call centre!Stop using spreadsheets to monitor quality in your call centre!
Stop using spreadsheets to monitor quality in your call centre!
 
After sales in a product failure scenario
After sales in a product failure scenarioAfter sales in a product failure scenario
After sales in a product failure scenario
 
Lect8
Lect8Lect8
Lect8
 
N4B SRL & SOLARWINDS MSP - 5 STEPS TO TAKE BACKUP FROM GOOD TO GREAT
N4B SRL & SOLARWINDS MSP - 5 STEPS TO TAKE BACKUP FROM GOOD TO GREATN4B SRL & SOLARWINDS MSP - 5 STEPS TO TAKE BACKUP FROM GOOD TO GREAT
N4B SRL & SOLARWINDS MSP - 5 STEPS TO TAKE BACKUP FROM GOOD TO GREAT
 
Keeping customer satisfaction high final
Keeping customer satisfaction high finalKeeping customer satisfaction high final
Keeping customer satisfaction high final
 
Akira brand strategy
Akira brand strategyAkira brand strategy
Akira brand strategy
 
Microsigns Platform Benefits
Microsigns Platform BenefitsMicrosigns Platform Benefits
Microsigns Platform Benefits
 
Arun Dutta: Digital Performance Monitoring
Arun Dutta: Digital Performance MonitoringArun Dutta: Digital Performance Monitoring
Arun Dutta: Digital Performance Monitoring
 
Finance Presentation
Finance PresentationFinance Presentation
Finance Presentation
 
"Lean startups : what happens when you put hardware into the mix?" - Gal Shau...
"Lean startups : what happens when you put hardware into the mix?" - Gal Shau..."Lean startups : what happens when you put hardware into the mix?" - Gal Shau...
"Lean startups : what happens when you put hardware into the mix?" - Gal Shau...
 
Lean Six Sigma
Lean Six SigmaLean Six Sigma
Lean Six Sigma
 
Business Process Optimization PowerPoint Presentation Slides
Business Process Optimization PowerPoint Presentation Slides Business Process Optimization PowerPoint Presentation Slides
Business Process Optimization PowerPoint Presentation Slides
 
Design of experiments (doe) for process development and validation
Design of experiments (doe) for process development and validationDesign of experiments (doe) for process development and validation
Design of experiments (doe) for process development and validation
 

Similar to Using data to inform design decisions in a UX immature organization - Big Design Conference 2017

Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
UserZoom
 
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...
UserZoom
 
Clover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive ExperimentationClover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive Experimentation
Optimizely
 
The ROI Of User Experience: Consider, Calculate & Measure Success
The ROI Of User Experience: Consider, Calculate & Measure SuccessThe ROI Of User Experience: Consider, Calculate & Measure Success
The ROI Of User Experience: Consider, Calculate & Measure Success
UserZoom
 
Is43 Developing A Control System Scope Of Work V2.0 Final
Is43 Developing A Control System Scope Of Work V2.0 FinalIs43 Developing A Control System Scope Of Work V2.0 Final
Is43 Developing A Control System Scope Of Work V2.0 Final
stevegreenblatt
 
Survey Results: Common Problems with Microsoft Teams Call Quality
Survey Results: Common Problems with Microsoft Teams Call QualitySurvey Results: Common Problems with Microsoft Teams Call Quality
Survey Results: Common Problems with Microsoft Teams Call Quality
panagenda
 
Strengthen your copy in 35 minutes: Proven strategies to boost the effectiven...
Strengthen your copy in 35 minutes: Proven strategies to boost the effectiven...Strengthen your copy in 35 minutes: Proven strategies to boost the effectiven...
Strengthen your copy in 35 minutes: Proven strategies to boost the effectiven...
MarketingExperiments
 
SearchLove London 2016 | Stephen Pavlovich | Habits of Advanced Conversion Op...
SearchLove London 2016 | Stephen Pavlovich | Habits of Advanced Conversion Op...SearchLove London 2016 | Stephen Pavlovich | Habits of Advanced Conversion Op...
SearchLove London 2016 | Stephen Pavlovich | Habits of Advanced Conversion Op...
Distilled
 
Ahead of the Curve: How 23andMe Improved UX with Performance Edge
Ahead of the Curve: How 23andMe Improved UX with Performance EdgeAhead of the Curve: How 23andMe Improved UX with Performance Edge
Ahead of the Curve: How 23andMe Improved UX with Performance Edge
Optimizely
 
4 ie 2015 rapid research
4   ie 2015 rapid research4   ie 2015 rapid research
4 ie 2015 rapid research
Maggie Nichols
 
How to build confidence in your release cycle
How to build confidence in your release cycleHow to build confidence in your release cycle
How to build confidence in your release cycle
DiUS
 
Growth Hacking at Philips Lighting
Growth Hacking at Philips LightingGrowth Hacking at Philips Lighting
Growth Hacking at Philips Lighting
KOOACH
 
Lessons Learned - Insights to Improve Support for MS Teams in a Hybrid Work E...
Lessons Learned - Insights to Improve Support for MS Teams in a Hybrid Work E...Lessons Learned - Insights to Improve Support for MS Teams in a Hybrid Work E...
Lessons Learned - Insights to Improve Support for MS Teams in a Hybrid Work E...
panagenda
 
Grothues ndc oslo_2016
Grothues ndc oslo_2016Grothues ndc oslo_2016
Grothues ndc oslo_2016
Katrin Grothues
 
User Acceptance Testing in the Testing Center of Excellence
User Acceptance Testing in the Testing Center of ExcellenceUser Acceptance Testing in the Testing Center of Excellence
User Acceptance Testing in the Testing Center of Excellence
TechWell
 
Creating Your Dashboard & Universal Measures with Userzoom – The Deep-Dive Ho...
Creating Your Dashboard & Universal Measures with Userzoom – The Deep-Dive Ho...Creating Your Dashboard & Universal Measures with Userzoom – The Deep-Dive Ho...
Creating Your Dashboard & Universal Measures with Userzoom – The Deep-Dive Ho...
UserZoom
 
Quality control and quality managment philosophies by Prof.Saurabh S. Bhange
Quality control and quality managment philosophies by Prof.Saurabh S. BhangeQuality control and quality managment philosophies by Prof.Saurabh S. Bhange
Quality control and quality managment philosophies by Prof.Saurabh S. Bhange
Saurabh Bhange
 
How BiggerPockets runs experiments faster with performance edge
How BiggerPockets runs experiments faster with performance edgeHow BiggerPockets runs experiments faster with performance edge
How BiggerPockets runs experiments faster with performance edge
Optimizely
 
Competitive UX Benchmarking: How Four Healthcare Insurance Sites Scored Acros...
Competitive UX Benchmarking: How Four Healthcare Insurance Sites Scored Acros...Competitive UX Benchmarking: How Four Healthcare Insurance Sites Scored Acros...
Competitive UX Benchmarking: How Four Healthcare Insurance Sites Scored Acros...
UserZoom
 
Growth Hacking at Philips Lighting
Growth Hacking at Philips LightingGrowth Hacking at Philips Lighting
Growth Hacking at Philips Lighting
Gunter Blanckaert
 

Similar to Using data to inform design decisions in a UX immature organization - Big Design Conference 2017 (20)

Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
 
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...
BENCHMARKING MINI-SERIES PART #1: Proving Value & Quantifying the Impact of U...
 
Clover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive ExperimentationClover Rings Up Digital Growth to Drive Experimentation
Clover Rings Up Digital Growth to Drive Experimentation
 
The ROI Of User Experience: Consider, Calculate & Measure Success
The ROI Of User Experience: Consider, Calculate & Measure SuccessThe ROI Of User Experience: Consider, Calculate & Measure Success
The ROI Of User Experience: Consider, Calculate & Measure Success
 
Is43 Developing A Control System Scope Of Work V2.0 Final
Is43 Developing A Control System Scope Of Work V2.0 FinalIs43 Developing A Control System Scope Of Work V2.0 Final
Is43 Developing A Control System Scope Of Work V2.0 Final
 
Survey Results: Common Problems with Microsoft Teams Call Quality
Survey Results: Common Problems with Microsoft Teams Call QualitySurvey Results: Common Problems with Microsoft Teams Call Quality
Survey Results: Common Problems with Microsoft Teams Call Quality
 
Strengthen your copy in 35 minutes: Proven strategies to boost the effectiven...
Strengthen your copy in 35 minutes: Proven strategies to boost the effectiven...Strengthen your copy in 35 minutes: Proven strategies to boost the effectiven...
Strengthen your copy in 35 minutes: Proven strategies to boost the effectiven...
 
SearchLove London 2016 | Stephen Pavlovich | Habits of Advanced Conversion Op...
SearchLove London 2016 | Stephen Pavlovich | Habits of Advanced Conversion Op...SearchLove London 2016 | Stephen Pavlovich | Habits of Advanced Conversion Op...
SearchLove London 2016 | Stephen Pavlovich | Habits of Advanced Conversion Op...
 
Ahead of the Curve: How 23andMe Improved UX with Performance Edge
Ahead of the Curve: How 23andMe Improved UX with Performance EdgeAhead of the Curve: How 23andMe Improved UX with Performance Edge
Ahead of the Curve: How 23andMe Improved UX with Performance Edge
 
4 ie 2015 rapid research
4   ie 2015 rapid research4   ie 2015 rapid research
4 ie 2015 rapid research
 
How to build confidence in your release cycle
How to build confidence in your release cycleHow to build confidence in your release cycle
How to build confidence in your release cycle
 
Growth Hacking at Philips Lighting
Growth Hacking at Philips LightingGrowth Hacking at Philips Lighting
Growth Hacking at Philips Lighting
 
Lessons Learned - Insights to Improve Support for MS Teams in a Hybrid Work E...
Lessons Learned - Insights to Improve Support for MS Teams in a Hybrid Work E...Lessons Learned - Insights to Improve Support for MS Teams in a Hybrid Work E...
Lessons Learned - Insights to Improve Support for MS Teams in a Hybrid Work E...
 
Grothues ndc oslo_2016
Grothues ndc oslo_2016Grothues ndc oslo_2016
Grothues ndc oslo_2016
 
User Acceptance Testing in the Testing Center of Excellence
User Acceptance Testing in the Testing Center of ExcellenceUser Acceptance Testing in the Testing Center of Excellence
User Acceptance Testing in the Testing Center of Excellence
 
Creating Your Dashboard & Universal Measures with Userzoom – The Deep-Dive Ho...
Creating Your Dashboard & Universal Measures with Userzoom – The Deep-Dive Ho...Creating Your Dashboard & Universal Measures with Userzoom – The Deep-Dive Ho...
Creating Your Dashboard & Universal Measures with Userzoom – The Deep-Dive Ho...
 
Quality control and quality managment philosophies by Prof.Saurabh S. Bhange
Quality control and quality managment philosophies by Prof.Saurabh S. BhangeQuality control and quality managment philosophies by Prof.Saurabh S. Bhange
Quality control and quality managment philosophies by Prof.Saurabh S. Bhange
 
How BiggerPockets runs experiments faster with performance edge
How BiggerPockets runs experiments faster with performance edgeHow BiggerPockets runs experiments faster with performance edge
How BiggerPockets runs experiments faster with performance edge
 
Competitive UX Benchmarking: How Four Healthcare Insurance Sites Scored Acros...
Competitive UX Benchmarking: How Four Healthcare Insurance Sites Scored Acros...Competitive UX Benchmarking: How Four Healthcare Insurance Sites Scored Acros...
Competitive UX Benchmarking: How Four Healthcare Insurance Sites Scored Acros...
 
Growth Hacking at Philips Lighting
Growth Hacking at Philips LightingGrowth Hacking at Philips Lighting
Growth Hacking at Philips Lighting
 

Recently uploaded

5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 

Recently uploaded (20)

5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 

Using data to inform design decisions in a UX immature organization - Big Design Conference 2017

  • 1. Using data to make informed design decisions. Mike Parker User Experience Director CA Technologies ?
  • 8. Gain a baseline understanding of your product’s performance. 
 8
  • 9. How?
 
 Usage analytics
 Call support logs
 Contextual interviews
 Sales representative interviews
 Usability testing
 Jira tickets
 Social media 
 3rd party measurement
 9
  • 10. How?
 
 Usage analytics
 Call support logs
 Contextual interviews
 Sales representative interviews
 Usability testing 
 Jira tickets
 Social media
 3rd party measurement
 10
  • 14. $5,535 support cost Look for KPIs that matter. 84 line setup18,297 customer calls 17 mins avg. length 1.1 avg. calls per customer 123 web portal support tickets $402,534 support cost $45 per ticket 28 AR setup 11 HG setup 52% web portal $209,308 support cost 9,514 web portal customer calls $22 per call
  • 15. $5,535 support cost Focus on biggest impact. 84 line setup18,297 customer calls 17 mins avg. length 1.1 avg. calls per customer 123 web portal support tickets $402,534 support cost $45 per ticket 28 AR setup 11 HG setup 52% web portal $209,308 support cost 9,514 web portal customer calls $22 per call
  • 16. 16 AR Issues Severity Effort Est. Mo. Cost Priority Locating the AR feature H L $3,586 H Special characters not allowed in Caller ID field M L $2,492 H Incorrect setup of advanced features (i.e., Simultaneous Ring, Bridging, etc.) H M $3,827 H Call Forwarding Busy feature set to on by default. M L $1,407 M Active status for AR when not setup M L $178 M Setting up AR call routing M H $1,157 M Recording AR custom greetings M H $1,068 M Setting up AR schedule M H $938 M Build a case.
  • 17. $5,535 support cost Sell your case. 84 line setup18,297 customer calls 17 mins avg. length 1.1 avg. calls per customer 123 web portal support tickets $402,534 support cost $45 per ticket 28 AR setup 11 HG setup 52% web portal $209,308 support cost 9,514 web portal customer calls $22 per call
  • 18. Sell your case. $5,535 support cost 84 line setup 18,297 customer calls 17 mins avg. length 1.1 avg. calls per customer 123 web portal support tickets $402,534 support cost $45 per ticket 28 AR setup 11 HG setup 52% web portal $209,308 support cost 9,514 web portal customer calls $22 per call ROI $59,846monthly savings EFFORT 4 weeks
  • 19. So who is this guy anyway? 19
  • 20. “Not only does this thing work great, it’s actually fun to use too.”
 - Jeffrey W. 20
  • 22. 22 How will you know it’s fixed?
  • 23. 23 Back to baseline. Test the before (Baseline) Test the after (Benchmark)
  • 24. 24 Quantitative Success / failure Time on task Number of clicks Which KPIs to measure? Qualitative Ease of use Confidence Consistency
  • 25. 25 1. I think that I would like to use this system frequently. 2. I found the system to be simple. 3. I thought the system was easy to use. 4. I can use the system without the support of a technical person. 5. The various functions in the system were well integrated. 6. I thought there was a lot of consistency in the system. 7. I would imagine that most people would learn to use the system very quickly. 8. I found the system very intuitive. 9. I felt confident using the system. 10. I could use the system without having to learn anything new. Don’t forget your SUS.
  • 26. 26 Online, Unmoderated. UserZoom. Clickable prototype.
 8 target participants, non-customers (no biases).
 5-7 tasks.
 Qualitative questionnaire following each task.
 SUS questionnaire at test completion. Same design for the before and after test. Decide on methodology.
  • 27. Put it all together. Task 1 Task 2 Test Screen 1 Test Screen 2 Task 1 Instruction Question- naire Test Screen 2 Test Screen 3 Task 2 Instruction Question- naire SUS Each task should be finite. Participant attempts task on screen 1. When testing a flow, break the larger task into bite size chunks that can be completed by the user 1 screen at a time. Screen 2 provides the participant with closure so they can provide meaningful answers to the questionnaire. Start the next task at the previous closure page for continuity. 27 Quantitative (Automatic): • Success / Fail • Time to complete • Number of clicks Qualitative: • Ease of Use • Confidence • Consistency
  • 28. 28 Do it. 1. Usability test. Establish baseline. Understand problems. 3. Usability test. Benchmark progress. Validate design decisions. 2. Design solutions.
  • 29. 29 improvement in user confidence (that they’ve completed the task correctly) The result. 32% 20% 37% improvement in flow efficiency (or time the user spent completing the AR setup flow) improvement in ease of use
  • 31. 3131 SUS comparison. BenchmarkBaseline 90.2562.25 0 10 20 30 40 50 60 70 80 90 100 F D C B A
  • 32. 3231 UX ROI. 45%Easier to use $40,096 Monthly cost savings 28Issues fixed Total UX effort5 wks
  • 33. 33 AR Redesign Benchmark usability study Call center visit w1 w2 w3 w4 w5 What it looked like. Improvements usability study Refine
  • 34. 34 Rock on!
 
 There’s no right way to include your users.
 Always start with a baseline.
 Look for impact. Sell it. Track. Measure. Sell it.
 

  • 35. Mike Parker User Experience Director, Agile Management, CA Technologies @parkahux 35 Thank you.