The document details a comparative evaluation workshop led by Sónia Sousa in 2015, focusing on iterative product benchmarking and user experience (UX) assessment. It outlines methodologies for evaluating concepts, collecting and analyzing data, and defining user targets, alongside strategies for comparing features among competing services. Ultimately, it emphasizes the importance of adapting ideas based on market analysis and presenting findings effectively.

1. Workshop 1
Comparative evaluation
2015 Sónia Sousa 1

2. Comparative evaluation
• Evaluation of concepts:
– Iterative product benchmarking
• lab studies, before usage, UX experts
• Aims of benchmarking
– Assess the competition
– Mature own idea
• When to stop
– When the resulting table is saturated (when
adding new products does not bring added value)
2015 Sónia Sousa 2

3. The result
2015 Sónia Sousa 3
Possible(feautures Service(1 service((2 service((3 service((4
feature(1 x x x x
feature(2 x
feature(3 x x x
feature(4 x x x x
feature(5 x x x
feature(6 x
feature(7 x x
feature(8 x
feature(9 x
feature(10 x
feature(11
feature(12
feature(13
feature(14
Extra
features

4. Methodology
• Benchmarking method
– Select subject
– Define the process
– Identify potential partners
– Identify data sources
– Collect data and select
partners
– Determine the gap
– Establish process differences
– Target future performance
– Communicate
– Adjust goal
– Implement
– Review and recalibrate
• Adaptation method
– Clearly define your idea
– Define the process
– Identify similar solution in the
market
2015 Sónia Sousa 4

5. Define your UX evaluation target
How to do that?
2015 Sónia Sousa 5

6. What to evaluate
1. Define your idea
– How to do that?
• Turn to your colleague and describe what your product
represents to you in 5 seconds
– Describe the service or product you wont to create.
• List the main ideas on the paper
2015 Sónia Sousa 6

7. Identify the users
2015 Sónia Sousa 7
3. Identify the participants
– Clearly identify who is going to use your system.
• Age, gender, technical knowledge, social status and so
on

8. What to observe
2. Define the process
– What you need to observe
• Tasks, content, layout, and so on
– Answer to the questions
• What is my strategy to achieve my objectives?
• What is the order of my actions?
• Will I need to use different resources in my actions?
2015 Sónia Sousa 8

9. Data collection
• Select the similar services you are going to
analyze
– Collect data
• Enumerating for each the most used features
– Follow this link and start to fill
• the Gdocs spread sheet table
2015 Sónia Sousa 9

10. Data analysis
• Compare the data collected
– Aggregate and describe the similar features
– Describe the specific features
– Propose additional features for your service
• Look for the one that add a extra value to your service
• Or differentiate your service from others
2015 Sónia Sousa 10

11. Present the results
• Follow this link and use this
– Gdocs presentation file to present your ideas
2015 Sónia Sousa 11