"To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information in this case study. All information in this case study is my own and does not necessarily reflect the views of ASUS." it's an internal project related to usability test. I reveal this project because this project have been released for two years. I think this research methodology is easy for beginner who wants to run a low budget research to examine their own product.

1. FaceKey(Facelogon)
Usability Report
Report published 2013
Software launched 2014
Report edition Jun 2018
40.17% of installs come from United Sates Should I remove ASUS FaceLogon?
76% of users will keep it.
To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information in this case study. All information in this case study is my own and
does not necessarily reflect the views of ASUS.
https://www.shouldiremoveit.com/. 10 Jun 2018

2. RESEARCH PLAN
PREPARATION INTERVIEW/DEBRIEF REPORT
•Define - Goals
•Scope
•Prepare (user testing process )
•Recruit (10-11people, Arrange
schedule)
•Materials
•Paperwork
(non- disclosure agreement,
questionnaire )
•Assessment
(Success rates,Task time,Satisfaction
questionnaire)
•Interview
•Debrief
•Result
•insights

3. Our high level goals were to
1. Make it fast and easy use for ASUS laptop user.
2. Give user more confident to use this Face recognition technology.
MY Role
My responsibility toward this project is conducted a usability test and I am a facilitator during test
and then my and coworker to deliver this report together.
I cooperated with a UX designer to interview 11 stakeholders in order to process the usability report
of FaceKey. Also, I worked alongside with couple product managers and supervisors.
The App launched globally on end of January, 2014.
Goal
Our goal for project was to examine the Usability of FaceKey. Due to the product manager
wants to preload FaceKey on ASUS PC, they request us to examine the Usability of
FaceKey. And deliver a report for the improvement.
PREPARATION

4. User scenario:
Test Level 1- Only one windows account, user can register FaceKey.
Task level 2- After user register FaceKey successfully, user able to use FaceKey to login the
Window account.
Task level 3- After user register FaceKey successfully, user able to Disable and Enable
FaceKey.
Task level 4- In two Windows accounts, user able to select an account, we assigned, to
register a FaceKey account.
PREPARATION

5. 2 Main Role of a test
Role for test
Participants /Facilitator/Observers
Responsibility
Facilitator
1. Responsible for safety & comfort of users
2. Introduce the testing process
3. When participant feels confused or need any help, facilitator will give tips.
2. Encourage then to think loud
3. Watch their body language, facial expression, emotions.
Observers
1. Taking picture and record everything we might needs.
2. Note taking
3. Watch their body language, facial expression, emotions.
4. Asking Facilitator to clarify some points with users, they wants to know.
1. User doesn’t have any experience to use FaceKey.
2. They need to accomplish four tasks:
Task level 1 / Task level 2 / Task level 3 / Task Level 4
3. User can process the very task intuitively.
4. Asking user to think aloud to let us understand
participant’s though.
User Testing Methodology
PREPARATION

6. Recruiting Participants
Age : 20 - 40 Number of Participants : 11people
Sampling frame:
1. They own at least one comical device such as laptop, desktop PC.
2. They usually use laptop at home or at their work place.
3. They have an experience of using laptop to process standard software packages
(eg Microsoft Office).
PREPARATION

7. Interview location:
13 F 雪梨梨會議室
Test set-up
We use a meeting room to create a real working environment (e.g.
office).
Equipment:
•Notebook x2
•Camera
•FaceKey software x1
•Pen
•Post-it
13 F 雪梨會議室
Video
camera
Notebook
Notebook
PREPARATION

8. PREPARATION - 30mins interview break down
3-5mins 25mins 5mins
Warn up Prototype Questionnaire
•Explain why and how
•Small talk/ ice breaking
•Warn up question
• Asking participants to complete tasks.
• Remind them to think out loud.
• Don’t ask users for a solution, and looking
for the reasons behind to inspire design
solutions.

9. PREPARATION -Define User Testing Process
First
time
User
Test Level 1: Only one windows account, user can register FaceKey.
1.使⽤者開始使⽤
FaceKey
UX Observer
1.User behavior issue
2. Time measurement
2.使⽤者enable FaceKey 3.輸⼊password
enable
FaceKey
4.使⽤者在Second
factor須選擇None
5.FacKey註冊成功 6.Sign in window
account
Start to register
FaceKey
Enter
password
Second
factor
Sign in window
account
FaceKey
successfully

10. First
time
Task level 2 After user register FaceKey successfully, user able to use FaceKey to login the
Window account.
2. 成功Sign in window account1.Lock Screen狀態下，使⽤者
利⽤FaceKey sign in
UX Observer
1.User behavior issue
2. Time measurement
User
Windows
Login page
Windows
Wallpaper
PREPARATION -Define User Testing Process

11. Task level 3 After user register FaceKey successfully, user able to Disable and Enable FaceKey.
First
time
1.Lock Screen狀態下，使⽤者
利⽤FaceKey sign in
2.使⽤者須先執⾏
disable FaceKey
3.使⽤者再執⾏
enable FaceKey
4.成功Enable FaceKey後，
會進⼊lock screen.
UX Observer
1.User behavior issue
2. Time measurement
User Enable
FaceKey
Registed
KeyKey
Disable
FaceKey
FaceKey on lock
screen
PREPARATION -Define User Testing Process

12. First
time
Task level 4 In two Windows accounts, user able to select an account, we assigned, to register a FaceKey account.
1. 使⽤者選擇⾃⼰帳號登⼊ 2.輸⼊Password 3.使⽤者在Second
factor須選擇None
4.FacKey註冊成功 5.Sign in window
account
UX Observer
1.User behavior issue
2. Time measurement
Second
factor
FaceKey register
successfully
log in window
account
User
Select one
windows
account
Enter
password
PREPARATION - Define User Testing Process

13. User
enable
FaceKey
Enter
password
Second
factor
Sign in window
account
Process
1. sometimes, user can’t find the register icon from windows login pageObservation
1. User Can’t enable application and launched register page.
2. User felt confused while the first time to register FaceKey.
3. User preferred to click Facekey icon to register rather than click text area.
Test Level 1: Only one windows account, user can register FaceKey.
Start to
register
FaceKey
Main issues:
First
time
FaceKey
successfully
INTERVIEW - User Testing Process
Difficult
Middle
Easy
Need asking for assistance

14. Task level 2 After user register FaceKey successfully, user able to use FaceKey to login the Window account.
User
Windows
Login page
Process
1. Log in the windows account successfullyObservation
Difficult
Middle
Easy
Need asking for assistance
1. When user use FaceKey to login first time, the pace of process is too fast, they
have a feeling of insecurity.
2. However, the pace of process will be accepted when they use this FaceKey
second time.
Main issues:
Windows
Wallpaper
INTERVIEW - User Testing Process

15. User Enable
FaceKey
Registed
KeyKey
Disable
FaceKey
Meddle- User found FaceKey in
tray icon, and able to right click to
disable/ enable APP.
009
Middle - User able to find the
FaceKey in Tray icon and launch APP
008
Asking for assistance - User cannot
Find FaceKey icon under desktop
mode.
010
Process
Observation
Task level 3 After user register FaceKey successfully, user able to Disable and Enable FaceKey.
Difficult
Middle
Easy
Need asking for assistance
1.User can disable on lock screen
2.User can’t find FaceKey icon in Desktop mode
3.User ask assistance to do them a favor to complete this tasks
FaceKey on lock
screen
INTERVIEW - User Testing Process

16. Second
factor
FaceKey register
successfully
log in window
account
User
Select one
windows
account
Difficult
Middle
Easy
Need asking for assistance
Task level 4 In two Windows accounts, user able to select an account, we assigned, to register a FaceKey account.
Enter
password
Click
Back
button
1. User don’t know needing to click Back button.
Main issues:
1. User don’t know needing to click BACK button to go back to
their own window account
INTERVIEW - User Testing Process

17. RESULT- Task Completion Rate
Participant Task1 Task2 Task3 Task4
1 V V - V
2 V V - V
3 V V V V
4 V V - V
5 - V V V
6 V - - V
7 V V - V
8 V V V -
9 V V V -
10 V V - V
11 V V - V
Success 10 10 4 9
Completion
Rate
0.91 0.91 0.36 0.81
9 of the 11 (91%) participants were
able to completed Task 1 and Task2.
9 of 11 (81%) participants were able
to completed Task 4.
However , only 4 out of 11 (36%)
participants able to complete Task3.
Conclusion:
1.User get confused when they try
to find out disable/ enable button.
2. User can’t switch the account
easily.

18. RESULT- Time on Task
Task1 Task2 Task3 Task4
1 55 5 - 115
2 133 5 120 56
3 23 8 35 25
4 29 29 29 28
5 51 15 63 70
6 91 40 190 114
7 49 14 100 52
8 47 10 185 52
9 81 7 128 35
10 37 10 238 17
11 42 27 60 17
Avg. TOT 58 15 115 53
Task 3 required participants to find
the FaceKey icon in the tray icon and
took longed time to complete
(mean=115 seconds).
However, completion time ranged
from 29 seconds to 238 seconds.
Conclusion:
1.Enable and disable function
needs explorable for user.
2. When user wants to register
another account needs to easy to
find out the registration button.
3. User feels unsafely, when they
login in too fast by using FaceKey.
(Sec)

19. RESULT- Questionnaire
Over all working process
User interface design
Register Process
Enable/ disable process
0 11
Confusing Clear1 2 3 4 5
9% 27% 36% 27%
9% 27%36%18%
18%55%27%
18% 18%18%45%

20. Give user feedback
when they complete a
task
During the Task 1, user can’t realize they
registered successfully after they register
FaceKey.
” 哇.. 這個字好多唷，⽽而且都是英
⽂文，要花很多時間讀 ”
Add some pictures or tutorial to guid user
instead of long description.
Insight 1: Insight 2:
” 字太多，感覺很難⽤用”
Keep user interface tidy .
Focus on one thing at the
time.
Insight 3:
Provide the information when user really need.
” 按鈕跟字很多，所以都亂點⼀一下，才註
冊成功”
INSIGHTS-REGISTER PROCESS
People rely on tutorial or
hints to talk them what
to do, so they don't want
to spend time to read
the instruction.
”頭像可以點嗎？”
” 要先enable 還是先Password setup”

21. ” 建議將註冊跟登⼊入操作分開，介⾯面
東⻄西太多”
The user interface combine registration and
login process at one page. Thus, Facekey
can only show register for set up process, and
then can put register in the setting page.
Keep user interface tidy .
Focus on one thing at the
time.
Separate the registration
and login page process.
Insight 4:
Use accurate
component of user
interface to guide user.
Use”Register FaceKey” button instead of
“Enable” check box to process registration .
Insight 5:
” 只有看到Enable 的check box,很難
知道他跟FaceKey 註冊有關 ”
For the task 4, we tend to examine if user able
to create a FaceKey account when they have
two windows account in one laptop. However,
after interview, we notice this scenario will not
happen frequently. People tend to not share
one laptop, they only share one laptop when
they have teenager or kids,. And they will just
buy reasonable (cheaper) price of laptop for
general use.
Insight 6:
INSIGHTS- Registered another account
Little amount of
people share one

22. User have a right to
disable/ enable APP
easily
Task 3 is the most difficult one, only 4 of 11
participants (36%) able to complete tasks. The
mani reasons :
1. FaceKey process really fast, so user
cannot disable from lock screen,
2. User can’t find the faceKey icon easily
from try icon. They tend to open MY
COMPUTER to find out the APP.
Can show FaceKey icon in the start menu, so
user can easy explore this APP.
Insight 7:
” Enable/ Disable 實在太難找了了”
INSIGHTS- Enable/ Disable FaceKey
” 如果在登⼊入⾴頁⾯面FaceKey跑得⽐比較
慢，就可以輕易易的disable,因爲按鈕
就在上⾯面”
” 如果辨識率不不⾼高，我會想要disbale
這個功能，可是卻很難找”
INSIGHTS- login
Insight 8:
User feel
It’s an really interesting insight, user feel
Insecure when FaceKey login too fast. During
the testing process , when user open the
laptop, the windows just shows wallpaper
right away without showing login page. How to
make user have more confident to user this
APP and how to build trust. Can be a topic to
have further discussion.
” 打開電腦，⾺馬上進⼊入windows桌
⾯面！登⼊入太快了了，以為沒有辨識”
”哇，怎麼那麼快，這樣安全嗎？“