CAPTCHA is a widely adopted security measure on the Web and is designed to effectively distinguish humans and bots by exploiting human’s ability to recognize patterns that an automated bot is incapable of. To counter this, bots are being designed to recognize patterns in CAPTCHAs. As a result, CAPTCHAs are now being designed to maximize the difficulty for bots to pass human interaction proof tests, while making it quite an arduous task even for humans as well. The approachability of CAPTCHA is increasingly being questioned because of the inconvenience it causes to legitimate users. Irrespective of the popularity, CAPTCHA is indispensable if one wants to avoid potential security threats. We investigated the usability issues associated with CAPTCHA. We built a holistic model by identifying the important concepts associated with CAPTCHAs and its usability. This model can be used as a guide for the design and evaluation of CAPTCHAs.

1. Security and User Experience:
A Holistic Model for CAPTCHA
Usability Issues
Jayalakshmi Raman, University of North Florida
Karthikeyan Umapathy, University of North
Florida
Haiyan Huang, Flagler College
March 23, 2018 Atlanta, GA
2018 Southern Association for Information Systems (SAIS) Annual Conference

2. CAPTCHA
Completely Automated Public Turing tests to tell
Computers and Humans Apart
A program that can distinguish humans from bots.
Picture source: https://www.letsnurture.com/blog/8-widely-used-captcha-examples.html

3. CAPTCHAs are Human Interaction Proofs
 CAPTCHA is designed as a challenge response test, that is,
 Simple enough for humans
 But hard for the bots
 These tests are typically a visual challenge as computers lack the ability
human eyes have, to process patterns.
 CAPTCHA design involves picking random string of characters (in case of
text-based CAPTCHAs) rendering into a distorted image.
HAT8M

4. Purpose of CAPTCHAs
 Websites featuring ability for visitors to comment,
register, signup, or post contents are exposed to
attacks from spam-robots.
 These malicious program’s harmful effects extend to
extracting private data, spamming web forms, and
swaying polls in websites.
 The purpose of CAPTCHA is to identify and block
malicious bots that may spam and/or make
unauthorized use of websites.
 CAPCTHAs are designed as the gateways of
websites to grant the access to “legitimate” site
visitors.
 CAPTCHA is widely adopted as a defense
mechanism across commercial websites to
determine whether a potential user is a human.
Source: http://ui-patterns.com/patterns/Captcha

5. Type of CAPTCHAs
Text-based
(Images of distorted text)
Image-based
(Set of images with
patterns among
them)
Source: https://www.letsnurture.com/blog/8-widely-used-captcha-examples.html, http://www.bespecular.com/blog/accessibility-of-captchas/
Audio-based
(Distorted sound
clips)
Math-based
(Basic math
problems)
3D CAPTCHAs
(animated texts or
verification code)
Puzzle-based
(Gamified puzzle
solvers)

6. Usability Issues of CAPTCHAs
 Usability of CAPTCHAs contributes significantly to the quality of user
experience one obtains from the website.
 With the advent of machine learning algorithms, deep learning
techniques and pattern recognition algorithms; bots are getting better
at reading CAPTCHAs.
 As a result, some additional features are incorporated into the design of
CAPTCHAs to make the tests harder for bots to pass.
 Improved CAPTCHAs sometimes are considered to be interfering with
usability and productivity because of their cumbersome nature.

7. Research Problem
 Limited amount of research studies on CAPTCHAs.
 As a widespread security measure encountered by most Internet users,
it is important to study CAPTCHAs state-of-the-art schemes and the
related usability issues.
 This research focuses on the usability factor in the domain of CAPTCHAs.
 The aim of this research is to develop a holistic framework that can shed
light on how to design effective and highly usable CAPTCHAs.
 This framework is developed based on empirical facts claimed in literature
thus serving as a model for evaluation for future CAPTCHA designs.

8. Research Methodology
 The aim of this research is to find the balance between usability and security
in CAPTCHAs.
 Conduct a comprehensive study to gain an in-depth understanding of user’s view of
CAPTCHA.
 Develop a holistic model that would in turn help in designing an effective and
adoptable CAPTCHA.
 We used a qualitative method proposed by Jabareen (2009) for conducting
systematic study of the phenomena of interest and building the conceptual
framework based on the analyzed concepts.
 A thorough understanding of relevant concepts are essential to gain
comprehensive understanding of the phenomena and to develop the
framework.
 Empirical evidence on the practical issues confronted by users when solving
CAPTCHA challenge was collected from findings reported in the peer-
reviewed literature.
 Thorough review of literature, we gathered evidences to form the basis for
developing a list of applicable usability features and concerns. These
identified features and concerns laid the foundations for developing the
holistic model of CAPTCHA usability.

9. Phases for Building Conceptual Framework
Conceptual framework analysis procedure consists of following steps:
1. Conduct extensive and systematic literature review on the phenomenon
to identify relevant literature
2. Reading and analyzing identified literature
3. Discover relevant concepts about the phenomenon from literature
4. Deconstruct and categorize the concepts
5. Integrate and group concepts based on similarities
6. Synthesize and re-synthesize concept groupings to build a holistic
framework that helps in making sense of the phenomenon
7. Validate the holistic framework by presenting to stakeholders
8. Rethink the holistic framework to keep it up to date

10. Holistic Model of CAPTCHA Usability
Usability of
CAPTCHA
Complexity
Content
Genericity
Presentation
Type of Input
Learnability
and ease of
use
Response
Time
Error Rate
User and
CAPTCHA
types
Culture and
familiarity
Language
Device Type
Distortion
Rate
Standardized
Scheme
Color
Schemes
Legends (*):
ConceptsAttributes
* Different colors are used to
distinguish concepts

11. Content Genericity
 CAPTCHA challenge tests must be generic enough to allow varied set of users to
take these challenges regardless of their geographic, culture, or content
knowledge.
 English language based challenges can pose barriers for non-English users to
solve the test.
 Recommend using generic contents like mathematical or image schema.
Language
 Challenge tests must abide by W3C Web Accessibility Initiative Guidelines.
 Alternative options to solve challenge tests must be provided.
 General knowledge varies across geographically and cultural regions.
 Combined with language barriers, these challenges can be unsolvable for
some.
 Recommend using animal images, geometric shapes, or other simple
entities that are globally recognized.
Culture and
familiarity
User and
CAPTCHA
types

12. Presentation
 Presentation of challenge response test schemes plays a vital role in learning and
usability of CAPTCHAs
Color
Schemes
Standardized
Scheme
 Colors can facilitate recognition, help user focus on objects, and get user’s
attention.
 However, color variations can complicate readability of CAPTCHAs.
 Recommend using simple color schemes or avoidance of color schemes can also
accomplish the job effectively.
 Variations in CAPTCHA schemes can pose substantial effort for users to learn
and solve the challenge tests.
 Since there is no single standard in use currently, designers can opt for the
most popular choice of CAPTCHA scheme to ensure familiarity among users.
 Recommend designing hybrid schemes that is easy for humans but harder
for bots.

13. Presentation (contd.)
Distortion
Rate
Device Type
 Excessive application of distortion and/or noise will make it hard for humans to
detect patterns as well.
 Recommend applying limited amount of distortion.
 Mobile users prefer touch inputs over audio.
 Presentation of a CAPTCHA can be different in mobile vs desktop machine.
 Recommend taking screen size and input mediums into consideration before
presentation CAPTCHA challenge.

14. Complexity
 Due to advancements with computer vision and machine learning, CAPTCHA
challenge complexity has been increased sacrificing usability.
Error Rate
Response
Time
 Studies indicate that despite users being familiar with CAPTCHAs only 48% of
the users were able to solve the CAPTCHA challenge in their first try.
 Every other attempt is inconvenience to user and system.
 Recommend designing challenges that can be solved by humans in one or two
attempts.
 Response time is the time taken by the users to solve a CAPTCHA challenge.
 When complexity is increased, users spend considerable amount of time
solving or need additional aids to solve the problem.
 Recommend designing CAPTCHAs that can be solved within 10 seconds in first
attempt, if not 20 seconds for multiple attempts.

15. Complexity (contd.)
Learnability
and ease of
use
Type of Input
 For complex challenges, user must be able to learn and adopt to the test from
their trail and quickly complete it in the next consecutive trials.
 Recommend designing challenges that have lower learning curve in regards to
identify patterns and solve the tests.
 Studies show users prefer mouse inputs over keyboard and touch over voice
inputs.
 Recommend using mouse input based challenges when accessing sites in
desktop and using touch inputs when accessing sites using mobile devices.

16. Conclusion
 CAPCTHA is a widely used security measure that is designed to distinguish
humans from bots, in order to prevent unauthorized access to websites which
would result in exploiting the Web resources.
 Contributions
 Holistic model that captures usability and CAPTCHA design factors.
 Holistic model can help designers and researchers make sense of the challenges
associated with balancing the effectiveness and the usability of CAPTCHAs.
 Limitations
 Study is based on secondary empirical evidences on the usability of CAPTCHA.
 While we attempted it to be systematic review of literature, peer-review articles found
were limited to search terms used.
 Holistic model makes aware of the most crucial characteristics of a CAPCTHA that
provides good user experience.

17. Thank You!

18. Designing CAPTCHA
 CAPTCHA design involves picking random string of characters (in case of text-
based CAPTCHAs) rendering into a distorted image.
HAT8M
 Inner workings of a CAPTCHA Source: Banday and Shah 2011, https://arxiv.org/ftp/arxiv/papers/1112/1112.5605.p

19. Holistic Model a.k.a Conceptual model
 Conceptual model is a product of systematic qualitative analysis of
multidisciplinary knowledge sources performed to gain better
understanding of a phenomenon.
 Conceptual model is
 interrelated concepts that together provides comprehensive
understanding of a phenomenon
 articulates 'the nature of reality' within a phenomenon
 explains 'how things really works' within a phenomenon
 A concept consists of a set of attributes which defines them.
 Every concept is in relation to the phenomenon under study, to other
relevant concepts, and to its own attributes.
 Concepts and attributes are identified through a systematic synthesis of
findings from multiple bodies of knowledge such as peer reviewed
research articles.