apidays LIVE Australia 2020 - Building Business Ecosystems Evaluating the usability of security APIs Dr Nalin Asanka Gamagedara Arachchilage, Senior Research Fellow in Cyber Security (DevOpsSec) at La Trobe University

1. Evaluating the usability of security
APIs
Nalin Asanka Gamagedara Arachchilage, PhD

2. Secure Coding or Programming issues

3. What we
investigated…
• Context: Programmers make mistakes when implementing
security APIs.
– Introduces security vulnerabilities into applications they
develop.
• There is no methodology to evaluate the usability of security
APIs.
• We developed a Cognitive Dimensions Framework (CDF)
based usability evaluation methodology to empirically evaluate
the usability of security APIs.
• We evaluated our CDF through 4 security APIs:
– Google authentication API
– Bouncy Castle lightweight crypto API
– Java Secure Socket Extension (JSSE) API
– OWASP Enterprise Security API (ESAPI)
• Results:
– Identified over 83% of the usability issues
– a considerably good validity and reliability.
• Recommendation:
– Our developed CDF provides a good platform to conduct
usability evaluation for security APIs.
– API developers can use our CDF to design security APIs
that are more usable.
3

4. Implementing
Security into
Software
ACM Reference Format:
Chamila Wijayarathna and Nalin A. G. Arachchilage. 2018. Why Johnny
Can’t Store Passwords Securely?: A Usability Evaluation of
Bouncycastle Password Hashing. In EASE’18: 22nd International
Conference on Evaluation and Assessment in Software Engineering
2018, June 28–29, 2018, Christchurch, New Zealand. ACM, New York,
NY, USA, 6 pages. https://doi.org/10.1145/3210459.3210483

5. What we did?
• In this work, we evaluated the usability of SCrypt password hashing
functionality of Bouncycastle API.
• Programmers to make mistakes (i.e. usability issues) while developing
applications that would result in security vulnerabilities.
• We conducted a study with 10 programmers where each of them spent
around 2 hours for the study and attempted to develop a secure password
storage solution using Bouncycastle API.
• We identified 63 usability issues that exist in the SCrypt implementation of
Bouncycastle API.
• Each participant had reported an average of approximately 15 usability
issues.
• Furthermore, we expect that this work will provide a guidance on how to
conduct usability evaluations for security APIs to identify usability issues exist
in them.

6. The issues we identified
• Selecting method parameters for SCrypt generate method invocation.
• If the programmer uses weak values for these parameters, security of the
password storage will be weak and it will be vulnerable to attacks.
• Using salt with SCrypt.generate() method
• Usage of salt provides protection against dictionary attacks and pre-computed rainbow
attacks.
• Usage of byte array vs String to store passwords
• Strings, password will be stored in the memory for longer and cannot be deleted until
garbage collector clears them.
• Issues observed about the documentation of Bouncycastle API
• Other observed issues
(a) Suggestions shown in Integrated Development Environment
for method parameters
(b) Method parameter names used in Bouncycastle SCrypt
source code
(c) Java API documentation of SCrypt.generate() method

7. Issues identified in all 4 APIs
7

8. We developed:
Cognitive
Dimensions
Framework for
security APIs
8

9. Cognitive Dimensions
Framework for security APIs
• Abstraction level: The minimum and maximum levels
of abstraction exposed by the API, and the minimum
and maximum levels usable by a targeted developer.
• Learning style: The knowledge about the API and its
security background that a programmer needs to have
before starting to use the API and how a programmer
would gain the knowledge about the API and its
security background.
• Working framework: The size of the conceptual
chunk (developer working set) needed to work
effectively.
9

10. Cognitive Dimensions
Framework for security APIs
• Work-step unit: How much of a programming task
must/can be completed in a single step.
• Progressive evaluation: To what extent partially
completed code can be executed to obtain feedback
on code behavior.
• Premature commitment: The amount of decisions
that developers have to make when writing code for a
given scenario and the consequences of those
decisions.
• Penetrability: How the API facilitates exploration,
analysis and understanding of its components and its
security related information, and the way a targeted
developer should go about retrieving what is needed.
10

11. Cognitive Dimensions
Framework for security APIs
• API elaboration: The extent to which the API
must be adapted to meet the needs of a targeted
developer.
• API viscosity: The barriers to change inherent in
the API, and how much effort a targeted developer
needs to expend to make a change.
• Consistency: How much of the rest of an API can
be inferred once part of it is learned.
• Role expressiveness: How apparent the
relationship is between each component exposed
by an API and the program as a whole.
11

12. Cognitive Dimensions
Framework for security APIs
• A Domain correspondence: How clearly the API components map
to the domain and any special tricks that the developer needs to be
aware of to accomplish some functionality.
• Hard-to Misuse: How hard it is to make mistakes while using the
API and how much help does the API provide to identify mistakes
that programmers make.
• End-user protection: How much does the security of the end user
of an application developed using the API depend on the
programmer who developed the application.
• Testability: The amount of support that the API provides for the
programmer to test the security of an application that was developed
using the API.
12

13. 13

14. Evaluating the usability of security
APIs
Nalin Asanka Gamagedara Arachchilage, PhD