This is an empirical investigation into the reasons why many developers misuse the Java cryptography APIs. Such API misuses can lead to non-intentional security vulnerabilities. We triangulate data from three different sources of information: StackOverflow, GitHub, and developer surveys.

1. “Jumping Through Hoops”
Why do Java Developers Struggle With
Cryptography APIs?
Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden
sarahnadi.org@sarahnadi
ICSE ’16 — May 20th, 2016

2. Sarah Nadi
An Application Developer’s World
2
Application

3. Sarah Nadi
User accounts
Payment info.
An Application Developer’s World
2
Application
Sensitive user
documents

4. Sarah Nadi
User accounts
Payment info.
How to encrypt data?
Encryption vs
Hashing?
Encryption mode?
Salted hashing?
How to securely connect
to a server?
An Application Developer’s World
2
Application
Sensitive user
documents

5. Sarah Nadi
Application Developers in The Wild
3
83% of 269 Vulnerabilities are due to misuse of crypto libraries
[Lazar et al., APSys ’14]
Even Amazon & Paypal misuse SSL certificate validation
[Georgiev et al., CCS ‘12]
88% of ~12,000 Android apps misuse crypto APIs
[Egele et al., CCS ‘13]

6. Sarah Nadi 4
So what exactly is a misuse?

7. Sarah Nadi
Example of an API Misuse
5
Sarah Nadi
Example of an API Misuse
5
SecretKey secretKey = …
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);

8. Sarah Nadi
Sarah Nadi
Example of an API Misuse
5
SecretKey secretKey = …
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);
Example of an API Misuse
6
“Algorithm/Mode/Padding”

9. Sarah Nadi
Sarah Nadi
Example of an API Misuse
5
SecretKey secretKey = …
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);
Example of an API Misuse
6
“Algorithm/Mode/Padding”
In some API implementations,
default mode for AES is
Electronic Codebook (ECB) —
which is insecure

10. Sarah Nadi
Sarah Nadi
Example of an API Misuse
5
SecretKey secretKey = …
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);
[https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation]
Example of an API Misuse
6
“Algorithm/Mode/Padding”
In some API implementations,
default mode for AES is
Electronic Codebook (ECB) —
which is insecure

11. Sarah Nadi 7
Why do Developers Struggle
With Cryptography APIs?

12. Sarah Nadi 7
Why do Developers Struggle
With Cryptography APIs?
What obstacles do
developers face?

13. Sarah Nadi 7
Why do Developers Struggle
With Cryptography APIs?
What are the common
cryptography tasks
developers perform?
What obstacles do
developers face?

14. Sarah Nadi 7
Why do Developers Struggle
With Cryptography APIs?
What are the common
cryptography tasks
developers perform?
What tools/methods would help
developers use cryptography
more effectively?
What obstacles do
developers face?

15. Sarah Nadi
Empirical Investigation
Study 1 Study 2
Study 4Study 3
8

16. Sarah Nadi
Study 1 (S1) Design
9
…
<java> <cryptography>
(Sorted by view count & score)
Goal: Find obstacles

17. Sarah Nadi
Study 1 (S1) Design
9
…
<java> <cryptography>
top
100
(Sorted by view count & score)
Goal: Find obstacles

18. Sarah Nadi
Study 1 (S1) Design
9
…
<java> <cryptography>
top
100
Question topic?
Likely obstacle?
(Sorted by view count & score)
Goal: Find obstacles

19. Sarah Nadi
Study 2 (S2) Design
10
javax.crypto
Goal: Find tasks

20. Sarah Nadi
Study 2 (S2) Design
10
random
100
repos
javax.crypto
Goal: Find tasks

21. Sarah Nadi
Study 2 (S2) Design
10
random
100
repos
What crypto task
is performed?
javax.crypto
Goal: Find tasks

22. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Goal: Find obstacles, tasks, & suggestions

23. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Goal: Find obstacles, tasks, & suggestions

24. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Frequency of Cryptography Use
Never Rarely
…
Occasionally
…
Frequently
…
Goal: Find obstacles, tasks, & suggestions

25. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Frequency of Cryptography Use
Never Rarely
…
Occasionally
…
Frequently
…
Cryptography Tasks Used/Needed
Goal: Find obstacles, tasks, & suggestions

26. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Frequency of Cryptography Use
Never Rarely
…
Occasionally
…
Frequently
…
Cryptography Tasks Used/Needed
Frequently-used Crypto
APIs/libraries & ease of use
Goal: Find obstacles, tasks, & suggestions

27. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Frequency of Cryptography Use
Never Rarely
…
Occasionally
…
Frequently
…
Cryptography Tasks Used/Needed
Frequently-used Crypto
APIs/libraries & ease of use
Obstacles (free-text +
rated)
Goal: Find obstacles, tasks, & suggestions

28. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Frequency of Cryptography Use
Never Rarely
…
Occasionally
…
Frequently
…
Cryptography Tasks Used/Needed
Frequently-used Crypto
APIs/libraries & ease of use
Obstacles (free-text +
rated)
Goal: Find obstacles, tasks, & suggestions

29. Sarah Nadi
Survey Process & Participant Recruitment
12
Study 4
Study 3

30. Sarah Nadi
Emailed
<java><cryptography> posters
Survey Process & Participant Recruitment
12
Study 4
Study 3

31. Sarah Nadi
Emailed
<java><cryptography> posters
Survey Process & Participant Recruitment
12
Study 4
11 participants
Study 3

32. Sarah Nadi
Emailed
<java><cryptography> posters
Survey Process & Participant Recruitment
12
Study 4
11 participants
Study 3
Refine

33. Sarah Nadi
Emailed
<java><cryptography> posters
Survey Process & Participant Recruitment
12
Snowball
sampling
Emailed
related
committers
+
Study 4
11 participants
Study 3
Refine

34. Sarah Nadi
Emailed
<java><cryptography> posters
Survey Process & Participant Recruitment
12
Snowball
sampling
Emailed
related
committers
37 participants+
Study 4
11 participants
Study 3
Refine

35. Sarah Nadi
Findings
13
TASKS
OBSTACLES
DESIRED
SUPPORT

36. Sarah Nadi
What Obstacles do Developers Face?
14

37. Sarah Nadi
What Obstacles do Developers Face?
15
Obstacle % Posts
API use 57%
Domain knowledge 15%
Provider & Setup 15%
Library Identification 7%
Domain knowledge + API use 6%

38. Sarah Nadi
What Obstacles do Developers Face?
15
S3 participants mention:
poor documentation, bad API design,and missing cryptography knowledge
Obstacle % Posts
API use 57%
Domain knowledge 15%
Provider & Setup 15%
Library Identification 7%
Domain knowledge + API use 6%

39. Sarah Nadi
What Obstacles do Developers Face?
15
S3 participants mention:
poor documentation, bad API design,and missing cryptography knowledge
65% of S4 Participants find the
APIs hard to use
Obstacle % Posts
API use 57%
Domain knowledge 15%
Provider & Setup 15%
Library Identification 7%
Domain knowledge + API use 6%

40. Sarah Nadi
Obstacle Category 1: Domain Knowledge
16
11%
24%
38%
27%
Frequently Occasionally Rarely Never Don't know
Identify correct cryptography algorithm
14%
41% 43%
3%
Frequently Occasionally Rarely Never Don't know
Identify relevant cryptography concepts

41. Sarah Nadi
Obstacle Category 1: Domain Knowledge
17
11%
24%
38%
27%
Frequently Occasionally Rarely Never Don't know
Identify correct cryptography algorithm
14%
41% 43%
3%
Frequently Occasionally Rarely Never Don't know
Identify relevant cryptography concepts
Developers may not always know the correct
cryptography algorithm to use

42. Sarah Nadi
Obstacle Category 2: Setting Up
18
27% 32% 27%
11% 3%
Frequently Occasionally Rarely Never Don't know
Identify relevant Java API
22%
41%
30%
5% 3%
Frequently Occasionally Rarely Never Don't know
Select provider
22% 27% 32%
16%
3%
Frequently Occasionally Rarely Never Don't know
Setup environment

43. Sarah Nadi
Obstacle Category 2: Setting Up
19
27% 32% 27%
11% 3%
Frequently Occasionally Rarely Never Don't know
Identify relevant Java API
22%
41%
30%
5% 3%
Frequently Occasionally Rarely Never Don't know
Select provider
22% 27% 32%
16%
3%
Frequently Occasionally Rarely Never Don't know
Setup environment
Finding the right API/library to use & setting it
up is often an obstacle in itself

44. Sarah Nadi
19%
54%
22%
5%
Frequently Occasionally Rarely Never Don't know
32%
43%
19%
5%
Frequently Occasionally Rarely Never Don't know
35%
46%
11% 8%
Frequently Occasionally Rarely Never Don't know
Obstacle Category 3: API Use
20
Identify sequence of method calls
Identify parameters
Understand API error messages

45. Sarah Nadi
19%
54%
22%
5%
Frequently Occasionally Rarely Never Don't know
32%
43%
19%
5%
Frequently Occasionally Rarely Never Don't know
35%
46%
11% 8%
Frequently Occasionally Rarely Never Don't know
Obstacle Category 3: API Use
21
Identify sequence of method calls
Identify parameters
Understand API error messages
The most frequently faced obstacle is
identifying the sequence of API method calls

46. Sarah Nadi
Common Cryptography Tasks
22

47. Sarah Nadi
Common Cryptography Tasks
23
10% of analyzed GitHub repos
Top task by 64% of S3 participants
Avg. Rank of 3.95 by 28 S4 participants
6% of analyzed GitHub repos
Top task by 34% of S3 participants
Avg. Rank 2.22, 35 S4 participants
64% of analyzed GitHub repos
37% of analyzed StackOverflow posts
Avg. rank 5.03, 25 S4 participants
User Authentication
Secure Communication
Symmetric Encryption

48. Sarah Nadi
What do Developers Want?
24

49. Sarah Nadi
What do Developers Want?
25
Better Documentation
“Better documentation with examples
and deeper layers of documentations
with theoretical knowledge” (S4-P23)
“better examples” (S4-P2)
“More/better example code, access to API source code and
high quality JavaDoc with cross references” (S4-P12)
“A real documentation would be very helpful. Some kind of best
practice methodology for a crypto API usage could also help” (S4-P18)

50. Sarah Nadi
What do Developers Want?
26
Higher Abstraction Level
“Make [the] API simpler, e.g., have a class with
the name AES or RSA[;] getInstance(‘whatever’) is
bad API design” (S4-P36)
“[I want something that] just takes input
[…]. Short and sweet” (S3-P3)
“Higher level task oriented APIs for things like
public key crypto, key exchanges, ..” (S4-P27)
“High level APIs [that] can't be used
incorrectly..” (S4-P8)
“A library providing simple API calls (one
or two methods and simple parameters)
for different use cases.” (S4-P7)
“Standardized task-based API” (S4-P1)
“…Providing higher-level APIs, .. would go
along way to making sure developers don't
do something dumb.” (S4-P27)

51. Sarah Nadi
What do Developers Want?
27
“Test tooling that understands encryption and
verifies if encryption is used where it should be.
Source scanners that identify configuration
mistakes, weak algorithms etc.” (S4-P17)
Tool Assistance
“Some kind of testing tool…” (S4-P13)
“Special CryptoDebugger” (S4-P11)
“Templates for common used patterns” (S4-P7)
“IDE Plugin generating code pattern
for specific use cases” (S4-P6)

52. Sarah Nadi 28
Better Documentation
Higher Abstraction Level
Tool Assistance

53. Sarah Nadi 29
Better Documentation
Higher Abstraction Level
Tool Assistance
How Can We Move Forward?

54. Sarah Nadi
The Ideal Tool Assistance
30
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT

55. Sarah Nadi
The Ideal Tool Assistance
30
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT

56. Sarah Nadi
The Ideal Tool Assistance
30
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT

57. Sarah Nadi
The Ideal Tool Assistance
30
BlockCipher
AES
keySize (128, 192, 256)
mode (ECB, CBC, …)
padding (PKCS5Padding, NoPadding)
DES
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT

58. Sarah Nadi
The Ideal Tool Assistance
30
BlockCipher
AES
keySize (128, 192, 256)
mode (ECB, CBC, …)
padding (PKCS5Padding, NoPadding)
DES
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256);
SecretKey secretKey = keyGen.generateKey();
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7PADDING");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);

59. Sarah Nadi
The Ideal Tool Assistance
30
BlockCipher
AES
keySize (128, 192, 256)
mode (ECB, CBC, …)
padding (PKCS5Padding, NoPadding)
DES
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256);
SecretKey secretKey = keyGen.generateKey();
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7PADDING");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);

60. Sarah Nadi
“The ability to just perform some
simple cryptographic tasks in Java
without jumping through hoops
would be brilliant…” (S4-P10)
31

61. “Jumping Through Hoops”
Why do Java Developers Struggle With
Cryptography APIs?
Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden
sarahnadi.org@sarahnadi