Why Java Developers Struggle With Cryptography APIs?

1. “Jumping Through Hoops” Why do Java Developers Struggle With Cryptography APIs? Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden sarahnadi.org@sarahnadi ICSE ’16 — May 20th, 2016

2. Sarah Nadi An Application Developer’s World 2 Application

3. Sarah Nadi User accounts Payment info. An Application Developer’s World 2 Application Sensitive user documents

4. Sarah Nadi User accounts Payment info. How to encrypt data? Encryption vs Hashing? Encryption mode? Salted hashing? How to securely connect to a server? An Application Developer’s World 2 Application Sensitive user documents

5. Sarah Nadi Application Developers in The Wild 3 83% of 269 Vulnerabilities are due to misuse of crypto libraries [Lazar et al., APSys ’14] Even Amazon & Paypal misuse SSL certiﬁcate validation [Georgiev et al., CCS ‘12] 88% of ~12,000 Android apps misuse crypto APIs [Egele et al., CCS ‘13]

6. Sarah Nadi 4 So what exactly is a misuse?

7. Sarah Nadi Example of an API Misuse 5 Sarah Nadi Example of an API Misuse 5 SecretKey secretKey = … Cipher cipher = Cipher.getInstance("AES"); cipher.init(Cipher.ENCRYPT_MODE, secretKey); cipher.doFinal(inputMsg);

8. Sarah Nadi Sarah Nadi Example of an API Misuse 5 SecretKey secretKey = … Cipher cipher = Cipher.getInstance("AES"); cipher.init(Cipher.ENCRYPT_MODE, secretKey); cipher.doFinal(inputMsg); Example of an API Misuse 6 “Algorithm/Mode/Padding”

9. Sarah Nadi Sarah Nadi Example of an API Misuse 5 SecretKey secretKey = … Cipher cipher = Cipher.getInstance("AES"); cipher.init(Cipher.ENCRYPT_MODE, secretKey); cipher.doFinal(inputMsg); Example of an API Misuse 6 “Algorithm/Mode/Padding” In some API implementations, default mode for AES is Electronic Codebook (ECB) — which is insecure

10. Sarah Nadi Sarah Nadi Example of an API Misuse 5 SecretKey secretKey = … Cipher cipher = Cipher.getInstance("AES"); cipher.init(Cipher.ENCRYPT_MODE, secretKey); cipher.doFinal(inputMsg); [https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation] Example of an API Misuse 6 “Algorithm/Mode/Padding” In some API implementations, default mode for AES is Electronic Codebook (ECB) — which is insecure

11. Sarah Nadi 7 Why do Developers Struggle With Cryptography APIs?

12. Sarah Nadi 7 Why do Developers Struggle With Cryptography APIs? What obstacles do developers face?

13. Sarah Nadi 7 Why do Developers Struggle With Cryptography APIs? What are the common cryptography tasks developers perform? What obstacles do developers face?

14. Sarah Nadi 7 Why do Developers Struggle With Cryptography APIs? What are the common cryptography tasks developers perform? What tools/methods would help developers use cryptography more effectively? What obstacles do developers face?

15. Sarah Nadi Empirical Investigation Study 1 Study 2 Study 4Study 3 8

16. Sarah Nadi Study 1 (S1) Design 9 … <java> <cryptography> (Sorted by view count & score) Goal: Find obstacles

17. Sarah Nadi Study 1 (S1) Design 9 … <java> <cryptography> top 100 (Sorted by view count & score) Goal: Find obstacles

18. Sarah Nadi Study 1 (S1) Design 9 … <java> <cryptography> top 100 Question topic? Likely obstacle? (Sorted by view count & score) Goal: Find obstacles

19. Sarah Nadi Study 2 (S2) Design 10 javax.crypto Goal: Find tasks

20. Sarah Nadi Study 2 (S2) Design 10 random 100 repos javax.crypto Goal: Find tasks

21. Sarah Nadi Study 2 (S2) Design 10 random 100 repos What crypto task is performed? javax.crypto Goal: Find tasks

22. Sarah Nadi Study 3 (S3) & Study 4 (S4) Survey Design 11 Goal: Find obstacles, tasks, & suggestions

23. Sarah Nadi Study 3 (S3) & Study 4 (S4) Survey Design 11 Background Goal: Find obstacles, tasks, & suggestions

24. Sarah Nadi Study 3 (S3) & Study 4 (S4) Survey Design 11 Background Frequency of Cryptography Use Never Rarely … Occasionally … Frequently … Goal: Find obstacles, tasks, & suggestions

25. Sarah Nadi Study 3 (S3) & Study 4 (S4) Survey Design 11 Background Frequency of Cryptography Use Never Rarely … Occasionally … Frequently … Cryptography Tasks Used/Needed Goal: Find obstacles, tasks, & suggestions

26. Sarah Nadi Study 3 (S3) & Study 4 (S4) Survey Design 11 Background Frequency of Cryptography Use Never Rarely … Occasionally … Frequently … Cryptography Tasks Used/Needed Frequently-used Crypto APIs/libraries & ease of use Goal: Find obstacles, tasks, & suggestions

27. Sarah Nadi Study 3 (S3) & Study 4 (S4) Survey Design 11 Background Frequency of Cryptography Use Never Rarely … Occasionally … Frequently … Cryptography Tasks Used/Needed Frequently-used Crypto APIs/libraries & ease of use Obstacles (free-text + rated) Goal: Find obstacles, tasks, & suggestions

28. Sarah Nadi Study 3 (S3) & Study 4 (S4) Survey Design 11 Background Frequency of Cryptography Use Never Rarely … Occasionally … Frequently … Cryptography Tasks Used/Needed Frequently-used Crypto APIs/libraries & ease of use Obstacles (free-text + rated) Goal: Find obstacles, tasks, & suggestions

29. Sarah Nadi Survey Process & Participant Recruitment 12 Study 4 Study 3

30. Sarah Nadi Emailed <java><cryptography> posters Survey Process & Participant Recruitment 12 Study 4 Study 3

31. Sarah Nadi Emailed <java><cryptography> posters Survey Process & Participant Recruitment 12 Study 4 11 participants Study 3

32. Sarah Nadi Emailed <java><cryptography> posters Survey Process & Participant Recruitment 12 Study 4 11 participants Study 3 Reﬁne

33. Sarah Nadi Emailed <java><cryptography> posters Survey Process & Participant Recruitment 12 Snowball sampling Emailed related committers + Study 4 11 participants Study 3 Reﬁne

34. Sarah Nadi Emailed <java><cryptography> posters Survey Process & Participant Recruitment 12 Snowball sampling Emailed related committers 37 participants+ Study 4 11 participants Study 3 Reﬁne

35. Sarah Nadi Findings 13 TASKS OBSTACLES DESIRED SUPPORT

36. Sarah Nadi What Obstacles do Developers Face? 14

37. Sarah Nadi What Obstacles do Developers Face? 15 Obstacle % Posts API use 57% Domain knowledge 15% Provider & Setup 15% Library Identiﬁcation 7% Domain knowledge + API use 6%

38. Sarah Nadi What Obstacles do Developers Face? 15 S3 participants mention: poor documentation, bad API design,and missing cryptography knowledge Obstacle % Posts API use 57% Domain knowledge 15% Provider & Setup 15% Library Identiﬁcation 7% Domain knowledge + API use 6%

39. Sarah Nadi What Obstacles do Developers Face? 15 S3 participants mention: poor documentation, bad API design,and missing cryptography knowledge 65% of S4 Participants ﬁnd the APIs hard to use Obstacle % Posts API use 57% Domain knowledge 15% Provider & Setup 15% Library Identiﬁcation 7% Domain knowledge + API use 6%

40. Sarah Nadi Obstacle Category 1: Domain Knowledge 16 11% 24% 38% 27% Frequently Occasionally Rarely Never Don't know Identify correct cryptography algorithm 14% 41% 43% 3% Frequently Occasionally Rarely Never Don't know Identify relevant cryptography concepts

41. Sarah Nadi Obstacle Category 1: Domain Knowledge 17 11% 24% 38% 27% Frequently Occasionally Rarely Never Don't know Identify correct cryptography algorithm 14% 41% 43% 3% Frequently Occasionally Rarely Never Don't know Identify relevant cryptography concepts Developers may not always know the correct cryptography algorithm to use

42. Sarah Nadi Obstacle Category 2: Setting Up 18 27% 32% 27% 11% 3% Frequently Occasionally Rarely Never Don't know Identify relevant Java API 22% 41% 30% 5% 3% Frequently Occasionally Rarely Never Don't know Select provider 22% 27% 32% 16% 3% Frequently Occasionally Rarely Never Don't know Setup environment

43. Sarah Nadi Obstacle Category 2: Setting Up 19 27% 32% 27% 11% 3% Frequently Occasionally Rarely Never Don't know Identify relevant Java API 22% 41% 30% 5% 3% Frequently Occasionally Rarely Never Don't know Select provider 22% 27% 32% 16% 3% Frequently Occasionally Rarely Never Don't know Setup environment Finding the right API/library to use & setting it up is often an obstacle in itself

44. Sarah Nadi 19% 54% 22% 5% Frequently Occasionally Rarely Never Don't know 32% 43% 19% 5% Frequently Occasionally Rarely Never Don't know 35% 46% 11% 8% Frequently Occasionally Rarely Never Don't know Obstacle Category 3: API Use 20 Identify sequence of method calls Identify parameters Understand API error messages

45. Sarah Nadi 19% 54% 22% 5% Frequently Occasionally Rarely Never Don't know 32% 43% 19% 5% Frequently Occasionally Rarely Never Don't know 35% 46% 11% 8% Frequently Occasionally Rarely Never Don't know Obstacle Category 3: API Use 21 Identify sequence of method calls Identify parameters Understand API error messages The most frequently faced obstacle is identifying the sequence of API method calls

46. Sarah Nadi Common Cryptography Tasks 22

47. Sarah Nadi Common Cryptography Tasks 23 10% of analyzed GitHub repos Top task by 64% of S3 participants Avg. Rank of 3.95 by 28 S4 participants 6% of analyzed GitHub repos Top task by 34% of S3 participants Avg. Rank 2.22, 35 S4 participants 64% of analyzed GitHub repos 37% of analyzed StackOverﬂow posts Avg. rank 5.03, 25 S4 participants User Authentication Secure Communication Symmetric Encryption

48. Sarah Nadi What do Developers Want? 24

49. Sarah Nadi What do Developers Want? 25 Better Documentation “Better documentation with examples and deeper layers of documentations with theoretical knowledge” (S4-P23) “better examples” (S4-P2) “More/better example code, access to API source code and high quality JavaDoc with cross references” (S4-P12) “A real documentation would be very helpful. Some kind of best practice methodology for a crypto API usage could also help” (S4-P18)

50. Sarah Nadi What do Developers Want? 26 Higher Abstraction Level “Make [the] API simpler, e.g., have a class with the name AES or RSA[;] getInstance(‘whatever’) is bad API design” (S4-P36) “[I want something that] just takes input […]. Short and sweet” (S3-P3) “Higher level task oriented APIs for things like public key crypto, key exchanges, ..” (S4-P27) “High level APIs [that] can't be used incorrectly..” (S4-P8) “A library providing simple API calls (one or two methods and simple parameters) for different use cases.” (S4-P7) “Standardized task-based API” (S4-P1) “…Providing higher-level APIs, .. would go along way to making sure developers don't do something dumb.” (S4-P27)

51. Sarah Nadi What do Developers Want? 27 “Test tooling that understands encryption and verifies if encryption is used where it should be. Source scanners that identify configuration mistakes, weak algorithms etc.” (S4-P17) Tool Assistance “Some kind of testing tool…” (S4-P13) “Special CryptoDebugger” (S4-P11) “Templates for common used patterns” (S4-P7) “IDE Plugin generating code pattern for specific use cases” (S4-P6)

52. Sarah Nadi 28 Better Documentation Higher Abstraction Level Tool Assistance

53. Sarah Nadi 29 Better Documentation Higher Abstraction Level Tool Assistance How Can We Move Forward?

54. Sarah Nadi The Ideal Tool Assistance 30 COMMON TASKS TO SUPPORT STEPS TO SUPPORT

57. Sarah Nadi The Ideal Tool Assistance 30 BlockCipher AES keySize (128, 192, 256) mode (ECB, CBC, …) padding (PKCS5Padding, NoPadding) DES COMMON TASKS TO SUPPORT STEPS TO SUPPORT

58. Sarah Nadi The Ideal Tool Assistance 30 BlockCipher AES keySize (128, 192, 256) mode (ECB, CBC, …) padding (PKCS5Padding, NoPadding) DES COMMON TASKS TO SUPPORT STEPS TO SUPPORT KeyGenerator keyGen = KeyGenerator.getInstance("AES"); keyGen.init(256); SecretKey secretKey = keyGen.generateKey(); Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7PADDING"); cipher.init(Cipher.ENCRYPT_MODE, secretKey); cipher.doFinal(inputMsg);

59. Sarah Nadi The Ideal Tool Assistance 30 BlockCipher AES keySize (128, 192, 256) mode (ECB, CBC, …) padding (PKCS5Padding, NoPadding) DES COMMON TASKS TO SUPPORT STEPS TO SUPPORT KeyGenerator keyGen = KeyGenerator.getInstance("AES"); keyGen.init(256); SecretKey secretKey = keyGen.generateKey(); Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7PADDING"); cipher.init(Cipher.ENCRYPT_MODE, secretKey); cipher.doFinal(inputMsg);

60. Sarah Nadi “The ability to just perform some simple cryptographic tasks in Java without jumping through hoops would be brilliant…” (S4-P10) 31

61. “Jumping Through Hoops” Why do Java Developers Struggle With Cryptography APIs? Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden sarahnadi.org@sarahnadi

Why Java Developers Struggle With Cryptography APIs?

You are reading a preview.

Check these out next

Why Java Developers Struggle With Cryptography APIs?

More Related Content

Viewers also liked (13)

Similar to Why Java Developers Struggle With Cryptography APIs? (20)

Recently uploaded (20)