Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Because you can’t fix what you don’t know is broken...

1,411 views

Published on

This talks introduces you to the Automated Error Reporting feature introduced in Eclipse Mars

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Because you can’t fix what you don’t know is broken...

  1. 1. Because you can’t fix what you don’t know is broken…
  2. 2. About me 2 • Initiator of the Automated Error Reporting Initiative • Project Lead of Eclipse Code Recommenders • Eclipse Committer since 2010 • Plug-in Developer for 10+ years (Eclipse 2.1) • Member of the Eclipse Architecture Council • Co-Lead of Java User Group Darmstadt • CEO of Codetrails • Passion to improve developers day-to-day work with intelligent and practical tools - mostly using Data Mining on Big (Software Engineering) Data. @marcelbruch marcelbruch
  3. 3. Adding a new character to the plot 3 Software
 User { ——–} Application
 Developer Product Manager
  4. 4. Imagine the following situation… 4 Software
 User { ——–} Application
 Developer Product Manager
  5. 5. But when the user starts the app… 5
  6. 6. 6Can you feel their pain…? This image is not available in your country
  7. 7. In the logs you’d find… 7
  8. 8. ✘ ✘ ✘ The tragedy of manual error reports 8 ✘ ✘ ! maintenance ✘ ✘ ✘ Release SR 2 production SR 1
  9. 9. ✘ ! The tragedy of manual error reports 9 maintenance Release SR 2 production SR 1 ✘ ✘ ✘
  10. 10. 10Not too happy but feeling okay… This image is not available in your country
  11. 11. What can we do? 11 Application Developer ? { ——–}
  12. 12. The Automated Error Reporting Initiative 12
  13. 13. Live Demo 13
  14. 14. About server responses,
 Bugzilla, and immediate feedback „Keep users informed about your progress…“ 14
  15. 15. Systems Overview 15 AERI Web UI Eclipse BugzillaEclipse IDEs „Direct Reporter Feedback“
  16. 16. Not yet tracked problems… 16
  17. 17. Committers needinfo… 17 Status: – – – Resolution: – – – Bug 442500 – ….
  18. 18. Fixed… 18 Status: – – – Resolution: – – – Bug 442500 – ….
  19. 19. Won’t Fix… 19 Status: – – – Resolution: – – – Bug 442500 – ….
  20. 20. Invalid (aka log messages) 20 Status: – – – Resolution: – – – Bug 442500 – ….
  21. 21. About caching, filtering,
 anonymization, and other shortcuts… „What goes to eclipse.org?“ 21
  22. 22. Do we send all errors to Eclipse…? 22 really
  23. 23. The error reporting funnel 23 ✘ structural filter
  24. 24. #1: Structural Filtering 24 java.net.UnknownHostException: marketplace.eclipse.org at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184) at java.net.Socket.connect(Socket.java:538) at com.example.HttpClient.openServer(HttpClient.java:527) at com.example.http.HttpClient.<init>(HttpClient.java:211) at com.example.HttpURLConnection.connect(HttpURLConnection.java:932) at org.eclipse.recommenders.internal.news.rcp.PollFeedJob.run(PollFeedJob.java:69) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55) Error connecting to ‘http://marketplace.eclipse.org/rss.xml’.
  25. 25. #2: Did I already sent this? 25 ✘ structural filter personal filter
  26. 26. #3: Did anyone else sent this? 26 ✘ structural filter personal filter global hist.
  27. 27. structural filter personal filter If you hit a known issue… 27 ✘ global hist.
  28. 28. #4: Review 28 review ✘ structural filter personal filter global hist.
  29. 29. #4: Review 29
  30. 30. The server side Walkthrough through the committer front-end 30
  31. 31. The core concepts 31
  32. 32. The Core Concepts 32 ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘✘ Incidents Problems Projects
  33. 33. The committer front-end 33
  34. 34. Creating bug reports 34
  35. 35. Creating bug reports 35
  36. 36. Notifying committers 36
  37. 37. Project Alerts 37
  38. 38. Weekly or daily email digests 38
  39. 39. Some fun facts e = mc2 —> errors = (more code)2 39
  40. 40. Disclaimer Don’t draw preliminary conclusions! Please consult your personal statistican…
  41. 41. How many milestones testers? 450 simrel committers 1200 eclipse committers Testers?~7000 error reporters
  42. 42. Milestones adoption 42 RC3 RC2 RC1 M7 M6 M5 M4 M3 M2 M1 1000 220 160 650 450 400 560 600 Do new milestones introduce new problems? 70 10 # reporters one week high SR2 SR1 R Luna Mars
  43. 43. 0 350 700 1050 1400 4.4.0 4.4.1 4.4.2 M1 M2 M3 M4 M5 M6 M7 RC1 RC2 RC3 Milestones adoption 43 Christmas Addition to all EPP packages RCs Addition to first EPP packages
  44. 44. How hard do they suffer? 44 300 - 3000 reports 25 testers 50-300 reports 330 testers 3-50 reports 3600 testers 1-2 reports 3000 testers
  45. 45. What do they report? 45 0 750 1500 2250 3000 ‚Normal‘ Errors Out of Memory Freezes Stack Overflows 3rd Party Involved Top 100 of 7000 Me
  46. 46. What kind of errors are send? 46 3rd Party Involved 16 % Stack Overflows 0 % Freezes 9 % Out Of Memory 1 % 'Normal' Errors 74 %
  47. 47. What are the most popular exceptions? 47 32 % 12 % 25 % 3 % 8 % 2 % 18 %
  48. 48. What are the most popular exceptions? 48 Other 32 % No {Class | Method | Field} DefFoundError 12 % NullPointerException 25 % IndexOutOfBoundsException 3 % IllegalArgumentException 8 % ClassCastException 2 % {File} IOException 18 %
  49. 49. NullPointerExceptions by Projects 49 0 1000 2000 3000 NullPointerException Code Recommenders 65 Statisticians, note: Projects automatically marked as „being responsible“ fixing these NullPointerExceptions but does not mean the NPE happened in their code.
  50. 50. Tip: Enable null analysis 50 @Override protected void run(@Nullable List<Incident> slice) { for (Incident incident : slice) {
  51. 51. JDT offers a lot… 51
  52. 52. Tip: Never return null 52 @Override public Optional<String> getReason() { return Optional.ofNullable(cause); } Use java.util.Optional or com.google.common.base.Optional instead
  53. 53. Oh Class, Were Art Thou? 53 Use proper import ranges and define uses on your exports to get rid of 
 {Class | Method | Field} DefNotFoundErrors, LinkageErrors, and some ClassCastExceptions. OSGI needs that information to work properly… Export-Package: org.eclipse.recommenders.utils; uses:="org.osgi.framework, org.eclipse.core.runtime, com.google.common.base, com.google.common.collect" Import-Package: com.google.common.annotations;version="[15.0.0,16.0.0)",
  54. 54. Index out of bounds? 54 a b c d e -1 and array.length do not work. No matter how often you try… There is no tool support for this. Just unit tests and code that tries to fail gracefully.
  55. 55. Tip: If error messages are not helpful… 55 try { // … } catch (Exception e) { log(„An error occurred“, e); } 21% of all logged errors do not contain stack traces. try { // … } catch (Exception e) { log(„Invalid argument {}“, value, e); }
  56. 56. Are your users ready for Java 8? 56 both
 2,5% Java 1.7 33% Java 1,8
 64% Java 1.9 0,3% Which VM do users run the Eclipse IDE with?
  57. 57. Do people anonymize their messages? 57 Sometimes 3 % No 81 % Yes 16 % Are you concerned that your exception message may reveal sensitive data?
  58. 58. Bugfixing Rates Speak plain: Does it work? 58
  59. 59. Bugzilla Summary 59 1170 bugs 275 fixed (24%) 100 invalid 70 not eclipse 467 in progress (40%) 172 duplicates (15%)69 works 4 me 28 wont fix Bugzilla
  60. 60. Fixed bugs by projects 60 More (35) 84 --- 11 EGit 15 Papyrus 19 Recommenders 41 Xtext 42 Oomph 58
  61. 61. In progress & fixed bugs 61 0 30 60 90 120 Oomph TMF Recommenders Papyrus EGit --- 97 1914 17 29 20 11 15 19 4142 58 Fixed In Progress
  62. 62. Future Work? Feature requests are tracked in bugzilla. Make sure your request is in there and vote for it. Client: • Separate view to „review, analyze and send later“ • Making client reusable in other (Eclipse- or SWT-based) applications • Additional client-side automated analysis Server: • Ensure scalability of the system • Improve duplicate detections • Go Neon if feedback is good. Hosting an error reporting service for Java software? • Need one? TTML. 62
  63. 63. How to reach EPP Logging 63 • Code: Check out the EPP Logging Oomph profile • Wiki: https://wiki.eclipse.org/EPP/Logging • Manual: https://dev.eclipse.org/recommenders/community/confess/#/about • Forum: https://www.eclipse.org/forums/index.php/f/69/ • Bugzilla: https://bugs.eclipse.org/bugs/buglist.cgi?component=logging&product=EPP • Mail: error-reports-inbox@eclipse.org
  64. 64. Credits • Daniel Haftstein, Codetrails – Eclipse client conception and implementation • Marcel Bruch, Codetrails – Server-side conception and implementation • Webmasters, Eclipse Foundation – Eclipse server infrastructure & integration • Thanks to many early adopters and committers providing feedback 64
  65. 65. Feedback 65
  66. 66. Contact Us 66 Codetrails GmbH Robert-Bosch-Str. 7 64293 Darmstadt, Germany www: codetrails.com phone: 06151 / 2767 092 email: info@codetrails.com Dr. Marcel Bruch CEO mobile: 0179 / 1317 721 email: marcel.bruch@codetrails.com
  67. 67. Feedback 67

×