Successfully reported this slideshow.
Your SlideShare is downloading. ×

The End of the world as we know it - AKA your last NullPointerException $1B bugs!

Loading in …3

Check these out next

1 of 46 Ad

More Related Content

Slideshows for you (20)


Similar to The End of the world as we know it - AKA your last NullPointerException $1B bugs! (20)


Recently uploaded (20)

The End of the world as we know it - AKA your last NullPointerException $1B bugs!

  1. 1. The End of the world as we know it? AKA your last NullPointerException $1B bugs! Michael EclipseCon Europe 2016, October 25, 2016
  2. 2. EclipseCon NullPointerException2 Agenda 1. Topic Intro 2. “How to” (incl. mvn build & IDE config) 3. Some experiences from ongoing work 4. Call to community action++
  3. 3. EclipseCon NullPointerException3 null analysis is not at all by Red Hat! AFAIK contributions mostly by Stephan Herrmann; as volunteer! #ACK is a Linux Foundation hosted platform, leading the future of open Software Defined Networking (SDN), based on OSGi (Karaf), backed by a strong multi vendor community with 100s of Devs. Red Hat contributes to ODL, for which I’ve started to use Eclipse null analysis @work, and extended some tooling, and will share some experiences from #dayjob. BTW: Red Hat is a great company to work for… #LifeAtRedHat
  4. 4. EclipseCon NullPointerException4 History C. A. R. Hoare (Mr. QuickSort et al) jokingly apologized in 2009 for “adding null reference to ALGOL [=> C => Java], which probably caused a billion dollars of pain and damage in the last 40 years” (AKA $1B mistake) NB: There actually are alt. (JVM) langs with types systems without any null at all. (Try, for fun.) Meanwhile in the real world, Java has null - and that ain’t never gonna change anymore…. but still not all hope is lost! ;-)
  5. 5. EclipseCon NullPointerException5 Why NPE Blog Post most viewed?
  6. 6. EclipseCon NullPointerException6 What? Static (compile-time) code analysis re. po. null-ness of method parameters, local variables, fields. Why? Aim to prove that your code is null safe, so NullPointerException will never occur anymore. How? Let’s see together...
  7. 7. EclipseCon NullPointerException7 Get started
  8. 8. EclipseCon NullPointerException <dependency> <groupId>org.eclipse.jdt</groupId> <artifactId>org.eclipse.jdt.annotation</artifactId> <version>2.0.0</version> </dependency> 8 @ which? Let’s get back to more about this (alternatives) in a moment…
  9. 9. EclipseCon NullPointerException, in EACH package, src/main before /test: @org.eclipse.jdt.annotation.NonNullByDefault package yourpackage; 9 @NonNullByDefault (Recommendation) Activating null analysis via Project Properties alone won’t do much yet. You’ll need to express what you want analysed with Java annotations. And occasionally, where you must, use @Nullable in (existing..) code.
  10. 10. EclipseCon NullPointerException10
  11. 11. EclipseCon NullPointerException11 Enjoy your extended coffee break… ;-) init() close() Done! Shortest presentation ever.
  12. 12. EclipseCon NullPointerException12 Done? 1. How will a (Maven) build enforce this, same as IDE? 2. How will each project get configured for this in-IDE? 3. Libraries, huh? (#existingcode #nogreenfield) 3.1 Library EEA & the build...
  13. 13. EclipseCon NullPointerException13 This was the most useful I was able to find, and (significantly...) extended and built upon: ● ● ● Curious: Sylvain lives literally next door, and we have common friends - what are the odds?? #FLOSS Research existing attempts to address this #ACK: Others have explored the same space before me - but I hadn’t found a full solution...
  14. 14. EclipseCon NullPointerException <plugin> <artifactId>maven-compiler-plugin</artifactId> <version>3.5.1</version> <configuration> ... <compilerId>jdt</compilerId> <compilerArguments> <properties>.settings/org.eclipse.jdt.core.prefs</properties> ? ^ ${basedir}/ ? <dependencies> <dependency> <!-- of maven-compiler-plugin, not project --> <groupId>org.eclipse.tycho</groupId> <artifactId>tycho-compiler-jdt</artifactId> 14 Build 1. 2. 3.
  15. 15. EclipseCon NullPointerException illustrates 3. in full example project. is a M2E AbstractProjectConfigurator which copies JDT compilerArguments properties from artifact in compiler-plugin dependency, or compilerArguments properties, into IDE. (TBD ⇒ ? #helpwanted) 15 Project Configuration Depends on where you sit on having .settings/** in SCM… ;-) Basically you can either: 1. use workspace wide default preferences (recommended: Oomph!); NOK for gradually.. 2. save .settings/*jdt.core.prefs project preferences properties per project in SCM 3. .settings/ in .gitignore & org.eclipse.jdt.core.prefs in Maven artifact, and: a. use maven-dependency-plugin to unpack it into target/ for build point maven-compiler-plugin to use that instead of .settings/ b. declare <dependency> and use M2E extension to set into each project (JDT API)
  16. 16. EclipseCon NullPointerException java.util java.util Optional.ofNullable(null) /* @Nullable */ V Map.get(Object key) Null type mismatch: required ... '@NonNull String' but the provided value is null 16 The underlying problem is interfacing with existing library code which you cannot modify... or... can we?! Libraries... You always use external 3rd-party libraries in all of your projects. Even if you think you don’t, you do - JDK! Look at 1st of many you’ll hit:
  17. 17. EclipseCon NullPointerException17
  18. 18. EclipseCon NullPointerException18 External *.eea annotation files “augment” (not in the Star Trek sense..) Eclipse Java Build Path entries with additional @NonNull VS @Nullablable information. Useful when working with 3rd-party library JARs you cannot easily change - including the JDK RT. Eclipse External null Annotations (EEA)
  19. 19. EclipseCon NullPointerException19
  20. 20. EclipseCon NullPointerException20
  21. 21. EclipseCon NullPointerException21 (TBD ⇒ ? #helpwanted #449653) Producing EEA JARs as Maven artifacts slf4j-api-eea ch.vorburger.null:eea-parent maven-resources-plugin config. maven-compiler-plugin disabled maven-surefire-plugin disabled /pom.xml /org/slf4j/LoggerFactory.eea /eea-for-gav NOT using Maven classifier - just naming convention of artifactId ending in *-eea (for filtering list of dependencies for performance in M2E plugin). Maven Classifiers don’t work great in M2E, and aren’t a fit here anyway - the EEA isn’t really an additional artifact for one (≉ GAV).
  22. 22. EclipseCon NullPointerException <plugin> <artifactId>maven-compiler-plugin</artifactId> <configuration> <compilerArguments> <annotationpath>CLASSPATH</annotationpath> <dependencies> <!-- of project, not maven-compiler-plugin --> <dependency> <groupId>ch.vorburger.null.eea</groupId> <artifactId>slf4j-api-eea</artifactId> <version>1.0.0-SNAPSHOT</version> <scope>provided</scope> 22 EEA & the build 1. Granularity not identical to in-IDE, but this works just fine… #pragmatic 2. Not REALLY needed on CP, but this approach avoids rebuilding infra. for DL, dist, etc. #itworks
  23. 23. EclipseCon NullPointerException External Annotations & M2E Maven in IDE 23 I’ve extended M2E Project Configurator (same which sets Compiler Properties) to set External annotations for Maven Dependencies classpath container. Problem: How to map JAR (or Project) with *.eea to Maven GAVs / JRE? I’ve used a file in JAR (eea-for-gav: org.slf4j:slf4j-api). It has N lines; useful e.g. for org.mockito:mockito-all & :mockito-core & “fat” J. Supports: Open/Close (=> binary) project - standard Maven Workspace Dependency Resolution (TBD ⇒ ? #helpwanted #457792)
  24. 24. EclipseCon NullPointerException24 In an ideal world, we should all have a common repo of EEA artifacts where we share null annotations... How-to new Git repo. @ GitHub PRs how? Or simply a GitHub Org?? (Soft on IP CLA?) I’m willing to kick something off… Is there anyone here who would be willing to contribute to this effort? External Annotations repository #helpwanted Bug 449653 and Bug 457792
  25. 25. Some Experiences & Wrapping Up
  26. 26. EclipseCon NullPointerException26 How-to move existing product to be NullPointerException free? 1. Set up your product’s org.eclipse.jdt.core.prefs & IDE Setup infra etc. It has to “just work” (OOB)… for Maven-based projects, see prez. part 1! ;) 2. Iterate submitting small incremental changes, project by project, src/main before src/test; @NonNullByDefault package by package: Add => lots red, adapt, some @Nullable, etc. 3. Contribute to (my) shared EEA library manually as required as you go 4. If push back, can start with IDE only, enforcement in build later (but do target build enforcement, else quickly regressions from others) Process (ODL: 60+ .git * 10s .project * 10s .java)
  27. 27. EclipseCon NullPointerException27 1. FindBugs: Oldest, but (supposedly) more limited null analysis. Still great for other static code analysis (just being ext. intro. in ODL) 2. Checker Framework: Interesting… actually. Extensive external annotations for JDK (*.jaif); IDE integration (TBC; must have IMHO) 3. IntelliJ IDEA: Some inference by bytecode analysis, also external annotations (XML); but no Java 8 generics, and how-to in build? Alternatives?
  28. 28. EclipseCon NullPointerException ● org.eclipse.jdt.annotation ■ Android’s, Lombok’s… ● javax.annotation from (NOT :jsr305 BTW...) ● org.jetbrains:annotations ■ org.checkerframework ● com.sun.istack.internal (rt.jar) 28 They’re NOT all the same… JDT’s & Checker’s are null type annotations (Java 8 @Target TYPE_USE), for generics! FB/JSR-305 is NOT, others IDK. The semantics differ, too; FB’s @Nullable :( You could also create your own. But simplest to get started is to just use JDT’s. Yeah, there are several “competing” annotations for (non) nullability… Accept it, pick one as primary for your project with your tool, move on! ;) @ which @Nullable & @NonNull (NotNull)
  29. 29. EclipseCon NullPointerException29 Gotcha! (Quiz Time) @Nullable File file() { return null; } @Nullable file() { return null; } File file() { return null; }
  30. 30. EclipseCon NullPointerException30 Gotcha! (Quiz Time) @Nullable File file() { return null; } @javax.annotation.Nullable file() { return null; } File file() { return null; }
  31. 31. EclipseCon NullPointerException31 FindBugs annotations interop? ● Configure javax.annotation.Nullable & NonNull (used e.g. in Google Guava! and ODL...) as secondary annotations in “Annotations for Null Specifications” preference intro. Since Neon (4.6) M5, save EEA. I’m still gaining experience with InterOp… diff. only evaluated when reading class files external to the project? That’s inconsistent… see prob with fileMayBeNull2. ● Use Eclipse’s null analysis instead of FindBugs’, in both build and IDE (use FB for other code analysis; exclude)
  32. 32. EclipseCon NullPointerException32 As of 2016.10.12: 441 closed VS 135 open bugs in CDT with “[null] …” in summary. (As usual: Not all 135 are true “bugs”; some are unclear, some lingering old “discussions”.) Sense of “not ideal solution, we had to compromise” in some areas (e.g. Bug #505828) Filed / found during evaluation: 1. Bug #443146 Code Completion unnecessary @NonNull when @NonNullByDefault 2. Bug #506375 Allow NonNullByDefault in external library annotation 3. Bug #505828 Inherit stronger than @NonNullByDefault 4. Bug #500885 NullPointerException at jdt.internal.compiler.lookup.BinaryTypeBinding 5. Bug #489900 annotation 2.1.0 in Maven central Status & Bugs
  33. 33. EclipseCon NullPointerException33 Annotations & @Override (with EEA…) @Override public boolean equals(Object obj) Illegal redefinition of parameter obj, inherited method from Object does not constrain this parameter @Override public boolean equals(@Nullable Object obj) Can be worked around by adding @Nullable - at every use! (Bug #505828 …)
  34. 34. EclipseCon NullPointerException34 ● ODL: (0) APPLY.. (1) OMG build on ecj… TBC! ;-) (2) @eclipse & @javax & (3) IntelliJ cohabitation? #me ● EEA: Repo with more contribs, releasing #me&you? Maybe conversion tool of Checker Framework JDK (*.jaif) metadata to EEA ?? Gen. EEA by automatic analysis? ● Plug-in Dependencies CP container? Gradle?? #you? ● Try it all out, contribute eea, provide feedback! #you Future Ideas (YOU? #helpwanted)
  35. 35. Q & A and YOUR feedback / experiences?
  36. 36. THANK YOU
  37. 37. EXTRA SLIDES
  38. 38. EclipseCon NullPointerException @NonNullByDefault({}) // deactivate it :-( public class Bean { private String name; public String getName() { return name; } public void setName(String name) { = name; } } 39 Zoom: Java Beans - null by default?
  39. 39. EclipseCon NullPointerException40 Zoom: Java Beans - Collection properties public class Bean { private final List<String> names = new ArrayList<>(); public List<String> getNames() { return names; } // no setNames() }
  40. 40. EclipseCon NullPointerException public class Bean { private Optional<String> name = Optional.empty(); public Optional<String> getName() { return name; } public void setName(String name) { = Optional.of(name); } // void unsetName() ? } 41 Zoom: Java Beans - Optional<> properties public class Bean { private @Nullable String name; public Optional<String> getName() { return Optional.ofNullable(name); } public void setName(String name) { = name; } // void unsetName() ? }
  41. 41. EclipseCon NullPointerException42 Zoom: Java Beans - mandatory properties Traditionally we don’t think about mandatory (required) bean properties.. if (some) clearly always are (beware..), use final and constructor or Builder pattern which throws Exception. public class Bean { private final String name; public Bean(String name) { = name; } public String getName() { return name; } } public class Bean { private final String name; public static Builder builder() { … public static class Builder { public Bean build() { … }
  42. 42. EclipseCon NullPointerException43 Zoom: Java Beans - mandatory properties (II) If you would like static compile-time checking of mandatory bean properties instead of (checked or unchecked) exceptions at run-time from the Builder, you can… It’s a PITA to hand-write, but ideal for code generators! Recommendation: Use For an example Builder of a Bean with some mandatory and some optional properties which enforces setting all required at compile time, in fixed order, by returning a particular (inner static) Build class for each (!) mandatory property, AKA “staged Builder”, follow (originally in in vorburger/eclipse-null-eea-augments)
  43. 43. EclipseCon NullPointerException44 Zoom: Fields TODO huh, seriously?! WTF. This is a significant obstacle to migrating existing code! Options? public int hashCode() { (...) + (field == null ? 0 : field.hashCode()); Potential null pointer access: this expression has a '@Nullable' type Quick Fix “Extract to local variable (all occurrences)” public int hashCode() { Type field2 = field; (...) + (field2 == null ? 0 : field2.hashCode()); PS: For hashCode() in particular, use Objects.hash(...) - but problem in general remains.
  44. 44. EclipseCon NullPointerException45 Zoom: Beans’ Fields: hashCode() & equals() In Beans’ hashCode() and equals() implementations, instead of introducing field indirection, use Java 7’s Objects.hash(...) and equalsHelper (* by yours truly). Or, again, consider using some bean code generator, such as e.g. Google’s AutoValue or, etc. (BTW public int hashCode() { Type field2 = field; (...) + (field2 == null ? 0 : field2.hashCode());
  45. 45. EclipseCon NullPointerException46 ● ● ● ... Recommended reading