Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lessons Learnt With Lambdas and Streams in JDK 8

2,203 views

Published on

A number of lessons learnt whilst understanding the realities of using Lambda expressions and the Streams API in JDK 8.

Published in: Software
  • Be the first to comment

Lessons Learnt With Lambdas and Streams in JDK 8

  1. 1. © Copyright Azul Systems 2016 © Copyright Azul Systems 2015 @speakjava Lessons Learnt With Lambdas and Streams in JDK 8 Simon Ritter Deputy CTO, Azul Systems 1
  2. 2. © Copyright Azul Systems 2016 A clever man learns from his mistakes... ...a wise man learns from other people’s
  3. 3. © Copyright Azul Systems 2016 Agenda  Lambdas and Streams Primer  Delaying Execution  Avoiding Loops In Streams  The Art Of Reduction  Parallel streams: myths and realities  Lambdas and Streams and JDK 9  Conclusions 3
  4. 4. © Copyright Azul Systems 2016 Lambdas And Streams Primer
  5. 5. © Copyright Azul Systems 2016 Lambda Expressions In JDK 8  Old style, anonymous inner classes  New style, using a Lambda expression  Can be used wherever the type is a functional interface 5 Simplified Parameterised Behaviour new Thread(new Runnable { public void run() { doSomeStuff(); } }).start(); new Thread(() -> doSomeStuff()).start();
  6. 6. © Copyright Azul Systems 2016 Functional Interface Definition  Is an interface  Must have only one abstract method – In JDK 7 this would mean only one method (like ActionListener)  JDK 8 introduced default methods – Adding multiple inheritance of types to Java – These are, by definition, not abstract  JDK 8 also now allows interfaces to have static methods  @FunctionalInterface to have the compiler check 6
  7. 7. © Copyright Azul Systems 2016 Type Inference  Compiler can often infer parameter types in a lambda expression – Inferrence based on target functional interface’s method signature  Fully statically typed (no dynamic typing sneaking in) – More typing with less typing static T void sort(List<T> l, Comparator<? super T> c); List<String> list = getList(); Collections.sort(list, (String x, String y) -> x.length() > y.length()); Collections.sort(list, (x, y) -> x.length() - y.length());
  8. 8. © Copyright Azul Systems 2016 Stream Overview  A stream pipeline consists of three types of things – A source – Zero or more intermediate operations – A terminal operation  Producing a result or a side-effect int total = transactions.stream() .filter(t -> t.getBuyer().getCity().equals(“London”)) .mapToInt(Transaction::getPrice) .sum(); Source Intermediate operation Terminal operation
  9. 9. © Copyright Azul Systems 2016 Stream Terminal Operations  The pipeline is only evaluated when the terminal operation is called – All operations can execute sequentially or in parallel – Intermediate operations can be merged  Avoiding multiple redundant passes on data  Short-circuit operations (e.g. findFirst)  Lazy evaluation – Stream characteristics help identify optimisations  DISTINT stream passed to distinct() is a no-op
  10. 10. © Copyright Azul Systems 2016 Optional Class  Terminal operations like min(), max(), etc do not return a direct result – Suppose the input Stream is empty?  Optional<T> – Container for an object reference (null, or real object) – Think of it like a Stream of 0 or 1 elements – use get(), ifPresent() and orElse() to access the stored reference – Can use in more complex ways: filter(), map(), etc
  11. 11. © Copyright Azul Systems 2016 Lambda Expressions And Delayed Execution
  12. 12. © Copyright Azul Systems 2016 Performance Impact For Logging  Heisenberg’s uncertainty principle  Setting log level to INFO still has a performance impact  Since Logger determines whether to log the message the parameter must be evaluated even when not used 12 logger.finest(getSomeStatusData()); Always executed
  13. 13. © Copyright Azul Systems 2016 Supplier<T>  Represents a supplier of results  All relevant logging methods now have a version that takes a Supplier  Pass a description of how to create the log message – Not the message  If the Logger doesn’t need the value it doesn’t invoke the Lambda  Can be used for other conditional activities 13 logger.finest(getSomeStatusData());logger.finest(() -> getSomeStatusData());
  14. 14. © Copyright Azul Systems 2016 Avoiding Loops In Streams
  15. 15. © Copyright Azul Systems 2016 Functional v. Imperative  For functional programming you should not modify state  Java supports closures over values, not closures over variables  But state is really useful… 15
  16. 16. © Copyright Azul Systems 2016 Counting Methods That Return Streams 16 Still Thinking Imperatively Set<String> sourceKeySet = streamReturningMethodMap.keySet(); LongAdder sourceCount = new LongAdder(); sourceKeySet.stream() .forEach(c -> sourceCount .add(streamReturningMethodMap.get(c).size()));
  17. 17. © Copyright Azul Systems 2016 Counting Methods That Return Streams 17 Functional Way sourceKeySet.stream() .mapToInt(c -> streamReturningMethodMap.get(c).size()) .sum();
  18. 18. © Copyright Azul Systems 2016 Printing And Counting 18 Still Thinking Imperatively LongAdder newMethodCount = new LongAdder(); functionalParameterMethodMap.get(c).stream() .forEach(m -> { output.println(m); if (isNewMethod(c, m)) newMethodCount.increment(); });
  19. 19. © Copyright Azul Systems 2016 Printing And Counting 19 More Functional, But Not Pure Functional int count = functionalParameterMethodMap.get(c).stream() .mapToInt(m -> { int newMethod = 0; output.println(m); if (isNewMethod(c, m)) newMethod = 1; return newMethod }) .sum(); There is still state being modified in the Lambda
  20. 20. © Copyright Azul Systems 2016 Printing And Counting 20 Even More Functional, But Still Not Pure Functional int count = functionalParameterMethodMap.get(nameOfClass) .stream() .peek(method -> output.println(method)) .mapToInt(m -> isNewMethod(nameOfClass, m) ? 1 : 0) .sum(); Strictly speaking printing is a side effect, which is not purely functional
  21. 21. © Copyright Azul Systems 2016 The Art Of Reduction (Or The Need to Think Differently)
  22. 22. © Copyright Azul Systems 2016 A Simple Problem  Find the length of the longest line in a file  Hint: BufferedReader has a new method, lines(), that returns a Stream 22 BufferedReader reader = ... int longest = reader.lines() .mapToInt(String::length) .max() .getAsInt();
  23. 23. © Copyright Azul Systems 2016 Another Simple Problem  Find the length of the longest line in a file 23
  24. 24. © Copyright Azul Systems 2016 Another Simple Problem  Find the length of the longest line in a file 24
  25. 25. © Copyright Azul Systems 2016 Naïve Stream Solution  That works, so job done, right?  Not really. Big files will take a long time and a lot of resources  Must be a better approach 25 String longest = reader.lines(). sort((x, y) -> y.length() - x.length()). findFirst(). get();
  26. 26. © Copyright Azul Systems 2016 External Iteration Solution  Simple, but inherently serial  Not thread safe due to mutable state 26 String longest = ""; while ((String s = reader.readLine()) != null) if (s.length() > longest.length()) longest = s;
  27. 27. © Copyright Azul Systems 2016 Recursive Approach 27 String findLongestString(String longest, BufferedReader reader) { String next = reader.readLine(); if (next == null) return longest; if (next.length() > longest.length()) longest = next; return findLongestString(longest, reader); }
  28. 28. © Copyright Azul Systems 2016 Recursion: Solving The Problem  No explicit loop, no mutable state, we’re all good now, right?  Unfortunately not: – larger data sets will generate an OOM exception – Too many stack frames 28 String longest = findLongestString("", reader);
  29. 29. © Copyright Azul Systems 2016 A Better Stream Solution  Stream API uses the well known filter-map-reduce pattern  For this problem we do not need to filter or map, just reduce Optional<T> reduce(BinaryOperator<T> accumulator)  BinaryOperator is a subclass of BiFunction – R apply(T t, U u)  For BinaryOperator all types are the same – T apply(T x, T y) 29
  30. 30. © Copyright Azul Systems 2016 A Better Stream Solution  The key is to find the right accumulator – The accumulator takes a partial result and the next element, and returns a new partial result – In essence it does the same as our recursive solution – But without all the stack frames 30
  31. 31. © Copyright Azul Systems 2016 A Better Stream Solution  Use the recursive approach as an accululator for a reduction 31 String longestLine = reader.lines() .reduce((x, y) -> { if (x.length() > y.length()) return x; return y; }) .get();
  32. 32. © Copyright Azul Systems 2016 A Better Stream Solution  Use the recursive approach as an accululator for a reduction 32 String longestLine = reader.lines() .reduce((x, y) -> { if (x.length() > y.length()) return x; return y; }) .get(); x in effect maintains state for us, by providing the partial result, which is the longest string found so far
  33. 33. © Copyright Azul Systems 2016 The Simplest Stream Solution  Use a specialised form of max()  One that takes a Comparator as a parameter  comparingInt() is a static method on Comparator Comparator<T> comparingInt( ToIntFunction<? extends T> keyExtractor) 33 reader.lines() .max(comparingInt(String::length)) .get();
  34. 34. © Copyright Azul Systems 2016 Parallel Streams: Myths And Realities
  35. 35. © Copyright Azul Systems 2016 35
  36. 36. © Copyright Azul Systems 2016 Serial And Parallel Streams  Syntactically very simple to change from serial to parallel 36 int sum = list.stream() .filter(w -> w.getColour() == RED) .mapToInt(w -> w.getWeight()) .sum(); int sum = list.parallelStream() .filter(w -> w.getColour() == RED) .mapToInt(w -> w.getWeight()) .sum();
  37. 37. © Copyright Azul Systems 2016 Serial And Parallel Streams  Syntactically very simple to change from serial to parallel  Use serial() or parallel() to change mid-stream 37 int sum = list.parallelStream() .filter(w -> w.getColour() == RED) .serial() .mapToInt(w -> w.getWeight()) .parallel() .sum();
  38. 38. © Copyright Azul Systems 2016 Serial And Parallel Streams  Syntactically very simple to change from serial to parallel  Use serial() or parallel() to change mid-stream  Whole stream is processed serially or in parallel – Last call wins  Operations should be stateless and independent 38 int sum = list.parallelStream() .filter(w -> w.getColour() == RED) .serial() .mapToInt(w -> w.getWeight()) .parallel() .sum();
  39. 39. © Copyright Azul Systems 2016 Common ForkJoinPool  Created when JVM starts up  Default number of threads is equal to CPUs reported by OS – Runtime.getRuntime().availableProcessors()  Can be changed manually 39 -Djava.util.concurrent.ForkJoinPool.common.parallelism=n
  40. 40. © Copyright Azul Systems 2016 Common ForkJoinPool 40 Set<String> workers = new ConcurrentSet<String>(); int sum = list.parallelStream() .peek(n -> workers.add(Thread.currentThread().getName())) .filter(w -> w.getColour() == RED) .mapToInt(w -> w.getWeight()) .sum(); System.out.println(”Worker thread count = ” + workers.size()); -Djava.util.concurrent.ForkJoinPool.common.parallelism=4 Worker thread count = 5
  41. 41. © Copyright Azul Systems 2016 Common ForkJoinPool 41 Set<String> workers = new ConcurrentSet<String>(); ... worker.stream().forEach(System.out::println); -Djava.util.concurrent.ForkJoinPool.common.parallelism=4 ForkJoinPool.commonPool-worker-0 ForkJoinPool.commonPool-worker-1 ForkJoinPool.commonPool-worker-2 ForkJoinPool.commonPool-worker-3 main
  42. 42. © Copyright Azul Systems 2016 Parallel Stream Threads  Invoking thread is also used as worker  Parallel stream invokes ForkJoin synchronously – Blocks until work is finished  Other application threads using parallel stream will be affected  Beware IO blocking actions inside stream operartions  Do NOT nest parallel streams
  43. 43. © Copyright Azul Systems 2016 Parallel Stream Custom Pool Hack 43 ForkJoinPool customPool = new ForkJoinPool(4); ForkJoinTask<Integer> hackTask = customPool.submit(() -> { return list.parallelStream() .peek(w -> workers.add(Thread.currentThread().getName())) .filter(w -> w.getColour() == RED) .mapToInt(w -> w.getWeight()) .sum(); }); int sum = hackTask.get(); Worker thread count = 4
  44. 44. © Copyright Azul Systems 2016 Is A Parallel Stream Faster?  Using a parallel stream guarantees more work – Setting up fork-join framework is overhead  This might complete more quickly  Depends on several factors – How many elements in the stream (N) – How long each element takes to process (T) – Parallel streams improve with N x T – Operation types  map(), filter() good  sort(), distinct() not so good 44
  45. 45. © Copyright Azul Systems 2016 Lambdas And Streams And JDK 9
  46. 46. © Copyright Azul Systems 2016 Additional APIs  Optional now has a stream() method – Returns a stream of one element or an empty stream  Collectors.flatMapping() – Returns a Collector that converts a stream from one type to another by applying a flat mapping function 46
  47. 47. © Copyright Azul Systems 2016 Additional APIs  Matcher stream support – Stream<MatchResult> results()  Scanner stream support – Stream<MatchResult> findAll(String pattern) – Stream<MatchResult> findAll(Pattern pattern) – Stream<String> tokens() 47
  48. 48. © Copyright Azul Systems 2016 Additional Stream Sources  java.net.NetworkInterface – Stream<InetAddress> inetAddresses() – Stream<NetworkInterface> subInterfaces() – Stream<NetworkInterface> networkInterfaces()  static  java.security.PermissionCollection – Stream<Permission> elementsAsStream() 48
  49. 49. © Copyright Azul Systems 2016 Parallel Support For Files.lines()  Memory map file for UTF-8, ISO 8859-1, US-ASCII – Character sets where line feeds easily identifiable  Efficient splitting of mapped memory region  Divides approximately in half – To nearest line feed 49
  50. 50. © Copyright Azul Systems 2016 Parallel Lines Performance 50
  51. 51. © Copyright Azul Systems 2016 Stream takeWhile  Stream<T> takeWhile(Predicate<? super T> p)  Select elements from stream until Predicate matches  Unordered stream needs consideration thermalReader.lines() .mapToInt(i -> Integer.parseInt(i)) .takeWhile(i -> i < 56) .forEach(System.out::println);
  52. 52. © Copyright Azul Systems 2016 Stream dropWhile  Stream<T> dropWhile(Predicate<? super T> p)  Ignore elements from stream until Predicate matches  Unordered stream still needs consideration thermalReader.lines() .mapToInt(i -> Integer.parseInt(i)) .dropWhile(i -> i < 56) .forEach(System.out::println);
  53. 53. © Copyright Azul Systems 2016 Conclusions
  54. 54. © Copyright Azul Systems 2016 Conclusions  Lambdas and Stream are a very powerful combination  Does require developers to think differently – Avoid loops, even non-obvious ones! – Reductions  Be careful with parallel streams  More to come in JDK 9 (and 10)  Join the Zulu.org community – www.zulu.org 54
  55. 55. © Copyright Azul Systems 2016 © Copyright Azul Systems 2015 @speakjava Q & A Simon Ritter Deputy CTO, Azul Systems 55

×