<Insert Picture Here>Project Lambda: To Multicore and BeyondAlexander PotochkinSenior software developerNovember 28, 2011
The following is intended to outline our general productdirection. It is intended for information purposes only, andmay no...
Introduction to Project Lambda•    OpenJDK Project Lambda started Dec 2009•    Targeted for Java SE 8 as JSR 335• Aims to ...
Hardware trends – the future is parallel• Chip designers havenowhere to go but parallel– Moores Law gives morecores, not f...
Developers need simple parallel libraries• One of Java‟s strengths has always been its libraries– Better libraries are key...
Without more language supportfor parallel idioms,people will instinctively reach forserial idioms
The biggest serial idiom of all: the for loopdouble highestScore = 0.0;for (Student s : students) {    if (s.gradYear == 2...
The biggest serial idiom of all: the for loopdouble highestScore = 0.0;for (Student s : students) {    if (s.gradYear == 2...
Let‟s try a more parallel idiom: internal iterationdouble highestScore =    students.filter(new Predicate<Student>() {    ...
Let‟s try a more parallel idiom: internal iterationdouble highestScore =    students.filter(new Predicate<Student>() {    ...
But … Yuck!double highestScore =    students.filter(new Predicate<Student>() {        public boolean op(Student s) {      ...
But … Yuck!A wise customer once said:“The pain of anonymous innerclasses makes us roll our eyes inthe back of our heads ev...
LAMBDA EXPRESSIONS
A better way to represent “code as data”double highestScore =  students.filter((Student s)-> s.gradYear == 2011)          ...
A better way to represent “code as data”double highestScore =  students.filter((Student s)-> s.gradYear == 2011)          ...
A better way to represent “code as data”double highestScore =  students.filter((Student s)-> s.gradYear == 2011)          ...
The science of lambda expressions• The name comes from the lambda calculus created by Alonzo Church(1936) and explored by ...
TYPING
What is the type of a lambda expression?(Student s) -> s.gradYear == 2011• Morally, a function type from Student to boolea...
“Use what you know”• Java already has an idiom for describing “functional things”:single-method interfacesinterface    Run...
Introducing: SAM types• A SAM type is an interface with a Single Abstract Methodinterface    Runnable             {   void...
The type of a lambda expression is a SAM type• A SAM type is an interface with a Single Abstract MethodPredicate<Student> ...
The science of SAM conversion • Lambda expression must have:  – Same parameter types and arity as SAM type‟s method  – Ret...
But wait, theres more• Lambdas solve the “vertical problem” of inner classes• Parameter types can still be a “horizontal p...
But wait, theres more• Lambdas solve the “vertical problem” of inner classes• Parameter types can still be a “horizontal p...
SAM conversion includes target typing• Target typing identifies parameter types for the lambdaexpression based on the cand...
Recap: SAM types• Self-documenting• Build on existing concepts• Ensure lambda expressions work easily with existing librar...
METHOD REFERENCES
Consider sorting a list of Person objects  by last name:class Person { String getLastName() {…} }List<Person> people = …Co...
A lambda expression helps, but only so muchList<Person> people = …Collections.sort(people,(a, b) -> a.getLastName().compar...
A lambda expression helps, but only so muchList<Person> people = …Collections.sort(people,(a, b) -> a.getLastName().compar...
How to express “Person‟s last name” in Java?• Assume an interface to extract a value from an object:interface Extractor<T,...
Is that the best we can do?Collections.sortBy(people, p -> p.getLastName() );• Writing little wrapper lambdas will be a pa...
Is that the best we can do?Collections.sortBy(people, p -> p.getLastName());• Writing little wrapper lambdas will be a pai...
Recap: Method references• When code outgrows a lambda expression, write a method andtake a method reference• Lambda expres...
A word about implementation• Lambda expressions are not sugar for inner classes– Implemented with MethodHandle from JSR 29...
LIBRARY EVOLUTION
As the language evolves,   the libraries should evolve with it• Java collections do not support internal iteration largely...
Without more library support forparallel idioms,people will instinctively reach forserial idioms
Library support for internal iteration• Sometimes, we want to add more types– Recall Java SE will likely define a “starter...
Extension methods: a measured step towards   more flexible inheritance  public interface Set<T> extends Collection<T> {  p...
Extension methods in a nutshellAn extension method is just an ordinary interface method• For a client:– Nothing new to lea...
Platform Evolution: A Collaborative EffortJSR 335 @ jcp.orgBrian Goetz is the spec leadJSR 166 @ oswego.eduDoug Lea’s conc...
Test it right now! http://jdk8.java.net/lambda/The early access implementation of Lambda enabled JDK
<Insert Picture Here>Project Lambda: To Multicore and BeyondAlexander PotochkinSenior software developerNovember 28, 2011
Upcoming SlideShare
Loading in …5
×

Project Lambda: To Multicore and Beyond

1,616 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,616
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
34
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Project Lambda: To Multicore and Beyond

  1. 1. <Insert Picture Here>Project Lambda: To Multicore and BeyondAlexander PotochkinSenior software developerNovember 28, 2011
  2. 2. The following is intended to outline our general productdirection. It is intended for information purposes only, andmay not be incorporated into any contract. It is not acommitment to deliver any material, code, or functionality,and should not be relied upon in making purchasingdecisions. The development, release, and timing of anyfeatures or functionality described for Oracle’s productsremains at the sole discretion of Oracle.
  3. 3. Introduction to Project Lambda• OpenJDK Project Lambda started Dec 2009• Targeted for Java SE 8 as JSR 335• Aims to support programming in a multicore environmentby adding closures and related features to the Java SE platform
  4. 4. Hardware trends – the future is parallel• Chip designers havenowhere to go but parallel– Moores Law gives morecores, not faster cores• We must learn to writesoftware that parallelizesgracefully
  5. 5. Developers need simple parallel libraries• One of Java‟s strengths has always been its libraries– Better libraries are key to making parallelization easier– Ideally, let the libraries worry about algorithmicdecomposition, scheduling, computation topology• Obvious place to start: parallel operations in collections– filter, sort, map/reduce– select, collect, detect, reject• High-level operations tend to improve the readability of code,as well as its performance
  6. 6. Without more language supportfor parallel idioms,people will instinctively reach forserial idioms
  7. 7. The biggest serial idiom of all: the for loopdouble highestScore = 0.0;for (Student s : students) { if (s.gradYear == 2011) { if (s.score > highestScore) { highestScore = s.score; } }}This code is inherently serial– Traversal logic is fixed (iterate serially from beginning to end)– Business logic is stateful (use of > and accumulator variable)
  8. 8. The biggest serial idiom of all: the for loopdouble highestScore = 0.0;for (Student s : students) { if (s.gradYear == 2011) { if (s.score > highestScore) { highestScore = s.score; } }}Existing collections impose external iteration– Client of collection determines mechanism of iteration– Implementation of accumulation is over-specified– Computation is achieved via side-effects
  9. 9. Let‟s try a more parallel idiom: internal iterationdouble highestScore = students.filter(new Predicate<Student>() { public boolean op(Student s) { return s.gradYear == 2011; }}).map(new Extractor<Student,Double>() { public Double extract(Student s) { return s.score; }}).max();Not inherently serial!– Traversal logic is not fixed by the language– Business logic is stateless (no stateful accumulator)
  10. 10. Let‟s try a more parallel idiom: internal iterationdouble highestScore = students.filter(new Predicate<Student>() { public boolean op(Student s) { return s.gradYear == 2011; }}).map(new Extractor<Student,Double>() { public Double extract(Student s) { return s.score; }}).max();Iteration and accumulation are embodied in the library– e.g. filtering may be done in parallel– Client is more flexible, more abstract, less error-prone
  11. 11. But … Yuck!double highestScore = students.filter(new Predicate<Student>() { public boolean op(Student s) { return s.gradYear == 2011; }}).map(new Extractor<Student,Double>() { public Double extract(Student s) { return s.score; }}).max();Boilerplate code!
  12. 12. But … Yuck!A wise customer once said:“The pain of anonymous innerclasses makes us roll our eyes inthe back of our heads every day.”
  13. 13. LAMBDA EXPRESSIONS
  14. 14. A better way to represent “code as data”double highestScore = students.filter((Student s)-> s.gradYear == 2011) .map((Student s) -> s.score) .max();• Zero or more formal parameters – Like a method• Body may be an expression or statements – Unlike a method – If body is an expression, no need for „return‟ or „;‟
  15. 15. A better way to represent “code as data”double highestScore = students.filter((Student s)-> s.gradYear == 2011) .map((Student s) -> s.score) .max();Code reads like the problem statement:“Find the highest score of the students who graduated in 2011”
  16. 16. A better way to represent “code as data”double highestScore = students.filter((Student s)-> s.gradYear == 2011) .map((Student s) -> s.score) .max();Shorter than nested for loops, and potentially faster becauseimplementation determines how to iterate– Virtual method lookup chooses the best filter() method– Opportunities for lazy evaluation in filter() and map()– Opportunities for parallelism
  17. 17. The science of lambda expressions• The name comes from the lambda calculus created by Alonzo Church(1936) and explored by Steele and Sussman (1975-1980)• A lambda expression is like a lexically scoped anonymous method – Lexical scoping: can read variables from the lexical environment – No shadowing of lexical scope, unlike with inner classes – Not a member of any class, unlike with inner classes
  18. 18. TYPING
  19. 19. What is the type of a lambda expression?(Student s) -> s.gradYear == 2011• Morally, a function type from Student to boolean• But Java does not have function types, so: – How would we write a function type? – How would it interact with autoboxing? – How would it interact with generics? – How would it describe checked exceptions?
  20. 20. “Use what you know”• Java already has an idiom for describing “functional things”:single-method interfacesinterface Runnable { void run(); }interface Callable<T> { T call(); }interface Comparator<T> { boolean compare(T x, T y); }interface ActionListener { void actionPerformed(…); } • Let‟s reuse these, rather than introduce function types Comparator<T> ~ a function type from (T,T) to boolean Predicate<T> ~ a function type from T to boolean
  21. 21. Introducing: SAM types• A SAM type is an interface with a Single Abstract Methodinterface Runnable { void run(); }interface Callable<T> { T call(); }interface Comparator<T> { boolean compare(T x, T y); }interface ActionListener { void actionPerformed(…); } • No special syntax to declare a SAM type • Recognition is automatic for suitable interfaces and abstract classes
  22. 22. The type of a lambda expression is a SAM type• A SAM type is an interface with a Single Abstract MethodPredicate<Student> p = (Student s)->s.gradYear == 2011;• Invoking the SAM type‟s method invokes the lambda‟s bodyboolean ok = p.isTrue(aStudent);• Instant compatibility with existing libraries!executor.execute(() -> println(“Boo”));btn.addActionListener( (ActionEvent e) -> println(“Boo”));
  23. 23. The science of SAM conversion • Lambda expression must have: – Same parameter types and arity as SAM type‟s method – Return type compatible with SAM type‟s method – Checked exceptions compatible with SAM type‟s method • SAM type‟s method name is not relevantinterface Predicate<T> { boolean op(T t); }Predicate<Student> p = (Student s) -> s.gradYear == 2011;interface StudentQualifier { Boolean check(Student s); }StudentQualifier c = (Student s) -> s.gradYear == 2011; • Lambda expressions may only appear in contexts where they can undergo SAM conversion (assignment, method call/return, cast)
  24. 24. But wait, theres more• Lambdas solve the “vertical problem” of inner classes• Parameter types can still be a “horizontal problem” double highestScore = students.filter((Student s)-> s.gradYear == 2011) .map( (Student s) -> s.score) .max();
  25. 25. But wait, theres more• Lambdas solve the “vertical problem” of inner classes• Parameter types can still be a “horizontal problem” double highestScore = students.filter((Student s)-> s.gradYear == 2011) .map( (Student s) -> s.score) .max();• SAM conversion can usually infer them! double highestScore = students.filter( s -> s.gradYear == 2011) .map( s -> s.score) .max();
  26. 26. SAM conversion includes target typing• Target typing identifies parameter types for the lambdaexpression based on the candidate SAM type‟s method interface Collection<T> { Collection<T> filter(Predicate<T> t); } Collection<Student> students = … … students.filter( s -> s.gradYear == 2011 )– students.filter() takes a Predicate<Student>– Predicate<Student> is a SAM type whose method takes Student– Therefore, s must be a Student
  27. 27. Recap: SAM types• Self-documenting• Build on existing concepts• Ensure lambda expressions work easily with existing libraries– Java SE will likely define a “starter kit” of SAM types such asPredicate, Filter, Extractor, Mapper, Reducer…• Type inference gets your eyes to the “beef” quickly• You could think of our lambda expressions as “SAM literals”
  28. 28. METHOD REFERENCES
  29. 29. Consider sorting a list of Person objects by last name:class Person { String getLastName() {…} }List<Person> people = …Collections.sort(people, new Comparator<Person>() { public int compare(Person a, Person b) { return a.getLastName().compareTo(b.getLastName()); }});– Yuck!
  30. 30. A lambda expression helps, but only so muchList<Person> people = …Collections.sort(people,(a, b) -> a.getLastName().compareTo(b.getLastName()));• More concise, but not more abstract– Performs data access (getLastName) and computation (compareTo)• More abstract if someone else handles computation – If we can extract the data dependency –“Person‟s last name” –from the code, then sort() can split data access and computation
  31. 31. A lambda expression helps, but only so muchList<Person> people = …Collections.sort(people,(a, b) -> a.getLastName().compareTo(b.getLastName())); SELECT * FROM PERSON ORDER BY LASTNAME• More concise, but not more abstract– Performs data access (getLastName) and computation (compareTo)• More abstract if someone else handles computation – If we can extract the data dependency –“Person‟s last name” –from the code, then sort() can split data access and computation
  32. 32. How to express “Person‟s last name” in Java?• Assume an interface to extract a value from an object:interface Extractor<T, U> { U get(T element); }• And a sortBy method keyed off an extractor:public <T, U extends Comparable<…>> void sortBy(Collection<T> coll, Extractor<T,U> ex) {…}}Then, pass a lambda expression that “wraps” a method call:Collections.sortBy(people, p -> p.getLastName());– SAM conversion types the lambda as Extractor<Person,String>– sortBy() can pre-query last names, cache them, build indices…
  33. 33. Is that the best we can do?Collections.sortBy(people, p -> p.getLastName() );• Writing little wrapper lambdas will be a pain• If only we could reuse an existing method...
  34. 34. Is that the best we can do?Collections.sortBy(people, p -> p.getLastName());• Writing little wrapper lambdas will be a pain• If only we could reuse an existing method...Collections.sortBy(people, Person#getLastName );• Method reference introduced with #• No need for () or parameter types in simple cases
  35. 35. Recap: Method references• When code outgrows a lambda expression, write a method andtake a method reference• Lambda expressions and method references have SAM types• Work easily with existing libraries• Can specify parameter types explicitly if needed• No field references (use method references to getters/setters)
  36. 36. A word about implementation• Lambda expressions are not sugar for inner classes– Implemented with MethodHandle from JSR 292• Method references are not sugar for wrapper lambdas– Implemented with enhanced ldc instruction from JSR 292
  37. 37. LIBRARY EVOLUTION
  38. 38. As the language evolves, the libraries should evolve with it• Java collections do not support internal iteration largely becausethe language made it so clunky at the time• Now the language can easily treat “code as data”, it‟s crucial tosupport parallel/functional idioms in the standard libraries• Continues a long theme of language/library co-evolution– synchronized {} blocks / Thread.wait()/notify()– for-each loop / Iterable<T>– Generic type inference / <T>Collection.toArray(T[] x)
  39. 39. Without more library support forparallel idioms,people will instinctively reach forserial idioms
  40. 40. Library support for internal iteration• Sometimes, we want to add more types– Recall Java SE will likely define a “starter kit” of SAM types• Sometimes, we want to augment existing interfaces• No good way to add methods to existing interfac`es today– Binary compatible: old clients continue to execute – Source incompatible: old implementations fail to compile • Existing techniques for interface evolution are insufficient– Adding to j.u.Collections diminishes the value of interface contracts– Using abstract classes instead of interfaces– Interface inheritance and naming conventions (e.g. IDocument,IDocumentExtension, IDocumentExtension2, IDocumentExtension3
  41. 41. Extension methods: a measured step towards more flexible inheritance public interface Set<T> extends Collection<T> { public int size(); … public extension T reduce(Reducer<T> r) default Collections.<T>setReducer; }Allows library maintainers to effectively add methods after thfact by specifying a default implementation– “If you cannot afford an implementation of reduce(),one will be provided for you”
  42. 42. Extension methods in a nutshellAn extension method is just an ordinary interface method• For a client:– Nothing new to learn –calling the extension method works as usual,and the default method is linked dynamically if needed• For an API implementer:– An implementation of an augmented interface may provide themethod, or not• For an API designer:– Default method can use public API of augmented interface
  43. 43. Platform Evolution: A Collaborative EffortJSR 335 @ jcp.orgBrian Goetz is the spec leadJSR 166 @ oswego.eduDoug Lea’s concurrency-interest list for parallel collectionsJCR Project Lambda @ openjdk.orgPrototype implementation ongoingMercurial repository available
  44. 44. Test it right now! http://jdk8.java.net/lambda/The early access implementation of Lambda enabled JDK
  45. 45. <Insert Picture Here>Project Lambda: To Multicore and BeyondAlexander PotochkinSenior software developerNovember 28, 2011

×