Martin Odersky: What's next for Scala
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Martin Odersky: What's next for Scala



Martin Odersky, the creator of Scala, talks about what's next for Scala.

Martin Odersky, the creator of Scala, talks about what's next for Scala.



Total Views
Views on SlideShare
Embed Views



16 Embeds 643 351 95
http://localhost 74 51 43 7 5 5 3 2 2 1 1 1 1 1


Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Martin Odersky: What's next for Scala Presentation Transcript

  • 1. Scala NextSF Scala meetup Dec 8th, 2011
  • 2. Scala Today 2
  • 3. Some adoption vectors: •  Web platforms •  Trading platforms •  Financial modeling •  Simulation Fast to first product, scalable afterwards 3
  • 4. Github vs. Stack Overflow RedMonk: Revisiting the Dataists Programming Language Rankings Typesafe Confidential 4
  • 5. Commercial Adoption •  Scala jobs tripled in last year •  Now at estimated 100,000 developers Typesafe Confidential 5
  • 6. (Only 17 months ago!)Scala New collections2.8: Package objects Context bounds Better implicits ... 6
  • 7. Parallel collections DelayedInit and AppScala Faster REPL Progress on IDEs:2.9: Eclipse, IntelliJ, Neatbeans, ENSIME Better docs Lots of bug fixes   7
  • 8. Parallel Collections•  Use Java 7 Fork Join framework•  Split work by number of Processors•  Each Thread has a work queue that is split exponentially. Largest on end of queue•  Granularity balance against scheduling overhead•  On completion threads “work steals” from end of other thread queues 8
  • 9. ... and its usage import  java.util.ArrayList;   ...   Person[]  people;   Person[]  minors;   Person[]  adults;   {    ArrayList<Person>  minorsList  =  new  ArrayList<Person>();        ArrayList<Person>  adultsList  =  new  ArrayList<Person>();  ... in Java:      for  (int  i  =  0;  i  <  people.length;  i++)                (people[i].age  <  18  ?  minorsList  :  adultsList)            .add(people[i]);        minors  =  minorsList.toArray(people);        adults  =  adultsList.toArray(people);   }   A function value     An infix method call... in Scala: val  people:  Array[Person]   val  (minors,  adults)  =  people  partition  (_.age  <  18)   A simple pattern match 9
  • 10. Going Parallel  ... in Java: ?  ... in Scala: val  people:  Array[Person]   val  (minors,  adults)  =  people.par  partition  (_.age  <  18)   10
  • 11. General Collection Hierarchy Remove this layer in 2.10? GenTraversable GenIterableTraversable GenSeq Iterable ParIterable Seq ParSeq 11
  • 12. Going Distributed•  Can we get the power of parallel collections to work on 10’000s of computers?•  Hot technologies: MapReduce (Google’s and Hadoop)•  But not everything is easy to fit into that mold•  Sometimes 100’s of map-reduce steps are needed.•  Distributed collections retain most operations, provide a powerful frontend for MapReduce computations.•  Scala’s uniform collection model is designed to also accommodate parallel and distributed.•  Projects at Google (Cascade), Berkeley (Spark), EPFL. 12
  • 13. Scala Eclipse IDEnext: Play web framework 2.0 Akka 2.0 Scala 2.10 13
  • 14. Scala Now in RC2 Final expected before the end of the year.EclipseIDE 14
  • 15. Goalsreliable (no crashes/lock ups)responsive (never wait when typing)work with large projects/files –  Scala compiler (80k LOC), 4-5000 LOC/file –  advanced use of the type system: path-dependent types, self-types, mix-ins
  • 16. FeaturesKeep it simple –  highlight errors as you type –  completions (including implicits) –  hyperlinking –  project builder (+ dependent projects)Support mixed Java-Scala projects –  all features should work between Java/Scala sourcesJUnit Test Runner should pick up testsMore stuff based on external libraries –  (some) refactoring, code formatter, mark occurrences, structured selections, show inferred semi-colons
  • 17. Features (3)based on external libraries –  (some) refactoring –  code formatter –  mark occurrences –  structured selections –  show inferred semi-colons
  • 18. @jonifreemanJoni FreemanLatest Scala Eclipse plugin works surprisingly well! Even manages ourmixed Java/Scala project. Kudos to the team! #scala@esorribasEduardo SorribasThe latest beta of the Scala IDE for eclipse is much better. Im startingto like it.@jannehietamakiJanne HietamäkiAfter years of misery, the Eclipse Scala plugin actually seems to workquite well.
  • 19. ArchitectureUse the full-blown Scala compiler for: –  interactive error highlight, completion, hyperlinking –  turning Scala symbols into Java model elementsWeave the JDT compiler when it needs help –  JDT was NOT meant to be extended
  • 20. Why rely on scalac? –  reuse (type-checker == 1-2 person years) –  consistency –  compiler pluginsWhy not? –  SPEED –  (very) tight dependency on the Scala version
  • 21. asynchronousinterruptibletargetedstop after type-checking Presentation Compiler
  • 22. Result is communicated through a SyncVar
  • 23. •  All compiler activity happens on PC thread•  compile loaded files when work queue is empty (in the background)•  Check work queue when type checker reaches safe-points in the AST•  Drop everything when a file is changed (AskReload)
  • 24. Implementation26
  • 25. 1 type-checker run / instance --> 100s of type-check runs / minute –  memory leaks –  side-effects/state –  out-of-order and targeted type-checkingneeded to improve the compiler –  2.9.x, 2.10 (trunk) –  what about 2.8? 2.8.2, 2.8.3-SNAPSHOT
  • 26. New: Play Framework 2.0•  Play Framework is an open source web application framework, inspired by Ruby on Rails, for Java and Scala•  Play Framework 2.0 retains full Java support while moving to a Scala core and builds on key pieces of the Typesafe Stack, including Akka middleware and SBT•  Play will be integrated in TypeSafe stack 2.0•  Typesafe will contribute to development and provide commercial support and maintenance.
  • 27. RoadmapMay 2011 Oct 2011 Q1 2012 Q3 2012Typesafe Typesafe Typesafe TypesafeStack 1.0 Stack 1.1 Stack 2.0 Stack 2.xScala 2.9.0 Scala 2.9.1 Scala 2.9.x Scala 2.10 Akka 1.1 Akka 1.2 Akka 2.0 Akka 2.x Play 2.0 Play 2.x Slick (DB)
  • 28. 1.  New reflection frameworkScala 2.  3.  Reification type Dynamic  2.10: 4.  More IDE improvements: find- references, debugger, worksheet. 5.  Faster builds 6.  SIPs: string interpolation, simpler implicits. ETA: Early 2012. 30
  • 29. New in Scala 2.10: DynamicType Dynamic bridges the gap between static and dynamic typing.Method calls get translated to applyDynamicGreat for interfacing with dynamic languages (e.g. JavaScript) class JS extends Dynamic { def applyDynamic(methName: String, args: Any*): Any = { println("apply dynamic "+methName+args.mkString("(", ",", ")")) } } val x = new JS // à x.applyDynamic( foo , 1) // à x.applyDynamic( bar )   31
  • 30. Proposed for Scala 2.10: SIP 11: String interpolationIdea: Instead of   Bob  is    +  n  +   years  old  write:  s Bob  is  $n  years  old  which gets translated to  new  StringContext( Bob  is ,   years  old ).s(n)  Here, s is a library-defined method for string interpolation.             32
  • 31. This can be generalized to other string processors besides s: xml        <body>      <a  href  =   some  link >  ${linktext}  </a>        </body>            scala        scala.concurrent.transaction.withinTransaction  {            (implicit  currentTransaction:  Transaction)  =>      $expr        }       33
  • 32. Proposed for Scala 2.10: SIP 12: Uncluttering controlShould be able to write:    if  x  <  0  then  –x  else  x      while  x  >  0  do  {  println(x);  x  -­‐=  1  }      for  x  <-­‐  xs  do  println(x)      for  x  <-­‐  xs  yield  x  *  x       34
  • 33. Proposed for Scala 2.10: SIP 13: Implicit classesVariation: Add @inline  to class def to get speed of extension methods. 35
  • 34. New in Scala 2.10: ReflectionPreviously: Needed to use Java reflection,no runtime info available on Scala s types.Now you can do: 36
  • 35. (Bare-Bones) Reflection in Java Why not add some meaningful operations? Need to write essential parts of a compiler (hard). Need to ensure that both compilers agree (almost impossible).Want to know whether type A conforms to B?Write your own Java compiler!   37
  • 36. How to do Better?•  Problem is managing dependencies between compiler and reflection.•  Time to look at DI again. Dependency Injection•  Idea: Avoid hard dependencies to specific classes.•  Instead of calling specific classes with new, have someone else do the wiring. 38
  • 37. Using Guice for Dependency Injection (Example by Jan Kriesten) 39
  • 38. ... plus some Boilerplate 40
  • 39. Dependency Injection in Scala Components are classes or traits Requirements are abstract values Wiring by implementing requirement values But what about cyclic dependencies? 41
  • 40. The Cake PatternComponents are traits Wiring by mixin composition Requirements are types of this   42
  • 41. Cake Pattern in the CompilerThe Scala compiler uses the cake pattern for everythingHere s a schema:(In reality there are about ~20 slices in the cake.) 43
  • 42. Towards Better ReflectionCan we unify the core parts of the compiler and reflection? Compiler ReflectionDifferent requirements: Error diagnostics, file access, classpath handling - but we are close! 44
  • 43. Compiler ArchitectureProblem: This exposes way too much detail! reflect.internal.Universe   nsc.Global  (scalac) reflect.runtime.Mirror   45
  • 44. Complete Reflection Architecture reflect.api.Universe  /   Cleaned-up facade: reflect.mirror   reflect.internal.Universe  Full implementation: nsc.Global  (scalac) reflect.runtime.Mirror   46
  • 45. How to Make a Facade The Facade Interfaces are not enough! The Implementation 47
  • 46. ConclusionScala is a very regular language when it comes to composition:1.  Everything can be nested: –  classes, methods, objects, types2.  Everything can be abstract: –  methods, values, types3.  The type of this can be declared freely, can thus express dependencies4.  This gives great flexibility for SW architecture, allows us to attack previously unsolvable problems. 48
  • 47. Going further: Parallel DSLsMid term, research project: How do we keep tomorrow s computers loaded? –  How to find and deal with 10000+ threads in an application? –  Parallel collections and actors are necessary but not sufficient for this.Our bet for the mid term future: parallel embedded DSLs. –  Find parallelism in domains: physics simulation, machine learning, statistics, ...Joint work with Kunle Olukuton, Pat Hanrahan @ Stanford.EPFL side funded by ERC. 49
  • 48. EPFL / Stanford Research Scientific Virtual Personal Data Applications Engineering Worlds Robotics informatics Domain Physics Probabilistic Machine Specific Rendering Scripting Learning Languages (Liszt) (RandomT) (OptiML) Domain Embedding Language (Scala) Polymorphic Embedding Staging Static Domain Specific Opt. DSLInfrastructure Parallel Runtime (Delite, Sequoia, GRAMPS) Dynamic Domain Spec. Opt. Task & Data Parallelism Locality Aware Scheduling Hardware ArchitectureHeterogeneous OOO Cores SIMD Cores Threaded Cores Specialized Cores Hardware Programmable Scalable Isolation & On-chip Pervasive Hierarchies Coherence Atomicity Networks Monitoring 50
  • 49. Example: Liszt - A DSL for Physics Simulation Combustion Turbulence Fuel injection Transition Thermal•  Mesh-based•  Numeric Simulation•  Huge domains Turbulence –  millions of cells•  Example: Unstructured Reynolds-averaged Navier Stokes (RANS) solver 51
  • 50. Liszt as Virtualized Scalaval // calculating scalar convection (Liszt)val Flux = new Field[Cell,Float] ASTval Phi = new Field[Cell,Float]val cell_volume = new Field[Cell,Float]val deltat = .001...untilconverged { for(f <- interior_faces) { val flux = calc_flux(f) Flux(inside(f)) -= flux Flux(outside(f)) += flux } for(f <- inlet_faces) { Optimisers Generators Flux(outside(f)) += calc_boundary_flux(f) } … for(c <- cells(mesh)) { Phi(c) += deltat * Flux(c) /cell_volume Schedulers (c) } … for(f <- faces(mesh)) Flux(f) = 0.f Hardware} DSL Library GPU, Multi-Core, etc 52
  • 51. Follow us on twitter: @typesafe 53