Scala 2013 review

2013 overview
Sagie Davidovich
singularityworld.com
linkedin.com/in/sagied

2013 JVM Languages Landscape
Static

Object Oriented
Functional

Dynamic

Performance study by
Google

Exponential growth in
demand

Thoughtworks
Technology Radar 2012

Evaluate

Consider

Hold

Language Design
Trade-‐‑oﬀs
Expressiveness
Performance

Complexity

Positioning

Object
Oriented
JVM

Statically
Typed
Functional

Prof. Martin Odersky
•  Designer of Java Generics
•  Creator of JavaC
•  Prof. at EPFL
•  ACM Fellow
•  Founder of

Class Employee -‐‑ Java
public class Employee implements Serializable { public boolean equals(Object o) {
private final String firstName; if (this == o) {
private final String lastName; return true;
}
public Employee (String firstName, String lastName) { if (o == null || getClass() != o.getClass()) {
this.firstName = firstName; return false;
this.lastName = lastName; }
} Employee employee = (Employee) o;
if (firstName != null ? !firstName.equals(employee.firstName) :
public String getFirstName() { person.firstName != null) {
return lastName; return false;
} }
if (lastName != null ? !lastName.equals(employee.lastName) :
employee.lastName != null) {
public String getLastName() {
return true;
return firstName;

} }
return true;
Oops, should be false
public Employee withFirstName(String firstName) {

} Anyone noticed?
return new Employee (firstName, lastName);
} public int hashCode() {
int result = firstName != null ? firstName.hashCode() : 0;

result = 31 * result + (lastName != null ?
public Employee withLastName(String lastName) {
lastName.hashCode() : 0);
return new Employee (firstName, lastName);
return result;
}
}

public String toString() {
return "Employee(" + firstName + "," + lastName + ")";
}
}

Class Employee -‐‑ Scala

case class Employee(firstName: String, lastName: String)!

•  Constructor
•  Copy constructors
•  Fields
•  GeVers / (seVers)
•  equals
•  hashCode
•  toString
•  Recursive decomposition
•  Companion object

Singletons
Java
public class OneAndOnly {
private final static OneAndOnly INSTANCE = new OneAndOnly();

private OneAndOnly() {}

public static OneAndOnly getInstance() { return OneAndOnly.INSTANCE; }
}

Scala
object OneAndOnly

Everything is an object

1.+(2)
Same as

1+2

Every block returns a value
(no statements, only expressions)
def add (x:Int, y:Int) = x + {val tmp = y; tmp * 3}!
!
if (x > 2) 3 else 4!
!
val doubled = for(i <- (1 to 10)) yield i * 2!

Type inference
scala> val x = 3
x: Int = 3

scala> def x(y:Int) = y * 9
x: (y: Int)Int

scala> def x(y:Int) =
if (y == 1) List(y) else Set(y)
x: (y: Int) Iterable[Int]

Higher order functions
List(1, 2, 3).map((x: Int) => x + 1)

=
List(1, 2, 3).map(x => x + 1) // type is inferred

=
List(1, 2, 3).map(_ + 1) // placeholder notation

Universal access principle
import System._!
!
class Clz {!
def w = currentTimeMillis / 1000!
val x = currentTimeMillis / 1000!
var y = currentTimeMillis / 1000!
lazy val z = currentTimeMillis / 1000!
}!

Default methods
apply and unapply
object Square{
def apply(d: Double) = d * d
def unapply(d: Double) = Some(math.sqrt(d))
}

//apply
val s = Square(3) // 9

// unaply
16 match { case Square(x) => x } // 4

Curried functions
•  Add new flow control and language constructs!
def loop(n: Int)(body: => Any) {!
(0 until n) foreach (n => body)!
}!
!

loop(2) {!
println("IM IN YR LOOP!")!
}!
•  Multiple varargs lists are possible!!
def crossProduct[L,R](left: L*)(right: R*) = 
for (l <- left; r <- right) yield (l,r)!
crossProduct(1,2,3)(‘a’,’b’,’c’)!

•  Partially applied functions!
!

Multiple inheritance without
the diamond problem

Type system

Strong
Explicit + Inferred
Traits
Self types
Static + Dynamic
Functional
Erasure
Nominal + Structural
Type aliases
Nonnullable
Monads
Implicit
Co/Contra-‐‑variant
Existential
Value + Reference
Case classes
Path dependent
Anonymous

Type safety (Java example)
// compiles fine in Java (arrays are co-variant!!)

int[] ints = {3};
Object[] objects = ints;

// ArrayStoreException in runtime

objects[0] = new Object();

Structural types
object Closer {
def using(closeable: { def close(): Unit }, f: => Unit) {
try {
f
} finally { closeable.close }
}
}

Simpler is safer
Indexing by ﬁrst character
Java
List<String> keywords = Arrays.asList("Apple", "Banana", "Beer");
Map<Character, List<String>> result = new HashMap<Character, List<String>>();
for(String k : keywords) {
char firstChar = k.charAt(1); Oops, here’s a bug.
if(!result.containsKey(firstChar)) {
Anyone noticed?
result.put(firstChar, new ArrayList<String>());
}
result.get(firstChar).add(k);
}
for (List<String> list : result.values()) {
Collections.sort(list);
}

Scala
val keywords = List("Apple", "Banana", "Beer”)
val result = keywords.sorted.groupBy(_.head)

Another example
!
Java!
public List<String> empNames(ArrayList<Employee> employees) {!
!ArrayList<String> result = new ArrayList<String>();!
!for (Employee emp: employees) {!
! !result.add(emp.getName());!
!}!
!return result;!
}!
!
!
!
Scala!
def empNames(employees: List[Employee]) = employees map getName!

Map combinator paVern
!
Java!
public List<String> empNames(ArrayList<Employee> employees) {!
!ArrayList<String> res = new ArrayList<String>();!
!for (Employee emp: employees) {!
! !res.add(emp.getName());!
!}!
!return res;!
}!
!
!
! Really need to encapsulate??
Scala!
def empNames(employees: List[Employee]) = employees map getName!

Another example
!
Java!
public static int factorial(int n) {!
int res = 1;!
for (int i = 1; i <= n; i++) {!
res *= i;!
}!
return res;!
}!
!
!

Scala!
def factorial(n: Int) = (1 to n).reduce(_*_)!

Reduce combinator paVern
!
Java!
public static int factorial(int n) {!
int res = 1;!
for (int i = 1; i <= n; i++) {!
res *= i;!
}!
return res;!
}!
!
!

Scala!
def factorial(n: Int) = (1 to n).reduce(_ * _)!

Combinatorics
o  (1 to 5) combinations 2

List(Vector(1, 2), Vector(1, 3), Vector(1, 4), Vector(1, 5), Vector(2,
3), Vector(2, 4), Vector(2, 5), Vector(3, 4), Vector(3, 5), Vector(4,
5))

o  (1 to 5).permutations

"George W. Bush".split(" ").permutations

List(Array(George, W., Bush), Array(George, Bush, W.), Array(W.,
George, Bush), Array(W., Bush, George), Array(Bush, George,
W.), Array(Bush, W., George))

o  "George W. Bush".split(" ").toSet.subsets

List(Set(), Set(George), Set(W.), Set(Bush), Set(George, W.),
Set(George, Bush), Set(W., Bush), Set(George, W., Bush))

++
Collection framework
++: +: /: /: !
:+ :: ::: : addString !
aggregate andThen apply applyOrElse asInstanceOf !
canEqual collect collectFirst combinations companion !
compose contains containsSlice copyToArray copyToBuffer !
corresponds count diff distinct drop !
dropRight dropWhile endsWith exists filter !
filterNot find flatMap flatten fold !
foldLeft foldRight forall foreach genericBuilder !
groupBy grouped hasDefiniteSize head headOption !
indexOf indexOfSlice indexWhere indices init !
inits intersect isDefinedAt isEmpty isInstanceOf !
isTraversableAgain iterator last lastIndexOf lastIndexOfSlice !
lastIndexWhere lastOption length lengthCompare lift !
map mapConserve max maxBy min !
minBy mkString nonEmpty orElse padTo !
par partition patch permutations prefixLength !
product productArity productElement productIterator productPrefix !
reduce reduceLeft reduceLeftOption reduceOption reduceRight !
reduceRightOption repr reverse reverseIterator reverseMap !
reverse_::: runWith sameElements scan scanLeft !
scanRight segmentLength seq size slice !
sliding sortBy sortWith sorted span !
splitAt startsWith stringPrefix sum tail !
tails take takeRight takeWhile to !
toArray toBuffer toIndexedSeq toIterable toIterator !
toList toMap toSeq toSet toStream !
toString toTraversable toVector transpose union !
unzip unzip3 updated view withFilter !
zip zipAll zipWithIndex !

Placeholder syntax for
anonymous functions
Placeholder
Regular
_ + 1 x => x + 1
_ * _ (x1, x2) => x1 * x2
(_: Int) * 2 (x: Int) => x * 2
if (_) x else y z => if (z) x else y
_.map(f) x => x.map(f)
_.map(_ + 1) x => x.map(y => y + 1)

Distributed & multicore
computing
Core

•  Parallel collections (myList.par.sum)
•  Futures & promises
•  Actors
•  STM – makes Java heap ACID compatible
3rd Party

•  Hadoop
•  Spark & Storm
•  Plain old Java threads
•  Scalding – Twitter’s Scala based MapReduce
implementation

Calling Hadoop from
Scala
Java
public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable>
output, Reporter reporter) throws IOException {
String line = value.toString();
StringTokenizer tokenizer = new StringTokenizer(line);
while (tokenizer.hasMoreTokens()) {
word.set(tokenizer.nextToken());
output.collect(word, one);
}
}

Scala
def map(key: LongWritable, value: Text, output: OutputCollector[Text,
IntWritable], reporter: Reporter) =
value split " " foreach (output collect (_, one))

Futures

val from1 = future {conn1.fetch}
val from2 = future {conn2.fetch}

from1 either from2 onSuccess println

Parallel collections
List(1, 2, 3, 4, 5).par filter (_ % 2 == 0)
// ParVector(2, 4)

Spark
val file = spark textFile "hdfs://...” 
!
file.flatMap(_ split " ") 
.map(word => (word, 1)) 
.reduceByKey(_ + _)!
!

Tail recursion optimization
Recursion in Java has an Achilles’ heel:
the call stack

def factorial(n: Int, res: Long = 1): Long =
if(n == 0) res
else factorial(n - 1, (res * n))

Safer string composition
•  Standard string composition:
“I’m “ + name + “, “ + age + “ years old”

•  Formatted:
“I’m %s, %d years old”.format(name, age)
java.util.IllegalFormatConversionException

•  Interpolated:
s“I’m $name, $age years old”

Create your own
interpolations
xml”””<body>
<a href = “http://…”> $text</a>
</body> ”””

Expression simpliﬁcation – Java
public Expr simplify(Expr expr) {
if (expr instanceof UnOp) {
UnOp unOp = (UnOp) expr;
if (unOp.getOperator().equals("-")) {
if (unOp.getArg() instanceof UnOp) {
UnOp arg = (UnOp) unOp.getArg();
if (arg.getOperator().equals("-"))
return arg.getArg();
}
}
}
if (expr instanceof BinOp) {
BinOp binOp = (BinOp) expr;
if (binOp.getRight() instanceof Number) {
Number arg = (Number) binOp.getRight();
if (binOp.getOperator().equals("+") && arg.getNum() == 0)
return binOp.getLeft();
}
}
return expr;
}

PaVern matching
Expression simpliﬁcation -‐‑ Scala

def simplify(expr: Expr): Expr = expr match {!
case UnOp("-", UnOp("-", e)) => simplify(e)!
case BinOp("+", e, Number(0)) => simplify(e)!
case _ => expr!
}!

Value classes
Extensibility minus boxing overhead!
!
class Meter(val v: Double) extends AnyVal {!
def plus (that: Meter) = new Meter(v + that.v) !
}!
!
Compiler generates!

object Meter {!
def plus(this: Meter, that: Meter) = new Meter(v + that.v)!
}!

Compile-‐‑time
Meta-‐‑programming
•  Language virtualization (overloading/overriding
semantics of the original programming language to
enable deep embedding of DSLs),

•  Program reification (providing programs with means to
inspect their own code),

•  Self-optimization (self-application of domain-specific
optimizations based on program reification),

•  Algorithmic program construction (generation of
code that is tedious to write with the abstractions
supported by a programming language).

Meta programming
Language
Operate on
Type Expressive Runtime
safe
ness
Performance
overhead
Textual C, Scala
Text
✗
Low
No
preprocessors
Template C++
Text
✗
Low
No
systems
Compile-‐‑time Haskell, Abstract (typed) ✔
High
No
meta-‐‑ Scala 2.10, Syntax trees

programming
Nemerle
Byte code Java, Scala
byte-‐‑code
✗
Medium
No
manipulation
“Dynamic” Ruby, Scala
objects
✗
High
Yes
reﬂection
Run-‐‑time Java, Scala
objects
✗
High
Yes
reﬂection

Scala macros
•  Full blown compile time meta-
programming
•  Access to the compiler API
•  Written in Scala
•  Hygienic
•  Type-safe

Macros – safe printf
printf(format: String, params: Any*)!
printf(“value is %d”, “boom”)!
// runtime exception!
java.util.IllegalFormatConversionException: d != java.lang.String!
!
!

Macro implementation!
def printf(format: String, params: Any*): Unit = macro printf_impl!
def printf_impl(c: Context)(format: c.Expr[String], params:
c.Expr[Any]*): c.Expr[Unit] = ...!

Eﬃcient Assert

assert(2 + 2 == 4, "weird arithmetic")

// 2 + 2 == 4 will be evaluated only if
assertions were enabled

Macros – units of measure
val gravityOnEarth = u(9.81, "m/s^2")!
!
val heightOfMyOfficeWindow = u(3.5, "m")!
!
val speedOfImpact = 
sqrt(2.0 * gravityOnEarth * heightOfMyOfficeWindow)!
!
val monthsInAYear = 10b12!

hVp://scalamacros.org/usecases/units-‐‑of-‐‑measure.html

Macros – type providers

type MySqlDb(connString: String) = macro ...!
type MyDb = Base with MySqlDb("Server=127.0.0.1;Database=Foo;”)!
val products = new MyDb().products!
products.filter(p => p.name.startsWith("foo"))!
!

!
!
!
!

http://scalamacros.org/usecases/type-providers.html!
Inspired by F# type providers!

Type macros
Injecting async methods
class D extends Lifter {!
!def x = 2!
!// def asyncX = future { 2 }!
}!
!
val d = new D!
d.asyncX onComplete {!
!case Success(x) => println(x)!
!case Failure(_) => println("failed")!
}!

hVp://scalamacros.org/talks/2012-‐‑12-‐‑18-‐‑MacroParadise.pdf

More uses of macros
•  Ad-hoc code style and code-smell detector
•  Advanced domain-specific languages
•  Logging with minimal overhead
•  Pre-compiled SQL queries
•  GPU optimized code (see Scalaxy, ScalaCL)
•  Macro annotations
o  @Cached

Macro annotations
class atomic extends MacroAnnotation {
def complete(defn: _) = macro(“backing field”)
def typeCheck(defn: _) = macro("return defn itself”)
}

@atomic var fld: Int

GPU compilation
•  Enablers: Immutability, Macros
•  ScalaCL Collections
OpenCL-backed collections that look and behave
like standard Scala collections (work in progress).

•  ScalaCL Compiler Plugin
optimizes Scala programs at compile-time,
transforming regular Scala loops into faster code
and transforming Scala functions given to ScalaCL
Collections into OpenCL kernels.

An extremely hack-‐‑able compiler
> scala -Xshow-phases!
phase name id description!
---------- -- -----------!
parser 1 parse source into ASTs, perform simple desugaring!
namer 2 resolve names, attach symbols to named trees!
packageobjects 3 load package objects!
typer 4 the meat and potatoes: type the trees!
patmat 5 translate match expressions!
superaccessors 6 add super accessors in traits and nested classes!
extmethods 7 add extension methods for inline classes!
pickler 8 serialize symbol tables!
refchecks 9 reference/override checking, translate nested objects!
selectiveanf 10 ANF pre-transform for @cps!
selectivecps 11 @cps-driven transform of selectiveanf assignments!
uncurry 12 uncurry, translate function values to anonymous classes!
tailcalls 13 replace tail calls by jumps!
specialize 14 @specialized-driven class and method specialization!
explicitouter 15 this refs to outer pointers, translate patterns!
erasure 16 erase types, add interfaces for traits!
posterasure 17 clean up erased inline classes!
lazyvals 18 allocate bitmaps, translate lazy vals into lazified defs!
lambdalift 19 move nested functions to top level!
constructors 20 move field definitions into constructors!
flatten 21 eliminate inner classes!
mixin 22 mixin composition!
cleanup 23 platform-specific cleanups, generate reflective calls!
icode 24 generate portable intermediate code!
inliner 25 optimization: do inlining!
inlinehandlers 26 optimization: inline exception handlers!
closelim 27 optimization: eliminate uncalled closures!
dce 28 optimization: eliminate dead code!
jvm 29 generate JVM bytecode!
terminal 30 The last phase in the compiler chain!

Type-‐‑safe failures
try { 2 / x } catch { case e:Throwable => 0}

in 2.10:
Try (2 / 0) // returns Failure
Try (2 / 1) // returns Success

Try (2 / x) getOrElse (0)
List(0,1,0,2).map (x=> Try(2 / x)) filter(_.isSuccess)
// List(Success(2), Success(1))

Implicit parameters
implicit val db = getDB!
def store(employee: Employee)(implicit db: DB)!
!
store(emp1oyee1) //db is passed implicitly!

Implicit methods
implicit def toEuro(x: Currency): Euro = …!
!
def transfer(amount: Euro) {!
println(”transferred” + amount)!
}!
!
// dollar is automatically converter to euro!
transfer(Dollar(3.0))!
transferred 2.25 Euros!

Implicit classes
!
implicit class StringExtensions(s: String) {!
def words = s split " "!
}!
!
!
“hello world”.words // Array(hello, world)!

Environments
-‐‑ Currently the most mature

-‐‑ Formally supported by TypeSafe

ublime
-‐‑ Through Ensime

Scala REPL

DSLs (accounting)
Rules to calculate an employee's paycheck:!
employee's gross salary for 2 weeks!
minus deductions for!
federalIncomeTax, which is 25% of gross!
stateIncomeTax, which is 5% of gross!
insurancePremiums, which are 500 in gross’s currency!
retirementFundContributions are 10% of gross!
!

hVp://ofps.oreilly.com/titles/9780596155957/DomainSpeciﬁcLanguages.html

DSLs (JSON)
("person" ->!
("name" -> "Joe") ~!
("age" -> 35) ~!
("spouse" ->!
("person" ->!
("name" -> "Marilyn") ~!
("age" -> 33)!
)!
)!
)!

DSL (XPath / json)

val titles = xml "channel" "item" "title“

val titles2 = json "feed" "entry" "title" @ "$t"

github.com/razie/snakked

DSLs
testing frameworks

val emptyStack = new Stack[String]
evaluating { emptyStack.pop() } should
produce [NoSuchElementException]

Full access to the compiler
import tools.reflect.ToolBox!
import reflect.runtime.{currentMirror => cm}!
!
val tb = cm.mkToolBox()!
!
scala> val tree = tb parse "(5 + 9) * 3"!
tree: tb.u.Tree = 5.$plus(9).$times(3)!
!
scala> tb eval tree!
res1: Any = 42!

Reﬂection

val take = typeOf[List[_]].member(TermName("take")).asMethod!
!
reflect(List(1,3,2,4)).reflectMethod(take)(2) // List(1,2)!

Testing
•  ScalaCheck
o  Auto-generation of inputs and mocks.
o  Composable check units
o  Inspired by Haskell QuickCheck

•  ScalaTest
o  TDD, BDD, FeatureSpec
o  Selenium DSL

•  Specs2
o  TDD, BDD, Datatables support
o  Integration with Mockito, EasyMock and JMock
o  "hello world" must be matching("h.* w.*")

•  JUnit

ScalaCheck
(inspired by Haskell QuickCheck)

scala> val sqrt = forAll {(n: Int) => math.sqrt(n*n) == n }

scala> propSqrt.check
! Falsified after 1 passed tests:
> -1

BDD with ScalaTest
describe("A Stack") {
it("should throw NoSuchElementException if an empty stack is popped") in {
val emptyStack = new Stack[String]
evaluating { emptyStack.pop() } should produce
[NoSuchElementException]
}
it("should pop values in last-in-first-out order") { … }
}

How to learn

•  twitter.github.com/scala_school
•  Effective Scala
•  Coursera Scala course (50K students last year)
•  Simply scala (interactive learning)
•  Programming in scala book
•  Stackoverflow

Contributing

docs.scala-‐‑lang.org/sips

Contributing
github.com/scala

Things to be aware of
✗  Scala Compiler has to do much more => slower

✗  Tooling is not perfect
(debugging, synthetic frames, macros support)
Getting better every day.

✗  Language is rapidly evolving.
Deprecations should be expected.

Scala 2013 review

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (8)

Similar to Scala 2013 review

Similar to Scala 2013 review (20)

Recently uploaded

Recently uploaded (20)

Scala 2013 review