Scala @ TomTom

Scala @ TomTom
Eric Bowman
eric.bowman@tomtom.com
26 May 2010

What’s He Building In There?
Not just PNDs
Traffic Services
Location Services
Address & Local Search
Tons of Data -> Tons of Analysis
Deep Commitment to Server-Side Java
2

5
Company Confidential
21,345 pages
40 reams of paper
5 laptops
8 keyboards
1 back operation
30,000 strands of hair

Tutorial on Good Lisp Programming Style
8
“Most algorithms can be characterized as:
Searching (some find find-if mismatch)
Sorting (sort merge remove-duplicates)
Filtering (remove remove-if mapcan)
Mapping (map mapcarmapc)
Combining (reduce mapcan)
Counting (count count-if)
These functions abstract common control patterns. Code that uses them is:
Concise
Self-documenting
Easy to understand
Often reusable
Usually efficient”
From Peter Norvig’sTutorial on Good Lisp Programming Style, 1993
(http://www.norvig.com/luv-slides.ps)

But...
“Lisp is good for:
Exploratory programming
Rapid prototyping
Minimizing time-to-market
Single-programmer (or single-digit team) projects
Source-to-source or data-to-data transformation” (ibid.)
9

I Like
Static types
Refactoring Tools
Lazy Sundays
Paying Some Kid To Cut The Grass
Spreadsheets
Not Flying The Plane
10

What is this Scala thing?
11

Scala Is...
An escaped research language from EPFL targeting JVM and CLR
Object-oriented and functional
Statically typed
Lisp, ML, Haskell, Erlang, etc.
DSL-friendly
12

Bi-Scalar Disorder
13

Functional Programming?
Functions as first class objects
Immutability
Closures
Binding free variables to enclosing lexical scope
Higher-order Functions
functions as input and/or output
A different set of idioms...
14

Keep Back
for (int i = 1; i <= 256; i++) {
if (array[i-1] == ...
15

LOLCrash
16

Idioms
Reduce cognitive overhead
Reduce bugs
Make intention clear
Mini-Patterns
17

Java Idioms
++C++
No more 0xDEADBEEF
Leads to lots of loops and copies, if you’re doing it right
Hard programs get complex doing common things
Nested loops begin to look Harmful...
18

For Comprehensions
for(inti=0; i<100; i++) { ... }
for (i <- 0 until 100) {
... /* do something with i */ }
(0.until(100)).foreach(i =>
/* something with i */)
20

For Comprehensions
Lists algorithmic “sweet spot”
Syntactic Sugar for:
foreach
map
filter
flatMap
21

The Easy Ones
for (i <- 1 to 6) yield i * 2
(1 to 6).map(_ * 2)
(2,4,6,8,10,12)
for (i <- 1 to 6 if i % 2 == 0) yield i
(1 to 6).filter(_ % 2 == 0)
(2,4,6)
for (i <- 1 to 6) { println(i + “ “) }
(1 to 6).foreach { i => print(i + “ “) }
1 2 3 4 5 6
22

A Harder One...
List<Integer> array = new ArrayList<Integer>();
for (i = 1; i <= 3; i++) {
for (j = i; j <= 3; j++) {
array.add(j);
}
}
System.out.println(array);
[1, 2, 3, 2, 3, 3]
for (i <- 1 to 3; j <- i to 3) yield j
(1, 2, 3, 2, 3, 3)
(1 to 3).flatMap(i => (i to 3).map(j => j))
23

flatMap
Subtle thing...
“Applies the given function f to each element, then concatenates the results”
Turns a list of lists into a list
List(List(1,2,3), List(4,5,6)).flatMap(x => x)
List(1, 2, 3, 4, 5, 6)
List(“tom”, “tom”).flatMap(_.capitalize).mkString
TomTom
“Special sauce” for nested looping constructs
(Equivalent to Haskell’s monadic “bind”)
24

IteratorIterator
package org.hyperic.sigar.util;
public static class IteratorIteratorimplements java.util.Iterator {
private java.util.ArrayListiterators;
public IteratorIterator() {}
public void add(java.util.Iteratoriterator) {}
public booleanhasNext() {}
public java.lang.Object next() {}
public void remove() {}
}
25

flatMapflatMap
def foo(arg: Iterable[Int]) {
... Do something with arg
}
val some: Iterable[Int] = getSomeIterable
foo(some)
val more: Iterable[Int] = getMore
foo(List(some, more).flatMap(x => x))
26

Real-Life Example
“mntstamarg”
Each term has aliases
Need all permutations
27

Java Version
static List<String> aliasReplace(String place) {
String[] bits = nospaces.split(place);
List<String> result = new ArrayList<String>();
List<StringBuilder> allAliases = new ArrayList<StringBuilder>();
for (String bit : bits) {
String[] ales = aliases.get(bit);
if (ales != null) {
if (allAliases.size() == 0) {
allAliases.add(new StringBuilder(bit));
for (String alias : ales) {
allAliases.add(new StringBuilder(alias));
}
}
else {
List<StringBuilder> clones = new
ArrayList<StringBuilder>();
for (StringBuilder a : allAliases) {
clones.add(new StringBuilder(a).append(" ").append(
bit));
for (String alias : ales) {
clones.add(new StringBuilder(a).append(" ").append(
alias));
}
}
allAliases = clones;
}
}
else {
if (allAliases.size() == 0) {
allAliases.add(new StringBuilder(bit));
}
else {
for (StringBuilder b : allAliases) {
b.append(" ").append(bit);
}
}
}
}
for (StringBuildermunge: allAliases) {
result.add(munge.toString());
}
return result;
}
28

Scala Version
def alias(query: String, aliases: String => List[String]): List[String] = {
def recurse(prefix: List[String], remainder: List[String]): List[List[String]] = {
remainder match {
case Nil => prefix :: Nil
case head :: tail => aliases(head).flatMap(term => recurse(term :: prefix, tail))
}
recurse(Nil,query.split(“”).toList.reverse).map(
_.mkString(“ “))
}
30

Yeah yeahyeah but
“Perl Whitespace Law”
“Each line of perl should be surrounded by whitespace equivalent to what it would take to achieve the same functionality in a normal programming language.” -- Don Hopkins
If it compiles, it nearly works. Really.
Visual Plane-Oriented Programming
I ♥ Idioms
“But in Java, each little part is so very simple...”
32

Discovering Fire
34

“Weak developers will move heaven and earth to do the wrong thing. You can’t limit the damage they do by locking up the sharp tools. They’ll just swing the blunt tools harder.” – Glenn Vandenburg
35

@TomTom
Testing
Middleware “Smart Content Switch”
We needed it quickly...
B2B/B2G Traffic Query Engine
Clustering Algorithm
DSL Templating Engine
Content Cleanup
Next-Generation Geocoder
36

How To Sneak It In
Start with Testing
ScalaCheckIS AWESOME.
Specs, ScalaTest testing DSLs
Start with something low risk
Ok, well, we didn’t do that
Prepare for steep learning curve
...followed by a productivity hockey stick
37

Another Example
8000 lines of broken Java -> 400 lines of broken Scala ->
hyp.take(1).flatMap(_.dropDistricts) match {
case Nil => hyp
case head => {
hyp.tail.foldLeft(head) {
case (run: List[Hypothesis], h: Hypothesis) => {
run.flatMap(_.merge(h)) match {
case Nil => run
case newRun => newRun.removeDuplicates
}
}
}
}
}
*Includes suggested improvements by Martin Odersky, I hope
38

Testing
Functional architect wrote a bunch of test cases, like:
Requirement R.17:
D1 -> N1, L1 -> N1, N1, D2 -> L2, L3, N3, D3 should cluster as (L1,N1,D1), (L2,D2), (L3), (N3), (D3)
Vim-macro’d into:
it should “satisfy R.17” in {
cluster(D1 -> N1,L1 -> N1,N1,D2 -> L2,L3,N3,D3) should equal {
groups(group(L1,N1,D1),group(L2,D2),group(L3),group(N3),
group(D3)))
}
39

ScalaCheck
The Dog’s Bollocks
Steep Learning Curve
object BinarySpecification extends Properties(“bin”) {
specify(“toBinary”,
Prop.forAllNoShrink(
for{n <- Gen.choose(0d, 0.99999999d)
m <- Gen.choose(20,31) } yield Pair(n, m)) {
p =>
Math.abs(toDecimal(toBinary(p._1, p._2)) – p._1 < 1e-10
})}
40

So little time...
Traits
Case classes
Pattern matching
Structural types
Self types
XML
Option
Covariance/Contravariance
@specialized
41

This slide left intentionally blank.

http://www.slideshare.net	65
http://www.linkedin.com	4
http://www.lmodules.com	3

Scala @ TomTom

by Eric Bowman

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 72

Statistics

Scala @ TomTom Presentation Transcript

Scala @ TomTom

by Eric Bowman

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 72

Statistics

Scala @ TomTom Presentation Transcript

Let LinkedIn power your SlideShare experience

Let LinkedIn power your SlideShare experience

Customize SlideShare content based on your interests

Keep up to date when your LinkedIn contacts post on SlideShare