talk to the Scala User group in Berlin

1. Scala in Nokia Places API
Lukasz Balamut - Places API Team
17 April 2013

2. Purpose of this talk
Why are we spending time migrating to a new language?
How are we doing that?
What have we learned?
and of course: we are hiring.

3. Our scala story.
Our migration in lines of code
We’ve discovered more and more things on the way.

4. The Concordion tests problem
In June 2011 we had problems with Concordion (HTML based acceptance test
framework),
hard to integrate
HTML tests were not readable
Solution:
Multi-line Strings in Scala
Continue using JUnit
val expectedBody = """
{
"id":"250u09wh-83d4c5479b7c4db9836602936dbc8cb8",
"title" : "Marfil (Le)",
"alternativeNames" : [
{
"name" : "Le Marfil",
"language" : "fr"
}
]
}
"""

5. The test names problem
JUnit + camel case - not that readable for long names
@Test
def preservesSearchResponseItemOrder() { /* ... */ }
JUnit + underscores - violates naming convention, but better to read
@Test
def preserves_search_response_item_order() { /* ... */ }
ScalaTest
test("preserves search response’s item order") { /* ... */ }
It is supported by tools we use and produces nice output when run (as a bonus)
[info] DiscoverAroundAcceptanceTest:
[info] - calls nsp with 15Km radius
[info] - preserves search response’s item order
[info] - when requesting zero results, no request to search is made

6. Lift-Json
JSON (de)serialisation can be done in really nice way
e.g. getting place id:
{
"place": {
"a_id": "276u33db-6f084959f97c45bc9db26348bafd4563"
}
}
Also there is a generic way of parsing json, that gives you XPath like access:
val id = (json "place" "a_id").extract[String]
Lift-json provides also easy and efficient case class JSON (de)serialisation,
hance case classes

7. Scala case classes in the model
We model our upstream and downstream datamodels as case classes.
case class Place (
name: String,
placeId: PlaceID
//(...)
)
Each case class is a proper immutable data type - the new Java Bean ;),
A lot of boilerplate code is generated by the compiler e.g.:
accessors,
equals/hashCode,
copy methods,
toString
apply/unapply (for pattern maching),
After decompiling the scala code above we will end up with 191 lines of Java:
package pbapi.model.api;
import scala.*;
import scala.collection.Iterator;
import scala.runtime.BoxesRunTime;
import scala.runtime.ScalaRunTime$;

8. Scala case classes - Configuration
Easy to update configuration
Easy to compare environments’ configuration
case class HttpClientConfig(
serviceName: String,
serviceBaseUrl: URL,
readTimeout: Int = 0,
connectionTimeout: Int,
proxy: Option[ProxyConfig] = None,
oauthKeys: Option[OAuthConfig] = None,
displayUrl: Option[String] = None
)
"httpConfig":{
"serviceName":"recommendations",
"serviceBaseUrl":"http://nose.svc.ovi.com/rest/v1/recommendations/near
"readTimeout":4000,
"connectionTimeout":500,
"displayUrl":"http://nose.svc.ovi.com/rest/v1/recommendations/nearby/"
}

9. The Scala Type System
A great way to express assumptions
Have them checked by the compiler through the whole codebase
At every point in the code know what to expect or you will be reminded by
compiler.

10. The Scala Type System - Options 1
Avoid null references (see The Billion Dollar Mistake )
This will not compile!
case class UpstreamResponse ( unreliableField: Option[String] )
case class MyResponse ( reliableField: String )
def transform(upstream: UpstreamResponse) =
MyResponse(
reliableField = upstream.unreliableField //<-- incompatible type
)

11. The Scala Type System - Options 2
But this will:
case class UpstreamResponse ( unreliableField: Option[String] )
case class MyResponse ( reliableField: String )
def transform(upstream: UpstreamResponse) =
MyResponse(
reliableField = upstream.unreliableField.getOrElse("n/a") // enf
)

12. The Scala Type System - Options 3
how we were learning e.g.: to transform string when it is defined
def transform(str: String): String
reliableField =
if (upstream.unreliableField.isDefined)
transform(upstream.unreliableField.get)
else "n/a"
reliableField =
upstream.unreliableField match {
case Some(f) => transform(f)
case None => "n/a"
}
reliableField =
upstream.unreliableField.map(transform) | "n/a"

13. The Scala Type System - Standard Types
Immutable Map, List, Sets
val map: Map[String, String] = Map("a" -> "a", "b" -> "b")
val list: List[String] = "a" :: "b" :: Nil
Tuples to express Pairs, Triplets etc.
val pair: Pair[String, Int] = ("a", 1)
val (a, b) = pair // defines a:String and b:Int

14. The Scala Type System - error handling
Types can help with exceptional cases
val parsed: Validation[NumberFormatException, Int] = someString.parseInt
val optional: Option[Int] = parsed.toOption
val withDefault: Int = optional.getOrElse(0) //if there was parsing prob
Exception to Option
val sub: Option[String] =
allCatch.opt(line.substring(line.indexOf("{"")))
or Either
val parsed: Either[Throwable, Incident] =
allCatch.either(new Incident(parse(jsonString))
and handle it later
parsed match {
case Right(j) => Some(j)
case Left(t) => {
println("Parse error %s".format(t.getMessage))
None

15. The Scala Type System - functions
we defer translation (Translated) and text rendering (FormattedText) to
serialisation time using function composition so we don’t need to pass user
context through the whole stack e.g.:
case class Place (
//(...)
attribution: Option[Translated[FormattedText]],
//(...)
)
case class Translated[A](translator: TransEnv => A) {
def apply(env: TransEnv): A = translator(env)
def map[B](f: A => B): Translated[B] =
new Translated[B](env => f(apply(env)))
//(...)
}

16. XML literals
XML can be part of the code and compiler is checking syntax of it:
def toKmlCoordinates(style: String): String =
(<Placemark>
<name>{"bbox " + this}</name>
<styleUrl>{style}</styleUrl>
<Polygon>
<outerBoundaryIs>
<LinearRing>
<coordinates>
{west + "," + north + ",0"}
{east + "," + north + ",0"}
{east + "," + south + ",0"}
{west + "," + south + ",0"}
{west + "," + north + ",0"}
</coordinates>
</LinearRing>
</outerBoundaryIs>
</Polygon>
</Placemark>).toString

17. Scalacheck
Automatic testing of assumptions, using set of generated values.
e.g. the code below tests if
BBox.parse(a.toString) == a
is true for any a
val generator: Gen[BBox] =
for {
s <- choose(-90.0, 90.0)
n <- choose(-90.0, 90.0)
if (n > s)
w <- choose(-180.0, 180.0)
e <- choose(-180.0, 180.0)
} yield new BBox(w, s, e, n)
property("parse . toString is identity") {
check {
(a: BBox) =>
BBox.parse(a.toString) == a
}
}
may yield this failing test message after several evaluations.

18. Scala (Scalding) in our analytics jobs
e.g. popular places job
class PopularPlacesJob(args: Args) extends AccessLogJob(args) {
implicit val mapMonoid = new MapMonoid[String, Long]()
val ord = implicitly[Ordering[(Long, String)]].reverse
readParseFilter()
.flatMap(’entry -> ’ppid) {
entry:AccessLogEntry => entry.request.ppid
}
.mapTo((’entry, ’ppid) -> (’ppid, ’time, ’app)) {
arg: (AccessLogEntry, String) => (arg._2, arg._1.dateTime, a
}
.groupBy((’ppid, ’app)) {
_.min(’time -> ’minTime)
.max(’time -> ’maxTime)
.size(’num)
}
.groupBy((’ppid)) {
_.min(’minTime)
.max(’maxTime)
.sum(’num)

19. Not so good in Scala
Almost whole team needed to learn the new language - luckily we have
Toralf ;)
Tools are not as mature as those in Java (but they are getting better very
fast)
Longer compilation, compiler has a lot more to do (e.g. type inference)
Multiple inheritance (traits)
Operators
changes in every language release

20. What we have learned, discovered?
When done right - it’s possible to painlessly migrate code to new
language, developing new feautres in the same time
How to use good typesystem and enjoy it
Take advantage form whole great immutable world of painless
programming
Use functions as first class citizens of language

21. Plans
Scala 2.10
Run Transformations in Futures, Validations etc.
Akka actors, Futures composition
Kill last Java files?
Stop using Spring

22. Thank you!
project site:
places.nlp.nokia.com
My contact:
lukasz.balamut@nokia.com
@lbalamut
open position in our team: