Scala in Places API

658 views

Published on

talk to the Scala User group in Berlin

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
658
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Scala in Places API

  1. 1. Scala in Nokia Places APILukasz Balamut - Places API Team 17 April 2013
  2. 2. Purpose of this talk Why are we spending time migrating to a new language? How are we doing that? What have we learned? and of course: we are hiring.
  3. 3. Our scala story. Our migration in lines of code We’ve discovered more and more things on the way.
  4. 4. The Concordion tests problem In June 2011 we had problems with Concordion (HTML based acceptance test framework), hard to integrate HTML tests were not readable Solution: Multi-line Strings in Scala Continue using JUnit val expectedBody = """ { "id":"250u09wh-83d4c5479b7c4db9836602936dbc8cb8", "title" : "Marfil (Le)", "alternativeNames" : [ { "name" : "Le Marfil", "language" : "fr" } ] } """
  5. 5. The test names problem JUnit + camel case - not that readable for long names @Test def preservesSearchResponseItemOrder() { /* ... */ } JUnit + underscores - violates naming convention, but better to read @Test def preserves_search_response_item_order() { /* ... */ } ScalaTest test("preserves search response’s item order") { /* ... */ } It is supported by tools we use and produces nice output when run (as a bonus) [info] DiscoverAroundAcceptanceTest: [info] - calls nsp with 15Km radius [info] - preserves search response’s item order [info] - when requesting zero results, no request to search is made
  6. 6. Lift-Json JSON (de)serialisation can be done in really nice way e.g. getting place id: { "place": { "a_id": "276u33db-6f084959f97c45bc9db26348bafd4563" } } Also there is a generic way of parsing json, that gives you XPath like access: val id = (json "place" "a_id").extract[String] Lift-json provides also easy and efficient case class JSON (de)serialisation, hance case classes
  7. 7. Scala case classes in the model We model our upstream and downstream datamodels as case classes. case class Place ( name: String, placeId: PlaceID //(...) ) Each case class is a proper immutable data type - the new Java Bean ;), A lot of boilerplate code is generated by the compiler e.g.: accessors, equals/hashCode, copy methods, toString apply/unapply (for pattern maching), After decompiling the scala code above we will end up with 191 lines of Java: package pbapi.model.api; import scala.*; import scala.collection.Iterator; import scala.runtime.BoxesRunTime; import scala.runtime.ScalaRunTime$;
  8. 8. Scala case classes - Configuration Easy to update configuration Easy to compare environments’ configuration case class HttpClientConfig( serviceName: String, serviceBaseUrl: URL, readTimeout: Int = 0, connectionTimeout: Int, proxy: Option[ProxyConfig] = None, oauthKeys: Option[OAuthConfig] = None, displayUrl: Option[String] = None ) "httpConfig":{ "serviceName":"recommendations", "serviceBaseUrl":"http://nose.svc.ovi.com/rest/v1/recommendations/near "readTimeout":4000, "connectionTimeout":500, "displayUrl":"http://nose.svc.ovi.com/rest/v1/recommendations/nearby/" }
  9. 9. The Scala Type System A great way to express assumptions Have them checked by the compiler through the whole codebase At every point in the code know what to expect or you will be reminded by compiler.
  10. 10. The Scala Type System - Options 1 Avoid null references (see The Billion Dollar Mistake ) This will not compile! case class UpstreamResponse ( unreliableField: Option[String] ) case class MyResponse ( reliableField: String ) def transform(upstream: UpstreamResponse) = MyResponse( reliableField = upstream.unreliableField //<-- incompatible type )
  11. 11. The Scala Type System - Options 2 But this will: case class UpstreamResponse ( unreliableField: Option[String] ) case class MyResponse ( reliableField: String ) def transform(upstream: UpstreamResponse) = MyResponse( reliableField = upstream.unreliableField.getOrElse("n/a") // enf )
  12. 12. The Scala Type System - Options 3 how we were learning e.g.: to transform string when it is defined def transform(str: String): String reliableField = if (upstream.unreliableField.isDefined) transform(upstream.unreliableField.get) else "n/a" reliableField = upstream.unreliableField match { case Some(f) => transform(f) case None => "n/a" } reliableField = upstream.unreliableField.map(transform) | "n/a"
  13. 13. The Scala Type System - Standard Types Immutable Map, List, Sets val map: Map[String, String] = Map("a" -> "a", "b" -> "b") val list: List[String] = "a" :: "b" :: Nil Tuples to express Pairs, Triplets etc. val pair: Pair[String, Int] = ("a", 1) val (a, b) = pair // defines a:String and b:Int
  14. 14. The Scala Type System - error handling Types can help with exceptional cases val parsed: Validation[NumberFormatException, Int] = someString.parseInt val optional: Option[Int] = parsed.toOption val withDefault: Int = optional.getOrElse(0) //if there was parsing prob Exception to Option val sub: Option[String] = allCatch.opt(line.substring(line.indexOf("{""))) or Either val parsed: Either[Throwable, Incident] = allCatch.either(new Incident(parse(jsonString)) and handle it later parsed match { case Right(j) => Some(j) case Left(t) => { println("Parse error %s".format(t.getMessage)) None
  15. 15. The Scala Type System - functions we defer translation (Translated) and text rendering (FormattedText) to serialisation time using function composition so we don’t need to pass user context through the whole stack e.g.: case class Place ( //(...) attribution: Option[Translated[FormattedText]], //(...) ) case class Translated[A](translator: TransEnv => A) { def apply(env: TransEnv): A = translator(env) def map[B](f: A => B): Translated[B] = new Translated[B](env => f(apply(env))) //(...) }
  16. 16. XML literals XML can be part of the code and compiler is checking syntax of it: def toKmlCoordinates(style: String): String = (<Placemark> <name>{"bbox " + this}</name> <styleUrl>{style}</styleUrl> <Polygon> <outerBoundaryIs> <LinearRing> <coordinates> {west + "," + north + ",0"} {east + "," + north + ",0"} {east + "," + south + ",0"} {west + "," + south + ",0"} {west + "," + north + ",0"} </coordinates> </LinearRing> </outerBoundaryIs> </Polygon> </Placemark>).toString
  17. 17. Scalacheck Automatic testing of assumptions, using set of generated values. e.g. the code below tests if BBox.parse(a.toString) == a is true for any a val generator: Gen[BBox] = for { s <- choose(-90.0, 90.0) n <- choose(-90.0, 90.0) if (n > s) w <- choose(-180.0, 180.0) e <- choose(-180.0, 180.0) } yield new BBox(w, s, e, n) property("parse . toString is identity") { check { (a: BBox) => BBox.parse(a.toString) == a } } may yield this failing test message after several evaluations.
  18. 18. Scala (Scalding) in our analytics jobs e.g. popular places job class PopularPlacesJob(args: Args) extends AccessLogJob(args) { implicit val mapMonoid = new MapMonoid[String, Long]() val ord = implicitly[Ordering[(Long, String)]].reverse readParseFilter() .flatMap(’entry -> ’ppid) { entry:AccessLogEntry => entry.request.ppid } .mapTo((’entry, ’ppid) -> (’ppid, ’time, ’app)) { arg: (AccessLogEntry, String) => (arg._2, arg._1.dateTime, a } .groupBy((’ppid, ’app)) { _.min(’time -> ’minTime) .max(’time -> ’maxTime) .size(’num) } .groupBy((’ppid)) { _.min(’minTime) .max(’maxTime) .sum(’num)
  19. 19. Not so good in Scala Almost whole team needed to learn the new language - luckily we have Toralf ;) Tools are not as mature as those in Java (but they are getting better very fast) Longer compilation, compiler has a lot more to do (e.g. type inference) Multiple inheritance (traits) Operators changes in every language release
  20. 20. What we have learned, discovered? When done right - it’s possible to painlessly migrate code to new language, developing new feautres in the same time How to use good typesystem and enjoy it Take advantage form whole great immutable world of painless programming Use functions as first class citizens of language
  21. 21. Plans Scala 2.10 Run Transformations in Futures, Validations etc. Akka actors, Futures composition Kill last Java files? Stop using Spring
  22. 22. Thank you! project site: places.nlp.nokia.com My contact: lukasz.balamut@nokia.com @lbalamut open position in our team:

×