Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scala Days Highlights | BoldRadius


Published on

Did you miss Scala Days 2015 in San Francisco? Have no fear! BoldRadius was there and we've compiled the best of the best! Here are the highlights of a great conference.

  • Be the first to comment

Scala Days Highlights | BoldRadius

  1. 1. ScalaDays Highlights 2015
  2. 2. Who? ● BoldRadius Solutions ○ ○ Typesafe Partner ○ Scala, Akka and Play specialists ○ Ottawa, Saskatoon, San Francisco, Boston, Chicago, Montreal, New York, Toronto ● Michael Nash, VP Capabilities ● Adam Murray, VP Business
  3. 3. Our Team at ScalaDays
  4. 4. What? What was ScalaDays all about? ● Held in San Francisco March 16th thru 18th ○ About 800 attendees ○ Three categories (intermediate, beginner, advanced), ○ Four tracks, 55 presentations, three keynotes ● Followed by Two Intensive Training days ○ All the Typesafe courses offered
  5. 5. Venue: Fort Mason
  6. 6. The main hall...
  7. 7. Keynotes ● Martin Odersky, Chief Architect & Co-Founder at Typesafe ○ Scala - where it came from, where it’s going ● Danese Cooper, Distinguished Member of Technical Staff - Open Source at Paypal ○ Open Languages ● Dianne March, Director of Engineering Tools at Netflix ○ Technical Leadership from wherever you are
  8. 8. Quote from James Gosling...
  9. 9. Highlights Too many great sessions to summarize them all, we have to extract a few recurring themes… ○ Distributed Application Design and Development ○ Big Data/Fast Data ○ Types and Safe Scala ○ Performance and Scalability
  10. 10. Distributed Applications ● Life Beyond the Illusion of Present ● Reactive Reference Architectures ● Akka in Production: Why and How ● Easy Scalability with Akka ● Scalable task distribution with Scala, Akka and Mesos ● A Guided Tour of a Distributed Application
  11. 11. Performance and Scalability ● Scala Collections and Performance ● Shattering Hadoop’s Large-Scala Sort Record with Spark and Scala ● Type-safe off-heap memory for Scala ● Akka in Production: Why and How ● Easy Scalability with Akka ● The JVM Backend and Optimizer in Scala 2.12
  12. 12. Big Data, Fast Data ● Shattering Hadoop’s Large Scala Sort Record with Spark and Scala ● Scala - The Real Spark of Data Science ● Apache Spark: A Large Community Project in Scala ● S3 at Scala: Async Scala Client with Play Iteratees and Composable Operations ● The Unreasonable Effectiveness of Scala for Big Data ● Scalable task distribution with scala, Akka and Mesos
  13. 13. Types and Safer Scala ● Keynote: Scala - where it came from, where it’s going ● Towards a Safer Scala ● Leveraging Scala Macros for Better Validation ● Type-level programming in Scala 101 ● Improving Correctness with Types ● Happy Paths: Functional Constructs in the Wild ● The Scalactic Way ● Delimited dependency-typed monadic checked exceptions
  14. 14. The Rest... Many excellent talks were outside these categories ● Why Scala.js ● Reactive Slick for Database Programming ● Exercise in machine learning ● Functional Natural Language Processing ● If I Only Had a Scala ● Akka HTTP: the Reactive Web Toolkit ● many many others
  15. 15. Highlights of Specific Sessions A quick taster of what was in some of the more popular sessions...
  16. 16. Scala - Where it came from, where it’s going
  17. 17. Where it’s from, where it’s going “Scala is a gateway drug to Haskell” (in actual fact it’s going well beyond Haskell.) Slides: Came from a practical combination of OOP and functional programming - Funny story about hipster syntax (..) instead of [..], <..> instead of [..], ??? instead of <..> - Trend in Type Systems - Scala JS is no longer experimental - TASTY: new scala-specific platform - Introduction to DOT - Type Parameters - Better treatment of effects with implicit Functions instead of Monads
  18. 18. TASTY Scala faces challenges: ● binary compatibility ● having to pick a platform: JDK( 7,8,9,10, ?) or Javascript. Proposing a scala-specific platform called TASTY, (serialized typed abstract syntax tree), as an intermediate representation before bytecode - carries metadata of types, can be compiled with different versions of JDK, and to Javascript.
  19. 19. Tasty will enable: ● instrumentation ● optimization ● code analysis ● refactoring ● publish once run anywhere ● automated remapping to solve binary compatibility issues.
  20. 20. Explorations: Implicits that compose
  21. 21. Explorations: Hope to find something cooler than Monads to handle effects. ● Monads don’t commute ● Require Monad transformers for composition ● Monad transformers make Martin’s head explode
  22. 22. Toward a Safer Scala Leif Wickland ● Scalac enables some error-prone code. ○ head of empty List? ● Using Static Analysis to detect errors early ● IDE based solutions ○ Inconsistencies ○ If not in release build process, doesn’t exist ● Web-based analysis ○ outside of compile loop ○ relatively immature analysis
  23. 23. Scalac switches
  24. 24. ● Scala Style ● Wart Remover ● Linter ● Watch ○ Abide ○ Scapegoat ○ Supersafe
  25. 25. Life Beyond the Illusion of Present Jonas Bonér: The idea of the present is an illusion. Everything we see, hear and feel is just an echo from the past. But this illusion has influenced us and the way we view the world in so many ways. There is no present, all we have is facts derived from the merging of multiple pasts. The truth is closer to Einstein’s physics where everything is relative to one’s perspective. As developers we need to wake up and break free from the perceived reality of living in a single globally consistent present.
  26. 26. The advent of multicore and cloud computing architectures meant that most applications today are distributed systems—multiple cores separated by the memory bus or multiple nodes separated by the network—which puts a harsh end to this illusion. The only way to design truly scalable and performant systems that can construct a sufficiently consistent view of history—and thereby our local “present”—is by treating time as a first class construct in our programming model and to model the present as facts derived from the merging of multiple concurrent pasts.
  27. 27. How do we deal with failure and communication unreliability in real life? Confirmation and repetition We can’t force the world into a globally consistent present (CRUD). Mentioned 2 paradigms/theories: ● CALM (consistency as logical monotonicity) ● CRDT (Commutative Replicated Data Type) CRDT (Commutative Replicated Data Type) eventually consistent data types ● minimize contention / coordination in a distributed system. ● set, maps, graphs: rich data types. ● monotonic merge function: all state change is monotonically increasing, no way back.
  28. 28. Improving Correctness with Types
  29. 29. ● Using Wrapper Types (aka Tiny Types) instead of primitives ● Never use null, or throw Exceptions ● use === org.scalactic.TypeCheckedTripleEquals ○ requires the types of the two values compared to be in a subtype/supertype ● Never Use Non-Empty Lists when a list must be populated (org.scalactic.Every) ● Use Type Tags (ala Shapeless, Scalaz) ● Use Path Dependent Types ● Other reading ○ Self recursive types ○ Phantom Types ○ Shapeless ○ Scalactic Types: Defensive Programming. Fail Fast. Design By Contract.
  30. 30. Function Passing Style Heather Miller: ● A new programming model called function passing designed to overcome many of imperative / weakly-typed issues found in traditional “big data” processing systems. ● Provides a more principled substrate on which to build data-centric distributed systems. ● Pass safe, well-typed serializable functions to immutable distributed data ● Based on her work on Pickling and Spores ● Uses Spores (Serializable Functions) for a distributed model. ● Kind of an Inverse of the Actor Model ● Stateless. data is stationary, functions are passed around. ● Uses Data Silos accessed through a Silo Ref.
  31. 31. The Unreasonable Effectiveness of Scala for Big Data Dean Wampler ● How Hadoop Works - Map Reduce ○ Problems ■ Hard to implement algorithms ■ The Hadoop API is horrible ● Scalding ○ An improved Hadoop API in Scala ○ Problems ■ Still uses a batch mode ● Spark ○ An elegant, functional API ○ Still in Batch Mode, but with mini batches which approach real time.
  32. 32. Akka HTTP: The Reactive Web Toolkit Roland Kuhn ● Replaces Spray ● Uses Akka Streams ○ Sources emit values to the stream ○ Sinks receive values, act on them ○ Sources can compose using Zip and Graph shapes ● “The pinball interpreter” ○ produce data ○ move downstream through transformations ○ get to the effect ○ go up and ask for more data ○ Filters interrupt the flow before getting to the effect, make the pinball go back upstream. ● A live coded demonstration of using Streams and Http ● Expected timeline for Streams - 4 weeks ● Expected timeline for Http - 8 weeks
  33. 33. The Scalactic Way Bill Venners ScalaTest: quality through tests Scalactic: quality through types SuperSafe: quality through static analysis
  34. 34. Reactive Slick for Database Programming Stefan Zeiger Slick 3.0 ● JDBC is inherently blocking (and blocking ties up threads) ● Traditional Model ○ Fully synchronous ○ One thread per web request ○ Contention for Connections (getConnection blocks) ○ Database back-pressure creates more blocked threads ○ Doesn’t scale
  35. 35. New Slick Architecture makes use of a new DataType to provide Asynchronous Database I/O ● based on State, IO and Free Monads. ● Returns a Future[R] ● Creates a separate ExecutionContext, avoiding blocking of current thread ● Works with akka-streams to create back pressure, so DB only gives as much data as client can process. ● For performance purposes it pre-fetches some data to keep client busy while it waits for the next portion from DB. sealed trait DBIOAction[+R, +S <: NoStream, -E <: Effect]{ def map[R2](f: R => R2)(implicit executor: ExecutionContext): DBIOAction[R2, NoStream, E] def flatMap[R2, S2 <: NoStream, E2 <: Effect](f: R => DBIOAction[R2, S2, E2])(implicit executor: ExecutionContext): DBIOAction[R2, S2, E with E2] ... }
  36. 36. Easy Scalability with Akka
  37. 37. Easy Scalability with Akka Michael Nash ● Reviewed Akka, CQRS, ES ● Introduced Distributed DDD ● Identical clustered system with DDDD and Without ● Gatling performance tests on both
  38. 38. ConductR ● Application manager that empowers ops to deploy distributed systems ● Uses Akka, Play, Aka Streams, Akka Cluster, FSM, Akka Data Replication ● How can we run cluster based apps ensuring the seed nodes are started first? ○ State replicated using Data Replication ● How can we consolidate logging? ○ Using Akka Streams ● How can we avoid batching? ○ Use Event Driven Architecture ● How can monitor/test multiple nodes? ○ Use the visualizer built into conductor ● How can we share state among the nodes? ○ Use Akka Data Replication
  39. 39. Scala.js Building a data-driven map of the world in 10 minutes….
  40. 40. Whats wrong with Webapps? ● Everything String/Map[String, String] ● Compiler cannot help you! ○ document.getElementByld("my-id") ● Javascript! ○ ["10", "10", "10", "10"].map(parseInt) [10, NaN, 2, 3] // WTF ScalaJS ● lets you write webapp in one language ● Swap String-typing for Strong-typing ● Type Everything!! ○ val res: Future[Seq[String]] = Ajax[Api].list(inputBox.value).call() ● IS NOT JAVASCRIPT ○ scala.js> List("10", "10", "10", "10").map(parseInt) List(10, 10, 10, 10) // Yay!
  41. 41. The BoldRadius Crew Packing up...
  42. 42. The End….
  43. 43. ScalaDays Amsterdam! ● June 8th to 10th ● Followed by Training
  44. 44. Q & A