Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Reactive database access with Slick3

4,046 views

Published on

#reactive_shinjuku

Published in: Software
  • Be the first to comment

Reactive database access with Slick3

  1. 1. Reactive database access with Slick3 Naoki Takezoe @takezoen BizReach, Inc
  2. 2. Who am I? ● Naoki Takezoe: @takezoen ● Scala programmer at BizReach, Inc ● Long time OSS programmer and technical writer
  3. 3. Slick3 (a.k.a. Reactive Slick) http://slick.typesafe.com/
  4. 4. What's Reactive Slick ● Data access framework for Scala ● Powerful type-safe query API ● Parallel execution and streaming
  5. 5. What's Reactive Slick ● Data access framework for Scala ● Powerful type-safe query API ● Parallel execution and streaming
  6. 6. History of Slick ● 2010 ScalaQuery 0.9 ● 2012 Slick 0.11 (Move to Typesafe) ● 2013 Slick 1.0 ● 2014 Slick 2.0 (Statically Session) ● 2015 Slick 3.0 (Reactive)
  7. 7. History of Slick ● 2010 ScalaQuery 0.9 ● 2012 Slick 0.11 (Move to Typesafe) ● 2013 Slick 1.0 ● 2014 Slick 2.0 (Statically Session) ● 2015 Slick 3.0 (Reactive) Large API changes. Migration was very hard...
  8. 8. Background ● C10K problem ● Non-blocking I/O based web framework ● Microservice architecture ● Big data
  9. 9. Parallel execution
  10. 10. Example 1: Simple query // Define query action val select: DBIO[Seq[String]] = Coffee // Table .filter(_.price <= 100.bind) // Where clause .map(_.name) // Select clause .result // Execute query db.run(select) // => Future[Seq[String]]
  11. 11. Example 2: Parallel execution // Define insert actions val inserts: Seq[DBIO[Int]] = Seq( Coffee("Colombian", 101, 7.99, 0, 0).insert, Coffee("French_Roast", 49, 8.99, 0, 0).insert, Coffee("Espresso", 150, 9.99, 0, 0).insert, Coffee("Colombian_Decaf", 101, 8.99, 0, 0).insert, Coffee("French_Roast_Decaf", 49, 9.99, 0, 0).insert ) // Combine above actions val combined: DBIO[Seq[Int]] = DBIO.sequence(inserts) // Run action transactionally db.run(combined.transactionally) // => Future[Seq[Int]]
  12. 12. Example 3: Work with other aync tasks def index = Action.async { for { res <- ws.url("http://xxx").get() name <- db.run( Coffee .filter(_.id === res.body.bind) .map(_.name).result.head ) } yield Ok(name) } Future[WSResponse] Future[String] Future[Result]
  13. 13. But JDBC is blocking. Why Slick3?
  14. 14. Separate ExecutionContext Controller Thread (Play) I/O Thread (Slick) JDBC
  15. 15. Separate ExecutionContext Controller Thread (Play) I/O Thread (Slick) JDBC for CPU bound tasks for I/O bound tasks
  16. 16. by separating ExecutionContext ● Use server resources effectively ● Avoid exhausting threads by I/O tasks
  17. 17. Future: Real non-blocking database access? ● postgresql-async ● mysql-async https://github.com/mauricio/postgresql-async
  18. 18. Streaming results (Reactive Streams)
  19. 19. Streaming results is ● for large data processing ● based on Reactive Streams
  20. 20. Reactive Streams public interface Publisher<T> { public void subscribe(Subscriber<? super T> s); } public interface Subscriber<T> { public void onSubscribe(Subscription s); public void onNext(T t); public void onError(Throwable t); public void onComplete(); } public interface Subscription { public void request(long n); public void cancel(); } public interface Processor<T, R> extends Subscriber<T>, Publisher<R> { }
  21. 21. Example: Slick API val publisher: DatabasePublisher[String] = db.stream(Coffee.map(_.message).result) // Iterate stream using Slick API publisher.foreach { name => println(name) }
  22. 22. DataPublisher implementation def foreach[U](f: T => U)(implicit ec: ExecutionContext): Future[Unit] = { val p = Promise[Unit]() @volatile var lastMsg: Future[Any] = null @volatile var subscr: Subscription = null subscribe(new Subscriber[T] { def onSubscribe(s: Subscription): Unit = { subscr = s s.request(1L) } def onComplete(): Unit = { ... } def onError(t: Throwable): Unit = { ... } def onNext(t: T): Unit = { lastMsg = Future(f(t)) lastMsg.onComplete { case Success(v) => subscr.request(1L) case Failure(t) => ... } } }) p.future } Implemented based on Reactive Streams API
  23. 23. Missing piece Need both of the publicher and the subscriber Publisher Subscriber / Publisher Subscriber
  24. 24. One idea: Slick3 + Akka Streams? http://www.slideshare.net/kazukinegoro5/akka-streams-100-scalamatsuri
  25. 25. Problem
  26. 26. Problem 1: Functionallity
  27. 27. Problem 2: Difficulty ● Programming ○ Monadic API is hard for non functional programmers ● Configuration ○ Connection pool and ExecutionContext
  28. 28. Sumary
  29. 29. Summary ● Slick3 provides 2 reactives: ○ Parallel database processing ○ Streaming large data based on Reactive Sreams ● Suitabe usecase for Slick are: ○ Composite database tasks and async tasks ○ Large data processing using Reactive Streams

×