Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Konrad 'ktoso' Malawski 
GeeCON 2014 @ Kraków, PL 
Akka Streams 
Konrad `@ktosopl` Malawski 
Tokyo Scala User Group @ Toky...
hAkker @ 
「アッカ」チームのメンバー。 
Konrad `@ktosopl` Malawski
hAkker @ 
Konrad `@ktosopl` Malawski 
typesafe.com 
geecon.org 
Java.pl / KrakowScala.pl 
sckrk.com / meetup.com/Paper-Cup...
ありがとうございました, Benedict! 
For translating the slides!
You? 
?
You? 
? 
z ?
You? 
? 
z ? 
?
You? 
? 
z ? 
? 
?
Streams
Streams
Streams 
“You cannot enter the same river twice” 
~ Heraclitus 
http://en.wikiquote.org/wiki/Heraclitus
Streams 
ライルタイムストリーム処理 
Real Time Stream Processing 
!パブリッシャーにつけることが遅ければ、 
データは川のように流れるため、最初の 
要素を逃がしてしまう可能性がある 
When you ...
Reactive Streams
Reactive Streams 
! 
! 
Stream processing
Reactive Streams 
Back-pressured 
! 
Stream processing
Reactive Streams 
Back-pressured 
Asynchronous 
Stream processing
Reactive Streams 
Back-pressured 
Asynchronous 
Stream processing 
Standardised (!)
Reactive Streams: Goals 
1. Back-pressured Asynchronous Stream processing 
! 
2. Standard implemented by many libraries
Reactive Streams - Specification & TCK 
http://reactive-streams.org
Reactive Streams - Who? 
Kaazing Corp. 
rxJava @ Netflix, 
reactor @ Pivotal (SpringSource), 
vert.x @ Red Hat, 
Twitter, ...
Reactive Streams - Inter-op 
システム達を協力させたい. 
We want to make different implementations 
co-operate with each other. 
http:/...
Reactive Streams - Inter-op 
リアクティブストリームのプロトコールを 
使ってシステム達を話し合わせる 
The different implementations “talk to each other” 
usi...
Reactive Streams - Inter-op 
リアクティブストリームSPIはユーザーAPIではない。 
対象システムのライブラリを使うべき。 
The Reactive Streams SPI is NOT meant to be ...
Back-pressure, なにですか?
Back-pressure? Example Without 
Publisher[T] Subscriber[T]
Back-pressure? Example Without 
速度が早いパブリッシャー 
Fast Publisher 速度が遅いサブスクライバー 
Slow Subscriber
Back-pressure? 
Push + NACK model
Back-pressure? Push + NACK model
Back-pressure? Push + NACK model 
通常の場合、サブスクライバーはバッファーがある. 
Subscriber usually has some kind of buffer.
Back-pressure? Push + NACK model
Back-pressure? Push + NACK model
Back-pressure? Push + NACK model 
バッファーが溢れてしまった場合、どうなる? 
What if the buffer overflows?
Back-pressure? Push + NACK model (a) 
有界バッファーを利用して、溢れてしまったメッセージを 
落とし、再送を求める. 
! 
Use bounded buffer, drop messages + requ...
Back-pressure? Push + NACK model (a) 
有界バッファーを利用して、溢れてしまったメッセージを 
落とし、再送を求める. 
! 
Use bounded buffer, drop messages + requ...
Back-pressure? Push + NACK model (b) 
バッファーの容量を増やす・・・まぁ、 
メモリ容量がある限り! 
Increase buffer size… 
Well, while you have memory ...
Back-pressure? Push + NACK model (b)
Back-pressure? 
Why NACKing is NOT enough
Back-pressure? Example NACKing 
たいへんですよ! 
バッファーが溢れるまで間もなく! 
Buffer overflow is imminent!
Back-pressure? Example NACKing 
データを送る速力を落とす、または送信を停止する、 
ようにパブリッシャーに告げる. 
Telling the Publisher to slow down / stop sendi...
Back-pressure? Example NACKing 
メッセージが送信されている最中のため、 
NACKが間に合わなかった! 
NACK did not make it in time, 
because M was in-fligh...
Back-pressure? 
! 
パブリッシャーの速度 < サブスクライバーの速度 
speed(publisher) < speed(subscriber)
Back-pressure? Fast Subscriber, No Problem 
速度が速いサブスクライバーなら、問題ない! 
No problem!
Back-pressure? 
Reactive-Streams 
= 
“Dynamic Push/Pull”
Back-pressure? RS: Dynamic Push/Pull 
プッシュ一方 - 速度が遅いサブスクライバーの場合、 
安全じゃない 
Just push – not safe when Slow Subscriber 
! 
! ...
Back-pressure? RS: Dynamic Push/Pull 
プッシュ一方 - 速度が遅いサブスクライバーの場合、 
安全じゃない 
Just push – not safe when Slow Subscriber 
! 
! ...
Back-pressure? RS: Dynamic Push/Pull 
速度が遅いサブスクライバーは自分のバッ 
ファーが3つの要素まで受け取られる。パブリッ 
シャーはサブスクライバーのバッファーが溢れ 
るほどデータを送らない。Slow...
Back-pressure? RS: Dynamic Push/Pull 
速度が速いパブリッシャーは最大でも3つの要 
素を送る。これは、プル型のback-pressure. 
Fast Publisher will send at-most...
Back-pressure? RS: Dynamic Push/Pull 
速度が速いサブスクライバーは、実際のデータ 
が届く前に、たくさんリクエストを送ることが 
できる!Fast Subscriber can issue more Req...
Back-pressure? RS: Dynamic Push/Pull 
パブリッシャーはサブスクライバーの全てのリ 
クエストを溜める. 
Fast Subscriber can issue more Request(n), before ...
Back-pressure? RS: Accumulate demand 
パブリッシャーはサブスクライバーの全ての 
リクエストを溜める. Publisher accumulates total demand per subscriber.
Back-pressure? RS: Accumulate demand 
溜まった要素をパブリッシュすることは安全だ。 
サブスクライバーのバッファーは溢れない。 
Total demand of elements is safe to pu...
Back-pressure? RS: Requesting “a lot” 
速度が速いサブスクライバーは、パブリッ 
シャーに対し要求が多くても良い。これは「パ 
ブリッシャー・プッシュ」だ。バッファー容量 
は知られているので、安全だ。
Back-pressure? RS: Requesting “a lot” 
Fast Subscriber, can request “a lot” from Publisher. 
This is effectively “publishe...
Back-pressure? RS: Dynamic Push/Pull 
MAX速度
Back-pressure? RS: Dynamic Push/Pull 
安全!溢れ 
ること絶対 
ない! 
MAX速度
わなにですか?
Akka = アッカ 
Akka is a high-performance concurrency 
library for Scala and Java. 
! 
At it’s core it focuses on the Actor M...
Akka = アッカ 
Akka is a high-performance concurrency 
library for Scala and Java. 
! 
At it’s core it focuses on the Actor M...
Akka 
Akka has multiple modules: 
! 
Akka-camel: integration 
Akka-remote: remote actors 
Akka-cluster: clustering 
Akka-p...
Akka Streams 
0.7 early preview
Akka Streams – Linear Flow
Akka Streams – Linear Flow
Akka Streams – Linear Flow
Akka Streams – Linear Flow
Akka Streams – Linear Flow 
FlowFrom[Double].map(_.toInt). [...] 
ソースはまだ付けられていない. 
パイプはダブルを処理する準備を整いた. 
No Source attached...
Akka Streams – Linear Flow 
implicit val sys = ActorSystem("tokyo-sys")! 
! 
アクターが住んでいる世界. 
AkkaStreamsはアクターを使うため、 
アクターシス...
Akka Streams – Linear Flow 
implicit val sys = ActorSystem("tokyo-sys")! 
implicit val mat = FlowMaterializer()! 
ストリームをどう...
Akka Streams – Linear Flow 
implicit val sys = ActorSystem("tokyo-sys")! 
implicit val mat = FlowMaterializer()! 
単純にアクターか...
Akka Streams – Linear Flow 
implicit val sys = ActorSystem("tokyo-sys")! 
implicit val mat = FlowMaterializer()! 
バッファーの容量...
Akka Streams – Linear Flow 
implicit val sys = ActorSystem("tokyo-sys")! 
implicit val mat = FlowMaterializer()! 
val fore...
Akka Streams – Linear Flow 
implicit val sys = ActorSystem("tokyo-sys")! 
implicit val mat = FlowMaterializer()! 
val fore...
Akka Streams – Linear Flow 
val mf = FlowFrom[Int].! 
map(_ * 2).! 
withSink(ForeachSink(println)) // needs source,! 
// c...
Akka Streams – Linear Flow 
val f = FlowFrom[Int].! 
map(_ * 2).! 
! ! ! withSink(ForeachSink(i => println(s"i = $i”))).! ...
Akka Streams – Linear Flow 
val f = FlowFrom[Int].! 
map(_ * 2).! 
! ! ! withSink(ForeachSink(i => println(s"i = $i”))).! ...
Akka Streams – Linear Flow 
val f = FlowFrom[Int].! 
map(_ * 2).! 
! ! ! withSink(ForeachSink(i => println(s"i = $i”))).! ...
Akka Streams – Linear Flow 
val f = FlowFrom[Int].! 
map(_ * 2).! 
! ! ! withSink(ForeachSink(i => println(s"i = $i”))).! ...
Akka Streams – Linear Flow 
val f = FlowFrom[Int].! 
map(_ * 2).! 
! ! ! withSink(ForeachSink(i => println(s"i = $i”))).! ...
Akka Streams – Flows are reusable 
! 
! ! ! f.withSource(IterableSource(1 to 10)).run()! 
! ! ! f.withSource(IterableSourc...
Akka Streams <-> Actors – Advanced 
val subscriber = system.actorOf(Props[SubStreamParent], ”parent")! 
! 
FlowFrom(1 to 1...
Akka Streams <-> Actors – Advanced 
val subscriber = system.actorOf(Props[SubStreamParent], ”parent")! 
! 
FlowFrom(1 to 1...
Akka Streams <-> Actors – Advanced 
! 
groupBy(_.last). 
「11」をグループ「1」に、「12」をグループ「2」になど 
GroupBy groups “11” to group “1”, ...
Akka Streams <-> Actors – Advanced 
! 
groupBy(_.last). 
サブスクライバーに[グループキー、サブストリームフロー]を提供する 
It offers (groupKey, subStream...
Akka Streams <-> Actors – Advanced 
! 
groupBy(_.last). 
子供を起動させ、サブーフローを扱わせる 
It can then start children, to handle the su...
Akka Streams <-> Actors – Advanced 
! 
groupBy(_.last). 
例えば、グループ毎に1人の子供 
For example, one child for each group.
Akka Streams <-> Actors – Advanced 
val subscriber = system.actorOf(Props[SubStreamParent], ”parent")! 
! 
FlowFrom(1 to 1...
Akka Streams <-> Actors – Advanced 
class SubStreamParent extends ActorSubscriber ! 
with ImplicitFlowMaterializer ! 
with...
Akka Streams <-> Actors – Advanced 
class SubStreamParent extends ActorSubscriber ! 
with ImplicitFlowMaterializer ! 
with...
Akka Streams <-> Actors – Advanced 
class SubStreamParent extends ActorSubscriber ! 
with ImplicitFlowMaterializer ! 
with...
Akka Streams <-> Actors – Advanced 
class SubStreamParent extends ActorSubscriber ! 
with ImplicitFlowMaterializer ! 
with...
Akka Streams <-> Actors – Advanced 
class SubStreamParent extends ActorSubscriber ! 
with ImplicitFlowMaterializer ! 
with...
Akka Streams <-> Actors – Advanced 
class SubStreamParent extends ActorSubscriber {! 
! 
override def requestStrategy = On...
Akka Streams – GraphFlow 
GraphFlow
Akka Streams – GraphFlow 
Linear Flows 
or 
non-akka pipelines 
Could be another RS implementation!
Akka Streams – GraphFlow 
Fan-out elements 
and 
Fan-in elements
Akka Streams – GraphFlow 
Fan-out elements 
and 
Fan-in elements 
Now you need a FlowGraph
Akka Streams – GraphFlow 
// first define some pipeline pieces! 
val f1 = FlowFrom[Input].map(_.toIntermediate)! 
val f2 =...
Akka Streams – GraphFlow
Akka Streams – GraphFlow 
val b3 = Broadcast[Int]("b3")! 
val b7 = Broadcast[Int]("b7")! 
val b11 = Broadcast[Int]("b11")!...
Akka Streams – GraphFlow
Akka Streams – GraphFlow 
// First layer! 
in7 ~> b7! 
b7 ~> m11! 
b7 ~> m8! 
! 
in5 ~> m11! 
! 
in3 ~> b3! 
b3 ~> m8! 
b3...
Akka Streams – GraphFlow 
! 
// Second layer! 
m11 ~> b11! 
b11 ~> FlowFrom[Int].grouped(1000) ~> resultFuture2 ! 
b11 ~> ...
Akka Streams – GraphFlow 
! 
// Third layer! 
m9 ~> FlowFrom[Int].grouped(1000) ~> resultFuture9! 
m10 ~> FlowFrom[Int].gr...
Akka Streams – GraphFlow 
! 
// Third layer! 
m9 ~> FlowFrom[Int].grouped(1000) ~> resultFuture9! 
m10 ~> FlowFrom[Int].gr...
Akka Streams – GraphFlow 
! 
// Third layer! 
m9 ~> FlowFrom[Int].grouped(1000) ~> resultFuture9! 
m10 ~> FlowFrom[Int].gr...
Akka Streams – GraphFlow 
Sinks and Sources are “keys” 
which can be addressed within the graph 
val resultFuture2 = Futur...
Akka Streams – GraphFlow 
Sinks and Sources are “keys” 
which can be addressed within the graph 
val resultFuture2 = Futur...
Akka Streams – GraphFlow 
! 
val g = FlowGraph {}! 
FlowGraphは不変で、安全に共有でき、 
何度も使いまわせる! 
! 
FlowGraph is immutable and safe...
Available Elements 
0.7 early preview
Available Sources 
• FutureSource 
• IterableSource 
• IteratorSource 
• PublisherSource 
• SubscriberSource 
• ThunkSourc...
Available operations 
• buffer 
• collect 
• concat 
• conflate 
• drop / dropWithin 
• take / takeWithin 
• filter 
• fol...
Available Sinks 
• BlackHoleSink 
• FoldSink 
• ForeachSink 
• FutureSink 
• OnCompleteSink 
• PublisherSink / FanoutPubli...
Spray => Akka-Http && ReactiveStreams 
Spray is now merged into Akka, as Akka-Http 
Works on Reactive Streams 
Streaming e...
Links 
• http://akka.io 
• http://reactive-streams.org 
• https://groups.google.com/group/akka-user <- ask questions here!...
ありがとう 
ございました! 
Ask questions, 
get Stickers! 
http://akka.io 
ktoso @ typesafe.com 
twitter: ktosopl 
github: ktoso 
team...
©Typesafe 2014 – All Rights Reserved
Upcoming SlideShare
Loading in …5
×

2014 akka-streams-tokyo-japanese

21,898 views

Published on

Akka Streams (0.7) talk for the Tokyo Scala User Group, hosted by Dwango.

Akka streams are an reactive streams implementation which allows for asynchronous back-pressured processing of data in complext pipelines. This talk aims to highlight the details about how reactive streams work as well as some of the ideas behind akka streams.

Published in: Technology
  • Be the first to comment

2014 akka-streams-tokyo-japanese

  1. 1. Konrad 'ktoso' Malawski GeeCON 2014 @ Kraków, PL Akka Streams Konrad `@ktosopl` Malawski Tokyo Scala User Group @ Tokyo 2014
  2. 2. hAkker @ 「アッカ」チームのメンバー。 Konrad `@ktosopl` Malawski
  3. 3. hAkker @ Konrad `@ktosopl` Malawski typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
  4. 4. ありがとうございました, Benedict! For translating the slides!
  5. 5. You? ?
  6. 6. You? ? z ?
  7. 7. You? ? z ? ?
  8. 8. You? ? z ? ? ?
  9. 9. Streams
  10. 10. Streams
  11. 11. Streams “You cannot enter the same river twice” ~ Heraclitus http://en.wikiquote.org/wiki/Heraclitus
  12. 12. Streams ライルタイムストリーム処理 Real Time Stream Processing !パブリッシャーにつけることが遅ければ、 データは川のように流れるため、最初の 要素を逃がしてしまう可能性がある When you attach “late” to a Publisher, you may miss initial elements – it’s a river of data. http://en.wikiquote.org/wiki/Heraclitus
  13. 13. Reactive Streams
  14. 14. Reactive Streams ! ! Stream processing
  15. 15. Reactive Streams Back-pressured ! Stream processing
  16. 16. Reactive Streams Back-pressured Asynchronous Stream processing
  17. 17. Reactive Streams Back-pressured Asynchronous Stream processing Standardised (!)
  18. 18. Reactive Streams: Goals 1. Back-pressured Asynchronous Stream processing ! 2. Standard implemented by many libraries
  19. 19. Reactive Streams - Specification & TCK http://reactive-streams.org
  20. 20. Reactive Streams - Who? Kaazing Corp. rxJava @ Netflix, reactor @ Pivotal (SpringSource), vert.x @ Red Hat, Twitter, akka-streams @ Typesafe, spray @ Spray.io, Oracle, java (?) – Doug Lea - SUNY Oswego … http://reactive-streams.org
  21. 21. Reactive Streams - Inter-op システム達を協力させたい. We want to make different implementations co-operate with each other. http://reactive-streams.org
  22. 22. Reactive Streams - Inter-op リアクティブストリームのプロトコールを 使ってシステム達を話し合わせる The different implementations “talk to each other” using the Reactive Streams protocol. http://reactive-streams.org
  23. 23. Reactive Streams - Inter-op リアクティブストリームSPIはユーザーAPIではない。 対象システムのライブラリを使うべき。 The Reactive Streams SPI is NOT meant to be user-api. You should use one of the implementing libraries. http://reactive-streams.org
  24. 24. Back-pressure, なにですか?
  25. 25. Back-pressure? Example Without Publisher[T] Subscriber[T]
  26. 26. Back-pressure? Example Without 速度が早いパブリッシャー Fast Publisher 速度が遅いサブスクライバー Slow Subscriber
  27. 27. Back-pressure? Push + NACK model
  28. 28. Back-pressure? Push + NACK model
  29. 29. Back-pressure? Push + NACK model 通常の場合、サブスクライバーはバッファーがある. Subscriber usually has some kind of buffer.
  30. 30. Back-pressure? Push + NACK model
  31. 31. Back-pressure? Push + NACK model
  32. 32. Back-pressure? Push + NACK model バッファーが溢れてしまった場合、どうなる? What if the buffer overflows?
  33. 33. Back-pressure? Push + NACK model (a) 有界バッファーを利用して、溢れてしまったメッセージを 落とし、再送を求める. ! Use bounded buffer, drop messages + require re-sending
  34. 34. Back-pressure? Push + NACK model (a) 有界バッファーを利用して、溢れてしまったメッセージを 落とし、再送を求める. ! Use bounded buffer, drop messages + require re-sending Kernel does this! Routers do this! (TCP)
  35. 35. Back-pressure? Push + NACK model (b) バッファーの容量を増やす・・・まぁ、 メモリ容量がある限り! Increase buffer size… Well, while you have memory available!
  36. 36. Back-pressure? Push + NACK model (b)
  37. 37. Back-pressure? Why NACKing is NOT enough
  38. 38. Back-pressure? Example NACKing たいへんですよ! バッファーが溢れるまで間もなく! Buffer overflow is imminent!
  39. 39. Back-pressure? Example NACKing データを送る速力を落とす、または送信を停止する、 ようにパブリッシャーに告げる. Telling the Publisher to slow down / stop sending…
  40. 40. Back-pressure? Example NACKing メッセージが送信されている最中のため、 NACKが間に合わなかった! NACK did not make it in time, because M was in-flight!
  41. 41. Back-pressure? ! パブリッシャーの速度 < サブスクライバーの速度 speed(publisher) < speed(subscriber)
  42. 42. Back-pressure? Fast Subscriber, No Problem 速度が速いサブスクライバーなら、問題ない! No problem!
  43. 43. Back-pressure? Reactive-Streams = “Dynamic Push/Pull”
  44. 44. Back-pressure? RS: Dynamic Push/Pull プッシュ一方 - 速度が遅いサブスクライバーの場合、 安全じゃない Just push – not safe when Slow Subscriber ! ! プル一方 - 速度早いサブスクライバーの場合、 遅すぎる Just pull – too slow when Fast Subscriber
  45. 45. Back-pressure? RS: Dynamic Push/Pull プッシュ一方 - 速度が遅いサブスクライバーの場合、 安全じゃない Just push – not safe when Slow Subscriber ! ! プル一方 - 速度早いサブスクライバーの場合、 遅すぎる Just pull – too slow when Fast Subscriber ! Solution: 動的な調整(リアクティブストリーム) Dynamic adjustment (Reactive Streams)
  46. 46. Back-pressure? RS: Dynamic Push/Pull 速度が遅いサブスクライバーは自分のバッ ファーが3つの要素まで受け取られる。パブリッ シャーはサブスクライバーのバッファーが溢れ るほどデータを送らない。Slow Subscriber sees it’s buffer can take 3 elements. Publisher will never blow up it’s buffer.
  47. 47. Back-pressure? RS: Dynamic Push/Pull 速度が速いパブリッシャーは最大でも3つの要 素を送る。これは、プル型のback-pressure. Fast Publisher will send at-most 3 elements. This is pull-based-backpressure.
  48. 48. Back-pressure? RS: Dynamic Push/Pull 速度が速いサブスクライバーは、実際のデータ が届く前に、たくさんリクエストを送ることが できる!Fast Subscriber can issue more Request(n), before more data arrives!
  49. 49. Back-pressure? RS: Dynamic Push/Pull パブリッシャーはサブスクライバーの全てのリ クエストを溜める. Fast Subscriber can issue more Request(n), before more data arrives. Publisher can accumulate demand.
  50. 50. Back-pressure? RS: Accumulate demand パブリッシャーはサブスクライバーの全ての リクエストを溜める. Publisher accumulates total demand per subscriber.
  51. 51. Back-pressure? RS: Accumulate demand 溜まった要素をパブリッシュすることは安全だ。 サブスクライバーのバッファーは溢れない。 Total demand of elements is safe to publish. Subscriber’s buffer will not overflow.
  52. 52. Back-pressure? RS: Requesting “a lot” 速度が速いサブスクライバーは、パブリッ シャーに対し要求が多くても良い。これは「パ ブリッシャー・プッシュ」だ。バッファー容量 は知られているので、安全だ。
  53. 53. Back-pressure? RS: Requesting “a lot” Fast Subscriber, can request “a lot” from Publisher. This is effectively “publisher push”, and is really fast. Buffer size is known and this is safe.
  54. 54. Back-pressure? RS: Dynamic Push/Pull MAX速度
  55. 55. Back-pressure? RS: Dynamic Push/Pull 安全!溢れ ること絶対 ない! MAX速度
  56. 56. わなにですか?
  57. 57. Akka = アッカ Akka is a high-performance concurrency library for Scala and Java. ! At it’s core it focuses on the Actor Model:
  58. 58. Akka = アッカ Akka is a high-performance concurrency library for Scala and Java. ! At it’s core it focuses on the Actor Model: アクターができること: • メッセージを送信・受信する (Send / receive messages) • アクターを作る (Create Actors) • 自分の動作を変える (Change it’s behaviour)
  59. 59. Akka Akka has multiple modules: ! Akka-camel: integration Akka-remote: remote actors Akka-cluster: clustering Akka-persistence: CQRS / Event Sourcing Akka-streams: stream processing …
  60. 60. Akka Streams 0.7 early preview
  61. 61. Akka Streams – Linear Flow
  62. 62. Akka Streams – Linear Flow
  63. 63. Akka Streams – Linear Flow
  64. 64. Akka Streams – Linear Flow
  65. 65. Akka Streams – Linear Flow FlowFrom[Double].map(_.toInt). [...] ソースはまだ付けられていない. パイプはダブルを処理する準備を整いた. No Source attached yet. “Pipe ready to work with Doubles”.
  66. 66. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys")! ! アクターが住んでいる世界. AkkaStreamsはアクターを使うため、 アクターシステムは必要. ! ActorSystem is the world in which Actors live in. AkkaStreams uses Actors, so it needs ActorSystem.
  67. 67. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys")! implicit val mat = FlowMaterializer()! ストリームをどうやって具体化するかの ロジックが含まれている. ! Contains logic on HOW to materialise the stream.
  68. 68. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys")! implicit val mat = FlowMaterializer()! 単純にアクターか、 もしくは実装されていれば、 Apache Spark (?!) ! A materialiser can choose HOW to materialise, it could even use Apache Spark (?!) if someone would implement that… :-)
  69. 69. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys")! implicit val mat = FlowMaterializer()! バッファーの容量を設定できるなど ! You can configure it’s buffer sizes etc.
  70. 70. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys")! implicit val mat = FlowMaterializer()! val foreachSink = ForeachSink[Int](println)! val mf = FlowFrom(1 to 3).withSink(foreachSink).run() Uses the implicit FlowMaterializer
  71. 71. Akka Streams – Linear Flow implicit val sys = ActorSystem("tokyo-sys")! implicit val mat = FlowMaterializer()! val foreachSink = ForeachSink[Int](println)! val mf = FlowFrom(1 to 3).withSink(foreachSink).run()(mat)
  72. 72. Akka Streams – Linear Flow val mf = FlowFrom[Int].! map(_ * 2).! withSink(ForeachSink(println)) // needs source,! // can NOT run 走らせるためにソースが必要!
  73. 73. Akka Streams – Linear Flow val f = FlowFrom[Int].! map(_ * 2).! ! ! ! withSink(ForeachSink(i => println(s"i = $i”))).! ! ! // needs Source to run! インプットが必要 走らせるためにソースが必要!
  74. 74. Akka Streams – Linear Flow val f = FlowFrom[Int].! map(_ * 2).! ! ! ! withSink(ForeachSink(i => println(s"i = $i”))).! ! ! // needs Source to run!
  75. 75. Akka Streams – Linear Flow val f = FlowFrom[Int].! map(_ * 2).! ! ! ! withSink(ForeachSink(i => println(s"i = $i”))).! ! ! // needs Source to run!
  76. 76. Akka Streams – Linear Flow val f = FlowFrom[Int].! map(_ * 2).! ! ! ! withSink(ForeachSink(i => println(s"i = $i”))).! ! ! // needs Source to run! ! ! ! ! f.withSource(IterableSource(1 to 10)).run() 準備完了!
  77. 77. Akka Streams – Linear Flow val f = FlowFrom[Int].! map(_ * 2).! ! ! ! withSink(ForeachSink(i => println(s"i = $i”))).! ! ! // needs Source to run! ! ! ! ! f.withSource(IterableSource(1 to 10)).run() 準備完了!
  78. 78. Akka Streams – Flows are reusable ! ! ! ! f.withSource(IterableSource(1 to 10)).run()! ! ! ! f.withSource(IterableSource(1 to 100)).run()! ! ! ! f.withSource(IterableSource(1 to 1000)).run()
  79. 79. Akka Streams <-> Actors – Advanced val subscriber = system.actorOf(Props[SubStreamParent], ”parent")! ! FlowFrom(1 to 100).! map(_.toString).! filter(_.length == 2).! drop(2).! groupBy(_.last).! publishTo(ActorSubscriber(subscriber))!
  80. 80. Akka Streams <-> Actors – Advanced val subscriber = system.actorOf(Props[SubStreamParent], ”parent")! ! FlowFrom(1 to 100).! map(_.toString).! filter(_.length == 2).! drop(2).! groupBy(_.last).! publishTo(ActorSubscriber(subscriber))! 各グループもストリームだよ! Each “group” is a stream too! It’s a “Stream of Streams”.
  81. 81. Akka Streams <-> Actors – Advanced ! groupBy(_.last). 「11」をグループ「1」に、「12」をグループ「2」になど GroupBy groups “11” to group “1”, “12” to group “2” etc.
  82. 82. Akka Streams <-> Actors – Advanced ! groupBy(_.last). サブスクライバーに[グループキー、サブストリームフロー]を提供する It offers (groupKey, subStreamFlow) to Subscriber
  83. 83. Akka Streams <-> Actors – Advanced ! groupBy(_.last). 子供を起動させ、サブーフローを扱わせる It can then start children, to handle the sub-flows!
  84. 84. Akka Streams <-> Actors – Advanced ! groupBy(_.last). 例えば、グループ毎に1人の子供 For example, one child for each group.
  85. 85. Akka Streams <-> Actors – Advanced val subscriber = system.actorOf(Props[SubStreamParent], ”parent")! ! FlowFrom(1 to 100).! map(_.toString).! filter(_.length == 2).! drop(2).! groupBy(_.last).! publishTo(ActorSubscriber(subscriber))! 普通 Akka Actor, will consume SubStream offers.
  86. 86. Akka Streams <-> Actors – Advanced class SubStreamParent extends ActorSubscriber ! with ImplicitFlowMaterializer ! with ActorLogging {! ! override def requestStrategy = OneByOneRequestStrategy! ! override def receive = {! case OnNext((groupId: String, subStream: FlowWithSource[_, _])) =>! ! val subSub = context.actorOf(Props[SubStreamSubscriber], ! s"sub-$groupId")! subStream.publishTo(ActorSubscriber(subSub))! }! }!
  87. 87. Akka Streams <-> Actors – Advanced class SubStreamParent extends ActorSubscriber ! with ImplicitFlowMaterializer ! with ActorLogging {! ! override def requestStrategy = OneByOneRequestStrategy! ! override def receive = {! case OnNext((groupId: String, subStream: FlowWithSource[_, _])) =>! ! val subSub = context.actorOf(Props[SubStreamSubscriber], ! s"sub-$groupId")! subStream.publishTo(ActorSubscriber(subSub))! }! }!
  88. 88. Akka Streams <-> Actors – Advanced class SubStreamParent extends ActorSubscriber ! with ImplicitFlowMaterializer ! with ActorLogging {! ! override def requestStrategy = OneByOneRequestStrategy! ! override def receive = {! case OnNext((groupId: String, subStream: FlowWithSource[_, _])) =>! ! val subSub = context.actorOf(Props[SubStreamSubscriber], ! s"sub-$groupId")! subStream.publishTo(ActorSubscriber(subSub))! }! }!
  89. 89. Akka Streams <-> Actors – Advanced class SubStreamParent extends ActorSubscriber ! with ImplicitFlowMaterializer ! with ActorLogging {! ! override def requestStrategy = OneByOneRequestStrategy! ! override def receive = {! case OnNext((groupId: String, subStream: FlowWithSource[_, _])) =>! ! val subSub = context.actorOf(Props[SubStreamSubscriber], ! s"sub-$groupId")! subStream.publishTo(ActorSubscriber(subSub))! }! }!
  90. 90. Akka Streams <-> Actors – Advanced class SubStreamParent extends ActorSubscriber ! with ImplicitFlowMaterializer ! with ActorLogging {! ! override def requestStrategy = OneByOneRequestStrategy! ! override def receive = {! case OnNext((groupId: String, subStream: FlowWithSource[_, _])) =>! ! val subSub = context.actorOf(Props[SubStreamSubscriber], ! s"sub-$groupId")! subStream.publishTo(ActorSubscriber(subSub))! }! }!
  91. 91. Akka Streams <-> Actors – Advanced class SubStreamParent extends ActorSubscriber {! ! override def requestStrategy = OneByOneRequestStrategy! ! override def receive = {! case OnNext(n: String) => println(s”n = $n”) ! }! }!
  92. 92. Akka Streams – GraphFlow GraphFlow
  93. 93. Akka Streams – GraphFlow Linear Flows or non-akka pipelines Could be another RS implementation!
  94. 94. Akka Streams – GraphFlow Fan-out elements and Fan-in elements
  95. 95. Akka Streams – GraphFlow Fan-out elements and Fan-in elements Now you need a FlowGraph
  96. 96. Akka Streams – GraphFlow // first define some pipeline pieces! val f1 = FlowFrom[Input].map(_.toIntermediate)! val f2 = FlowFrom[Intermediate].map(_.enrich)! val f3 = FlowFrom[Enriched].filter(_.isImportant)! val f4 = FlowFrom[Intermediate].mapFuture(_.enrichAsync)! ! // then add input and output placeholders! val in = SubscriberSource[Input]! val out = PublisherSink[Enriched]!
  97. 97. Akka Streams – GraphFlow
  98. 98. Akka Streams – GraphFlow val b3 = Broadcast[Int]("b3")! val b7 = Broadcast[Int]("b7")! val b11 = Broadcast[Int]("b11")! val m8 = Merge[Int]("m8")! val m9 = Merge[Int]("m9")! val m10 = Merge[Int]("m10")! val m11 = Merge[Int]("m11")! val in3 = IterableSource(List(3))! val in5 = IterableSource(List(5))! val in7 = IterableSource(List(7))!
  99. 99. Akka Streams – GraphFlow
  100. 100. Akka Streams – GraphFlow // First layer! in7 ~> b7! b7 ~> m11! b7 ~> m8! ! in5 ~> m11! ! in3 ~> b3! b3 ~> m8! b3 ~> m10!
  101. 101. Akka Streams – GraphFlow ! // Second layer! m11 ~> b11! b11 ~> FlowFrom[Int].grouped(1000) ~> resultFuture2 ! b11 ~> m9! b11 ~> m10! ! m8 ~> m9!
  102. 102. Akka Streams – GraphFlow ! // Third layer! m9 ~> FlowFrom[Int].grouped(1000) ~> resultFuture9! m10 ~> FlowFrom[Int].grouped(1000) ~> resultFuture10!
  103. 103. Akka Streams – GraphFlow ! // Third layer! m9 ~> FlowFrom[Int].grouped(1000) ~> resultFuture9! m10 ~> FlowFrom[Int].grouped(1000) ~> resultFuture10!
  104. 104. Akka Streams – GraphFlow ! // Third layer! m9 ~> FlowFrom[Int].grouped(1000) ~> resultFuture9! m10 ~> FlowFrom[Int].grouped(1000) ~> resultFuture10!
  105. 105. Akka Streams – GraphFlow Sinks and Sources are “keys” which can be addressed within the graph val resultFuture2 = FutureSink[Seq[Int]]! val resultFuture9 = FutureSink[Seq[Int]]! val resultFuture10 = FutureSink[Seq[Int]]! ! val g = FlowGraph { implicit b =>! // ...! m10 ~> FlowFrom[Int].grouped(1000) ~> resultFuture10! // ...! }.run()! ! Await.result(g.getSinkFor(resultFuture2), 3.seconds).sorted! should be(List(5, 7))
  106. 106. Akka Streams – GraphFlow Sinks and Sources are “keys” which can be addressed within the graph val resultFuture2 = FutureSink[Seq[Int]]! val resultFuture9 = FutureSink[Seq[Int]]! val resultFuture10 = FutureSink[Seq[Int]]! ! val g = FlowGraph { implicit b =>! // ...! m10 ~> FlowFrom[Int].grouped(1000) ~> resultFuture10! // ...! }.run()! ! Await.result(g.getSinkFor(resultFuture2), 3.seconds).sorted! should be(List(5, 7))
  107. 107. Akka Streams – GraphFlow ! val g = FlowGraph {}! FlowGraphは不変で、安全に共有でき、 何度も使いまわせる! ! FlowGraph is immutable and safe to share and re-use! Think of it as “the description” which then gets “run”.
  108. 108. Available Elements 0.7 early preview
  109. 109. Available Sources • FutureSource • IterableSource • IteratorSource • PublisherSource • SubscriberSource • ThunkSource • TickSource (timer based) • 簡単に自分のものを追加できる! … easy to add your own! 0.7 early preview
  110. 110. Available operations • buffer • collect • concat • conflate • drop / dropWithin • take / takeWithin • filter • fold • foreach • groupBy • grouped • map • onComplete • prefixAndTail • broadcast • merge / “generalised merge” • zip • 自分のオペレーションを追加する事は可能! … possible to add your own! 0.7 early preview
  111. 111. Available Sinks • BlackHoleSink • FoldSink • ForeachSink • FutureSink • OnCompleteSink • PublisherSink / FanoutPublisherSink • SubscriberSink • 簡単に自分のものを追加できる! … easy to add your own! 0.7 early preview
  112. 112. Spray => Akka-Http && ReactiveStreams Spray is now merged into Akka, as Akka-Http Works on Reactive Streams Streaming end-to-end!
  113. 113. Links • http://akka.io • http://reactive-streams.org • https://groups.google.com/group/akka-user <- ask questions here! • http://akka.io/news/2014/09/12/akka-streams-0.7-released.html
  114. 114. ありがとう ございました! Ask questions, get Stickers! http://akka.io ktoso @ typesafe.com twitter: ktosopl github: ktoso team blog: letitcrash.com
  115. 115. ©Typesafe 2014 – All Rights Reserved

×