SCALA - organizing control
flow (between imperative
and declarative approaches)
Ruslan Shevchenko
GoSave
!
ruslan@shevchen...
Styles of control flow organizations:
!
• Future-s …
• async/await
• Actors
• Channels
• Reactive Collections
• DSLs for C...
Control Flow
val x = readX
val y = readY
val z = x*y
scala
X
Y
Z
Imperative = we explicitly set one
directed by evaluation strategy
z = x*y
where x = readX
y = readX
haskell
X Y
Z
let x = readX
y = readX
in x*y
Control Flow:
!
• Imperative/Declarative (?)
!
• Declarative + Understanding =
Imperative
Control flow: what we need ?
!
• Manage multi{core,machine} control-
flows.
!
• Optimize resource utilization
( see reacti...
Reactivity
!
• Ugly situation in industry
!
• In ideal world - {operation system/
language VM} must care about resource
ut...
Control Flow
val x = readX
val y = readY
val z = x*y
Can we readX and Y in parallel
X Y
Z
Low level
@volatile var x = _
val xThread = new Thread(
public void run() {
x = readX
}
).start();
!
@volatile var y = _
v...
Low level with thread pool
val xTask = new Task(
public X run() {
readX
}
);
pool.submit(xTask)
!
val yTask = new Task(
pu...
Scala Future
X Y
Z
val x = Future{ readX }
val y = Future{ readY }
val z = Await.result(x, 1 minute) + Await.result(y, 1 m...
Future
!
• Future[T] = pointer to the value in the
future
!
• isComplete: Boolean
!
• Await.result(feature, duration): T
!...
Future { readX }
object Future
{
!
def apply[T](body: =>T) : Future[T] =
…………
!
}
Call by name syntax = [call by function ...
Future-s are composable.
!
• map: X=>Y Future[X] => Future[Y]
!
• Future[X].map[Y](f:X=>Y): Future[Y]
!
• flatMap:
• X => ...
Future-s are composable: map
!
• Future[X].map[Y](f:X=>Y): Future[Y]
!
Future{ calculatePi() } map ( _ + 1) =
!
Future{ ca...
Future-s are composable: flatMap
!
• Future[X]:
• flatMap[Y](f:X=>Future[Y]): Future[Y]
!
Future{ calculatePi() } flatMap ...
Scala Future
val x = Future{ readX }
val y = Future{ readY }
val z = Future{ … after X and Y }
val z = Future{ readX } fla...
Scala Future
val x = Future{ readX }
val y = Future{ readY }
val z = Future{ … after X and Y }
val xFuture = Future{ readX...
Scala: monadic syntax
for{ x <- Future{ readX },
y <- Future{ readY }) yield x+y
!
Future{ readX } flatMap{ x =>
for (y <-...
Future
!
• Good for simple cases
!
• Hard when we want to implement
complex logic
!
SIP22 (Async/Await)
!
• when we want to implement complex
logic
!
• Ozz style -> (limited) F# ->(limited) C#
!
• in scala ...
SIP22 (Async/Await)
!
• async(body: =>T):Future[T]
!
• await(future: Future[T]): T
• can be used only inside async
!
• asy...
Async/Await
val x = Future{ readX }
val y = Future{ readY }
val z = Future{ … after X and Y }
val z = async{
val x = futur...
Async/Await
val z = async{
val x = Future{ readX }
val y = Future{ readY }
await(x) + await(y)
}
val z = {
var state=0, aw...
Async/Await
val z = {
var state=0, awaitX=false, awaitY=false
val res = Promise[X]()
def f = state match {
case 0 =>
state...
Async/Await
!
• Good for relative complex logic
!
• No support for awaits inside closures
inside async block.
!
• Still lo...
Akka http://www.akka.io
!
• Erlang-style concurrency
!
• Actor - active object
• available by name in akka cluster.
• send...
Actor
Mailbox
Processor (with state) 1
Akka
!
• tell(x:Any)=>Unit — send and forget
actor ! x
!
!
• ack(x:Any)=>Future[Any] — send and
receive future to answer
a...
Actor
!
class EventsProcessor extends Actor
{
var nMessages=0
!
def receive = {
case Event(msg) => println(msg)
nMessages+...
Akka
!
• Actor Supervising (restart if fail)
!
• Utils (Scheduler, EventBus, … )
!
• Common patterns
• Load balancing
• Th...
Akka
!
• Scale on cluster
!
• Persistent Queue/ Actor State
!
• Optional monitoring console
(commercial)
!
!
Akka : Differences from Erlang Model
!
• No blocking inside actors. [Use
additional tread-pool]
!
• Scheduler switch to ot...
Go-like channels
!
• Come from Go language
• http://golang.org/
!
!
Go-like channels
!
• Bounded Queue
!
• coroutines (different threads) can
• write to queue (wait if full)
• read from queu...
Go-like channels
!
• Implemented as library on top of Akka
and SIP22
!
• Fully async (blocked reads must be in
async block...
Rx streams
!
• Reactive extensions
• http://rxscala.github.io/
!
• Rx collection call you.
!
Rx scala
!
trait Observer[E]
{
def onNext(e: E)
def onError(e: Throwable)
def onCompleted()
!
}
!
!
!
trait Observable[E]
...
Rx scala: Observer/Iterator duality
!
trait Observer[-E]
{
def onNext(e: E)
def onError(e: Throwable)
def onCompleted()
!
...
Rx streams
!
• Use - if event-source provide this format.
!
• Typical pattern - collect event-sources
into you collection ...
Computation plan DSL
!
• Collections operations can be distributed
!
• Simple form: .par
for( x <- collection.par) yield x...
Computation plan DSL for Hadoop
!
• Scalding
• https://github.com/twitter/scalding
!
• Scoobi
• http://nicta.github.io/sco...
Scoobi
!
val lines = fromTextFile("hdfs://in/...")
!
val counts = lines.mapFlatten(_.split(" "))
.map(word => (word, 1))
....
Mahout: computation plan DSL for Spark
!
• https://mahout.apache.org/
!
• Scalable machine learning library.
• R-like matr...
Mahout
!
// R-like operations (linear algebra)
!
val g = bt.t %*% bt - c - c.t + (s_q cross s_q) * (xi dot xi)
!
drmA.mapB...
Mahout
!
val inCoreA = dense(!
(1, 2, 3, 4),!
(2, 3, 4, 5),!
(3, -4, 5, 6),!
(4, 5, 6, 7),!
(8, 6, 7, 8)!
)
val A = drmPar...
Scala: organization of control flow
!
• Many styles. No one is better.
!
• Low-level: futures & callbacks
!
• Middle-level...
Scala: organization of control flow
• Possibilities:
• Flexible syntax
• Call-by-name
• Macroses
!
• Limitations:
• JVM
• ...
Scala: organization of control flow
!
• Thanks for attention.
!
• Questions (?)
!
• //Ruslan Shevchenko @rssh1
<ruslan@she...
Upcoming SlideShare
Loading in …5
×

Ruslan.shevchenko: most functional-day-kiev 2014

598 views

Published on

Lecture about available methods for scala control-flow organization on
http://frameworksdays.com/event/most-functional-day

Published in: Software
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
598
On SlideShare
0
From Embeds
0
Number of Embeds
34
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Ruslan.shevchenko: most functional-day-kiev 2014

  1. 1. SCALA - organizing control flow (between imperative and declarative approaches) Ruslan Shevchenko GoSave ! ruslan@shevchenko.kiev.ua @rssh1 https://github.com/rssh
  2. 2. Styles of control flow organizations: ! • Future-s … • async/await • Actors • Channels • Reactive Collections • DSLs for Computation Plans
  3. 3. Control Flow val x = readX val y = readY val z = x*y scala X Y Z Imperative = we explicitly set one
  4. 4. directed by evaluation strategy z = x*y where x = readX y = readX haskell X Y Z let x = readX y = readX in x*y
  5. 5. Control Flow: ! • Imperative/Declarative (?) ! • Declarative + Understanding = Imperative
  6. 6. Control flow: what we need ? ! • Manage multi{core,machine} control- flows. ! • Optimize resource utilization ( see reactive manifesto http://www.reactivemanifesto.org/ )
  7. 7. Reactivity ! • Ugly situation in industry ! • In ideal world - {operation system/ language VM} must care about resource utilization, human - about logic ! • Our world is far from ideal
  8. 8. Control Flow val x = readX val y = readY val z = x*y Can we readX and Y in parallel X Y Z
  9. 9. Low level @volatile var x = _ val xThread = new Thread( public void run() { x = readX } ).start(); ! @volatile var y = _ val yThread = new Thread( public void run() { y = readY } ).start(); yThread.join() xThread.join() z = x+y As GOTO from 60-s
  10. 10. Low level with thread pool val xTask = new Task( public X run() { readX } ); pool.submit(xTask) ! val yTask = new Task( public Y run() { readY } ) pool.submit(yTask) ! z = xTask.get()+yTask.get() X Y Z
  11. 11. Scala Future X Y Z val x = Future{ readX } val y = Future{ readY } val z = Await.result(x, 1 minute) + Await.result(y, 1 minute)
  12. 12. Future ! • Future[T] = pointer to the value in the future ! • isComplete: Boolean ! • Await.result(feature, duration): T ! • onComplete(Try[T] => U) : Unit
  13. 13. Future { readX } object Future { ! def apply[T](body: =>T) : Future[T] = ………… ! } Call by name syntax = [call by function .. ] by name - term from Algol 68 Own functions like control flow constructions
  14. 14. Future-s are composable. ! • map: X=>Y Future[X] => Future[Y] ! • Future[X].map[Y](f:X=>Y): Future[Y] ! • flatMap: • X => Future[Y] Future[X]=>Future[Y] ! • Future[X].flatMap[Y](f: x => Future[Y]):Future[Y]
  15. 15. Future-s are composable: map ! • Future[X].map[Y](f:X=>Y): Future[Y] ! Future{ calculatePi() } map ( _ + 1) = ! Future{ calculatePi() } map (x => x+1) = ! Future{ => 4.1415926 }
  16. 16. Future-s are composable: flatMap ! • Future[X]: • flatMap[Y](f:X=>Future[Y]): Future[Y] ! Future{ calculatePi() } flatMap ( calculateExp(_) ) = ! Future{ => e^pi … }
  17. 17. Scala Future val x = Future{ readX } val y = Future{ readY } val z = Future{ … after X and Y } val z = Future{ readX } flatMap { x => readY map ( y=> x+y ) } !
  18. 18. Scala Future val x = Future{ readX } val y = Future{ readY } val z = Future{ … after X and Y } val xFuture = Future{ readX } val yFuture = Future{ readY } val z = xFuture flatMap { x => yFuture map ( y=> x+y ) } for{ x <- Future{ readX }, y <- Future{ readY }) yield x+y ! Using monadic syntax:
  19. 19. Scala: monadic syntax for{ x <- Future{ readX }, y <- Future{ readY }) yield x+y ! Future{ readX } flatMap{ x => for (y <- Future{ readY }) yield x+y } Future{ readX } flatMap{ x => Future{ readY } map ( y => x+y ) }
  20. 20. Future ! • Good for simple cases ! • Hard when we want to implement complex logic !
  21. 21. SIP22 (Async/Await) ! • when we want to implement complex logic ! • Ozz style -> (limited) F# ->(limited) C# ! • in scala as library: https://github.com/ scala/async !
  22. 22. SIP22 (Async/Await) ! • async(body: =>T):Future[T] ! • await(future: Future[T]): T • can be used only inside async ! • async macro rewrite body as state machine. !
  23. 23. Async/Await val x = Future{ readX } val y = Future{ readY } val z = Future{ … after X and Y } val z = async{ val x = future{ readX } val y = future{ readY } await(x) + await(y) }
  24. 24. Async/Await val z = async{ val x = Future{ readX } val y = Future{ readY } await(x) + await(y) } val z = { var state=0, awaitX=false, awaitY=false def f = state match { case 0 => state = 1 Future{ x=readX(); awaitX=true } onComplete f Future{ x=readY(); awaitY=true} onComplete f case 1 // Just show the idea, not actual
  25. 25. Async/Await val z = { var state=0, awaitX=false, awaitY=false val res = Promise[X]() def f = state match { case 0 => state = 1 Future{ x=readX(); awaitX=true } onComplete f Future{ x=readY(); awaitY=true} onComplete f case 1 => if (awaitX && awaitY) { res.successful(x + y)} } f() res.future() } // Just show the idea, not actual
  26. 26. Async/Await ! • Good for relative complex logic ! • No support for awaits inside closures inside async block. ! • Still low-level, can’t be used for organizing program structure !
  27. 27. Akka http://www.akka.io ! • Erlang-style concurrency ! • Actor - active object • available by name in akka cluster. • send message (opt. receive answer) • have local state ! ! ! !
  28. 28. Actor Mailbox Processor (with state) 1
  29. 29. Akka ! • tell(x:Any)=>Unit — send and forget actor ! x ! ! • ack(x:Any)=>Future[Any] — send and receive future to answer actor ? x ! !
  30. 30. Actor ! class EventsProcessor extends Actor { var nMessages=0 ! def receive = { case Event(msg) => println(msg) nMessages+=1 case Echo(msg) => sender ! Echo(msg) case Ping => sender ! Pong case Stop => context.stop(self) } ! }
  31. 31. Akka ! • Actor Supervising (restart if fail) ! • Utils (Scheduler, EventBus, … ) ! • Common patterns • Load balancing • Throttling messages • ….. ! !
  32. 32. Akka ! • Scale on cluster ! • Persistent Queue/ Actor State ! • Optional monitoring console (commercial) ! !
  33. 33. Akka : Differences from Erlang Model ! • No blocking inside actors. [Use additional tread-pool] ! • Scheduler switch to other actor after processing <N> messages (<N> instructions in Erlang) ! • Supervising is more robust. ! !
  34. 34. Go-like channels ! • Come from Go language • http://golang.org/ ! !
  35. 35. Go-like channels ! • Bounded Queue ! • coroutines (different threads) can • write to queue (wait if full) • read from queue (wait if full) • select/wait one from possible operations
  36. 36. Go-like channels ! • Implemented as library on top of Akka and SIP22 ! • Fully async (blocked reads must be in async block) ! • (yet not ready) ;))) https://github.com/rssh/scala-gopher branch “async/unsugared.”
  37. 37. Rx streams ! • Reactive extensions • http://rxscala.github.io/ ! • Rx collection call you. !
  38. 38. Rx scala ! trait Observer[E] { def onNext(e: E) def onError(e: Throwable) def onCompleted() ! } ! ! ! trait Observable[E] { ! def subscribe(o:Observer) ! ………. ! map, flatMap, zip, filter … } !
  39. 39. Rx scala: Observer/Iterator duality ! trait Observer[-E] { def onNext(e: E) def onError(e: Throwable) def onCompleted() ! } ! ! ! trait Iterator[+E] { ! def next: E // throw …. def hasNext() ! } !
  40. 40. Rx streams ! • Use - if event-source provide this format. ! • Typical pattern - collect event-sources into you collection via hight-level operations, than process. !
  41. 41. Computation plan DSL ! • Collections operations can be distributed ! • Simple form: .par for( x <- collection.par) yield x+1 ! ! • Same idea for hadoop map/reduce !
  42. 42. Computation plan DSL for Hadoop ! • Scalding • https://github.com/twitter/scalding ! • Scoobi • http://nicta.github.io/scoobi/ ! • Scrunch • http://crunch.apache.org/scrunch.html
  43. 43. Scoobi ! val lines = fromTextFile("hdfs://in/...") ! val counts = lines.mapFlatten(_.split(" ")) .map(word => (word, 1)) .groupByKey .combine(Sum.int) ! counts.toTextFile(“hdfs://out/…", overwrite=true).persist(ScoobiConfiguration()) map, groupByKey, combine => Map/Reduce tasks
  44. 44. Mahout: computation plan DSL for Spark ! • https://mahout.apache.org/ ! • Scalable machine learning library. • R-like matrix operations • Optimizer for algebraic expression
  45. 45. Mahout ! // R-like operations (linear algebra) ! val g = bt.t %*% bt - c - c.t + (s_q cross s_q) * (xi dot xi) ! drmA.mapBlock(ncol = r) { case (keys, blockA) => val blockY = blockA %*% Matrices.symmetricUniformView(n, r, omegaSeed) keys -> blockY } Match operations => computations plans on storm claster
  46. 46. Mahout ! val inCoreA = dense(! (1, 2, 3, 4),! (2, 3, 4, 5),! (3, -4, 5, 6),! (4, 5, 6, 7),! (8, 6, 7, 8)! ) val A = drmParallelize(inCoreA, numPartitions = 2) ! val inCoreB = drmB.collect In core => out core transformation
  47. 47. Scala: organization of control flow ! • Many styles. No one is better. ! • Low-level: futures & callbacks ! • Middle-level: actors, channels, streams ! • Hight-level: declarative DSL
  48. 48. Scala: organization of control flow • Possibilities: • Flexible syntax • Call-by-name • Macroses ! • Limitations: • JVM • Complex language constructions (hard to change structure.)
  49. 49. Scala: organization of control flow ! • Thanks for attention. ! • Questions (?) ! • //Ruslan Shevchenko @rssh1 <ruslan@shevchenko.kiev.ua> ! • GoSave, inc. !

×