Implementation of 'go-like' language constructions in scala [english version]

584 views

Published on

Presentation about implementation of go-like language constructions in scala: https://github.com/rssh/scala-gopher

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
584
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Implementation of 'go-like' language constructions in scala [english version]

  1. 1. Go / Scala Implementation of go language constructions in scala Ruslan Shevchenko <ruslan@shevchenko.kiev.ua> https://github.com/rssh/scala-gopher
  2. 2. Whats in go ?  scope  ( defer/ panic / destroy )  parallelism  channels  select statement  Differences from 'native' scala approach  How to implement go variant Future directions...
  3. 3. Scope def copy(inf: File, outf: File): Long = goScope { val in = new FileInputStream(inf) defer{ in.close() } val out = new FileOutputStream(outf); defer{ out.close() } out.getChannel() transferFrom(in.getChannel(), 0, Long.MaxValue) } def copy(inf: File, outf: File): Long = { val in = new FileInputStream(inf) try { val out = new FileOutputStream(outf) try { out.getChannel() transferFrom(in.getChannel(), 0, Long.MaxValue) } finally { out.close(); } } finally { in.close(); } } Old way:
  4. 4. Traditional way: def copy(inf: File, outf: File): Long = { val in = new FileInputStream(inf) try { val out = new FileOutputStream(out) try { out.getChannel() transferFrom(in.getChannel(), 0,Long.MaxValue) } finally { out.close(); } } finally { in.close(); } }
  5. 5. def copy(inf: File, outf: File): Long = goScope { val in = new FileInputStream(inf) defer{ in.close() } val out = new FileOutputStream(outf); defer{ out.close() } out.getChannel() transferFrom(in.getChannel(), 0, Long.MaxValue) } Go way
  6. 6. try try try finally finally finally op/defer
  7. 7. How this done is scala:  Macroses  Call-by-name-params  Traditional try/cath goScope = .... (macro wich rewrite arg) defer => @compileTimeOnly
  8. 8. def copy(inf: File, outf: File): Long = { val sc0 = new ScopeContext(); sc0.eval { val in = new FileInputStream(inf) sc0.pushDefer{ in.close() } val out = new FileOutputStream(outf); sc0.pushDefer{ out.close() } out.getChannel() transferFrom(in.getChannel(), 0, Long.MaxValue) } sc0.unwindDefers[Long]() } Unsugared:
  9. 9. Scope: defer / panic / recover  panic => throw new PanicException()  Recover => return try { .... } catch { case ex: Exception => return x } defer{ ......... recover(x) }
  10. 10. Channels Read: go: channel -> x Scala: channel ~> x x = channel ? ?! ?? Write go: channel <- x Scala: channel <~ x channel ! x !! (immediatly) !? (with timeout)
  11. 11. Select Channels connect execution flows
  12. 12. go func() { for { select { case msg1 := <- c1: fmt.Println(msg1) case msg2 := <- c2: fmt.Println(msg2) } } }() Go: typical pattern: select inside for inside goroutine
  13. 13. go for(s <- select) s match { case c1 ~> (msg1: Msg1Type) => println(msg1) case c2 ~>(msg2:Msg2Type) => println(msg2) } Scala Analog: - implemented as foreach macros - ~> pattern: require typing - active queue Open question: native non-macro syntax
  14. 14. Unsugared (simplicified) [1]: go { val s = new SelectContext(); s.addReader(c1) { (msg1:Msg1Type) => println(msg1) } s.addReader(c2) { (msg2:Msg2Type) => println(msg2) } s.run() }
  15. 15. Unsugared (simplicified) [2]: val scope = new ScopeContext(); val s = new SelectContext(); Future { s.addReader(c1) { (msg1:Msg1Type) => println(msg1) } s.addReader(c2) { (msg2:Msg2Type) => println(msg2) } } map { s.runUnblocked() } map { scope.finish() } (Future work) S
  16. 16. Just select: for(s <- select.once) s match { case c1 ~> (msg1: Msg1Type) => println(msg1) case c2 ~>(msg2:Msg2Type) => println(msg2) } select { case msg1 := <- c1: fmt.Println(msg1) case msg2 := <- c2: fmt.Println(msg2) } GO SCALA
  17. 17. Internal implementation:  Active channel  Channel = [Blocked Query + Execution Context ]  Coordination primitives + wait for select  Count down latch  Memory barrier  Extensions over Go model  async operations  context shutdown.
  18. 18. val consumer = go { for(s <- select) { s match { case `channel` ~> (i:Int) => sum = sum + i if (i==1000) s.shutdown() } } sum } Future[Int]
  19. 19. Akka (Erlang-like actors) Channels (go-like)  Reactive  No blocking operations  Request/Reply paradigm.  Mailbox overflow on overload  Handle something.  Flow oriented  Wait is normal (switch during wait)  One-direction pipe paradigm  wait on send On overload  Compute something
  20. 20. x1, x2, x3, ............ x1*y1, x2*y2, x3*y3, ............ y1, y2, y3, ............ go { while(;) { val x = channelX ? val y = channelY ? channelZ ! x*y } } Example: Easy with channels, hard with Akka
  21. 21. bindWrite bindRead ? [blocking operation]
  22. 22. Future directions  Redefining of base primitivies [select in loop in go] is too fundamental, to be combination of 3 constructions.  Native non-macro syntax  Make scala type inference works  Pluggable implementations  Optimize corner cases
  23. 23. Pluggable implementations.  Exists many strategies. (current implemented is not best)  Schedule/callback instead waits  Play with own dispatcher  .....  Performance tests,  Implement/Prototype/Test loop,  Problems:  Lack of import-based configuration in scala  @exported import  Community help needed.
  24. 24. Rewriting blocking API into reactive - like CPS (continuations passing style)
  25. 25. go { ... val x = channelX ? val y = channelY ? channelZ ! x*y ... } go { ..... channelX.read map { x => channelY.read map { y => channelZ.write map { ch => ch <~! x*y } } } } }
  26. 26. someCollection foldLeft { (s,x) => s + (x* channel ?) } Problem – same as with CPS: - foldLeft – implemented not here. - foreach for(x <- something) channel <~~ x Continuations not supported directly by JVM
  27. 27. Continuations in JVM - Patched JVM - Byte-code rewriting. - Do nothing – thread pools now big (utilize wait by running Flow of non-blocking tasks here)
  28. 28. The Da Vinci machine http://openjdk.java.net/projects/mlvm/index.html AVIAN VM http://oss.readytalk.com/avian/
  29. 29. Someday everything will be fine. For now - here is a cat
  30. 30. Continuations: Bytecode rewriting. Frames: Let's save frame during function end, instead deleting one. foldLeft ? channel + *
  31. 31. Continuations: Bytecode rewriting. Frames: foldLeft + *
  32. 32. Continuations: Bytecode rewriting. Frames: Suspended CPS transformation on bytecode level foldLeft + * Channel ?
  33. 33. Community help needed. https://github.com/rssh/scala-gopher  Build examples of typical usage.  Instrument ones for performance tests  Provide better implementations. Let's think together. Thanks for attention. Ruslan Shevchenko <Ruslan@Shevchenko.Kiev.UA>

×