Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

F# in real world (LambdaCon15)

2,511 views

Published on

Published in: Technology
  • Be the first to comment

F# in real world (LambdaCon15)

  1. 1. Yan Cui (@theburningmonk) BOLOGNA 28 march 2015 LambdaCon F# in the Real World
  2. 2. agenda
  3. 3. Hi, my name is Yan Cui.
  4. 4. 1MILLION USERS ACTIVE DAILY
  5. 5. 250MILLION DAY PER REQUEST
  6. 6. 2MONTH TB P E R sec ops 25,000
  7. 7. why F#?
  8. 8. time to market
  9. 9. correctness
  10. 10. efficient
  11. 11. tame complexity
  12. 12. 17th April! London
  13. 13. Collectables Wager Size Special Symbol Avg Wager Size Web Server call
  14. 14. • Line Win! – X number of matching symbols on adjacent columns! – Positions have to be a ‘line’! – Wild symbols substitute for other symbols! ! • Scatter Win! – X number of matching symbols anywhere! – Triggers bonus game
  15. 15. What symbols should land?! Any special symbol wins?! Did the player win anything?
  16. 16. What the player’s new average wager?
  17. 17. Should the player receive collectables?
  18. 18. type Symbol = Standard! of string! ! ! ! | Wild
  19. 19. type Symbol = Standard! of string! ! ! ! | Wild e.g.! Standard “hat”! ! Standard “shoe”! ! Standard “bonus”! ! …
  20. 20. type Symbol = Standard! of string! ! ! ! | Wild i.e.! Wild
  21. 21. ! ! ! type Win = LineWin of int * Symbol * int! ! | ScatterWin of Symbol * int
  22. 22. ! ! ! type Win = LineWin of int * Symbol * int! ! | ScatterWin of Symbol * int e.g.! LineWin (5, Standard “shoe”, 4)
  23. 23. ! ! ! type Win = LineWin of int * Symbol * int! ! | ScatterWin of Symbol * int e.g.! ScatterWin (Standard “bonus”, 3)
  24. 24. type LineNum = int! type Count! = int! ! type Win = LineWin of LineNum * Symbol * Count! ! | ScatterWin of Symbol * Count
  25. 25. type LineNum = int! type Count! = int! ! type Win = LineWin of LineNum * Symbol * Count! ! | ScatterWin of Symbol * Count e.g.! LineWin (5, Standard “shoe”, 4)
  26. 26. type LineNum = int! type Count! = int! ! type Win = LineWin of LineNum * Symbol * Count! ! | ScatterWin of Symbol * Count e.g.! ScatterWin (Standard “bonus”, 3)
  27. 27. make invalid states unrepresentable
  28. 28. NASA orbiter crashed because one engineer accidentally used miles instead of kilometres
  29. 29. you’re never too smart to make mistakes
  30. 30. [<Measure>]! type Pence e.g.! 42<Pence>! ! 153<Pence>! ! …
  31. 31. 10<Meter> / 2<Second> ! = 5<Meter/Second>! 10<Meter> * 2<Second> ! = 20<Meter Second> ! 10<Meter> + 10<Meter> ! = 20<Meter>! 10<Meter> * 10! ! ! = 100<Meter>! 10<Meter> * 10<Meter> ! = 100<Meter2>! 10<Meter> + 2<Second> ! // error! 10<Meter> + 2 ! ! ! // error
  32. 32. 10<Meter> / 2<Second> ! = 5<Meter/Second>! 10<Meter> * 2<Second> ! = 20<Meter Second> ! 10<Meter> + 10<Meter> ! = 20<Meter>! 10<Meter> * 10! ! ! = 100<Meter>! 10<Meter> * 10<Meter> ! = 100<Meter2>! 10<Meter> + 2<Second> ! // error! 10<Meter> + 2 ! ! ! // error
  33. 33. type Wager = int64<Pence>! type Multiplier = int! ! type Payout = Coins! of Wager! ! ! | MultipliedCoins of Multiplier * Wager! ! ! | Multi! of Payout list! ! ! | BonusGame
  34. 34. type Wager = int64<Pence>! type Multiplier = int! ! type Payout = Coins! of Wager! ! ! | MultipliedCoins of Multiplier * Wager! ! ! | Multi! of Payout list! ! ! | BonusGame
  35. 35. type Wager = int64<Pence>! type Multiplier = int! ! type Payout = Coins! of Wager! ! ! | MultipliedCoins of Multiplier * Wager! ! ! | Multi! of Payout list! ! ! | BonusGame
  36. 36. type State = ! {! AvgWager! : Wager! SpecialSymbol : Symbol! Collectables : Map<Collectable, Count>! }
  37. 37. New avg wager Got a Collectable! A pay line win!
  38. 38. Betting small reduces avg wager! Bonus Game!
  39. 39. type MonopolyBoardSpace = ! | RailRoad! ! of bool! | Property! ! of int! | ElectricCompany! of bool! | WaterCompany! of bool! | Chance! | CommunityChest! | GotoJail! | Jail! | FreeParking! | Go
  40. 40. type MonopolyBoardSpace = ! | RailRoad! ! of bool! | Property! ! of int! | ElectricCompany! of bool! | WaterCompany! of bool! | Chance! | CommunityChest! | GotoJail! | Jail! | FreeParking! | Go
  41. 41. type MonopolyBoardSpace = ! | RailRoad! ! of bool! | Property! ! of int! | ElectricCompany! of bool! | WaterCompany! of bool! | Chance! | CommunityChest! | GotoJail! | Jail! | FreeParking! | Go
  42. 42. type MonopolyBoardSpace = ! | RailRoad! ! of bool! | Property! ! of int! | ElectricCompany! of bool! | WaterCompany! of bool! | Chance! | CommunityChest! | GotoJail! | Jail! | FreeParking! | Go
  43. 43. [<Measure>]! type Position! ! type MonopolyBoard =! {! Spaces : MonopolyBoardSpace [ ]! }! member this.Item with ! get (pos : int<Position>) =! this.Spaces.[int pos]
  44. 44. [<Measure>]! type Position! ! type MonopolyBoard =! {! Spaces : MonopolyBoardSpace [ ]! }! member this.Item with ! get (pos : int<Position>) = ! this.Spaces.[int pos]
  45. 45. And a pay line win! Coin size brought over from main game Houses = multiplier on wins GAME OVER Collected in the bonus game. Gives player extra ‘lives’.
  46. 46. type BonusGameState =! {! Board ! : MonopolyBoard! Position ! : int<Position>! Lives ! : int<Life>! Doubles! : int<Double>! CoinSize! : Wager! TotalWin! : Wager! }
  47. 47. Coin size brought over from main game
  48. 48. type BonusGameState =! {! Board ! : MonopolyBoard! Position ! : int<Position>! Lives ! : int<Life>! Doubles! : int<Double>! CoinSize! : Wager! TotalWin! : Wager! }
  49. 49. And a pay line win! Houses = multiplier on wins
  50. 50. type BonusGameState =! {! Board ! : MonopolyBoard! Position ! : int<Position>! Lives ! : int<Life>! Doubles! : int<Double>! CoinSize! : Wager! TotalWin! : Wager! }
  51. 51. Collected in the bonus game. Gives player extra ‘lives’.
  52. 52. type BonusGameState =! {! Board ! : MonopolyBoard! Position ! : int<Position>! Lives ! : int<Life>! Doubles! : int<Double>! CoinSize! : Wager! TotalWin! : Wager! }
  53. 53. 0<Position>
  54. 54. let jail = 6<Position>
  55. 55. let rec move newPos state =! match state.Board.[newPos] with! …! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! …
  56. 56. let rec move newPos state =! match state.Board.[newPos] with! …! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! …
  57. 57. let rec move newPos state =! match state.Board.[newPos] with! …! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! …
  58. 58. let rec move newPos state =! match state.Board.[newPos] with! …! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! …
  59. 59. let rec move newPos state =! match state.Board.[newPos] with! …! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! …
  60. 60. let rec move newPos state =! match state.Board.[newPos] with! …! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! …
  61. 61. let rec move newPos state =! match state.Board.[newPos] with! …! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! …
  62. 62. let rec move newPos state =! match state.Board.[newPos] with! …! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! …
  63. 63. let rec move newPos state =! match state.Board.[newPos] with! …! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! …
  64. 64. let rec move newPos state =! match state.Board.[newPos] with! …! | RailRoad true ! -> awardRailRoadMultiplier state! | Property n when n > 0 -> awardPropertyMultiplier n state! | ElectricityCompany true! | WaterCompany true -> awardUtilityMultiplier state! …
  65. 65. let rec move newPos state =! match state.Board.[newPos] with! …! | CommunityChest! -> playCommunityChestCard state! | Chance ! ! -> playChanceCard state! …
  66. 66. let rec move newPos state =! match state.Board.[newPos] with! | RailRoad true ! -> awardRailRoadMultiplier state! | Property n when n > 0 -> awardPropertyMultiplier n state! | ElectricityCompany true! | WaterCompany true -> awardUtilityMultiplier state! | CommunityChest! -> playCommunityChestCard state! | Chance ! ! -> playChanceCard state! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! | _ ! ! ! -> state
  67. 67. let rec move newPos state =! match state.Board.[newPos] with! | RailRoad true ! -> awardRailRoadMultiplier state! | Property n when n > 0 -> awardPropertyMultiplier n state! | ElectricityCompany true! | WaterCompany true -> awardUtilityMultiplier state! | CommunityChest! -> playCommunityChestCard state! | Chance ! ! -> playChanceCard state! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! | _ ! ! ! -> state
  68. 68. let rec move newPos state =! match state.Board.[newPos] with! | RailRoad true ! -> awardRailRoadMultiplier state! | Property n when n > 0 -> awardPropertyMultiplier n state! | ElectricityCompany true! | WaterCompany true -> awardUtilityMultiplier state! | CommunityChest! -> playCommunityChestCard state! | Chance ! ! -> playChanceCard state! | GotoJail when state.Lives = 0<Life> -> ! ! move jail state |> gameOver! | GotoJail ! ! -> move jail state! | _ ! ! ! -> state
  69. 69. Recap
  70. 70. lightweight syntax to create types and hierarchies
  71. 71. great for domain modelling
  72. 72. make invalid states unrepresentable
  73. 73. clear, concise way to handle branching
  74. 74. better correctness
  75. 75. order of magnitude increase in productivity
  76. 76. player states are big
  77. 77. Stateless Server DatabaseClient
  78. 78. 1:1 read-write ratio
  79. 79. stateless = ! Inefficient & Expensive
  80. 80. ServerClient Session 1 Session 2
  81. 81. ServerClient Database
  82. 82. Elastic Load Balancer S3 Auto scaling Group Server A Server B ... EC2 CloudFront Stateful Server
  83. 83. The Actor Model An actor is the fundamental unit of computation which embodies the 3 things! • Processing! • Storage! • Communication! that are essential to computation.! ! -Carl Hewitt* * http://bit.ly/HoNHbG
  84. 84. The Actor Model • Everything is an actor! • An actor has a mailbox! • When an actor receives a message it can:! – Create new actors! – Send messages to actors! – Designate how to handle the next message
  85. 85. Stateful Server • Gatekeeper! – Manages the local list of active workers! – Spawns new workers! ! • Worker! – Manages the states for a player! – Optimistic locking! – Persist state after period of inactivity
  86. 86. Stateful Server Game Server Player A Player B S3 Worker C Worker B Gatekeeper RequestHandlers Asynchronous
  87. 87. Stateful Server Game Server Worker C Worker B Gatekeeper Worker A ok RequestHandlers Player A Player B Asynchronous S3
  88. 88. Stateful Server Game Server S3 Worker C Worker B Gatekeeper Worker A RequestHandlers Player A Player B Asynchronous
  89. 89. Stateful Server Game Server S3 Worker C Gatekeeper Worker A RequestHandlers Player A Player B Asynchronous
  90. 90. Stateful Server Game Server Worker C Worker A Gatekeeper error RequestHandlers Player A Player B S3 Asynchronous
  91. 91. type Agent<‘T> = MailboxProcessor<‘T>! ! ! !
  92. 92. type Agent<‘T> = MailboxProcessor<‘T>! ! type Message = ! | Get of AsyncReplyChannel<…>! | Put of State * Version * AsyncReplyChannel<…>
  93. 93. type Agent<‘T> = MailboxProcessor<‘T>! ! type Message = ! | Get of AsyncReplyChannel<…>! | Put of State * Version * AsyncReplyChannel<…>
  94. 94. type Result<‘T> =! | Success! of ’T! | Failure! of Exception! ! ! !
  95. 95. type Result<‘T> =! | Success! of ’T! | Failure! of Exception! ! type GetResult = Result<State * Version>! type PutResult = Result<unit>!
  96. 96. type Agent<‘T> = MailboxProcessor<‘T>! ! type Message = ! | Get of AsyncReplyChannel<GetResult>! | Put of State * Version * AsyncReplyChannel<PutResult>
  97. 97. type Agent<‘T> = MailboxProcessor<‘T>! ! type Message = ! | Get of AsyncReplyChannel<GetResult>! | Put of State * Version * AsyncReplyChannel<PutResult>
  98. 98. type Worker (playerId) =! let agent = Agent<Message>.Start(fun inbox ->! let state = getCurrentState playerId! ! let rec workingState (state, version) = ! async { … }! and closedState () =! async { … }! ! workingState (state, 1))
  99. 99. type Worker (playerId) =! let agent = Agent<Message>.Start(fun inbox ->! let state = getCurrentState playerId! ! let rec workingState (state, version) = ! async { … }! and closedState () =! async { … }! ! workingState (state, 1))
  100. 100. type Worker (playerId) =! let agent = Agent<Message>.Start(fun inbox ->! let state = getCurrentState playerId! ! let rec workingState (state, version) = ! async { … }! and closedState () =! async { … }! ! workingState (state, 1))
  101. 101. type Worker (playerId) =! let agent = Agent<Message>.Start(fun inbox ->! let state = getCurrentState playerId! ! let rec workingState (state, version) = ! async { … }! and closedState () =! async { … }! ! workingState (state, 1))
  102. 102. type Worker (playerId) =! let agent = Agent<Message>.Start(fun inbox ->! let state = getCurrentState playerId! ! let rec workingState (state, version) = ! async { … }! and closedState () =! async { … }! ! workingState (state, 1))
  103. 103. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!
  104. 104. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!
  105. 105. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!
  106. 106. non-blocking I/O
  107. 107. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!
  108. 108. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!
  109. 109. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! …! | Some(Get(reply)) ->! reply.Reply <| Success(state, version)! return! workingState(state, version)! …! }!
  110. 110. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! …! | Some(Put(newState, v, reply)) when version = v ->! reply.Reply <| Success()! return! workingState(newState, version+1)! …! }
  111. 111. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! …! | Some(Put(_, v, reply)) ->! reply.Reply <| Failure(VersionMismatch(version, v))! return! workingState(state, version)! }
  112. 112. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None ! ! ! ! ! ! -> …! | Some(Get(reply)) ! ! ! ! -> …! | Some(Put(newState, v, reply)) when version = v -> …! | Some(Put(_, v, reply)) ! ! ! -> …! }
  113. 113. and closedState () = ! async { ! let! msg = inbox.Receive()! match msg with! | Get(reply) ->! reply.Reply <| Failure(WorkerStopped)! return! closedState()! | Put(_, _, reply) ->! reply.Reply <| Failure(WorkerStopped)! return! closedState()! }
  114. 114. and closedState () = ! async { ! let! msg = inbox.Receive()! match msg with! | Get(reply) ->! reply.Reply <| Failure(WorkerStopped)! return! closedState()! | Put(_, _, reply) ->! reply.Reply <| Failure(WorkerStopped)! return! closedState()! }
  115. 115. 5x efficient improvement
  116. 116. 60% latency drop
  117. 117. no databases
  118. 118. 90+% cost saving
  119. 119. need to ensure server affinity
  120. 120. need to balance load
  121. 121. need to avoid hotspots
  122. 122. • Persist player state after short inactivity! • Move player to another server after persistence
  123. 123. need to gracefully scale down
  124. 124. Recap
  125. 125. Agents! • no locks! • async message passing! !
  126. 126. Agents! • no locks! • async message passing! • self-contained! • easy to reason with
  127. 127. Agents! • low level! • exceptions kills agents! • primitive error monitoring! • no supervision! • no distribution
  128. 128. MS Orleans Cricket akka.Net
  129. 129. Async Workflow! • non-blocking I/O! • no callbacks
  130. 130. Pattern Matching! • clear & concise
  131. 131. Caught a Gnome
  132. 132. EXP Item Gold Quest Progress Caught a Gnome
  133. 133. Level Up Quest Progress EXP Item Gold Caught a Gnome Quest Complete
  134. 134. Level Up Quest Progress EXP Item Gold Caught a Gnome New Quest Quest Complete
  135. 135. Quest Progress EXP Item Gold Caught a Gnome Quest Complete New Quest Level Up
  136. 136. Quest Progress New Quest Achievement Progress EXP Item Gold Quest Complete Level Up Caught a Gnome
  137. 137. Level Up Quest Progress EXP Item Gold Caught a Gnome Quest Complete New Quest Achievement Progress Achievement Complete
  138. 138. Level Up Quest Progress EXP Item Gold Caught a Gnome Quest Complete New Quest Achievement Progress Achievement Complete
  139. 139. 100+ actions
  140. 140. triggered by different abstraction layers
  141. 141. non-functional requirements
  142. 142. message-broker pattern
  143. 143. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting
  144. 144. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting Ignore Process Process Process Process Ignore
  145. 145. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold
  146. 146. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold
  147. 147. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold Process Ignore Ignore Ignore Process Ignore
  148. 148. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold Level Up
  149. 149. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold Level Up
  150. 150. need lots of facts
  151. 151. type Fact =! | GotExp! ! of Exp! | GotGold! ! of Gold! | GotItem! ! of Item * Count! | CaughtMonster! of Monster * Bait * Location! | LevelUp! ! of OldLevel * NewLevel! …
  152. 152. type Reward =! | GotExp! ! of Exp! | GotGold! ! of Gold! | GotItem! ! of Item * Count! ! type StateChange =! | LevelUp! ! of OldLevel * NewLevel! …
  153. 153. type Fact =! | StateChange! of StateChange! | Reward! ! of Reward! | Trapping! ! of Trapping! | Fishing! ! of Fishing! …
  154. 154. let process fact =! match fact with! | StateChange(stateChange)! -> …! | Reward(reward)! ! ! -> …! | Trapping(trapping)! ! -> …! …
  155. 155. C# interop
  156. 156. …! var fact = Fact.NewStateChange(! ! StateChange.NewLevelUp(oldLvl, newLvl));! …
  157. 157. type IFact = interface end! ! type Reward =! | GotExp! ! of Exp! | GotGold! ! of Gold! | GotItem! ! of Item * Count! interface IFact
  158. 158. type IFact = interface end! ! type Reward =! | GotExp! ! of Exp! | GotGold! ! of Gold! | GotItem! ! of Item * Count! interface IFact
  159. 159. let process (fact : IFact) =! match fact with! | :? StateChange ! as stateChange!-> …! | :? Reward ! ! as reward! ! -> …! | :? Trapping! ! as trapping! -> …! …! | _ -> raise <| NotSupportedFact fact
  160. 160. let process (fact : IFact) =! match fact with! | :? StateChange ! as stateChange!-> …! | :? Reward ! ! as reward! ! -> …! | :? Trapping! ! as trapping! -> …! …! | _ -> raise <| NotSupportedFact fact
  161. 161. simple
  162. 162. flexible
  163. 163. saver
  164. 164. @theburningmonk github.com/theburningmonk theburningmonk.com
  165. 165. http://hire.jobvite.com/m?3OHq4hwL F# C# AWS Erlang Docker micro-services elasticsearch redis Google AppEngine Google BigQuery PostSharpNeo4j Python DevOps PagerDuty NewRelic
  166. 166. BOLOGNA 28 march 2015 LambdaCon Leave your feedback on Joind.in! https://joind.in/13652

×