Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

F# in the real world (CodeMotion Dubai)

2,470 views

Published on

In this talk we discussed a number of real world use cases of F# at Gamesys for domain modelling, async programming, writing game logic & genetic algorithms, and creating DSLs to tame complexities with AWS APIs.

Published in: Technology
  • Be the first to comment

F# in the real world (CodeMotion Dubai)

  1. 1. Hi, my name is Yan Cui.
  2. 2. Jan 2010 Nov 2015 Apr 2016
  3. 3. F#in the Real World
  4. 4. agenda
  5. 5. Domain Modelling Infrastructure Game Logic Algorithms DSLs
  6. 6. 1MILLION USERS ACTIVE DAILY
  7. 7. 250MILLION DAY PER REQUEST
  8. 8. 2MONTH TB P E R sec ops 25,000
  9. 9. we are polyglot C#
  10. 10. why F#?
  11. 11. time to market
  12. 12. correctness
  13. 13. efficient
  14. 14. tame complexity
  15. 15. Collectables Wager Size Special Symbol Avg Wager Size Web Server call
  16. 16. • Line Win – X number of matching symbols on adjacent columns – Positions have to be a ‘line’ – Wild symbols substitute for other symbols • Scatter Win – X number of matching symbols anywhere – Triggers bonus game
  17. 17. What symbols should land? Any special symbol wins? Did the player win anything?
  18. 18. What the player’s new average wager?
  19. 19. Should the player receive collectables?
  20. 20. type Symbol = Standard of string | Wild
  21. 21. type Symbol = Standard of string | Wild e.g. Standard “hat” Standard “shoe” Standard “bonus” …
  22. 22. type Symbol = Standard of string | Wild i.e. Wild
  23. 23. type Win = LineWin of int * Symbol * int | ScatterWin of Symbol * int
  24. 24. type Win = LineWin of int * Symbol * int | ScatterWin of Symbol * int e.g. LineWin (5, Standard “shoe”, 4)
  25. 25. type Win = LineWin of int * Symbol * int | ScatterWin of Symbol * int e.g. ScatterWin (Standard “bonus”, 3)
  26. 26. type LineNum = int type Count = int type Win = LineWin of LineNum * Symbol * Count | ScatterWin of Symbol * Count
  27. 27. closed hierarchy
  28. 28. no Nulls
  29. 29. make invalid states unrepresentable
  30. 30. [<Measure>] type Pence e.g. 42<Pence> 153<Pence> …
  31. 31. 10<Meter> / 2<Second> = 5<Meter/Second> 10<Meter> * 2<Second> = 20<Meter Second> 10<Meter> + 10<Meter> = 20<Meter> 10<Meter> * 10 = 100<Meter> 10<Meter> * 10<Meter> = 100<Meter2> 10<Meter> + 2<Second> // error 10<Meter> + 2 // error
  32. 32. 10<Meter> / 2<Second> = 5<Meter/Second> 10<Meter> * 2<Second> = 20<Meter Second> 10<Meter> + 10<Meter> = 20<Meter> 10<Meter> * 10 = 100<Meter> 10<Meter> * 10<Meter> = 100<Meter2> 10<Meter> + 2<Second> // error 10<Meter> + 2 // error
  33. 33. type Wager = int64<Pence> type Multiplier = int type Payout = Coins of Wager | MultipliedCoins of Multiplier * Wager | Multi of Payout list | BonusGame
  34. 34. type Wager = int64<Pence> type Multiplier = int type Payout = Coins of Wager | MultipliedCoins of Multiplier * Wager | Multi of Payout list | BonusGame
  35. 35. type Wager = int64<Pence> type Multiplier = int type Payout = Coins of Wager | MultipliedCoins of Multiplier * Wager | Multi of Payout list | BonusGame
  36. 36. type State = { AvgWager : Wager SpecialSymbol : Symbol Collectables : Map<Collectable, Count> }
  37. 37. immutable by default
  38. 38. Recap
  39. 39. lightweight syntax for types & hierarchies
  40. 40. great for domain modelling
  41. 41. make invalid states unrepresentable
  42. 42. better correctness
  43. 43. order of magnitude increase in productivity
  44. 44. player states are big
  45. 45. Stateless Server DatabaseClient
  46. 46. 1:1 read-write ratio
  47. 47. ServerClient Session 1 Session 2
  48. 48. ServerClient Database
  49. 49. Elastic Load Balancer S3 Auto scaling Group Server A Server B ... EC2 CloudFront Stateful Server
  50. 50. Stateful Server Game Logic Stateless Middle-tier Infrastructure logging circuit breaker perf tracking retry …
  51. 51. Stateful Server Game Logic Stateless Middle-tier Infrastructure logging circuit breaker perf tracking retry …
  52. 52. Stateful Server Game Logic Stateful Middle-tier Infrastructure logging circuit breaker perf tracking retry …
  53. 53. The Actor Model An actor is the fundamental unit of computation which embodies the 3 things • Processing • Storage • Communication that are essential to computation. -Carl Hewitt* * http://bit.ly/HoNHbG
  54. 54. The Actor Model • Everything is an actor • An actor has a mailbox • When an actor receives a message it can: – Create new actors – Send messages to actors – Designate how to handle the next message
  55. 55. Stateful Server • Gatekeeper – Manages the local list of active workers – Spawns new workers • Worker – Manages the states for a player – Optimistic locking – Persist state after period of inactivity
  56. 56. Stateful Server Game Server Player A Player B S3 Worker C Worker B Gatekeeper RequestHandlers Asynchronous
  57. 57. Stateful Server Game Server Worker C Worker B Gatekeeper Worker A ok RequestHandlers Player A Player B Asynchronous S3
  58. 58. Stateful Server Game Server S3 Worker C Worker B Gatekeeper Worker A RequestHandlers Player A Player B Asynchronous
  59. 59. Stateful Server Game Server S3 Worker C Gatekeeper Worker A RequestHandlers Player A Player B Asynchronous
  60. 60. Stateful Server Game Server Worker C Worker A Gatekeeper error RequestHandlers Player A Player B S3 Asynchronous
  61. 61. type Agent<‘T> = MailboxProcessor<‘T>
  62. 62. type Agent<‘T> = MailboxProcessor<‘T> type Message = | Get of AsyncReplyChannel<…> | Put of State * Version * AsyncReplyChannel<…>
  63. 63. type Agent<‘T> = MailboxProcessor<‘T> type Message = | Get of AsyncReplyChannel<…> | Put of State * Version * AsyncReplyChannel<…>
  64. 64. type Result<‘T> = | Success of ’T | Failure of Exception
  65. 65. type Result<‘T> = | Success of ’T | Failure of Exception type GetResult = Result<State * Version> type PutResult = Result<unit>
  66. 66. type Agent<‘T> = MailboxProcessor<‘T> type Message = | Get of AsyncReplyChannel<GetResult> | Put of State * Version * AsyncReplyChannel<PutResult>
  67. 67. type Agent<‘T> = MailboxProcessor<‘T> type Message = | Get of AsyncReplyChannel<GetResult> | Put of State * Version * AsyncReplyChannel<PutResult>
  68. 68. type Worker (playerId) = let agent = Agent<Message>.Start(fun inbox -> let state = getCurrentState playerId let rec workingState (state, version) = async { … } and closedState () = async { … } workingState (state, 1))
  69. 69. type Worker (playerId) = let agent = Agent<Message>.Start(fun inbox -> let state = getCurrentState playerId let rec workingState (state, version) = async { … } and closedState () = async { … } workingState (state, 1))
  70. 70. type Worker (playerId) = let agent = Agent<Message>.Start(fun inbox -> let state = getCurrentState playerId let rec workingState (state, version) = async { … } and closedState () = async { … } workingState (state, 1))
  71. 71. type Worker (playerId) = let agent = Agent<Message>.Start(fun inbox -> let state = getCurrentState playerId let rec workingState (state, version) = async { … } and closedState () = async { … } workingState (state, 1))
  72. 72. type Worker (playerId) = let agent = Agent<Message>.Start(fun inbox -> let state = getCurrentState playerId let rec workingState (state, version) = async { … } and closedState () = async { … } workingState (state, 1))
  73. 73. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with | None -> do! persist state return! closedState() … }
  74. 74. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with | None -> do! persist state return! closedState() … }
  75. 75. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with | None -> do! persist state return! closedState() … }
  76. 76. non-blocking I/O
  77. 77. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with | None -> do! persist state return! closedState() … }
  78. 78. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with | None -> do! persist state return! closedState() … }
  79. 79. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with … | Some(Get(reply)) -> reply.Reply <| Success(state, version) return! workingState(state, version) … }
  80. 80. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with … | Some(Get(reply)) -> reply.Reply <| Success(state, version) return! workingState(state, version) … } type Message = | Get of AsyncReplyChannel<GetResult> | Put of State * Version * AsyncReplyChannel<PutResult>
  81. 81. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with … | Some(Get(reply)) -> reply.Reply <| Success(state, version) return! workingState(state, version) … } type GetResult = Result<State * Version> type PutResult = Result<unit>
  82. 82. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with … | Some(Get(reply)) -> reply.Reply <| Success(state, version) return! workingState(state, version) … }
  83. 83. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with … | Some(Put(newState, v, reply)) when version = v -> reply.Reply <| Success() return! workingState(newState, version+1) … }
  84. 84. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with … | Some(Put(newState, v, reply)) when version = v -> reply.Reply <| Success() return! workingState(newState, version+1) … } type Message = | Get of AsyncReplyChannel<GetResult> | Put of State * Version * AsyncReplyChannel<PutResult>
  85. 85. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with … | Some(Put(newState, v, reply)) when version = v -> reply.Reply <| Success() return! workingState(newState, version+1) … }
  86. 86. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with … | Some(Put(newState, v, reply)) when version = v -> reply.Reply <| Success() return! workingState(newState, version+1) … }
  87. 87. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with … | Some(Put(_, v, reply)) -> reply.Reply <| Failure(VersionMismatch(version, v)) return! workingState(state, version) }
  88. 88. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with … | Some(Put(_, v, reply)) -> reply.Reply <| Failure(VersionMismatch(version, v)) return! workingState(state, version) } type Result<‘T> = | Success of ’T | Failure of Exception
  89. 89. let rec workingState (state, version) = async { let! msg = inbox.TryReceive(60000) match msg with | None -> … | Some(Get(reply)) -> … | Some(Put(newState, v, reply)) when version = v -> … | Some(Put(_, v, reply)) -> … }
  90. 90. and closedState () = async { let! msg = inbox.Receive() match msg with | Get(reply) -> reply.Reply <| Failure(WorkerStopped) return! closedState() | Put(_, _, reply) -> reply.Reply <| Failure(WorkerStopped) return! closedState() }
  91. 91. and closedState () = async { let! msg = inbox.Receive() match msg with | Get(reply) -> reply.Reply <| Failure(WorkerStopped) return! closedState() | Put(_, _, reply) -> reply.Reply <| Failure(WorkerStopped) return! closedState() }
  92. 92. 5x efficient improvement
  93. 93. 60% latency drop
  94. 94. no databases
  95. 95. 90% cost saving
  96. 96. Recap
  97. 97. Agents • no locks • async message passing
  98. 98. Agents • no locks • async message passing • self-contained • easy to reason with
  99. 99. akka.Net Orleans
  100. 100. Pattern Matching • clear & concise
  101. 101. Async Workflow • non-blocking I/O • no callbacks
  102. 102. Caught a Gnome
  103. 103. EXP Item Gold Quest Progress Caught a Gnome
  104. 104. Level Up Quest Progress EXP Item Gold Caught a Gnome Quest Complete
  105. 105. Level Up Quest Progress EXP Item Gold Caught a Gnome New Quest Quest Complete
  106. 106. Quest Progress EXP Item Gold Caught a Gnome Quest Complete New Quest Level Up
  107. 107. Quest Progress New Quest Achievement Progress EXP Item Gold Quest Complete Level Up Caught a Gnome
  108. 108. Level Up Quest Progress EXP Item Gold Caught a Gnome Quest Complete New Quest Achievement Progress Achievement Complete
  109. 109. Level Up Quest Progress EXP Item Gold Caught a Gnome Quest Complete New Quest Achievement Progress Achievement Complete
  110. 110. 100+ actions
  111. 111. triggered by different abstraction layers
  112. 112. non-functional requirements
  113. 113. message-broker pattern
  114. 114. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting
  115. 115. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting Ignore Process Process Process Process Ignore
  116. 116. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold
  117. 117. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold
  118. 118. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold Process Ignore Ignore Ignore Process Ignore
  119. 119. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold Level Up
  120. 120. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold Level Up
  121. 121. need lots of facts
  122. 122. type Fact = | GotExp of Exp | GotGold of Gold | GotItem of Item * Count | CaughtMonster of Monster * Bait * Location | LevelUp of OldLevel * NewLevel …
  123. 123. type Reward = | GotExp of Exp | GotGold of Gold | GotItem of Item * Count type StateChange = | LevelUp of OldLevel * NewLevel …
  124. 124. type Fact = | StateChange of StateChange | Reward of Reward | Trapping of Trapping | Fishing of Fishing …
  125. 125. let process fact = match fact with | StateChange(stateChange) -> … | Reward(reward) -> … | Trapping(trapping) -> … …
  126. 126. C# interop
  127. 127. … var fact = Fact.NewStateChange( StateChange.NewLevelUp(oldLvl, newLvl)); …
  128. 128. type IFact = interface end type Reward = | GotExp of Exp | GotGold of Gold | GotItem of Item * Count interface IFact
  129. 129. type IFact = interface end type Reward = | GotExp of Exp | GotGold of Gold | GotItem of Item * Count interface IFact
  130. 130. let process (fact : IFact) = match fact with | :? StateChange as stateChange -> … | :? Reward as reward -> … | :? Trapping as trapping -> … … | _ -> raise <| NotSupportedFact fact
  131. 131. let process (fact : IFact) = match fact with | :? StateChange as stateChange -> … | :? Reward as reward -> … | :? Trapping as trapping -> … … | _ -> raise <| NotSupportedFact fact
  132. 132. simple
  133. 133. flexible
  134. 134. saver
  135. 135. location bait attraction rate catch rate
  136. 136. auto-tuning trapping stats
  137. 137. Monster strength speed intelligence Trap strength speed technology
  138. 138. Monster strength speed intelligence Trap strength speed technology Catch Rate %
  139. 139. trial-and-error
  140. 140. “if you require constant diligence you’re setting everyone up for failure and hurt” - Bryan Hunter
  141. 141. genetic algorithms
  142. 142. F#-powered DSLs. Awesome!
  143. 143. wanna find correlations?
  144. 144. wanna find correlations? you can DIY it! ;-)
  145. 145. Amazon .CloudWatch .Selector github.com/fsprojects/Amazon.CloudWatch.Selector
  146. 146. Find metrics whose 5 min average exceeded 1 second during last 12 hours
  147. 147. cloudWatch.Select( unitIs “milliseconds” + average (>) 1000.0 @ last 12 hours |> intervalOf 5 minutes)
  148. 148. cloudWatch.Select(“ unitIs ‘milliseconds’ and average > 1000.0 duringLast 12 hours at intervalOf 5 minutes”)
  149. 149. Did any cache nodes’ CPU spike yesterday?
  150. 150. cloudWatch.Select( namespaceLike “elasticache” + nameLike “cpu” + max (>) 80.0 @ last 24 hours |> intervalOf 15 minutes)
  151. 151. cloudWatch.Select( namespaceLike “elasticache” + nameLike “cpu” + max (>) 80.0 @ last 24 hours |> intervalOf 15 minutes) regex support
  152. 152. Amazing F# =
  153. 153. managed key-value store
  154. 154. redundancy 9-9s guarantee
  155. 155. great performance
  156. 156. name your throughput
  157. 157. select GameTitle, UserId, TopScore from GameScores where GameTitle = “Starship X” and TopScore >= 1000 order desc limit 3 with (NoConsistentRead, Index(GameTitleIndex, true))
  158. 158. DynamoDB.SQL github.com/fsprojects/DynamoDb.SQL
  159. 159. GOAL Disguise complexity
  160. 160. GOAL SELECT UserId, TopScore FROM GameScore WHERE GameTitle CONTAINS “Zelda” ORDER DESC LIMIT 3 WITH (NoConsistentRead)
  161. 161. Query AST Execution
  162. 162. F# & FParsec* *www.quanttec.com/fparsec External DSL via
  163. 163. SELECT * FROM GameScore Abstract Syntax Tree (AST) FParsec
  164. 164. select GameTitle, UserId, TopScore from GameScores where GameTitle = “Starship X” and TopScore >= 1000 order desc limit 3 with (NoConsistentRead, Index(GameTitleIndex, true))
  165. 165. < 50 LOC
  166. 166. S3 Provider github.com/fsprojects/S3Provider F# type provider for Amazon S3
  167. 167. intellisense over S3
  168. 168. compile time validation
  169. 169. no code generation
  170. 170. Domain Modelling Infrastructure Game Logic Algorithms DSLs
  171. 171. why F#?
  172. 172. time to market
  173. 173. correctness
  174. 174. efficient
  175. 175. tame complexity
  176. 176. @theburningmonk theburningmonk.com github.com/theburningmonk

×