F# at GameSys
•
13 likes•5,059 views
F# was used to build game infrastructure and logic for a company serving millions of users daily. Some key benefits of F# included: 1) Domain modeling using discriminated unions, types, and measures helped increase correctness and productivity. 2) An actor model with asynchronous message passing was used to build stateful game servers in a scalable and efficient way without locks. 3) Processing game events as immutable facts allowed different systems to process specific facts independently and flexibly.
More Related Content
What's hot
Similar to F# at GameSys
Similar to F# at GameSys (20)
More from Yan Cui
More from Yan Cui (20)
F# at GameSys
- 1. F# at GameSys! (how we used F# to build games for millions)
- 2. Hi, my name is Yan Cui.
- 4. agenda
- 10. why F#?
- 11. time to market
- 12. correctness
- 13. efficient
- 14. tame complexity
- 16. Collectables Wager Size Special Symbol Avg Wager Size Web Server call
- 17. • Line Win! – X number of matching symbols on adjacent columns! – Positions have to be a ‘line’! – Wild symbols substitute for other symbols! ! • Scatter Win! – X number of matching symbols anywhere! – Triggers bonus game
- 18. What symbols should land?! Any special symbol wins?! Did the player win anything?
- 19. What the player’s new average wager?
- 20. Should the player receive collectables?
- 21. type Symbol = Standard! of string! ! ! ! | Wild
- 22. type Symbol = Standard! of string! ! ! ! | Wild e.g.! Standard “hat”! ! Standard “shoe”! ! Standard “bonus”! ! …
- 23. type Symbol = Standard! of string! ! ! ! | Wild i.e.! Wild
- 24. ! ! ! type Win = LineWin of int * Symbol * int! ! | ScatterWin of Symbol * int
- 25. ! ! ! type Win = LineWin of int * Symbol * int! ! | ScatterWin of Symbol * int e.g.! LineWin (5, Standard “shoe”, 4)
- 26. ! ! ! type Win = LineWin of int * Symbol * int! ! | ScatterWin of Symbol * int e.g.! ScatterWin (Standard “bonus”, 3)
- 27. type LineNum = int! type Count! = int! ! type Win = LineWin of LineNum * Symbol * Count! ! | ScatterWin of Symbol * Count
- 28. type LineNum = int! type Count! = int! ! type Win = LineWin of LineNum * Symbol * Count! ! | ScatterWin of Symbol * Count e.g.! LineWin (5, Standard “shoe”, 4)
- 29. type LineNum = int! type Count! = int! ! type Win = LineWin of LineNum * Symbol * Count! ! | ScatterWin of Symbol * Count e.g.! ScatterWin (Standard “bonus”, 3)
- 31. [<Measure>]! type Pence e.g.! 42<Pence>! ! 153<Pence>! ! …
- 32. 10<Meter> / 2<Second> ! = 5<Meter/Second>! 10<Meter> * 2<Second> ! = 20<Meter Second> ! 10<Meter> + 10<Meter> ! = 20<Meter>! 10<Meter> * 10! ! ! = 100<Meter>! 10<Meter> * 10<Meter> ! = 100<Meter2>! 10<Meter> + 2<Second> ! // error! 10<Meter> + 2 ! ! ! // error
- 33. 10<Meter> / 2<Second> ! = 5<Meter/Second>! 10<Meter> * 2<Second> ! = 20<Meter Second> ! 10<Meter> + 10<Meter> ! = 20<Meter>! 10<Meter> * 10! ! ! = 100<Meter>! 10<Meter> * 10<Meter> ! = 100<Meter2>! 10<Meter> + 2<Second> ! // error! 10<Meter> + 2 ! ! ! // error
- 34. type Wager = int64<Pence>! type Multiplier = int! ! type Payout = Coins! of Wager! ! ! | MultipliedCoins of Multiplier * Wager! ! ! | Multi! of Payout list! ! ! | BonusGame
- 35. type Wager = int64<Pence>! type Multiplier = int! ! type Payout = Coins! of Wager! ! ! | MultipliedCoins of Multiplier * Wager! ! ! | Multi! of Payout list! ! ! | BonusGame
- 36. type Wager = int64<Pence>! type Multiplier = int! ! type Payout = Coins! of Wager! ! ! | MultipliedCoins of Multiplier * Wager! ! ! | Multi! of Payout list! ! ! | BonusGame
- 37. type State = ! {! AvgWager! : Wager! SpecialSymbol : Symbol! Collectables : Map<Collectable, Count>! }
- 39. New avg wager Got a Collectable! A pay line win!
- 41. Recap
- 42. lightweight syntax for types & hierarchies
- 46. order of magnitude increase in productivity
- 49. player states are big
- 55. Elastic Load Balancer S3 Auto scaling Group Server A Server B ... EC2 CloudFront Stateful Server
- 56. The Actor Model An actor is the fundamental unit of computation which embodies the 3 things! • Processing! • Storage! • Communication! that are essential to computation.! ! -Carl Hewitt* * http://bit.ly/HoNHbG
- 57. The Actor Model • Everything is an actor! • An actor has a mailbox! • When an actor receives a message it can:! – Create new actors! – Send messages to actors! – Designate how to handle the next message
- 58. Stateful Server • Gatekeeper! – Manages the local list of active workers! – Spawns new workers! ! • Worker! – Manages the states for a player! – Optimistic locking! – Persist state after period of inactivity
- 59. Stateful Server Game Server Player A Player B S3 Worker C Worker B Gatekeeper RequestHandlers Asynchronous
- 60. Stateful Server Game Server Worker C Worker B Gatekeeper Worker A ok RequestHandlers Player A Player B Asynchronous S3
- 61. Stateful Server Game Server S3 Worker C Worker B Gatekeeper Worker A RequestHandlers Player A Player B Asynchronous
- 62. Stateful Server Game Server S3 Worker C Gatekeeper Worker A RequestHandlers Player A Player B Asynchronous
- 63. Stateful Server Game Server Worker C Worker A Gatekeeper error RequestHandlers Player A Player B S3 Asynchronous
- 64. type Agent<‘T> = MailboxProcessor<‘T>! ! ! !
- 65. type Agent<‘T> = MailboxProcessor<‘T>! ! type Message = ! | Get of AsyncReplyChannel<…>! | Put of State * Version * AsyncReplyChannel<…>
- 66. type Agent<‘T> = MailboxProcessor<‘T>! ! type Message = ! | Get of AsyncReplyChannel<…>! | Put of State * Version * AsyncReplyChannel<…>
- 67. type Result<‘T> =! | Success! of ’T! | Failure! of Exception! ! ! !
- 68. type Result<‘T> =! | Success! of ’T! | Failure! of Exception! ! type GetResult = Result<State * Version>! type PutResult = Result<unit>!
- 69. type Agent<‘T> = MailboxProcessor<‘T>! ! type Message = ! | Get of AsyncReplyChannel<GetResult>! | Put of State * Version * AsyncReplyChannel<PutResult>
- 70. type Agent<‘T> = MailboxProcessor<‘T>! ! type Message = ! | Get of AsyncReplyChannel<GetResult>! | Put of State * Version * AsyncReplyChannel<PutResult>
- 71. type Worker (playerId) =! let agent = Agent<Message>.Start(fun inbox ->! let state = getCurrentState playerId! ! let rec workingState (state, version) = ! async { … }! and closedState () =! async { … }! ! workingState (state, 1))
- 72. type Worker (playerId) =! let agent = Agent<Message>.Start(fun inbox ->! let state = getCurrentState playerId! ! let rec workingState (state, version) = ! async { … }! and closedState () =! async { … }! ! workingState (state, 1))
- 73. type Worker (playerId) =! let agent = Agent<Message>.Start(fun inbox ->! let state = getCurrentState playerId! ! let rec workingState (state, version) = ! async { … }! and closedState () =! async { … }! ! workingState (state, 1))
- 74. type Worker (playerId) =! let agent = Agent<Message>.Start(fun inbox ->! let state = getCurrentState playerId! ! let rec workingState (state, version) = ! async { … }! and closedState () =! async { … }! ! workingState (state, 1))
- 75. type Worker (playerId) =! let agent = Agent<Message>.Start(fun inbox ->! let state = getCurrentState playerId! ! let rec workingState (state, version) = ! async { … }! and closedState () =! async { … }! ! workingState (state, 1))
- 76. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!
- 77. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!
- 78. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!
- 79. non-blocking I/O
- 80. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!
- 81. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None -> ! do! persist state! return! closedState()! …! }!
- 82. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! …! | Some(Get(reply)) ->! reply.Reply <| Success(state, version)! return! workingState(state, version)! …! }!
- 83. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! …! | Some(Put(newState, v, reply)) when version = v ->! reply.Reply <| Success()! return! workingState(newState, version+1)! …! }
- 84. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! …! | Some(Put(_, v, reply)) ->! reply.Reply <| Failure(VersionMismatch(version, v))! return! workingState(state, version)! }
- 85. let rec workingState (state, version) = ! async { ! let! msg = inbox.TryReceive(60000)! match msg with! | None ! ! ! ! ! ! -> …! | Some(Get(reply)) ! ! ! ! -> …! | Some(Put(newState, v, reply)) when version = v -> …! | Some(Put(_, v, reply)) ! ! ! -> …! }
- 86. and closedState () = ! async { ! let! msg = inbox.Receive()! match msg with! | Get(reply) ->! reply.Reply <| Failure(WorkerStopped)! return! closedState()! | Put(_, _, reply) ->! reply.Reply <| Failure(WorkerStopped)! return! closedState()! }
- 87. and closedState () = ! async { ! let! msg = inbox.Receive()! match msg with! | Get(reply) ->! reply.Reply <| Failure(WorkerStopped)! return! closedState()! | Put(_, _, reply) ->! reply.Reply <| Failure(WorkerStopped)! return! closedState()! }
- 91. 60% latency drop
- 92. no databases
- 93. 90% cost saving
- 95. need to ensure server affinity
- 96. need to balance load
- 98. • Persist player state after short inactivity! • Move player to another server after persistence
- 99. need to gracefully scale down
- 100. Recap
- 101. Agents! • no locks! • async message passing! !
- 102. Agents! • no locks! • async message passing! • self-contained! • easy to reason with
- 103. Agents! • low level! • exceptions kills agents! • primitive error monitoring! • no supervision! • no distribution
- 106. Async Workflow! • non-blocking I/O! • no callbacks
- 107. Pattern Matching! • clear & concise
- 110. Caught a Gnome
- 111. EXP Item Gold Quest Progress Caught a Gnome
- 112. Level Up Quest Progress EXP Item Gold Caught a Gnome Quest Complete
- 113. Level Up Quest Progress EXP Item Gold Caught a Gnome New Quest Quest Complete
- 114. Quest Progress EXP Item Gold Caught a Gnome Quest Complete New Quest Level Up
- 115. Quest Progress New Quest Achievement Progress EXP Item Gold Quest Complete Level Up Caught a Gnome
- 116. Level Up Quest Progress EXP Item Gold Caught a Gnome Quest Complete New Quest Achievement Progress Achievement Complete
- 117. Level Up Quest Progress EXP Item Gold Caught a Gnome Quest Complete New Quest Achievement Progress Achievement Complete
- 118. 100+ actions
- 120. triggered by different abstraction layers
- 126. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold
- 127. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting EXP Item Gold
- 131. need lots of facts
- 132. type Fact =! | GotExp! ! of Exp! | GotGold! ! of Gold! | GotItem! ! of Item * Count! | CaughtMonster! of Monster * Bait * Location! | LevelUp! ! of OldLevel * NewLevel! …
- 133. type Reward =! | GotExp! ! of Exp! | GotGold! ! of Gold! | GotItem! ! of Item * Count! ! type StateChange =! | LevelUp! ! of OldLevel * NewLevel! …
- 134. type Fact =! | StateChange! of StateChange! | Reward! ! of Reward! | Trapping! ! of Trapping! | Fishing! ! of Fishing! …
- 135. let process fact =! match fact with! | StateChange(stateChange)! -> …! | Reward(reward)! ! ! -> …! | Trapping(trapping)! ! -> …! …
- 136. C# interop
- 137. …! var fact = Fact.NewStateChange(! ! StateChange.NewLevelUp(oldLvl, newLvl));! …
- 138. type IFact = interface end! ! type Reward =! | GotExp! ! of Exp! | GotGold! ! of Gold! | GotItem! ! of Item * Count! interface IFact
- 139. type IFact = interface end! ! type Reward =! | GotExp! ! of Exp! | GotGold! ! of Gold! | GotItem! ! of Item * Count! interface IFact
- 140. let process (fact : IFact) =! match fact with! | :? StateChange ! as stateChange!-> …! | :? Reward ! ! as reward! ! -> …! | :? Trapping! ! as trapping! -> …! …! | _ -> raise <| NotSupportedFact fact
- 141. let process (fact : IFact) =! match fact with! | :? StateChange ! as stateChange!-> …! | :? Reward ! ! as reward! ! -> …! | :? Trapping! ! as trapping! -> …! …! | _ -> raise <| NotSupportedFact fact
- 142. simple
- 143. flexible
- 144. saver
- 146. location bait attraction rate catch rate
- 150. trial-and-error
- 153. genetic algorithms
- 169. wanna find correlations? you can DIY it! ! ;-)
- 170. why was payment service slow last night?
- 175. Find metrics whose 5 min average exceeded 1 second during last 12 hours
- 176. cloudWatch.Select( unitIs “milliseconds” + average (>) 1000.0 @ last 12 hours |> intervalOf 5 minutes)
- 177. cloudWatch.Select(“ unitIs ‘milliseconds’ and average > 1000.0 duringLast 12 hours at intervalOf 5 minutes”)
- 178. did any cache nodes’ CPU spike yesterday?
- 179. cloudWatch.Select( namespaceLike “elasticache” + nameLike “cpu” + max (>) 80.0 @ last 24 hours |> intervalOf 15 minutes)
- 180. cloudWatch.Select( namespaceLike “elasticache” + nameLike “cpu” + max (>) 80.0 @ last 24 hours |> intervalOf 15 minutes) regex support
- 186. Amazing F# =
- 187. usable from anywhere you can run F# code e.g. F# REPL, executable, .. Internal DSL
- 188. useful for building tools e.g. CLI, … External DSL
- 192. Recap
- 198. great performance
- 199. name your throughput
- 201. hash key + range key
- 204. Hash Key Range Key
- 205. Hash Key Range Key Local Secondary Index
- 206. Hash Key Range Key Local Secondary Index Global Secondary Index
- 207. query:! ! given a hash key! ! filter on! ! ! range key, or! ! ! local secondary index
- 209. Hash Key Range Key Local Secondary Index Global Secondary Index Who are the TOP 3 players in “Starship X” with a score of at least 1000?
- 214. GOAL SELECT UserId, TopScore FROM GameScore WHERE GameTitle CONTAINS “Zelda” ORDER DESC! LIMIT !! 3 WITH (NoConsistentRead)
- 215. Query AST Execution
- 216. F# & FParsec* *www.quanttec.com/fparsec External DSL via
- 218. SELECT * FROM GameScore Abstract Syntax Tree (AST) FParsec
- 220. select GameTitle, UserId, TopScore from GameScores where GameTitle = “Starship X” and TopScore >= 1000 order desc limit 3 with (NoConsistentRead, Index(GameTitleIndex, true))
- 221. S3 Provider github.com/fsprojects/S3Provider F# type provider for Amazon S3
- 225. no code generation
- 226. a long time ago, in a language far far away from .Net - Erlang. there was an idea to let you pattern match again binary payloads and work with them at a bit level…
- 227. <<SourcePort:16, DestinationPort:16, ! SeqNumber:32,! AckNumber:32, DataOffset:4, ! Reserved:3, Flags:9, WindSize:16,! CheckSum:16, UrgentPointer:16,! Payload/binary>> = SomeBinary
- 228. <<SourcePort:16, DestinationPort:16, ! SeqNumber:32,! AckNumber:32, DataOffset:4, ! Reserved:3, Flags:9, WindSize:16,! CheckSum:16, UrgentPointer:16,! Payload/binary>> = SomeBinary
- 229. <<SourcePort:16, DestinationPort:16, ! SeqNumber:32,! AckNumber:32, DataOffset:4, ! Reserved:3, Flags:9, WindSize:16,! CheckSum:16, UrgentPointer:16,! Payload/binary>> = SomeBinary
- 230. <<SourcePort:16, DestinationPort:16, ! SeqNumber:32,! AckNumber:32, DataOffset:4, ! Reserved:3, Flags:9, WindSize:16,! CheckSum:16, UrgentPointer:16,! Payload/binary>> = SomeBinary
- 231. <<SourcePort:16, DestinationPort:16, ! SeqNumber:32,! AckNumber:32, DataOffset:4, ! Reserved:3, Flags:9, WindSize:16,! CheckSum:16, UrgentPointer:16,! Payload/binary>> = SomeBinary
- 232. <<SourcePort:16, DestinationPort:16, ! SeqNumber:32,! AckNumber:32, DataOffset:4, ! Reserved:3, Flags:9, WindSize:16,! CheckSum:16, UrgentPointer:16,! Payload/binary>> = SomeBinary
- 234. BitSyntax github.com/theburningmonk/bitsyntax Working with Stream at bit level
- 235. do bitWriter stream {! do! BitWriter.WriteInt16(12345s) ! // source port! do! BitWriter.WriteInt16(12321s)! // destination port! do! BitWriter.WriteInt32(1) ! // sequence number! do! BitWriter.WriteInt32(1) ! // ack number! do! BitWriter.WriteInt32(2, numBits = 4) // data offset! do! BitWriter.WriteInt32(0, numBits = 3) // reserved! do! BitWriter.WriteBool(false) ! // NS! …! do! BitWriter.WriteInt16(42s)! ! // Checksum! do! BitWriter.WriteInt16(1208s)! // Urgent Pointer! do! BitWriter.WriteString("Hello World!") // payload! }
- 236. do bitWriter stream {! do! BitWriter.WriteInt16(12345s) ! // source port! do! BitWriter.WriteInt16(12321s)! // destination port! do! BitWriter.WriteInt32(1) ! // sequence number! do! BitWriter.WriteInt32(1) ! // ack number! do! BitWriter.WriteInt32(2, numBits = 4) // data offset! do! BitWriter.WriteInt32(0, numBits = 3) // reserved! do! BitWriter.WriteBool(false) ! // NS! …! do! BitWriter.WriteInt16(42s)! ! // Checksum! do! BitWriter.WriteInt16(1208s)! // Urgent Pointer! do! BitWriter.WriteString("Hello World!") // payload! } 16 bits
- 237. do bitWriter stream {! do! BitWriter.WriteInt16(12345s) ! // source port! do! BitWriter.WriteInt16(12321s)! // destination port! do! BitWriter.WriteInt32(1) ! // sequence number! do! BitWriter.WriteInt32(1) ! // ack number! do! BitWriter.WriteInt32(2, numBits = 4) // data offset! do! BitWriter.WriteInt32(0, numBits = 3) // reserved! do! BitWriter.WriteBool(false) ! // NS! …! do! BitWriter.WriteInt16(42s)! ! // Checksum! do! BitWriter.WriteInt16(1208s)! // Urgent Pointer! do! BitWriter.WriteString("Hello World!") // payload! } 32 bits
- 238. do bitWriter stream {! do! BitWriter.WriteInt16(12345s) ! // source port! do! BitWriter.WriteInt16(12321s)! // destination port! do! BitWriter.WriteInt32(1) ! // sequence number! do! BitWriter.WriteInt32(1) ! // ack number! do! BitWriter.WriteInt32(2, numBits = 4) // data offset! do! BitWriter.WriteInt32(0, numBits = 3) // reserved! do! BitWriter.WriteBool(false) ! // NS! …! do! BitWriter.WriteInt16(42s)! ! // Checksum! do! BitWriter.WriteInt16(1208s)! // Urgent Pointer! do! BitWriter.WriteString("Hello World!") // payload! } take ’first’ N bits
- 240. bitReader stream {! let! srcPort ! = BitReader.ReadInt32(numBits = 16)! let! destPort ! = BitReader.ReadInt32(numBits = 16)! let! seqNum ! = BitReader.ReadInt32(numBits = 32)! let! ackNum ! = BitReader.ReadInt32(numBits = 32)! let! dataOffset ! = BitReader.ReadInt32(numBits = 4)! let! _ ! = BitReader.ReadByte(numBits = 3)! …! let! payload ! = BitReader.Rest(UTF8.GetString)! ! return srcPort, destPort, … payload! }
- 241. bitReader stream {! let! srcPort ! = BitReader.ReadInt32(numBits = 16)! let! destPort ! = BitReader.ReadInt32(numBits = 16)! let! seqNum ! = BitReader.ReadInt32(numBits = 32)! let! ackNum ! = BitReader.ReadInt32(numBits = 32)! let! dataOffset ! = BitReader.ReadInt32(numBits = 4)! let! _ ! = BitReader.ReadByte(numBits = 3)! …! let! payload ! = BitReader.Rest(UTF8.GetString)! ! return srcPort, destPort, … payload! } packs into 32 bit int
- 243. bitReader stream {! let! srcPort ! = BitReader.ReadInt32(numBits = 16)! let! destPort ! = BitReader.ReadInt32(numBits = 16)! let! seqNum ! = BitReader.ReadInt32(numBits = 32)! let! ackNum ! = BitReader.ReadInt32(numBits = 32)! let! dataOffset ! = BitReader.ReadInt32(numBits = 4)! let! _ ! = BitReader.ReadByte(numBits = 3)! …! let! payload ! = BitReader.Rest(UTF8.GetString)! ! return srcPort, destPort, … payload! } packs into a byte
- 244. 0 1 0 1 1 0 1 1 …! let! dataOffset = BitReader.ReadInt32(numBits = 4)! …
- 245. 0 1 0 1 1 0 1 1 …! let! dataOffset = BitReader.ReadInt32(numBits = 4)! … 0 1 1 0 1 1 0 0 <<< 2
- 246. 0 1 0 1 1 0 1 1 …! let! dataOffset = BitReader.ReadInt32(numBits = 4)! … 0 1 1 0 1 1 0 0 <<< 2 0 0 0 0 0 1 1 0 >>> 4
- 247. 0 0 0 0 0 1 1 0 …! let! dataOffset = BitReader.ReadInt32(numBits = 4)! … 6uy
- 248. …! let! dataOffset = BitReader.ReadInt32(numBits = 4)! … 6uy 6 padding padding padding byte 0 0 0 0 0 1 1 0
- 249. 1 0 0 1 0 0 0 1 …! let! dataOffset = BitReader.ReadInt32(numBits = 4)! … 1 0 0 0 1 0 1 1
- 250. 1 0 0 1 0 0 0 1 …! let! dataOffset = BitReader.ReadInt32(numBits = 17)! … 1 0 0 0 1 0 1 1 1 1 0 1 1 0 1 0 0 1 0 0 1 0 0 0
- 252. Filbert github.com/fsprojects/Filbert BERT serializer and BERT-RPC client