Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Adopting F# at SBTech

478 views

Published on

Adopting F# at SBTech

Published in: Software
  • Be the first to comment

  • Be the first to like this

Adopting F# at SBTech

  1. 1. About me: @AntyaDev like types*
  2. 2. genda • Why F#? • Adopting F# for C#/OOP developers (inconveniences, C# interoperability, code style, domain modeling, testing)
  3. 3. - Biggest provider in sport offering - Supports a lot of regulated markets - About 1000 microservices (200 distinct types) - 5 datacenters maintained fully by SBTech - About 500 concurrent live events at pick time - On average we handle about 100K+ RPS
  4. 4. highload and near real time
  5. 5. Why F#?
  6. 6. Why F#? If we have sexy C#
  7. 7. If we have sexy C# class Person { public string FirstName; // Not null public string? MiddleName; // May be null public string LastName; // Not null }
  8. 8. If we have sexy C# switch (shape) { case Circle c: WriteLine($"circle with radius {c.Radius}"); break; case Square s when (s.Length == s.Height): WriteLine($"{s.Length} x {s.Height} square"); break; case Rectangle r: WriteLine($"{r.Length} x {r.Height} rectangle"); break; }
  9. 9. If we have sexy C# public (string, string) LookupName(long id) { // some code.. return (first, middle); } var names = LookupName(id); WriteLine($"found {names.first} {names.last}.");
  10. 10. Why F#? If we have sexy C#
  11. 11. 1 day
  12. 12. Story of a new C# project Get("/api/bets", async request => { var result = Validator.Validate(request); // unit testable if (result.IsOk) { var response = await _httpClient.Call(request); return BetsMapper.MapResponse(response); // unit testable } else return Errors.MapError(result); // unit testable });
  13. 13. In 1 week
  14. 14. public class BetsProviderService : IBetsProviderService { readonly IOpenBetsRepository _openBetsRepository; readonly ISettledBetsRepository _settledBetsRepository; public BetsProviderService(IOpenBetsRepository openBetsRepository, ISettledBetsRepository settledBetsRepository) { _openBetsRepository = openBetsRepository; _settledBetsRepository = settledBetsRepository; } } Story of a new C# project
  15. 15. Different Culture OOP FP abstraction extensibility purity composability correctness
  16. 16. Triggers: Much better tooling support Community growth Recursive modules
  17. 17. Tooling support Visual Studio RiderVisual Studio Code
  18. 18. Recursive modules type Plane() = member x.Start() = PlaneLogic.start(x) // types and modules can't currently be mutually // referential at all module private PlaneLogic = let start (x: Plane) = ...
  19. 19. Recursive modules module M type First = | T of Second and Second = | T of First
  20. 20. Recursive modules module M type First = | T of Second type Second = | T of First
  21. 21. Recursive modules module rec M type First = | T of Second type Second = | T of First
  22. 22. Recursive modules module M let private helper () = ... let publicApiFunc () = helper()
  23. 23. Recursive modules module rec M let publicApiFunc () = helper() let private helper () = ...
  24. 24. Community growth • To begin, F# has grown to be bigger than ever, at least as far as we can measure, through product telemetry, twitter activity, GitHub activity, and F# Software Foundation activity. • Active unique users of F# we can measure are in the tens of thousands. • Measured unique users of Visual Studio Code with Ionide increased by over 50% this year, to become far larger than ever. • Measured unique users of Visual Studio who use F# increased by over 20% since last year to be larger than ever, despite quality issues earlier in the year that we believe have inhibited growth. • Much of the measured growth coincides with the release of .NET Core 2.0, which has shown significant interest in the F# community.
  25. 25. namespace rec SportData.ETL.Core type ItemRawUpdate<'Item> = { Id: string Data: 'Item option } module TransformationFlow = let getUpdates (feed: ChangeFeed, lastOffset: uint32) = Code Style
  26. 26. public Game CreateGame(MasterEventInfo masterEventInfo) { var league = ItemsCacheReference.LeagueInfoCache.GetItem(masterEventInfo.LeagueID); var region = league != null ? ItemsCacheReference.CountryInfoCache.GetItem(league.Country) : null; var branch = ItemsCacheReference.BranchInfoCache.GetItem(masterEventInfo.BranchID); var isLive = DateTime.UtcNow.AddMinutes(5) > masterEventInfo.GameDate; return new Game(masterEventInfo.MasterEventID.ToString()) { LastUpdateDateTime = DateTime.UtcNow, SportId = masterEventInfo.BranchID.ToString(), SportName = branch?.Name, SportOrder = branch?.OrderId ?? 0, LeagueId = masterEventInfo.LeagueID.ToString(), LeagueName = league?.Name, RegionId = region?.RegionID.ToString(), RegionName = region?.CountryName, RegionCode = region?.CountryCode, LeagueOrder = league?.OrderID ?? 0, IsTopLeague = league?.IsHot ?? false } }
  27. 27. let createGame (globalCache: GlobalItemsCache, masterEvent: MasterEventItem, currentTime: DateTime) = maybe { let! league = Cache.get(globalCache.Leagues, masterEvent.LeagueID) let! country = Cache.get(globalCache.Countries, league.CountryID) let! sport = Cache.get(globalCache.Branches, masterEvent.BranchID) let! participants = getParticipants(masterEvent, globalCache) let isTeamSwap = isTeamSwapEnabled(globalCache, masterEvent.LeagueID, masterEvent.BranchID) let isLive = isEventLive(currentTime, masterEvent.GameDate) let eventTags = getEventTags(globalCache, masterEvent.ID) let mainEventId = getMainEventId(masterEvent) let metadata = getEventMetadata(globalCache, mainEventId, masterEvent.ID) let liveGameState = if isLive then getLiveGameStateByMasterEvent(globalCache, masterEvent) else None return { Id = masterEvent.ID.ToString() Type = getEventType(masterEvent) } } Maybe Monad
  28. 28. I need website Website SBTech story
  29. 29. I need an API to build unique UI The Problem Oops?!
  30. 30. The Problem Oops?! 1. well defined contracts (Event, Market, Selection) 2. flexible way to query data and subscribe on changes 3. push updates notifications about changes 4. near real time change delivery (1 sec delay) We need to provide:
  31. 31. PUSH based Queryable API (Change Feed)
  32. 32. select * from Games where isLive = true order by totalBets desc limit 10 PUSH changes
  33. 33. Change Feed (Actor) Change Stream Feed View 2:2 1:0 Chelsea - Milan Milan - Liverpool
  34. 34. Subscribers Feed View 2:2 1:0 Chelsea - Milan Milan - Liverpool Change Log Query Change Feed (Actor)
  35. 35. Subscribers type NodeName = string type LastHeartBeat = DateTime type NodeSubscribers = Map<NodeName, LastHeartBeat> type FeedStatus = | ReadyForDeactivation | HasSubscribers | NoSubscribers type ChangeFeed = { Id: FeedId View: FeedView ChangeLog: ChangeLog Subscribers: NodeSubscribers Query: Query Status: FeedStatus } Query Feed View 2:2 1:0 Chelsea - Milan Milan - Liverpool Change Feed Change Log
  36. 36. module ChangeFeed let getSnapshot (feed: ChangeFeed, fastPreloadAmount: uint32) = ... let getUpdates (feed: ChangeFeed, lastOffset: uint32) = ... let getLatestUpdate (feed: ChangeFeed) = ... let subscribe (feed: ChangeFeed, name: NodeName, currentTime: DateTime) = ... let subscribeByOffset (feed: ChangeFeed, name: NodeName, lastOffset: uint32) = ... let unsubscribe (feed: ChangeFeed, name: NodeName) = ... let getInactiveNodes (feed: ChangeFeed, currentTime: DateTime, inactivePeriod: TimeSpan) = ... let reloadView (feed: ChangeFeed, queryResults: QueryResult seq) = ...
  37. 37. Feed View 2:2 1:0 Chelsea - Milan Milan - Liverpool type PartialView = { EntityType: EntityType Entities: IEntity seq } type FeedView = { OrderType: EntityType OrderIds: string seq OrderFilter: FilterFn option Views: Map<EntityType, PartialView> } let tryCreate (queryResults: QueryResult seq, orderType: EntityType, orderFilter: FilterFn option) = match queryResults with | NotMatchFor orderType -> fail <| Errors.OrderTypeIsNotMatch | Empty -> ok <| { OrderType = orderType; OrderIds = Seq.empty OrderFilter = orderFilter; Views = views } | _ -> ok <| ...
  38. 38. [<Property>] let ``empty ChangeLog should start with offset 0 and requested maxSize`` (maxSize: uint32) = let changeLog = ChangeLog.create(maxSize) Assert.Equal(0u, changeLog.Offset) Assert.Equal(maxSize, changeLog.MaxSize) [<Property(Arbitrary=[|typeof<Generators.Generator>|])>] let ``all changes in changeLog.stream should have UpdateType.Update`` (payloads: Payload array) = let mutable changeLog = ChangeLog.create(5u) for p in payloads do changeLog <- ChangeLog.append(changeLog, p) let result = changeLog.Stream.All(fun change -> change.Type = UpdateType.Update) Assert.True(result) TDD without Mocks
  39. 39. We have our own frontend but we need a Data The Next Challenge Oops?!
  40. 40. SBTech All changes Operator
  41. 41. RDBMS All changes from 40 tables ETL Operator Well defined entities compose entities react on changes recalculate odds (data locality)
  42. 42. C# F#Asp NET Core Orleans DB Drivers [ DDD; Tests] Infrastructure business logic
  43. 43. public Task<FullFeedUpdate> GetSnapshot(GetSnapshot msg) { // invoke F# code from C# var updates = ChangeFeedModule.getSnapshot(FeedState, msg.FastPreloadAmount); return updates; }
  44. 44. Applied at the heart of the system Zero Null Reference exceptions Domain errors instead of exceptions DDD with types (strong determinism) Dependency Rejection TDD without mocks (property based testing)
  45. 45. let result = [ChaosMonkey LatencyMonkey CpuMonkey MemoryMonkey] |> induceDamage
  46. 46. NDamage feature "Github Activity" [ step "list repository events" (GET "/repos/WeKnowSports/SportsDataAPI/events") // action (checkStatus HttpStatusCode.OK) // validator step "list issue events for a repository" (GET "/repos/WeKnowSports/SportsDataAPI/issues/events") (checkStatus HttpStatusCode.OK) pause "00:00:05" ]
  47. 47. let platform = Docker(host = "http://localhost:2375") let chaosMonkey = { Name = "DockerChaosMonkey" Frequency = "00:00:05" Probability = 1.0 // 100% that alive node will be terminated TargetGroups = ["api"; "mongo"; "rabbit"] Platform = platform } let httpBase = http |> baseURL("https://api.github.com") |> header("User-Agent", "NDamage") |> header("Accept", "application/json") |> header("Accept-Encoding", "gzip") [ feature "Github Reactions" [ step "list reactions for an issue"
  48. 48. NDamage http://ndamage.com @n_damage Supported Platforms: [Docker; Google Cloud; Amazon; Azure] Test your resilience! CI Pipeline: [Jenkins; Team City] Alerting: [Email; Slack]

×