About me:
@AntyaDev
like types*
genda
• Why F#?
• Adopting F# for C#/OOP
developers (inconveniences, C#
interoperability, code style,
domain modeling, testing)
- Biggest provider in sport offering
- Supports a lot of regulated markets
- About 1000 microservices (200 distinct types)
- 5 datacenters maintained fully by SBTech
- About 500 concurrent live events at pick time
- On average we handle about 100K+ RPS
highload and
near real time
Why F#?
Why F#?
If we have
sexy C#
If we have sexy C#
class Person
{
public string FirstName; // Not null
public string? MiddleName; // May be null
public string LastName; // Not null
}
If we have sexy C#
switch (shape)
{
case Circle c:
WriteLine($"circle with radius {c.Radius}");
break;
case Square s when (s.Length == s.Height):
WriteLine($"{s.Length} x {s.Height} square");
break;
case Rectangle r:
WriteLine($"{r.Length} x {r.Height} rectangle");
break;
}
If we have sexy C#
public (string, string) LookupName(long id)
{
// some code..
return (first, middle);
}
var names = LookupName(id);
WriteLine($"found {names.first} {names.last}.");
Why F#?
If we have
sexy C#
1 day
Story of a new C# project
Get("/api/bets", async request =>
{
var result = Validator.Validate(request); // unit testable
if (result.IsOk)
{
var response = await _httpClient.Call(request);
return BetsMapper.MapResponse(response); // unit testable
}
else return Errors.MapError(result); // unit testable
});
In 1 week
public class BetsProviderService : IBetsProviderService
{
readonly IOpenBetsRepository _openBetsRepository;
readonly ISettledBetsRepository _settledBetsRepository;
public BetsProviderService(IOpenBetsRepository openBetsRepository,
ISettledBetsRepository settledBetsRepository)
{
_openBetsRepository = openBetsRepository;
_settledBetsRepository = settledBetsRepository;
}
}
Story of a new C# project
Different Culture
OOP FP
abstraction
extensibility
purity
composability
correctness
Triggers:
Much better tooling support
Community growth
Recursive modules
Tooling support
Visual Studio
RiderVisual Studio Code
Recursive modules
type Plane() = member x.Start() = PlaneLogic.start(x)
// types and modules can't currently be mutually
// referential at all
module private PlaneLogic =
let start (x: Plane) = ...
Recursive modules
module M
type First =
| T of Second
and Second =
| T of First
Recursive modules
module M
type First =
| T of Second
type Second =
| T of First
Recursive modules
module rec M
type First =
| T of Second
type Second =
| T of First
Recursive modules
module M
let private helper () = ...
let publicApiFunc () = helper()
Recursive modules
module rec M
let publicApiFunc () = helper()
let private helper () = ...
Community growth
• To begin, F# has grown to be bigger than ever, at least as far as
we can measure, through product telemetry, twitter activity,
GitHub activity, and F# Software Foundation activity.
• Active unique users of F# we can measure are in the tens of
thousands.
• Measured unique users of Visual Studio Code
with Ionide increased by over 50% this year, to become far larger
than ever.
• Measured unique users of Visual Studio who use F# increased by
over 20% since last year to be larger than ever, despite quality
issues earlier in the year that we believe have inhibited growth.
• Much of the measured growth coincides with the release of .NET
Core 2.0, which has shown significant interest in the F#
community.
namespace rec SportData.ETL.Core
type ItemRawUpdate<'Item> = {
Id: string
Data: 'Item option
}
module TransformationFlow =
let getUpdates (feed: ChangeFeed, lastOffset: uint32) =
Code Style
public Game CreateGame(MasterEventInfo masterEventInfo)
{
var league = ItemsCacheReference.LeagueInfoCache.GetItem(masterEventInfo.LeagueID);
var region = league != null
? ItemsCacheReference.CountryInfoCache.GetItem(league.Country)
: null;
var branch = ItemsCacheReference.BranchInfoCache.GetItem(masterEventInfo.BranchID);
var isLive = DateTime.UtcNow.AddMinutes(5) > masterEventInfo.GameDate;
return new Game(masterEventInfo.MasterEventID.ToString())
{
LastUpdateDateTime = DateTime.UtcNow,
SportId = masterEventInfo.BranchID.ToString(),
SportName = branch?.Name,
SportOrder = branch?.OrderId ?? 0,
LeagueId = masterEventInfo.LeagueID.ToString(),
LeagueName = league?.Name,
RegionId = region?.RegionID.ToString(),
RegionName = region?.CountryName,
RegionCode = region?.CountryCode,
LeagueOrder = league?.OrderID ?? 0,
IsTopLeague = league?.IsHot ?? false
}
}
let createGame (globalCache: GlobalItemsCache,
masterEvent: MasterEventItem, currentTime: DateTime) = maybe {
let! league = Cache.get(globalCache.Leagues, masterEvent.LeagueID)
let! country = Cache.get(globalCache.Countries, league.CountryID)
let! sport = Cache.get(globalCache.Branches, masterEvent.BranchID)
let! participants = getParticipants(masterEvent, globalCache)
let isTeamSwap = isTeamSwapEnabled(globalCache, masterEvent.LeagueID, masterEvent.BranchID)
let isLive = isEventLive(currentTime, masterEvent.GameDate)
let eventTags = getEventTags(globalCache, masterEvent.ID)
let mainEventId = getMainEventId(masterEvent)
let metadata = getEventMetadata(globalCache, mainEventId, masterEvent.ID)
let liveGameState = if isLive then getLiveGameStateByMasterEvent(globalCache, masterEvent)
else None
return { Id = masterEvent.ID.ToString()
Type = getEventType(masterEvent) }
}
Maybe Monad
I need website
Website
SBTech story
I need an API
to build
unique UI
The Problem
Oops?!
The Problem
Oops?!
1. well defined contracts (Event, Market, Selection)
2. flexible way to query data and subscribe on changes
3. push updates notifications about changes
4. near real time change delivery (1 sec delay)
We need to provide:
PUSH based
Queryable API
(Change Feed)
select * from Games where isLive = true
order by totalBets desc
limit 10
PUSH changes
Change Feed
(Actor)
Change Stream
Feed View
2:2
1:0 Chelsea - Milan
Milan - Liverpool
Subscribers
Feed View
2:2
1:0 Chelsea - Milan
Milan - Liverpool
Change Log
Query
Change Feed
(Actor)
Subscribers
type NodeName = string
type LastHeartBeat = DateTime
type NodeSubscribers = Map<NodeName, LastHeartBeat>
type FeedStatus =
| ReadyForDeactivation
| HasSubscribers
| NoSubscribers
type ChangeFeed = {
Id: FeedId
View: FeedView
ChangeLog: ChangeLog
Subscribers: NodeSubscribers
Query: Query
Status: FeedStatus
}
Query
Feed View
2:2
1:0 Chelsea - Milan
Milan - Liverpool
Change Feed
Change Log
module ChangeFeed
let getSnapshot (feed: ChangeFeed, fastPreloadAmount: uint32) = ...
let getUpdates (feed: ChangeFeed, lastOffset: uint32) = ...
let getLatestUpdate (feed: ChangeFeed) = ...
let subscribe (feed: ChangeFeed, name: NodeName, currentTime: DateTime) = ...
let subscribeByOffset (feed: ChangeFeed, name: NodeName, lastOffset: uint32) = ...
let unsubscribe (feed: ChangeFeed, name: NodeName) = ...
let getInactiveNodes (feed: ChangeFeed, currentTime: DateTime, inactivePeriod: TimeSpan) = ...
let reloadView (feed: ChangeFeed, queryResults: QueryResult seq) = ...
Feed View
2:2
1:0 Chelsea - Milan
Milan - Liverpool
type PartialView = {
EntityType: EntityType
Entities: IEntity seq
}
type FeedView = {
OrderType: EntityType
OrderIds: string seq
OrderFilter: FilterFn option
Views: Map<EntityType, PartialView>
}
let tryCreate (queryResults: QueryResult seq, orderType: EntityType,
orderFilter: FilterFn option) =
match queryResults with
| NotMatchFor orderType -> fail <| Errors.OrderTypeIsNotMatch
| Empty -> ok <| { OrderType = orderType; OrderIds = Seq.empty
OrderFilter = orderFilter; Views = views }
| _ -> ok <| ...
[<Property>]
let ``empty ChangeLog should start with offset 0 and requested maxSize`` (maxSize: uint32) =
let changeLog = ChangeLog.create(maxSize)
Assert.Equal(0u, changeLog.Offset)
Assert.Equal(maxSize, changeLog.MaxSize)
[<Property(Arbitrary=[|typeof<Generators.Generator>|])>]
let ``all changes in changeLog.stream should have UpdateType.Update`` (payloads: Payload array) =
let mutable changeLog = ChangeLog.create(5u)
for p in payloads do
changeLog <- ChangeLog.append(changeLog, p)
let result = changeLog.Stream.All(fun change -> change.Type = UpdateType.Update)
Assert.True(result)
TDD without Mocks
We have our own frontend
but
we need a Data
The Next Challenge
Oops?!
SBTech
All changes
Operator
RDBMS
All changes
from
40 tables
ETL
Operator
Well defined entities
compose entities
react on changes
recalculate odds
(data locality)
C#
F#Asp NET Core
Orleans
DB Drivers
[ DDD; Tests]
Infrastructure
business logic
public Task<FullFeedUpdate> GetSnapshot(GetSnapshot msg)
{
// invoke F# code from C#
var updates = ChangeFeedModule.getSnapshot(FeedState, msg.FastPreloadAmount);
return updates;
}
Applied at the heart of the system
Zero Null Reference exceptions
Domain errors instead of exceptions
DDD with types (strong determinism)
Dependency Rejection
TDD without mocks (property based testing)
let result = [ChaosMonkey
LatencyMonkey
CpuMonkey
MemoryMonkey]
|> induceDamage
NDamage
feature "Github Activity" [
step "list repository events"
(GET "/repos/WeKnowSports/SportsDataAPI/events") // action
(checkStatus HttpStatusCode.OK) // validator
step "list issue events for a repository"
(GET "/repos/WeKnowSports/SportsDataAPI/issues/events")
(checkStatus HttpStatusCode.OK)
pause "00:00:05"
]
let platform = Docker(host = "http://localhost:2375")
let chaosMonkey = {
Name = "DockerChaosMonkey"
Frequency = "00:00:05"
Probability = 1.0 // 100% that alive node will be terminated
TargetGroups = ["api"; "mongo"; "rabbit"]
Platform = platform
}
let httpBase = http
|> baseURL("https://api.github.com")
|> header("User-Agent", "NDamage")
|> header("Accept", "application/json")
|> header("Accept-Encoding", "gzip")
[
feature "Github Reactions" [
step "list reactions for an issue"
NDamage
http://ndamage.com
@n_damage
Supported Platforms: [Docker; Google Cloud; Amazon; Azure]
Test your resilience!
CI Pipeline: [Jenkins; Team City]
Alerting: [Email; Slack]
Adopting F# at SBTech

Adopting F# at SBTech

  • 2.
  • 3.
    genda • Why F#? •Adopting F# for C#/OOP developers (inconveniences, C# interoperability, code style, domain modeling, testing)
  • 4.
    - Biggest providerin sport offering - Supports a lot of regulated markets - About 1000 microservices (200 distinct types) - 5 datacenters maintained fully by SBTech - About 500 concurrent live events at pick time - On average we handle about 100K+ RPS
  • 5.
  • 6.
  • 7.
    Why F#? If wehave sexy C#
  • 8.
    If we havesexy C# class Person { public string FirstName; // Not null public string? MiddleName; // May be null public string LastName; // Not null }
  • 9.
    If we havesexy C# switch (shape) { case Circle c: WriteLine($"circle with radius {c.Radius}"); break; case Square s when (s.Length == s.Height): WriteLine($"{s.Length} x {s.Height} square"); break; case Rectangle r: WriteLine($"{r.Length} x {r.Height} rectangle"); break; }
  • 10.
    If we havesexy C# public (string, string) LookupName(long id) { // some code.. return (first, middle); } var names = LookupName(id); WriteLine($"found {names.first} {names.last}.");
  • 11.
    Why F#? If wehave sexy C#
  • 12.
  • 13.
    Story of anew C# project Get("/api/bets", async request => { var result = Validator.Validate(request); // unit testable if (result.IsOk) { var response = await _httpClient.Call(request); return BetsMapper.MapResponse(response); // unit testable } else return Errors.MapError(result); // unit testable });
  • 14.
  • 15.
    public class BetsProviderService: IBetsProviderService { readonly IOpenBetsRepository _openBetsRepository; readonly ISettledBetsRepository _settledBetsRepository; public BetsProviderService(IOpenBetsRepository openBetsRepository, ISettledBetsRepository settledBetsRepository) { _openBetsRepository = openBetsRepository; _settledBetsRepository = settledBetsRepository; } } Story of a new C# project
  • 17.
  • 18.
    Triggers: Much better toolingsupport Community growth Recursive modules
  • 19.
  • 22.
    Recursive modules type Plane()= member x.Start() = PlaneLogic.start(x) // types and modules can't currently be mutually // referential at all module private PlaneLogic = let start (x: Plane) = ...
  • 23.
    Recursive modules module M typeFirst = | T of Second and Second = | T of First
  • 24.
    Recursive modules module M typeFirst = | T of Second type Second = | T of First
  • 25.
    Recursive modules module recM type First = | T of Second type Second = | T of First
  • 26.
    Recursive modules module M letprivate helper () = ... let publicApiFunc () = helper()
  • 27.
    Recursive modules module recM let publicApiFunc () = helper() let private helper () = ...
  • 28.
    Community growth • Tobegin, F# has grown to be bigger than ever, at least as far as we can measure, through product telemetry, twitter activity, GitHub activity, and F# Software Foundation activity. • Active unique users of F# we can measure are in the tens of thousands. • Measured unique users of Visual Studio Code with Ionide increased by over 50% this year, to become far larger than ever. • Measured unique users of Visual Studio who use F# increased by over 20% since last year to be larger than ever, despite quality issues earlier in the year that we believe have inhibited growth. • Much of the measured growth coincides with the release of .NET Core 2.0, which has shown significant interest in the F# community.
  • 30.
    namespace rec SportData.ETL.Core typeItemRawUpdate<'Item> = { Id: string Data: 'Item option } module TransformationFlow = let getUpdates (feed: ChangeFeed, lastOffset: uint32) = Code Style
  • 31.
    public Game CreateGame(MasterEventInfomasterEventInfo) { var league = ItemsCacheReference.LeagueInfoCache.GetItem(masterEventInfo.LeagueID); var region = league != null ? ItemsCacheReference.CountryInfoCache.GetItem(league.Country) : null; var branch = ItemsCacheReference.BranchInfoCache.GetItem(masterEventInfo.BranchID); var isLive = DateTime.UtcNow.AddMinutes(5) > masterEventInfo.GameDate; return new Game(masterEventInfo.MasterEventID.ToString()) { LastUpdateDateTime = DateTime.UtcNow, SportId = masterEventInfo.BranchID.ToString(), SportName = branch?.Name, SportOrder = branch?.OrderId ?? 0, LeagueId = masterEventInfo.LeagueID.ToString(), LeagueName = league?.Name, RegionId = region?.RegionID.ToString(), RegionName = region?.CountryName, RegionCode = region?.CountryCode, LeagueOrder = league?.OrderID ?? 0, IsTopLeague = league?.IsHot ?? false } }
  • 32.
    let createGame (globalCache:GlobalItemsCache, masterEvent: MasterEventItem, currentTime: DateTime) = maybe { let! league = Cache.get(globalCache.Leagues, masterEvent.LeagueID) let! country = Cache.get(globalCache.Countries, league.CountryID) let! sport = Cache.get(globalCache.Branches, masterEvent.BranchID) let! participants = getParticipants(masterEvent, globalCache) let isTeamSwap = isTeamSwapEnabled(globalCache, masterEvent.LeagueID, masterEvent.BranchID) let isLive = isEventLive(currentTime, masterEvent.GameDate) let eventTags = getEventTags(globalCache, masterEvent.ID) let mainEventId = getMainEventId(masterEvent) let metadata = getEventMetadata(globalCache, mainEventId, masterEvent.ID) let liveGameState = if isLive then getLiveGameStateByMasterEvent(globalCache, masterEvent) else None return { Id = masterEvent.ID.ToString() Type = getEventType(masterEvent) } } Maybe Monad
  • 33.
  • 34.
    I need anAPI to build unique UI The Problem Oops?!
  • 35.
    The Problem Oops?! 1. welldefined contracts (Event, Market, Selection) 2. flexible way to query data and subscribe on changes 3. push updates notifications about changes 4. near real time change delivery (1 sec delay) We need to provide:
  • 36.
  • 37.
    select * fromGames where isLive = true order by totalBets desc limit 10 PUSH changes
  • 38.
    Change Feed (Actor) Change Stream FeedView 2:2 1:0 Chelsea - Milan Milan - Liverpool
  • 39.
    Subscribers Feed View 2:2 1:0 Chelsea- Milan Milan - Liverpool Change Log Query Change Feed (Actor)
  • 40.
    Subscribers type NodeName =string type LastHeartBeat = DateTime type NodeSubscribers = Map<NodeName, LastHeartBeat> type FeedStatus = | ReadyForDeactivation | HasSubscribers | NoSubscribers type ChangeFeed = { Id: FeedId View: FeedView ChangeLog: ChangeLog Subscribers: NodeSubscribers Query: Query Status: FeedStatus } Query Feed View 2:2 1:0 Chelsea - Milan Milan - Liverpool Change Feed Change Log
  • 41.
    module ChangeFeed let getSnapshot(feed: ChangeFeed, fastPreloadAmount: uint32) = ... let getUpdates (feed: ChangeFeed, lastOffset: uint32) = ... let getLatestUpdate (feed: ChangeFeed) = ... let subscribe (feed: ChangeFeed, name: NodeName, currentTime: DateTime) = ... let subscribeByOffset (feed: ChangeFeed, name: NodeName, lastOffset: uint32) = ... let unsubscribe (feed: ChangeFeed, name: NodeName) = ... let getInactiveNodes (feed: ChangeFeed, currentTime: DateTime, inactivePeriod: TimeSpan) = ... let reloadView (feed: ChangeFeed, queryResults: QueryResult seq) = ...
  • 42.
    Feed View 2:2 1:0 Chelsea- Milan Milan - Liverpool type PartialView = { EntityType: EntityType Entities: IEntity seq } type FeedView = { OrderType: EntityType OrderIds: string seq OrderFilter: FilterFn option Views: Map<EntityType, PartialView> } let tryCreate (queryResults: QueryResult seq, orderType: EntityType, orderFilter: FilterFn option) = match queryResults with | NotMatchFor orderType -> fail <| Errors.OrderTypeIsNotMatch | Empty -> ok <| { OrderType = orderType; OrderIds = Seq.empty OrderFilter = orderFilter; Views = views } | _ -> ok <| ...
  • 43.
    [<Property>] let ``empty ChangeLogshould start with offset 0 and requested maxSize`` (maxSize: uint32) = let changeLog = ChangeLog.create(maxSize) Assert.Equal(0u, changeLog.Offset) Assert.Equal(maxSize, changeLog.MaxSize) [<Property(Arbitrary=[|typeof<Generators.Generator>|])>] let ``all changes in changeLog.stream should have UpdateType.Update`` (payloads: Payload array) = let mutable changeLog = ChangeLog.create(5u) for p in payloads do changeLog <- ChangeLog.append(changeLog, p) let result = changeLog.Stream.All(fun change -> change.Type = UpdateType.Update) Assert.True(result) TDD without Mocks
  • 44.
    We have ourown frontend but we need a Data The Next Challenge Oops?!
  • 45.
  • 46.
    RDBMS All changes from 40 tables ETL Operator Welldefined entities compose entities react on changes recalculate odds (data locality)
  • 47.
    C# F#Asp NET Core Orleans DBDrivers [ DDD; Tests] Infrastructure business logic
  • 48.
    public Task<FullFeedUpdate> GetSnapshot(GetSnapshotmsg) { // invoke F# code from C# var updates = ChangeFeedModule.getSnapshot(FeedState, msg.FastPreloadAmount); return updates; }
  • 49.
    Applied at theheart of the system Zero Null Reference exceptions Domain errors instead of exceptions DDD with types (strong determinism) Dependency Rejection TDD without mocks (property based testing)
  • 50.
    let result =[ChaosMonkey LatencyMonkey CpuMonkey MemoryMonkey] |> induceDamage
  • 51.
    NDamage feature "Github Activity"[ step "list repository events" (GET "/repos/WeKnowSports/SportsDataAPI/events") // action (checkStatus HttpStatusCode.OK) // validator step "list issue events for a repository" (GET "/repos/WeKnowSports/SportsDataAPI/issues/events") (checkStatus HttpStatusCode.OK) pause "00:00:05" ]
  • 52.
    let platform =Docker(host = "http://localhost:2375") let chaosMonkey = { Name = "DockerChaosMonkey" Frequency = "00:00:05" Probability = 1.0 // 100% that alive node will be terminated TargetGroups = ["api"; "mongo"; "rabbit"] Platform = platform } let httpBase = http |> baseURL("https://api.github.com") |> header("User-Agent", "NDamage") |> header("Accept", "application/json") |> header("Accept-Encoding", "gzip") [ feature "Github Reactions" [ step "list reactions for an issue"
  • 53.
    NDamage http://ndamage.com @n_damage Supported Platforms: [Docker;Google Cloud; Amazon; Azure] Test your resilience! CI Pipeline: [Jenkins; Team City] Alerting: [Email; Slack]