# Time for Functions

Feb. 24, 2014
1 of 54

### Time for Functions

• 1. Time for Functions @simontcousins
• 4. let rec qsort = function | [] -> [] | hd :: tl -> let lesser, greater = List.partition ((>=) hd) tl List.concat [qsort lesser; [hd]; qsort greater] http://rosettacode.org/wiki/Sorting_algorithms/Quicksort#F.23
• 5. private static List<int> quicksort(List<int> arr) { List<int> loe = new List<int>(), gt = new List<int>(); if (arr.Count < 2) return arr; int pivot = arr.Count / 2; int pivot_val = arr[pivot]; arr.RemoveAt(pivot); foreach (int i in arr) { if (i <= pivot_val) loe.Add(i); else if (i > pivot_val) gt.Add(i); } List<int> resultSet = new List<int>(); resultSet.AddRange(quicksort(loe)); if (loe.Count == 0){ loe.Add(pivot_val); }else{ gt.Add(pivot_val); } resultSet.AddRange(quicksort(gt)); return resultSet; }
• 6. Functional code is… • Clear • • Concise • • closer to a statement of the algorithm less noise and accidental complexity Correct • the type system works with the developer
• 7. and this is a really bi g but but… http://xkcd.com/
• 8. Some new things to learn… recursion pure functions immutable data partial application let rec qsort = function | [] -> [] | hd :: tl -> let lesser, greater = List.partition ((>=) hd) tl List.concat [qsort lesser; [hd]; qsort greater] pattern matching higher-order functions type inference generics by default 'a list -> 'a list when 'a : comparison
• 9. let rec qsort = function | [] -> [] | hd :: tl -> let lesser, greater = List.partition ((>=) hd) tl List.concat [qsort lesser; [hd]; qsort greater] gotcha!
• 10. Some real-world concerns… • Good for demos but what about large programs? • Good for academics but what about us? • Elegant code but what about performance? • Does it work with legacy software? • Where do I ﬁnd functional programmers?
• 12. Bespoke Enterprise Applications for the Energy Sector • lots of data • forecasts • • • metered data market data lots of types • units of measure • lots of computations • schedules • contracts • analysis rates • • station parameters … all changing over time
• 13. stay at 50Hz by adjusting these to make this zero { www.statnett.no THE ENERGY SECTOR
• 14. Project: Balancing Services • Blackstart • BMSU • Faststart • Frequency Response • Reactive Power • STOR contracted services provided by energy companies to ensure the security and stability of supply http://www2.nationalgrid.com/uk/services/balancing-services/
• 15. Old System New System • C# • F# • OO / Imperative • Functional / OO / Imperative • Relational Database • Document Store • Untestable • Highly testable • Slow • Fast • Contracts not fully implemented • Contracts fully implemented defeated by complexity tinyurl.com/stor-contract
• 16. Dynamic API Market API Test Console Contract Evaluation Job API Asset API Contract Evaluation API Scheduler Document Store Web UI View Model API
• 18. Elegant beautiful simple efficient functional
• 19. Not Elegant “… but hey, it’s object-oriented!”
• 20. Real-world OO • Struggles to be elegant • top down designs • • high ceremony • • coarse abstractions data and behaviour tightly coupled Mutating state is a powerful and dangerous technique • hard to reason about • requires synchronised access Lots of accidental complexity • abstraction event horizon • • • ORMs, IoCs, Mocks, Design Patterns, UML … Hard to ﬁnd developers who have mastered all of this
• 21. me preparing to mutate some state
• 22. Not-Only SQL • most applications do not require the ﬂexibility a relational schema affords • • applications are written in terms of aggregates not relational schemas • • as-of store inputs and outputs • avoid accidental complexity: ORM, normal form persist aggregates making aggregates immutable affords • • separate reporting concerns from application concerns what-if, easy test and debug fits well with functional programs
• 23. JSON Documents Input • RunID • Contract Parameters • Asset Parameters • Dynamic Parameters • Market Parameters JobRequest • RunID • Contract • Interval Pure Function Contract Evaluation API Scheduler Contract Evaluation Job API Output • RunID • Revenue • Additional Information Document Store “Pure I/0” Input Output
• 25. Adoption: F# • Low risk • Runs on CLR and mono • Open source • Inter-op with legacy software and libraries • Back-out to C#
• 26. Adoption: Developers • Self taught • Hire good .NET developers, not language x developers • .NET developer cutting F# production code in a week • Functional programmer in a month
• 28. Approach exploratory DRYer REPL driven repeatedly re-factor test driven documented development
• 29. Self-host Web API let config = new HttpSelfHostConfiguration(baseAddress) config.MapHttpAttributeRoutes() config.Formatters.JsonFormatter.SerializerSettings <- JsonSerializerSettings( PreserveReferencesHandling = PreserveReferencesHandling.None, Converters = [| Json.TupleConverter() Json.OptionConverter() Json.ArrayConverter() Json.ListConverter() Json.MapTypeConverter() Json.UnionTypeConverter() |]) config.DependencyResolver <- new UnityResolver(container) F# type JSON converters
• 30. Topshelf Windows Service F# working with an existing OO framework HostFactory.Run(fun hc -> hc.UseLog4Net("log4net.config") hc.SetServiceName("Job.Api.Host") hc.SetDisplayName("E.ON Ancillary Services Job API Host") hc.SetDescription("An API service for Ancillary Services Jobs.") hc.RunAsNetworkService() |> ignore hc.Service<ApiService>(fun (s: ServiceConfigurator<ApiService>) -> s.ConstructUsing(fun (name: string) -> new ApiService(config)) |> ignore s.WhenStarted(fun (svc: ApiService) -> jobRequestQueue.Start() svc.Start()) |> ignore s.WhenStopped(fun (svc: ApiService) -> svc.Stop() jobRequestQueue.Stop()) |> ignore) |> ignore)
• 31. Web API Service an F# class!!! type ApiService(config: HttpSelfHostConfiguration) = ! member val Server = new HttpSelfHostServer(config) with get, set ! member this.Start() = this.Server.OpenAsync().Wait() member this.Stop() = if this.Server <> null then this.Server.CloseAsync().Wait() this.Server.Dispose()
• 32. Web API Controller another F# class!!! type JobController(log: ILog, jobRequestQueue: JobRequestQueue) inherit ApiController() ! [<Route("job/ping")>] member x.Get() = log.Debug("ping!!!") "pong" [<Route("job")>] member x.Post(request:JobRequest) = jobRequestQueue.Add(request)
• 33. agents: the safe way to manage state Job Queue let requests = BlockingQueueAgent<JobRequest>(config.JobRequestQueueLength) ! let workerName (i: int) = String.Format("worker[{0}]", i) ! let worker (workerName: string) = async { while true do log.DebugFormat("{0} free", workerName) let! request = requests.AsyncGet() log.DebugFormat("{0} busy: job {1}", workerName, request.JobId) run request } async, efficient use of threads ! for i in 1 .. config.JobRequestWorkers do Async.Start(workerName i |> worker, CancellationToken.Token) ! requests.Add(request) scale workers github.com/fsprojects/fsharpx/blob/master/src/FSharpx.Core/Agents/ BlockingQueueAgent.fs
• 34. Execute Job composition of async computations async { let! input = buildRequest dataProvider let! output = sendToCompute input let result = buildModel input output do! store result } {
• 35. Post dispose of resource when done async { use! response = httpClient.PostAsync(uri, toContent request) |> Async.AwaitTask return! response.EnsureSuccessStatusCode().Content.ReadAsStringAsync() |> Async.AwaitTask } F# async works with TPL Tasks
• 36. API Call catch exceptions as Choice2Of2 Async.RunSynchronously( post client config.JobUri request |> Async.Catch, config.Timeout) |> Choice.choice (fun _ -> log.InfoFormat("Executed Job [{0}]", request.JobId)) (fun exn -> log.Error(String.Format("Failed Job [{0}]", request.JobId), exn)) FSharpx github.com/fsprojects/fsharpx/blob/master/src/FSharpx.Core/ ComputationExpressions/Monad.fs
• 37. FR Calculation Request type FrequencyResponseCalculationRequest = { Interval: Interval.Time.T InitialState: FRUnitState ContractParameters: Line.Time.T<ContractParameters> Instruction: Line.Time.T<Instruction> Mel: Line.Time.T<float<MW>> Sel: Line.Time.T<float<MW>> AdjustedPN: Line.Time.T<float<MW>> ActualFrequencies: Line.Time.T<float<Hz>> TargetFrequencies: Line.Time.T<float<Hz>> MarketPrices: Line.Time.T<float<``£``/(MW h)>> } ubiquitous language
• 38. Ubiquitous Language [<Measure>] [<Measure>] [<Measure>] [<Measure>] type type type type min hh h MW units of measure ! type type type type Interval<'t> = 't * 't Point<'x,'y> = 'x * 'y Segment<'x,'y> = Point<'x,'y> * Point<'x,'y> Line<'x,'y> = Segment<'x,'y> list all missing concepts from C# solution
• 39. Ubiquitous Language (Revised) segment is an event module Segment = ! segment holds a value over an interval type T<'x,'y> = | Instantaneous of Point.T<'x,'y> | Discrete of IntervalType.T * Interval.T<'x> * 'y | Continuous of Point.T<'x,'y> * Point.T<'x,'y> segment between two data points
• 40. module Units = ! Units of Measure [<AutoOpen>] module UnitNames = /// a unit of time [<Measure>] type minute /// a unit of time [<Measure>] type halfhour /// a unit of time [<Measure>] type hour /// a unit of active power [<Measure>] type megawatt /// a unit of energy [<Measure>] type poundssterling /// a unit of frequency [<Measure>] type hertz [<AutoOpen>] module UnitSymbols = /// a synonym for halfhour, a unit of time [<Measure>] type min = minute /// a synonym for halfhour, a unit of time [<Measure>] type hh = halfhour /// a synonym for hour, a unit of time [<Measure>] type h = hour /// a synonym for megawatt, a unit of power [<Measure>] type MW = megawatt /// a synonym for pounds sterling, a unit of currency [<Measure>] type ``£`` = poundssterling /// a synonym for hertz, a unit of frequency [<Measure>] type Hz = hertz https://github.com/fsharp/fsharp/blob/master/src/fsharp/FSharp.Core/SI.fs
• 41. Units of Measure // Conversion constants let minutePerHalfhour = 30.0<min>/1.0<hh> let minutePerHour = 60.0<min>/1.0<h> let halfhourPerMinute = 1.0<hh>/30.0<min> let halfhourPerHour = 2.0<hh>/1.0<h> let hourPerMinute = 1.0<h>/60.0<min> let hourPerHalfhour = 1.0<h>/2.0<hh> module Minute = let toHalfhour (a:float<min>) = a * halfhourPerMinute let toHour (a:float<min>) = a * hourPerMinute let inline lift a = LanguagePrimitives.FloatWithMeasure<min>(float a) let liftTimeSpan (t:TimeSpan) = lift t.TotalMinutes
• 42. Contract Evaluation to p let run interval initialState parameters actualFrequencies targetFrequencies marketPrices pdtmLine = let deloadLine = DeloadLineCalculation.run … let holdingPayments = holdingPayments … let referencePrices = ReferencePriceCalculation.run … responseEnergyPayments … se cr et … but it involves a fold
• 43. Testing // Straight forward implementation ! let rec reverse = function | [] -> [] | x::xs -> reverse xs @ [x] ! // Efficient implementation ! let rec revAcc xs acc = match xs with | [] -> acc | h::t -> revAcc t (h::acc) ! let rev xs = match xs with | [] -> xs | [_] -> xs | h1::h2::t -> revAcc t [h2;h1] ! // Generate random tests to see if they behave the same ! Check.Quick(fun (xs:int list) -> reverse xs = rev xs) github.com/fsharp/FsCheck
• 44. Testing nice names open NUnit.Framework open FsUnit ! [<TestFixture; Category("Unit")>] type ``When I run the deload line calculation`` () = ! [<Test>] member x.``with empty MEL line and empty PN line then the deload line is correct`` () = let melLine = Line.empty let pnLine = Line.empty let actual = DeloadLineCalculation.run melLine pnLine let expected : Line.Time.T<float<MW>> = Line.empty actual |> should equal expected structural equality for free github.com/fsharp/FsUnit
• 46. C# F# Two Implementations of the Same Application 400000 30,801 348,430 21,442 Lines of Code 300000 305,566 200000 16,667 163,276 100000 643 56,929 487 53,270 9,359 42,864 3,630 29,080 0 15 3,011 Braces Blanks Null Checks Comments Useful Code App Code Test Code Total Code
• 47. … things aren’t looking good for the old way of doing things
• 48. Logging LOC
• 49. Exception Handling LOC
• 50. Test Code Ratio
• 51. Performance
• 52. … and ﬁnally say yes to NOOO
• 53. Manifesto for Not Only Object-Oriented Development! We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: ! • • • • • 0b ov Functions and Types over classes 0, er si gn 00 Purity over mutability 0, at Composition over inheritance or 0 0 0 Higher-order functions over method dispatch ie s! Options over nulls 10 ! That is, while there is value in the items on the right (except for nulls), we value the items on the left more. notonlyoo.org
• 54. @simontcousins simontylercousins.net www.slideshare.net/simontcousins/time-for-functions