Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Jon Harrop, Flying Frog Consultancy
 Jon Harrop
 Started programming in 1981
BASIC, Pascal, 6502, ARM, C, C++
 At British Petroleum in 1995
UFI
 Physics a...
 One of the world’s largest insurance
companies.
 One of the world’s largest insurance
companies.
 Actually, almost everyone else too…
1. An ordinary delivery process
2. Streamlining the delivery process
3. Technical aspects of the solution
 Automate repetition
◦ Traders in a pit
◦ Actuaries with calculators
 Use computers!
 Much better performance
◦ Latency
◦ Throughput
 More sophisticated algorithms
1. Business idea!
1. Business idea!
2. Model idea: MS Excel.
1. Business idea!
2. Model idea: MS Excel.
3. Build idea: C++/Java/C#.
1. Business idea!
2. Model idea: MS Excel.
3. Build idea: C++/Java/C#.
4. Test idea: NUnit (feedback to 2)
1. Business idea!
2. Model idea: MS Excel.
3. Build idea: C++/Java/C#.
4. Test idea: NUnit (feedback to 2)
5. Deploy produ...
Room for improvement in:
 Classic waterfall approach.
◦ Iterations are an absolute killer.
 Delivery speed and cost.
◦ T...
 First iteration highlights issues.
◦ Civilized discussion.
 First iteration highlights issues.
◦ Civilized discussion.
 Second iteration, more issues.
◦ Stressful disagreements.
 First iteration highlights issues.
◦ Civilized discussion.
 Second iteration, more issues.
◦ Stressful disagreements.
...
Big problems:
 Accuracy.
◦ Excel/Word are too free-form.
 Complexity.
◦ Models are complicated, lots of dispatch.
 Late...
Old process:
 See the forest for the trees.
◦ Don’t just rewrite code in language du jour!!!
◦ Common mistake when adopting new techno...
A new process:
Observations
Observations
 Separate people/jobs
◦ One person uses Tool to turn ideas into products.
◦ Another person maintains Tool.
...
Advantages:
 Business people ship products themselves!
 Less contention
◦ Faster iterations
◦ Quicker time-to-market
◦ C...
Challenges:
 Designing bespoke Tool
◦ Hard to gather requirements:“Define the language
you dream in?”
 Training
◦ Users
...
Build a new tool for business users that:
 Has a great user experience.
◦ Graphical application.
◦ Interoperates with Exc...
DSLs have a rich history but GUIs are notably
absent…
DSL Patterns by Martin Fowler
No need to ship a DSL that looks worse than
this:
No need to ship a DSL that looks worse than
this:
1974
Started with a working demo
 Developed in one week!
 Slick looking
 Easy to use
 Bespoke language
 Integrated testing...
Final solution:
 Development effort
◦ 3 months to design
◦ 3 months to build
 Code size
◦ 10kLOC of F#
 Performance
◦ 2...
A different 3 month project:
 114µs latency.
 200,000 tps.
 With fault tolerance!
 Async message passing over Infiniba...
 Training
◦ On-the-fly
◦ 30 people
◦ 5 teams
 Emergency
◦ Valuable change came in late
◦ Estimated 9 months and £1m
◦ Co...
“Pattern matching is quite neat, as is the partial
application of functions”
“rapidly coming to appreciate the benefits
wh...
Benefits
 Delivery speed
◦ From idea to production 100x faster
 Delivery costs
◦ Maybe 10,000x lower?!
 F# is good for programmatic GUIs
 Approach described in the F# Journal
 Custom Control union type
 Compiled to WPF Co...
Nice name!
Input
Variables
Patterns
Output
Variables
Expressions
Take a leaf out of Excel:
 Editable content:
 Display version:
Display version:
Editable content:
A bit like type checking…
type Control =
| Gap
| Label of string
| Title of string
| Dock of (Position * Control) list
| Ctrl of FrameworkElement
| …
let rec make : Control -> FrameworkElement = function
| Gap ->
let ctrl = new FrameworkElement()
ctrl.HorizontalAlignment ...
let ui =
Dock
[ Top, Title "Bespoke business rules engine"
Bottom,
Tabs
[ Label "Inputs",
Dock[ Top, Label "This version w...
let StringValue(value: Value) =
let label = Controls.Label()
let textbox = Controls.TextBox()
let setContent = function
| ...
let UnionValue(allCases, value: Value) =
let label = Controls.Label()
let combo = Controls.ComboBox()
let itemOf case = ca...
The expression language:
type Expr =
| EBool of bool
| ENumber of decimal
| EString of string
| EDate of System.DateTime
|...
The expression interpreter:
let rec eval vars expr =
match expr with
| EBool b -> VBool b
| ENumber x -> VNumber x
| EStri...
Challenges
 Slow startup times
 Poor GUI performance (grid of numbers)
 Minor perf bugs
 Brittle .NET serialization
Startup times
Problems:
 64-bit NGEN bug
 Poor WPF Grid performance
Solutions:
 Use 32-bit client
 Custom WPF control…
Startup times
 Took two days to fix
 Startup times down from 15s to 0.1s
GUI performance
Problem:
 The WPF Grid control is:
◦ very slow to start up
◦ very slow to scroll
Solution:
 Custom FastG...
GUI performance
 16x faster startup than vanilla WPF.
 Solution in the F# Journal.
 Still a lot of room for improvement…
Minor perf bugs
 sprintf “%A”
 Decimal slow (but ok)
Serialization
Problem:
 .NET serialization not great for F#.
 Bloated format (includes types).
 Brittle when loading (i...
Serialization
write true = "<bool>True</bool>"
write {anInt=2; aReal=3.4m} = "<record><
field><name>anInt</name><value><in...
Structurally-typed serialization handles:
 unit
 bool
 byte, int, float, decimal
 System.DateTime
 string
 Tuples, r...
Any questions, please contact me:
Jon Harrop <jon@ffconsultancy.com>
Using F# to change the way we work
Using F# to change the way we work
Upcoming SlideShare
Loading in …5
×

Using F# to change the way we work

4,058 views

Published on

This slide deck describes the most successful software project of my career: automating a 40-person human compiler yielding a reduction in project time from months to hours and cost from $1,000,000 to $100s for our client. The system is still in operation years later and has never had a single bug filed against it.

Published in: Software
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Using F# to change the way we work

  1. 1. Jon Harrop, Flying Frog Consultancy
  2. 2.  Jon Harrop  Started programming in 1981 BASIC, Pascal, 6502, ARM, C, C++  At British Petroleum in 1995 UFI  Physics and Chemistry at Cambridge Fortran, SML, OCaml, Mathematica, F#  Co-founded Flying Frog in 2005 ◦ Sell books, software and consultancy services ◦ Over 1,100 clients in industry using OCaml & F#
  3. 3.  One of the world’s largest insurance companies.
  4. 4.  One of the world’s largest insurance companies.  Actually, almost everyone else too…
  5. 5. 1. An ordinary delivery process 2. Streamlining the delivery process 3. Technical aspects of the solution
  6. 6.  Automate repetition ◦ Traders in a pit ◦ Actuaries with calculators  Use computers!
  7. 7.  Much better performance ◦ Latency ◦ Throughput  More sophisticated algorithms
  8. 8. 1. Business idea!
  9. 9. 1. Business idea! 2. Model idea: MS Excel.
  10. 10. 1. Business idea! 2. Model idea: MS Excel. 3. Build idea: C++/Java/C#.
  11. 11. 1. Business idea! 2. Model idea: MS Excel. 3. Build idea: C++/Java/C#. 4. Test idea: NUnit (feedback to 2)
  12. 12. 1. Business idea! 2. Model idea: MS Excel. 3. Build idea: C++/Java/C#. 4. Test idea: NUnit (feedback to 2) 5. Deploy production code. Room for improvement?
  13. 13. Room for improvement in:  Classic waterfall approach. ◦ Iterations are an absolute killer.  Delivery speed and cost. ◦ This is a long journey. So what does an iteration look like?
  14. 14.  First iteration highlights issues. ◦ Civilized discussion.
  15. 15.  First iteration highlights issues. ◦ Civilized discussion.  Second iteration, more issues. ◦ Stressful disagreements.
  16. 16.  First iteration highlights issues. ◦ Civilized discussion.  Second iteration, more issues. ◦ Stressful disagreements.  Later iterations. ◦ Wild fighting.
  17. 17. Big problems:  Accuracy. ◦ Excel/Word are too free-form.  Complexity. ◦ Models are complicated, lots of dispatch.  Latency. ◦ Complected testing.
  18. 18. Old process:
  19. 19.  See the forest for the trees. ◦ Don’t just rewrite code in language du jour!!! ◦ Common mistake when adopting new technologies.  Overhaul the delivery process…
  20. 20. A new process: Observations
  21. 21. Observations  Separate people/jobs ◦ One person uses Tool to turn ideas into products. ◦ Another person maintains Tool.  Separate tests ◦ Tests for the model. ◦ Unit test for the Tool. Advantages
  22. 22. Advantages:  Business people ship products themselves!  Less contention ◦ Faster iterations ◦ Quicker time-to-market ◦ Cheaper delivery ◦ Happier employees  Scalable delivery process ◦ Add more tool users ◦ Add more tool developers
  23. 23. Challenges:  Designing bespoke Tool ◦ Hard to gather requirements:“Define the language you dream in?”  Training ◦ Users ◦ Developers  Plumbing  Deployment
  24. 24. Build a new tool for business users that:  Has a great user experience. ◦ Graphical application. ◦ Interoperates with Excel.  Creates precise specifications. ◦ Catch errors as early as possible.  Mirrors the production environment. ◦ Code reuse.  Integrates testing.  Is tuned to their needs.
  25. 25. DSLs have a rich history but GUIs are notably absent… DSL Patterns by Martin Fowler
  26. 26. No need to ship a DSL that looks worse than this:
  27. 27. No need to ship a DSL that looks worse than this: 1974
  28. 28. Started with a working demo  Developed in one week!  Slick looking  Easy to use  Bespoke language  Integrated testing  Their own worked examples included  Demo runs on their machines (WPF)
  29. 29. Final solution:  Development effort ◦ 3 months to design ◦ 3 months to build  Code size ◦ 10kLOC of F#  Performance ◦ 20,000 msg/s ◦ Under 35ms door-to-door latency
  30. 30. A different 3 month project:  114µs latency.  200,000 tps.  With fault tolerance!  Async message passing over Infiniband  Low-latency allocationless F# code
  31. 31.  Training ◦ On-the-fly ◦ 30 people ◦ 5 teams  Emergency ◦ Valuable change came in late ◦ Estimated 9 months and £1m ◦ Completed in 24 hours  Maintenance ◦ All done by client now ◦ After one week of F# training ◦ Ordinary developers
  32. 32. “Pattern matching is quite neat, as is the partial application of functions” “rapidly coming to appreciate the benefits when it comes to refactoring code” “Our plans are getting bigger each week as we can see more and more flexibility, it’s great!”
  33. 33. Benefits  Delivery speed ◦ From idea to production 100x faster  Delivery costs ◦ Maybe 10,000x lower?!
  34. 34.  F# is good for programmatic GUIs  Approach described in the F# Journal  Custom Control union type  Compiled to WPF Control  Custom Editable Control  Reuse for editable strings, unions etc.  Compose into a UI tree  Peek Value property to get the “rules”
  35. 35. Nice name!
  36. 36. Input Variables
  37. 37. Patterns
  38. 38. Output Variables
  39. 39. Expressions
  40. 40. Take a leaf out of Excel:  Editable content:  Display version:
  41. 41. Display version:
  42. 42. Editable content: A bit like type checking…
  43. 43. type Control = | Gap | Label of string | Title of string | Dock of (Position * Control) list | Ctrl of FrameworkElement | …
  44. 44. let rec make : Control -> FrameworkElement = function | Gap -> let ctrl = new FrameworkElement() ctrl.HorizontalAlignment <- HorizontalAlignment.Stretch ctrl.VerticalAlignment <- VerticalAlignment.Stretch ctrl | Label s -> upcast Controls.Label(Content=s) | Title s -> upcast Controls.TextBlock(Text=s, FontSize=24.0) | Dock xs -> let panel = Controls.DockPanel() for p, ctrl in xs do let ctrl = make ctrl Controls.DockPanel.SetDock(ctrl, p.AsWPF) panel.Children.Add ctrl |> ignore upcast panel | …
  45. 45. let ui = Dock [ Top, Title "Bespoke business rules engine" Bottom, Tabs [ Label "Inputs", Dock[ Top, Label "This version was…" Bottom, Scroll inputs ] Label "Rules", Dock[ Top, Label "The following rules…" Bottom, Scroll(Ctrl rulesView) ]]]
  46. 46. let StringValue(value: Value) = let label = Controls.Label() let textbox = Controls.TextBox() let setContent = function | VString text -> label.Content <- text textbox.Text <- text | _ -> failwith "Invalid value" let ctrl = Editable<Value>(value, label, textbox, setContent) textbox.TextChanged.Add(fun _ -> ctrl.SetValue(Some textbox.Text |> Option.map VString)) ctrl
  47. 47. let UnionValue(allCases, value: Value) = let label = Controls.Label() let combo = Controls.ComboBox() let itemOf case = case, Controls.ComboBoxItem(Content=case) let items = Map[for case in allCases -> itemOf case] let setContent = function | VCase case -> label.Content <- case combo.SelectedItem <- items.[case] | value -> failwith "Invalid value" let ctrl = Editable<Value>(value, label, combo, setContent) combo.ItemsSource <- [ for KeyValue(_, ctrl) in items -> ctrl ] for KeyValue(case, x) in items do x.Selected.Add(fun _ -> ctrl.SetValue(Some(VCase case))) ctrl
  48. 48. The expression language: type Expr = | EBool of bool | ENumber of decimal | EString of string | EDate of System.DateTime | ETuple of Expr list | EVar of string | …
  49. 49. The expression interpreter: let rec eval vars expr = match expr with | EBool b -> VBool b | ENumber x -> VNumber x | EString s -> VString s | EDate d -> VDate d | ETuple fs -> VTuple(List.map (eval vars) fs) | EVar v -> match Map.tryFind v vars with | None -> failwithf "Unknown variable '%s'" v | Some x -> x | …
  50. 50. Challenges  Slow startup times  Poor GUI performance (grid of numbers)  Minor perf bugs  Brittle .NET serialization
  51. 51. Startup times Problems:  64-bit NGEN bug  Poor WPF Grid performance Solutions:  Use 32-bit client  Custom WPF control…
  52. 52. Startup times  Took two days to fix  Startup times down from 15s to 0.1s
  53. 53. GUI performance Problem:  The WPF Grid control is: ◦ very slow to start up ◦ very slow to scroll Solution:  Custom FastGrid control…  Low-level rendering in F# ◦ Calls DrawGlyphs
  54. 54. GUI performance  16x faster startup than vanilla WPF.  Solution in the F# Journal.  Still a lot of room for improvement…
  55. 55. Minor perf bugs  sprintf “%A”  Decimal slow (but ok)
  56. 56. Serialization Problem:  .NET serialization not great for F#.  Bloated format (includes types).  Brittle when loading (incompatible types). Solution:  Custom structural serialization library.  Handles values of all F# types.  Solution in the F# Journal.
  57. 57. Serialization write true = "<bool>True</bool>" write {anInt=2; aReal=3.4m} = "<record>< field><name>anInt</name><value><int>2</i nt></value></field><field><name>aReal</n ame><value><decimal>3.4</decimal></value ></field></record>"
  58. 58. Structurally-typed serialization handles:  unit  bool  byte, int, float, decimal  System.DateTime  string  Tuples, records, unions  Lists, arrays, sets and maps
  59. 59. Any questions, please contact me: Jon Harrop <jon@ffconsultancy.com>

×