Railway Oriented Programming

276,427 views

Published on

(Video of these slides here http://fsharpforfunandprofit.com/rop)
(My response to "this is just Either" here: http://fsharpforfunandprofit.com/rop/#monads)

Many examples in functional programming assume that you are always on the "happy path". But to create a robust real world application you must deal with validation, logging, network and service errors, and other annoyances.

So, how do you handle all this in a clean functional way? This talk will provide a brief introduction to this topic, using a fun and easy-to-understand railway analogy.

Published in: Technology
  • This is a beautifully clear tutorial Scott. Thanks for putting it together. Should be a lot more programming using this model. We just rewrote a chunk of our DNA compiler using this pattern and dramatically improved error reporting and reduced size of code.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • More than 5000 IT Certified ( SAP,Oracle,Mainframe,Microsoft and IBM Technologies etc...)Consultants registered. Register for IT courses at http://www.todaycourses.com Most of our companies will help you in processing H1B Visa, Work Permit and Job Placements
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Brilliant Document. I write F# code and love it. Railway programming as a frame of reference makes it easier to sell F# / functional story.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I don't know F#. but it helps me a lot to understand functional approach. thank you.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @Ricardo Thanks!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Railway Oriented Programming

  1. 1. Railway Oriented Programming A functional approach to error handling What do railways have to do with programming?
  2. 2. Railway Oriented Programming A functional approach to error handling Scott Wlaschin @ScottWlaschin fsharpforfunandprofit.com FPbridge.co.uk...but OCaml and Haskell are very similar. Examples will be in F#...
  3. 3. Overview Topics covered: • Happy path programming • Straying from the happy path • Introducing "Railway Oriented Programming" • Using the model in practice • Extending and improving the design
  4. 4. Happy path programming Implementing a simple use case
  5. 5. A simple use case Receive request Validate and canonicalize request Update existing user record Send verification email Return result to user type Request = { userId: int; name: string; email: string } "As a user I want to update my name and email address"
  6. 6. Imperative code string ExecuteUseCase() { var request = receiveRequest(); validateRequest(request); canonicalizeEmail(request); db.updateDbFromRequest(request); smtpServer.sendEmail(request.Email); return "Success"; }
  7. 7. Functional flow let executeUseCase = receiveRequest >> validateRequest >> canonicalizeEmail >> updateDbFromRequest >> sendEmail >> returnMessage F# left-to-right composition operator
  8. 8. Straying from the happy path... What do you do when something goes wrong?
  9. 9. Straying from the happy path
  10. 10. “A program is a spell cast over a computer, turning input into error messages”
  11. 11. “A program is a spell cast over a computer, turning input into error messages”
  12. 12. Straying from the happy path Name is blank Email not valid Receive request Validate and canonicalize request Update existing user record Send verification email Return result to user User not found Db error Authorization error Timeout "As a user I want to update my name and email address" type Request = { userId: int; name: string; email: string } - and see sensible error messages when something goes wrong!
  13. 13. Imperative code with error handling string UpdateCustomerWithErrorHandling() { var request = receiveRequest(); validateRequest(request); canonicalizeEmail(request); db.updateDbFromRequest(request); smtpServer.sendEmail(request.Email) return "OK"; }
  14. 14. Imperative code with error handling string UpdateCustomerWithErrorHandling() { var request = receiveRequest(); var isValidated = validateRequest(request); if (!isValidated) { return "Request is not valid" } canonicalizeEmail(request); db.updateDbFromRequest(request); smtpServer.sendEmail(request.Email) return "OK"; }
  15. 15. Imperative code with error handling string UpdateCustomerWithErrorHandling() { var request = receiveRequest(); var isValidated = validateRequest(request); if (!isValidated) { return "Request is not valid" } canonicalizeEmail(request); var result = db.updateDbFromRequest(request); if (!result) { return "Customer record not found" } smtpServer.sendEmail(request.Email) return "OK"; }
  16. 16. Imperative code with error handling string UpdateCustomerWithErrorHandling() { var request = receiveRequest(); var isValidated = validateRequest(request); if (!isValidated) { return "Request is not valid" } canonicalizeEmail(request); try { var result = db.updateDbFromRequest(request); if (!result) { return "Customer record not found" } } catch { return "DB error: Customer record not updated" } smtpServer.sendEmail(request.Email) return "OK"; }
  17. 17. Imperative code with error handling string UpdateCustomerWithErrorHandling() { var request = receiveRequest(); var isValidated = validateRequest(request); if (!isValidated) { return "Request is not valid" } canonicalizeEmail(request); try { var result = db.updateDbFromRequest(request); if (!result) { return "Customer record not found" } } catch { return "DB error: Customer record not updated" } if (!smtpServer.sendEmail(request.Email)) { log.Error "Customer email not sent" } return "OK"; }
  18. 18. Imperative code with error handling string UpdateCustomerWithErrorHandling() { var request = receiveRequest(); var isValidated = validateRequest(request); if (!isValidated) { return "Request is not valid" } canonicalizeEmail(request); try { var result = db.updateDbFromRequest(request); if (!result) { return "Customer record not found" } } catch { return "DB error: Customer record not updated" } if (!smtpServer.sendEmail(request.Email)) { log.Error "Customer email not sent" } return "OK"; }
  19. 19. Functional flow with error handling let updateCustomer = receiveRequest >> validateRequest >> canonicalizeEmail >> updateDbFromRequest >> sendEmail >> returnMessage Before
  20. 20. Functional flow with error handling let updateCustomerWithErrorHandling = receiveRequest >> validateRequest >> canonicalizeEmail >> updateDbFromRequest >> sendEmail >> returnMessage Does this look familiar? Don't believe me? Stay tuned! After
  21. 21. Designing the unhappy path
  22. 22. Request/response (non-functional) design Request Response Validate Update Send Request handling service Request Response Validate Update Send Request handling service Request Errors Response Validate Update Send Request handling service Imperative code can return early
  23. 23. Data flow (functional) design ResponseValidate Update SendA single function representing the use caseRequest Request ResponseValidate Update Send A single function representing the use case Request Errors Success Response Validate Update Send Error Response A single function representing the use case Q: How can you bypass downstream functions when an error happens?
  24. 24. Functional design How can a function have more than one output? type Result = | Success | ValidationError | UpdateError | SmtpError Request Errors SuccessValidate Update Send Failure A single function representing the use case
  25. 25. Functional design How can a function have more than one output? type Result = | Success | Failure Request Errors SuccessValidate Update Send Failure A single function representing the use case
  26. 26. Functional design How can a function have more than one output? type Result<'TEntity> = | Success of 'TEntity | Failure of string Request Errors SuccessValidate Update Send Failure A single function representing the use case
  27. 27. Functional design Request Errors SuccessValidate Update Send Failure A single function representing the use case • Each use case will be equivalent to a single function • The function will return a sum type with two cases: "Success" and "Failure". • The use case function will be built from a series of smaller functions, each representing one step in a data flow. • The errors from each step will be combined into a single "failure" path.
  28. 28. How do I work with errors in a functional way?
  29. 29. Monad dialog
  30. 30. v
  31. 31. Monads are confusing? Monads are not really that confusing http://bit.ly/monad-paper
  32. 32. Railway oriented programming This has absolutely nothing to do with monads.
  33. 33. A railway track analogy The Tunnel of Transformation Function apple -> banana
  34. 34. A railway track analogy Function 1 apple -> banana Function 2 banana -> cherry
  35. 35. A railway track analogy >> Function 1 apple -> banana Function 2 banana -> cherry
  36. 36. A railway track analogy New Function apple -> cherry Can't tell it was built from smaller functions!
  37. 37. An error generating function Request SuccessValidate Failure let validateInput input = if input.name = "" then Failure "Name must not be blank" else if input.email = "" then Failure "Email must not be blank" else Success input // happy path
  38. 38. Introducing switches Success! Failure Input ->
  39. 39. Connecting switches Validate UpdateDbon success bypass
  40. 40. Connecting switches Validate UpdateDb
  41. 41. Connecting switches Validate UpdateDb SendEmail
  42. 42. Connecting switches Validate UpdateDb SendEmail
  43. 43. The two-track model in practice
  44. 44. Composing switches Validate UpdateDb SendEmail
  45. 45. Composing switches Validate UpdateDb SendEmail
  46. 46. Composing switches Validate UpdateDb SendEmail >> >> Composing one-track functions is fine...
  47. 47. Composing switches Validate UpdateDb SendEmail >> >> ... and composing two-track functions is fine...
  48. 48. Composing switches Validate UpdateDb SendEmail   ... but composing switches is not allowed!
  49. 49. Composing switches Validate Two-track input Two-track input Validate One-track input Two-track input  
  50. 50. Building an adapter block Two-track input Slot for switch function Two-track output
  51. 51. Building an adapter block Two-track input Two-track output Validate Validate
  52. 52. let adapt switchFunction = fun twoTrackInput -> match twoTrackInput with | Success s -> switchFunction s | Failure f -> Failure f Building an adapter block Two-track input Two-track output
  53. 53. let adapt switchFunction = fun twoTrackInput -> match twoTrackInput with | Success s -> switchFunction s | Failure f -> Failure f Building an adapter block Two-track input Two-track output
  54. 54. let adapt switchFunction = fun twoTrackInput -> match twoTrackInput with | Success s -> switchFunction s | Failure f -> Failure f Building an adapter block Two-track input Two-track output
  55. 55. let adapt switchFunction = fun twoTrackInput -> match twoTrackInput with | Success s -> switchFunction s | Failure f -> Failure f Building an adapter block Two-track input Two-track output
  56. 56. Bind as an adapter block Two-track input Two-track output let bind switchFunction = fun twoTrackInput -> match twoTrackInput with | Success s -> switchFunction s | Failure f -> Failure f bind : ('a -> TwoTrack<'b>) -> TwoTrack<'a> -> TwoTrack<'b>
  57. 57. Bind as an adapter block Two-track input Two-track output let bind switchFunction twoTrackInput = match twoTrackInput with | Success s -> switchFunction s | Failure f -> Failure f bind : ('a -> TwoTrack<'b>) -> TwoTrack<'a> -> TwoTrack<'b>
  58. 58. name50 Bind example let nameNotBlank input = if input.name = "" then Failure "Name must not be blank" else Success input let name50 input = if input.name.Length > 50 then Failure "Name must not be longer than 50 chars" else Success input let emailNotBlank input = if input.email = "" then Failure "Email must not be blank" else Success input nameNotBlank emailNotBlank
  59. 59. Bind example nameNotBlank (combined with) name50 (combined with) emailNotBlank nameNotBlank name50 emailNotBlank
  60. 60. Bind example bind nameNotBlank bind name50 bind emailNotBlank nameNotBlank name50 emailNotBlank
  61. 61. Bind example bind nameNotBlank >> bind name50 >> bind emailNotBlank nameNotBlank name50 emailNotBlank
  62. 62. Bind example let validateRequest = bind nameNotBlank >> bind name50 >> bind emailNotBlank // validateRequest : TwoTrack<Request> -> TwoTrack<Request> validateRequest
  63. 63. Bind example let (>>=) twoTrackInput switchFunction = bind switchFunction twoTrackInput let validateRequest twoTrackInput = twoTrackInput >>= nameNotBlank >>= name50 >>= emailNotBlank validateRequest
  64. 64. Bind doesn't stop transformations FunctionB type TwoTrack<'TEntity> = | Success of 'TEntity | Failure of string FunctionA
  65. 65. Composing switches - review Validate UpdateDb SendEmail Validate UpdateDb SendEmail
  66. 66. Comic Interlude What do you call a train that eats toffee? I don't know, what do you call a train that eats toffee? A chew, chew train!
  67. 67. More fun with railway tracks... Working with other functions
  68. 68. More fun with railway tracks... Fitting other functions into this framework: • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions
  69. 69. Converting one-track functions Fitting other functions into this framework: • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions
  70. 70. Converting one-track functions // trim spaces and lowercase let canonicalizeEmail input = { input with email = input.email.Trim().ToLower() } canonicalizeEmail
  71. 71. Converting one-track functions UpdateDb SendEmailValidate canonicalizeEmail Won't compose
  72. 72. Converting one-track functions Two-track input Slot for one-track function Two-track output
  73. 73. Converting one-track functions Two-track input Two-track output CanonicalizeCanonicalize
  74. 74. Converting one-track functions Two-track input Two-track output let map singleTrackFunction = fun twoTrackInput -> match twoTrackInput with | Success s -> Success (singleTrackFunction s) | Failure f -> Failure f map : ('a -> 'b) -> TwoTrack<'a> -> TwoTrack<'b>` Single track function
  75. 75. Converting one-track functions Two-track input Two-track output let map singleTrackFunction = bind (singleTrackFunction >> Success) map : ('a -> 'b) -> TwoTrack<'a> -> TwoTrack<'b> Single track function
  76. 76. Converting one-track functions UpdateDb SendEmailValidate canonicalizeEmail Will compose
  77. 77. Converting dead-end functions Fitting other functions into this framework: • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions
  78. 78. Converting dead-end functions let updateDb request = // do something // return nothing at all updateDb
  79. 79. Converting dead-end functions SendEmailValidate UpdateDb Won't compose
  80. 80. Converting dead-end functions One-track input Slot for dead end function One-track output
  81. 81. Converting dead-end functions One-track input One-track output let tee deadEndFunction oneTrackInput = deadEndFunction oneTrackInput oneTrackInput tee : ('a -> unit) -> 'a -> 'a Dead end function
  82. 82. Converting dead-end functions SendEmailValidate UpdateDb Will compose
  83. 83. Functions that throw exceptions Fitting other functions into this framework: • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions
  84. 84. Functions that throw exceptions One-track input Two-track output SendEmail SendEmail Add try/catch to handle timeouts, say Looks innocent, but might throw an exception
  85. 85. Functions that throw exceptions EvenYoda recommends not to use exception handling for control flow: Guideline: Convert exceptions into Failures "Do or do not, there is no try".
  86. 86. Supervisory functions Fitting other functions into this framework: • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions
  87. 87. Supervisory functions Two-track input Two-track output Slot for one-track function for Success case Slot for one-track function for Failure case
  88. 88. Putting it all together
  89. 89. Putting it all together Validate UpdateDb SendEmail Canonicalize Input Output??
  90. 90. Putting it all together Validate UpdateDb SendEmail Canonicalize returnMessage Input Output let returnMessage result = match result with | Success obj -> OK obj.ToJson() | Failure msg -> BadRequest msg
  91. 91. Putting it all together - review The "two-track" framework is a useful approach for most use-cases. You can fit most functions into this model.
  92. 92. Putting it all together - review The "two-track" framework is a useful approach for most use-cases. let updateCustomer = receiveRequest >> validateRequest >> updateDbFromRequest >> sendEmail >> returnMessage let updateCustomer = receiveRequest >> validateRequest >> updateDbFromRequest >> sendEmail >> returnMessage Let's look at the code -- before and after adding error handling
  93. 93. Comic Interlude Why can't a steam locomotive sit down? I don't know, why can't a steam locomotive sit down? Because it has a tender behind!
  94. 94. Extending the framework: • Designing for errors • Parallel tracks • Domain events
  95. 95. Designing for errors Unhappy paths are requirements too
  96. 96. Designing for errors let validateInput input = if input.name = "" then Failure "Name must not be blank" else if input.email = "" then Failure "Email must not be blank" else Success input // happy path type TwoTrack<'TEntity> = | Success of 'TEntity | Failure of string Using strings is not good
  97. 97. Designing for errors let validateInput input = if input.name = "" then Failure NameMustNotBeBlank else if input.email = "" then Failure EmailMustNotBeBlank else Success input // happy path type TwoTrack<'TEntity> = | Success of 'TEntity | Failure of ErrorMessage type ErrorMessage = | NameMustNotBeBlank | EmailMustNotBeBlank Special type rather than string
  98. 98. Designing for errors let validateInput input = if input.name = "" then Failure NameMustNotBeBlank else if input.email = "" then Failure EmailMustNotBeBlank else if (input.email doesn't match regex) then Failure EmailNotValid input.email else Success input // happy path type ErrorMessage = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress Add invalid email as data
  99. 99. Designing for errors type ErrorMessage = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress // database errors | UserIdNotValid of UserId | DbUserNotFoundError of UserId | DbTimeout of ConnectionString | DbConcurrencyError | DbAuthorizationError of ConnectionString * Credentials // SMTP errors | SmtpTimeout of SmtpConnection | SmtpBadRecipient of EmailAddress Documentation of everything that can go wrong -- And it's type-safe documentation that can't go out of date!
  100. 100. Designing for errors type ErrorMessage = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress // database errors | UserIdNotValid of UserId | DbUserNotFoundError of UserId | DbTimeout of ConnectionString | DbConcurrencyError | DbAuthorizationError of ConnectionString * Credentials // SMTP errors | SmtpTimeout of SmtpConnection | SmtpBadRecipient of EmailAddress Documentation of everything that can go wrong -- And it's type-safe documentation that can't go out of date! As we develop the code, we can build up a complete list of everything that could go wrong
  101. 101. Designing for errors – converting to strings No longer works – each case must now be explicitly converted to a string returnMessage let returnMessage result = match result with | Success _ -> "Success" | Failure msg -> msg
  102. 102. Designing for errors – converting to strings let returnMessage result = match result with | Success _ -> "Success" | Failure err -> match err with | NameMustNotBeBlank -> "Name must not be blank" | EmailMustNotBeBlank -> "Email must not be blank" | EmailNotValid (EmailAddress email) -> sprintf "Email %s is not valid" email // database errors | UserIdNotValid (UserId id) -> sprintf "User id %i is not a valid user id" id | DbUserNotFoundError (UserId id) -> sprintf "User id %i was not found in the database" id | DbTimeout (_,TimeoutMs ms) -> sprintf "Could not connect to database within %i ms" ms | DbConcurrencyError -> sprintf "Another user has modified the record. Please resubmit" | DbAuthorizationError _ -> sprintf "You do not have permission to access the database" // SMTP errors | SmtpTimeout (_,TimeoutMs ms) -> sprintf "Could not connect to SMTP server within %i ms" ms | SmtpBadRecipient (EmailAddress email) -> sprintf "The email %s is not a valid recipient" email Each case must be converted to a string – but this is only needed once, and only at the last step. All strings are in one place, so translations are easier. returnMessage (or use resource file)
  103. 103. Designing for errors - review type ErrorMessage = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress // database errors | UserIdNotValid of UserId | DbUserNotFoundError of UserId | DbTimeout of ConnectionString | DbConcurrencyError | DbAuthorizationError of ConnectionString * Credentials // SMTP errors | SmtpTimeout of SmtpConnection | SmtpBadRecipient of EmailAddress Documentation of everything that can go wrong. Type-safe -- can't go out of date!
  104. 104. Parallel tracks
  105. 105. Parallel validation nameNotBlank name50 emailNotBlank Problem: Validation done in series. So only one error at a time is returned
  106. 106. Parallel validation nameNotBlank name50 emailNotBlank Split input Combine output Now we do get all errors at once! ... But how to combine?
  107. 107. Combining switches + Trick: if we create an operation that combines pairs into a new switch, we can repeat to combine as many switches as we like.
  108. 108. Combining switches + + Trick: if we create an operation that combines pairs into a new switch, we can repeat to combine as many switches as we like.
  109. 109. Combining switches + + Trick: if we create an operation that combines pairs into a new switch, we can repeat to combine as many switches as we like.
  110. 110. Domain events Communicating information to downstream functions
  111. 111. Events are not errors Validate UpdateDb SendEmail Tell CRM that email was sent
  112. 112. Events are not errors Validate UpdateDb SendEmail Tell CRM that email was sent type MyUseCaseMessage = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress // database errors | UserIdNotValid of UserId // SMTP errors | SmtpTimeout of SmtpConnection // Domain events | UserSaved of AuditInfo | EmailSent of EmailAddress * MsgId
  113. 113. Events are not errors Validate UpdateDb SendEmail Tell CRM that email was sent type MyUseCaseMessage = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress // database errors | UserIdNotValid of UserId // SMTP errors | SmtpTimeout of SmtpConnection // Domain events | UserSaved of AuditInfo | EmailSent of EmailAddress * MsgId type TwoTrack<'TEntity> = | Success of 'TEntity * Message list | Failure of Message list
  114. 114. Comic Interlude Why can't a train driver be electrocuted? I don't know, why can't a train driver be electrocuted? Because he's not a conductor!
  115. 115. Some topics not covered... ... but could be handled in an obvious way.
  116. 116. Topics not covered • Errors across service boundaries • Async on success path (instead of sync) • Compensating transactions (instead of two phase commit) • Logging (tracing, app events, etc.)
  117. 117. Summary A recipe for handling errors in a functional way
  118. 118. Recipe for handling errors in a functional way type TwoTrack<'TEntity> = | Success of 'TEntity * Message list | Failure of Message list Validate UpdateDb SendEmail type Message = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress
  119. 119. Demo http://bit.ly/rop-example
  120. 120. I don’t always have errors...
  121. 121. I don’t always have errors... Railway Oriented Programming @ScottWlaschin fsharpforfunandprofit.com FPbridge.co.uk http://bit.ly/rop-example

×