From Zero to Haskell: Lessons Learned

1. From Zero to Haskell: Lessons Learned ZuriHac 2019 Susan Potter 2019-06-15

2. finger $(whoami) Name: Susan Potter Logged in since Sun Jan 18 18:30 1996 (GMT) on tty1 - 23 years writing software in industry - Server-side/backend/infrastructure engineering, mostly - Former SRE; really care about operational ergonomics Today: - Building new backend services, test & automation in Haskell - Infrastructure and CI/CD with Nix/NixOS/Hydra - Still babysit a bloated Rails webapp Previously: trading systems and SaaS products Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 1 / 44

3. Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 2 / 44

4. Overview

5. Overview Clickstreams (collecting & reporting API) Figure 1: High write throughput and unreliable runtime dependencies killed original Ruby service Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 3 / 44

6. Overview Streaming data replication with scrubbing Figure 2: Soft real-time replication needs to ensure PII data is scrubbed Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 4 / 44

7. Overview Developer, testing & deploy tools Figure 3: Developer tools scattered across copy-pasta Bash/Ruby scripts across multiple repos Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 5 / 44

8. Overview Agenda Illustrated examples of team feedback: • Security/Privacy Awareness • Operational Debuggability Human factors review • Teaching/Learning • Managing Up • Setting Expectations Take-Aways Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 6 / 44

9. Security/Privacy Awareness

10. Security/Privacy Awareness Security/Privacy Awareness: Motivation Need to scrub sensitive PII: • between production and staging • before sharing with third-parties Security gates Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 7 / 44

11. Security/Privacy Awareness Security/Privacy Awareness: The Big Idea ”Susan, you keep telling us about the powers of the Haskell type system, when do we actually use it for great good?” –Team Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 8 / 44

12. Security/Privacy Awareness Security/Privacy Awareness: Challenge Accepted Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 9 / 44

13. Security/Privacy Awareness Security/Privacy Awareness: The Big Idea How do we get type errors when we want them? • Reiﬁcation turning terms into types Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 10 / 44

14. Security/Privacy Awareness Security/Privacy Awareness: The Big Idea How do we get type errors when we want them? • Reiﬁcation turning terms into types Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 10 / 44

15. Security/Privacy Awareness Security/Privacy Awareness: Language Extensions {-# LANGUAGE DataKinds #-} {-# LANGUAGE KindSignatures #-} {-# LANGUAGE GADTs #-} -- OPTIONAL {-# LANGUAGE NoImplicitPrelude #-} -- OPTIONAL {-# LANGUAGE OverloadedStrings #-} -- OPTIONAL {-# LANGUAGE TypeApplications #-} -- OPTIONAL Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 11 / 44

16. Security/Privacy Awareness Security/Privacy Awareness: Imports import Control.Applicative import Control.Monad import Data.Eq import Data.Function import Data.Maybe import Data.Semigroup import Data.Text import GHC.Err (error) import GHC.Show import GHC.Types Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 12 / 44

17. Security/Privacy Awareness Security/Privacy Awareness: Data Types EmailState data EmailState = Unscrubbed | Scrubbed deriving (Eq, Show) Email data Email (s :: EmailState) = MkEmail { emailAddress :: Text } deriving (Eq, Show) Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 13 / 44

18. Security/Privacy Awareness Security/Privacy Awareness: Data Types EmailState data EmailState = Unscrubbed | Scrubbed deriving (Eq, Show) Email data Email (s :: EmailState) = MkEmail { emailAddress :: Text } deriving (Eq, Show) Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 13 / 44

19. Security/Privacy Awareness Security/Privacy Awareness: -XTypeApplications -- Before enabling TypeApplications -- >>> MkEmail "me@realemailaddress.com" :: Email 'Scrubbed -- MkEmail {emailAddress = "me@realemailaddress.com"} -- it :: Email 'Scrubbed -- After: -- >>> MkEmail @'Unscrubbed "me@realemailaddress.com" -- MkEmail {emailAddress = "me@realemailaddress.com"} -- it :: Email 'Unscrubbed Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 14 / 44

20. Security/Privacy Awareness Security/Privacy Awareness: -XDataKinds -- >>> import Scrubbed -- >>> :set -XDataKinds -- >>> :kind! 'Scrubbed -- 'Scrubbed :: EmailState -- = 'Scrubbed Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 15 / 44

21. Security/Privacy Awareness Security/Privacy Awareness: -XKindSignatures Remember this? data Email (s :: EmailState) -- this is the kind signature! = MkEmail { emailAddress :: Text } deriving (Eq, Show) -- >>> :kind! Email Int -- <interactive>:1:7: error: -- • Expected kind ‘EmailState’, but ‘Int’ has kind ‘*’ -- • In the first argument of ‘Email’, namely ‘Int’ -- In the type ‘Email Int’ Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 16 / 44

22. Security/Privacy Awareness Security/Privacy Awareness: -XKindSignatures data Email (s :: EmailState) -- this is the kind signature! = MkEmail { emailAddress :: Text } deriving (Eq, Show) -- >>> :kind! Email 'Unscrubbed -- Email 'Unscrubbed :: * -- = Email 'Unscrubbed Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 17 / 44

23. Security/Privacy Awareness Security/Privacy Awareness: Back to the team … ”Nice tricks but what do they buy us, Susan?” – Team Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 18 / 44

24. Security/Privacy Awareness Security/Privacy Awareness: Introducer -- >>> Just unscrubbedEmail = readEmail "user@email.address"→ -- unscrubbedEmail :: Email 'Unscrubbed readEmail :: Text -> Maybe (Email 'Unscrubbed) readEmail = pure . MkEmail @'Unscrubbed -- silly version for slides→ Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 19 / 44

25. Security/Privacy Awareness Security/Privacy Awareness: State Transitions -- >>> scrubEmail (MkUsername "username0") <$> readEmail "somethingelse@email.address"→ -- Just (MkEmail {emailAddress = "username0@some.testing.domain"})→ -- it :: Maybe (Email 'Scrubbed) scrubEmail :: Username -> Email 'Unscrubbed -> Email 'Scrubbed→ scrubEmail username (MkEmail unscrubbedEmail) = MkEmail @'Scrubbed (mkEmailAddress $ getUsername username)→ Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 20 / 44

26. Security/Privacy Awareness Security/Privacy Awareness: State Transitions -- >>> import Text.StringRandom -- >>> stringRandomIO "[0-9a-zA-Z][0-9a-zA-Z.+]{2,8}" >>= pure . randomizeEmail→ -- MkEmail {emailAddress = "DIR9qO@some.testing.domain"} -- it :: Email 'Scrubbed randomizeEmail :: Text -> Email 'Scrubbed randomizeEmail randomizedPrefix = MkEmail @'Scrubbed (mkEmailAddress randomizedPrefix) -- ^^^^^^^^^^ using -XTypeApplications Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 21 / 44

27. Security/Privacy Awareness Security/Privacy Awareness: Relies on two things… • Hiding MkEmail data constructor • Only providing functions for valid state transitions Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 22 / 44

30. Security/Privacy Awareness Security/Privacy Awareness: -XGADTs Now I started going wild (reﬂection, going back to terms): -- OPTIONAL: -- GADT can act as a witness of EmailState -- if we need to reflect at runtime data SEmailState :: EmailState -> Type where SUnscrubbed :: SEmailState 'Unscrubbed SScrubbed :: SEmailState 'Scrubbed Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 23 / 44

31. Security/Privacy Awareness Security/Privacy Awareness: Team Verdict ”Wow, stop! Too much type acrobatics, Susan.” – Team Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 24 / 44

32. Security/Privacy Awareness Security/Privacy Awareness: Oh no! Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 25 / 44

33. Security/Privacy Awareness Security/Privacy Awareness: Backtrack to just enough • Didn’t really need to reﬂect using a witness in this case. • Keep is simple, Susan (KISS) • Removed GADT witness and reﬂection functions and then… ”Cool, we like this just enough solution.” – Team Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 26 / 44

34. Operational Debuggability

35. Operational Debuggability Operational Debuggability: Motivation TREND Our systems are larger today: • more external integrations • more ”micro”-services • more storage & compute needs (for new categories of problems) • more runtime dependencies (monitoring, metrics, orchestration) • more deployment strategies And … Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 27 / 44

36. Operational Debuggability Operational Debuggability: Semi-Structured Logging Figure 4: Getting even semi-strutured log records in a Rails app is like herding cats Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 28 / 44

37. Operational Debuggability Operational Debuggability: Semi-Structured Logging 1 data LogMessage a 2 = MkLogMessage 3 { logMessageService :: !Service -- compile 4 , logMessageVersion :: !Version -- compile 5 , logMessageEnv :: !Env -- on start 6 , logMessageRuntimeId :: !RuntimeId -- on start 7 , logMessageTime :: !UTCTime -- runtime 8 , logMessageThreadId :: !ThreadId -- runtime 9 , logMessagePayload :: a -- runtime } deriving (Generic) Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 29 / 44

38. Operational Debuggability Operational Debuggability: Semi-Structured Logging mkLog :: (Loggable a, MonadIO m) => a -> m (LogMessage a) class Loggable a where formatLog :: a -> LogOutputType instance Loggable a => Loggable (LogMessage a) where formatLog (MkLogMessage t tid rid v s c e p) = ... Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 30 / 44

39. Operational Debuggability Operational Debuggability: Semi-Structured Logging -- Request/Response specific types type Username = Text data PasswordReset = PasswordReset Username instance Loggable PasswordReset where formatLog (PasswordReset username) = ... Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 31 / 44

40. Operational Debuggability Operational Debuggability: Semi-Structured Logging Use formatted protocol and ingest structure. We us CEE-enhanced syslog @cee: {"time":"20190613T041811654", "tid":167, "pid":33451, "ver":"f73cfffe", "svc":"dk", "cmp":"tel", "env":"s1", "p":{"username":"someuser", "action":"password-reset"}} → → Now we can query structured logs not search for arbitrary text @severity:"error" svc:"dk" env:"prod" payload_status:[500 TO 599]→ Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 32 / 44

41. Operational Debuggability Operational Debuggability: Semi-Structured Logging Use formatted protocol and ingest structure. We us CEE-enhanced syslog @cee: {"time":"20190613T041811654", "tid":167, "pid":33451, "ver":"f73cfffe", "svc":"dk", "cmp":"tel", "env":"s1", "p":{"username":"someuser", "action":"password-reset"}} → → Now we can query structured logs not search for arbitrary text @severity:"error" svc:"dk" env:"prod" payload_status:[500 TO 599]→ Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 32 / 44

42. Operational Debuggability Operational Debuggability: Semi-Structured Logging • BAD: Team found logging setup more involved (compared to Rails) • GOOD: First service work easily extracted in to a shared library for the next • GOOD: Simple type features used so far but some boilerplate • GOOD: Structured log record consistency (unlike ﬁrst pass in Rails) • TODO: Trees That Grow to extend LogMessage in new ways • TODO: Type families to change output type • TODO: instance Generic a => Loggable a where ... Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 33 / 44

45. Operational Debuggability Operational Debuggability: WIP Kill Switches Kill switches are like feature ﬂags EXCEPT: • they prevent access to a run-time resources like a database or cache or ﬂaky API getKillSwitch :: ??? f => f (Either a b) gracefullyFail :: ??? f => f (a -> c) normalOps :: ??? f => f (b -> c) branch getKillSwitch gracefullyFail normalOps Look familiar? ;) Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 34 / 44

46. Operational Debuggability Operational Debuggability: WIP Kill Switches Selectives allow: • static over- and under-estimation of computation tree • under-estimation can be used to ﬁnd computations that always run • over-estimation can be used to ﬁnd computations that sometimes run • generate an on-call endpoint mapping to help tired brain debug at 3AM • Still a work-in-progress Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 35 / 44

52. Human Factors

53. Human Factors Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 36 / 44

54. Human Factors Teaching/Learning in industry • Industry technical leaders untrained at teaching • ”Success” criteria often ﬂawed or unmeasurable • Bottom-up foundational learning with top-down practical practice • Setup simple working dev envs; introduce the tools of trade • Team-level learning, not just individual Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 37 / 44

59. Human Factors Team-level learning (safe-to-fail experiments) • Deﬁne hypothesis • Design experiment • Document results • Share recommendations back to team • Team Discusses • Ticket cleanup for failures and successes Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 38 / 44

60. Human Factors Managing Up • Past project failures (postmortems) • Propose solution to core problems • Offer PoC evidence that it satisﬁes a core requirement • Frame results to your audience (technical vs non-technical) • Incremental rollout, show results early, increase trust Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 39 / 44

65. Human Factors Set Expectations With: • Management (above) • Your technical peers • Your business peers • Your team By: • Be realistic • Iterative delivery • It will never be rainbows and unicorns, sorry! Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 40 / 44

66. Human Factors Set Expectations With: • Management (above) • Your technical peers • Your business peers • Your team By: • Be realistic • Iterative delivery • It will never be rainbows and unicorns, sorry! Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 40 / 44

67. In Closing

68. In Closing Recap: Takeaways from our experience so far • Stay within a novelty budget • Give people new to Haskell working dev envs from day one! (Frustration budget) • Teach thinking tools over syntax • Promote team experiments and learning over time Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 41 / 44

72. Fertig/Fin/The End

73. Fertig/Fin/The End Questions? Accounts GitHub @mbbx6spp Keybase @mbbx6spp LinkedIn /in/susanpotter Twitter @SusanPotter I <3 silly hats. Thank you! Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 42 / 44

74. Appendix: Answers/links to answers in Q&A (post-talk)

75. Appendix: Answers/links to answers in Q&A (post-talk) Materials on thinking tools vs syntax • Parametricity • https://www.well-typed.com/blog/2015/05/parametricity/ • https://yowconference.com/slides/yowlambdajam2014/ Morris-ParametricityTypesAreDocumentation.pdf • Theorems for free! https: //homepages.inf.ed.ac.uk/wadler/topics/parametricity.html • The algebra of [algebraic] data types • https://codewords.recurse.com/issues/three/ algebra-and-calculus-of-algebraic-data-types • Abstractions of typed functional programming (aka typeclassopedia) • https://wiki.haskell.org/Typeclassopedia • https: //www.cs.tufts.edu/comp/150FP/archive/brent-yorgey/tc.pdfSusan Potter From Zero to Haskell: Lessons Learned 2019-06-15 43 / 44

76. Appendix: Answers/links to answers in Q&A (post-talk) Examples of Haskell/Nix development environments • Pinned on an aging nixpkgs version though: https://gist.github. com/mbbx6spp/adc6610892f50f342995d3f5f40e7f6c • NEWER more all-encompassing example coming soon Susan Potter From Zero to Haskell: Lessons Learned 2019-06-15 44 / 44

From Zero to Haskell: Lessons Learned

Recommended

Recommended

More Related Content

Similar to From Zero to Haskell: Lessons Learned

Similar to From Zero to Haskell: Lessons Learned (20)

More from Susan Potter

More from Susan Potter (17)

Recently uploaded

Recently uploaded (20)

From Zero to Haskell: Lessons Learned