Successfully reported this slideshow.
Your SlideShare is downloading. ×

2 Years of Real World FP at REA

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 160 Ad

More Related Content

Slideshows for you (20)

Similar to 2 Years of Real World FP at REA (20)

Advertisement

Recently uploaded (20)

Advertisement

2 Years of Real World FP at REA

  1. 1. 2 Years of Real World FP at REA @KenScambler Scala Developer at λ
  2. 2. Me 14 years 5 years 5 years when possible when bored when forced
  3. 3. @ Jul 13 Jan 14 Jul 14 Jan 15 - 3 teams - 17 codebases - 43K LOC
  4. 4. Why Functional Programming?
  5. 5. Compelling, tangible software engineering benefits
  6. 6. Modularity Abstraction Composability
  7. 7. Modular, abstract, composable programs are simple programs
  8. 8. Modularity
  9. 9. Can you reason about something in isolation?
  10. 10. Or do you need to fit everything in your head at once?
  11. 11. A B C K Is the cost of replacing B with K just writing K?
  12. 12. A B CK Is the cost of replacing B with K just writing K?
  13. 13. AK B CKK …or rewriting half the program? glue glue
  14. 14. A pure function is just input  output; no side effects You can tell what it does without Looking at surrounding context.
  15. 15. Consider: Let’s parse a string like “Richmond, VIC 3121” def parseLocation(str: String): Location = { val parts = str.split(“,”) val secondStr = parts(1) val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) }
  16. 16. Could be null Possible errors: 1 def parseLocation(str: String): Location = { val parts = str.split(“,”) val secondStr = parts(1) val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) }
  17. 17. def parseLocation(str: String): Location = { val parts = str.split(“,”) val secondStr = parts(1) val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) } Might not have enough elements Possible errors: 2
  18. 18. def parseLocation(str: String): Location = { val parts = str.split(“,”) val secondStr = parts(1) val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) } Might not have enough elements Possible errors: 3
  19. 19. def parseLocation(str: String): Location = { val parts = str.split(“,”) val secondStr = parts(1) val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) } Might not have enough elements Possible errors: 4
  20. 20. def parseLocation(str: String): Location = { val parts = str.split(“,”) val secondStr = parts(1) val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) } Might not have enough elements Possible errors: 5
  21. 21. def parseLocation(str: String): Location = { val parts = str.split(“,”) val secondStr = parts(1) val parts2 = secondStr.split(“ “) Location( parts(0), parts2(0), parts(1).toInt) } Might not be an int Possible errors: 6
  22. 22. def doSomethingElse(): Unit = { // ...Do other stuff parseLocation(“Melbourne, VIC 3000”) } Possible errors: 6 + 3 = 9 6 3
  23. 23. def anotherThing(): Unit = { // ...Do even more stuff doSomethingElse() } Possible errors: 9 + 4 = 13 4 9
  24. 24. Code inherits all the errors and side-effects of code it calls. Local reasoning becomes impossible; modularity is lost.
  25. 25. def parseLocation(str: String): Option[Location] = { val parts = str.split(”,") for { locality <- parts.optGet(0) theRestStr <- parts.optGet(1) theRest = theRestStr.split(" ") subdivision <- theRest.optGet(0) postcodeStr <- theRest.optGet(1) postcode <- postcodeStr.optToInt } yield Location(locality, subdivision, postcode) } Possible errors: 0
  26. 26. All possibilities have been elevated into the type system Local reasoning is possible! Local reasoning is possible about things that call it, etc…
  27. 27. Abstraction
  28. 28. Know as little as you need
  29. 29. Know as little as you need Produce as little as you can
  30. 30. def sumInts( list: List[Int]): Int = { var result = 0 for (x <- list) { result = result + x } return result }
  31. 31. def flatten[A]( list: List[List[A]]): List[A] = { var result = List() for (x <- list) { result = result ++ x } return result }
  32. 32. def all[A]( list: List[Boolean]): Boolean = { var result = true for (x <- list) { result = result && x } return result }
  33. 33. def sumInts( list: List[Int]): Int = { var result = 0 for (x <- list) { result = result + x } return result }
  34. 34. def flatten[A]( list: List[List[A]]): List[A] = { var result = List() for (x <- list) { result = result ++ x } return result }
  35. 35. def all[A]( list: List[Boolean]): Boolean = { var result = true for (x <- list) { result = result && x } return result }
  36. 36. trait Monoid[M] { def zero: M def append(m1: M, m2: M): M } Extract the essence!
  37. 37. def fold[M](list: List[M]) (implicit m: Monoid[M]): M = { var result = m.zero for (x <- list) { result = m.append(result,x) } result }
  38. 38. def fold[M](list: List[M]) (implicit m: Monoid[M]): M = list.foldLeft(m.zero)(m.append)
  39. 39. fold(List(1, 2, 3, 4))  10 fold(List(List("a", "b"), List("c")))  List("a", "b", "c") fold(List(true, true, false, true))  false
  40. 40. Abstraction is always a good thing! Less repetition More reuse Less decay, because code can’t grow tumours around unnecessary detail
  41. 41. Composability
  42. 42. Functions compose. A => B B => C C => D D => E
  43. 43. A => E Functions compose.
  44. 44. Sideways too. A => E X => Y
  45. 45. (A,X) => (E,Y) Sideways too.
  46. 46. This works in the large, as well as the small! Entire systems can be composable like functions…. without side effects
  47. 47. Truly composable systems can accrue more and more stuff without getting more complex!
  48. 48. Simplicity • Modularity – Reason locally • Abstraction – say only what you need, hide everything you don’t • Composability – scale without accruing complexity
  49. 49. The human process
  50. 50. Technical excellence isn’t enough! Software development is a human process Coffee Tea Quiet time GSD Ponies Reconsider!
  51. 51. Xi’an offshore team Team Team Team Team
  52. 52. Xi’an offshore team • Many teams are partly based in Xi’an. • They’re very good, but… • Communication is hard! • It works, but requires great investment of time and money
  53. 53. Bottom-up tech decisions
  54. 54. GET /foo/bar PUT {"mode": "banana"} POST {"partyTime": "5:00"} GET /buzz/5/
  55. 55. Architect Mountain
  56. 56. Architect Mountain
  57. 57. Architect Mountain
  58. 58. Architect Mountain Just needs more Agile Don’t forget your velocity More meetings, but littler NO No no no no no no no no no no no no no no no no no no no no no no no no no no no no no no no no not like this. Wake up it’s a school day
  59. 59. Bottom-up tech decisions You have to win
  60. 60. Bottom-up tech decisions You have to win
  61. 61. Bottom-up tech decisions You have to win
  62. 62. Software Paleontology Everyone’s got a history…
  63. 63. Scriptozoic era 1995 – 2010 Mostly Perl
  64. 64. Monolithocene epoch 2010 – 2012 Ruby, Java Scriptozoic era 1995 – 2010 Mostly Perl
  65. 65. AWS Monolithocene epoch 2010 – 2012 Ruby, Java Scriptozoic era 1995 – 2010 Mostly Perl Microservices 2012 – Ruby, Scala, JS
  66. 66. Adoption
  67. 67. June, 2013
  68. 68. λ
  69. 69. λ λ
  70. 70. λ λ λ
  71. 71. λ λ λ
  72. 72. λ λ
  73. 73. λ λ
  74. 74. λ
  75. 75. Language choice Object Oriented Powerful static types Functional JVM
  76. 76. Object Oriented Powerful static types Functional JVM
  77. 77. Functional JVM
  78. 78. Functional JVM
  79. 79. Functional JVM
  80. 80. Functional JVM
  81. 81. Whatever works for you!
  82. 82. The journey
  83. 83. Jul 13 Jan 14 1
  84. 84. LOC Web JSON DB Dep Inj Play2 Play2 6K Squeryl / Play2 Constructors Type API #1
  85. 85. LOC Web JSON DB Dep Inj Play2 Play2 6K Squeryl / Play2 Constructors Type API • Mentor • Code reviews Learning Investment #1
  86. 86. LOC Web JSON DB Dep Inj Play2 Play2 6K Squeryl / Play2 Constructors Type API • Mentor • Code reviews Learning Investment Report card Learning curve Technical result Productivity Sentiment Steep but ok OK; slight dip Great; but FWs too heavy #1
  87. 87. Jul 13 Jan 14 1 2 Some infrastructure
  88. 88. LOC Web JSON DB Dep Inj Play2 Argonaut 3K Squeryl / Play2 Constructors Type API #2
  89. 89. LOC Web JSON DB Dep Inj Play2 Argonaut 3K Squeryl / Play2 Constructors Type API • Almost none Learning Investment #2
  90. 90. LOC Web JSON DB Dep Inj Play2 Argonaut 3K Squeryl / Play2 Constructors Type API • Almost none Learning Investment Report card Learning curve Technical result Productivity Sentiment Learning? Meh Needed rework; OK in the end #2 !!!
  91. 91. Lesson #1 New tech, mindset requires investment in learning Invest in your people!
  92. 92. λ
  93. 93. Another team! λ
  94. 94. “We’ll have some of that!” λ
  95. 95. Jul 13 Jan 14 1 2 4 6 5 New team; new ideas! 3
  96. 96. LOC Web JSON DB Dep Inj Unfinagled Argonaut 2K, 3K, 4K, 1K Slick Constructors Type Web app, libs, API x 2 • 2 x Mentor Learning Investment Report card Learning curve Technical result Productivity Sentiment Not bad Great Great #3,4, 5,6
  97. 97. Jul 13 Jan 14 1 2 4 6 5 3 7 Theft & innovation Jul 14
  98. 98. Lesson #2 Having multiple teams is great, because you can steal from each other
  99. 99. LOC Web JSON DB Dep Inj Unfinagled Argonaut 4K Slick Monad Transformers Type API #7
  100. 100. LOC Web JSON DB Dep Inj Unfinagled Argonaut 4K Slick Monad Transformers Type API • 2 x Mentor Learning Investment #7
  101. 101. LOC Web JSON DB Dep Inj Unfinagled Argonaut 4K Slick Monad Transformers Type API • 2 x Mentor Learning Investment Report card Learning curve Technical result Productivity Sentiment Vertical Great Great #7
  102. 102. Monad Transformers Good technical benefits, but… Only 2 people could understand the code Experienced engineers felt totally helpless Learning curve way too steep
  103. 103. Devops All-rounders Gurus
  104. 104. All-rounders Gurus JS / CSS / HTML
  105. 105. AWS All-rounders Gurus
  106. 106. Scala (originally!) Gurus
  107. 107. All-rounders Scala (now) Gurus • Smooth learning curve is utterly essential • We need more all- rounders • We can’t leave people behind
  108. 108. Lesson #3 Familiar, but technically unsound concepts have limited value. However… if we can’t make a concept learnable, then we can’t use it.
  109. 109. λ A Ruby team considers its options…
  110. 110. λ
  111. 111. λ
  112. 112. Jul 13 Jan 14 1 2 4 6 5 3 7 A 3rd team dips its toe in the water 8 Jul 14
  113. 113. LOC Web JSON DB Dep Inj Play2 Play2 2K Anorm Constructors Type Web app #8
  114. 114. LOC Web JSON DB Dep Inj Play2 Play2 2K Anorm Constructors Type Web app • Trial and error • Code Katas Learning Investment #8
  115. 115. LOC Web JSON DB Dep Inj Play2 Play2 2K Anorm Constructors Type Web app • Trial and error • Code Katas Learning Investment Report card Learning curve Technical result Productivity Sentiment Steep Meh OK #8
  116. 116. Lesson #4 It’s really hard learning from scratch Be prepared for pain up front
  117. 117. λ
  118. 118. Jul 13 Jan 14 1 2 4 6 5 3 7 Latest iteration 8 Jul 14 Jan 15 17
  119. 119. Design trends Inheritance/mixins Static functions
  120. 120. Design trends Inheritance/mixins Static functions Partial functions Total functions
  121. 121. Design trends Inheritance/mixins Static functions Partial functions Total functions Exceptions Sum types
  122. 122. Design trends Inheritance/mixins Static functions Partial functions Total functions Strings/primitives Wrapper types Exceptions Sum types
  123. 123. FP Guild Every Thursday, in work hours 7 – 12 people each week Reading, exercises, talks, live coding
  124. 124. LOC Web JSON DB Dep Inj Unfiltered Argonaut 3K Slick Free Monads Type API #17
  125. 125. LOC Web JSON DB Dep Inj Unfiltered Argonaut 3K Slick Free Monads Type API • 2 x Mentors • Pull Requests • Code reviews • Pairing • FP Guild Learning Investment #17
  126. 126. LOC Web JSON DB Dep Inj Unfiltered Argonaut 3K Slick Free Monads Type API • 2 x Mentors • Pull Requests • Code reviews • Pairing • FP Guild Learning Investment Report card Learning curve Technical result Productivity Sentiment Smooth Great Brilliant #17
  127. 127. Pure core Routes Controllers Logic Script Interpreter Actual DB “Authenticate” “Use config” “Get from DB” “Update DB” “Log result” Web Framework Server App runtime
  128. 128. Pure core Routes Controllers Logic Script Pure Interpreter Pure tests Input Output Assert “Authenticate” “Use config” “Get from DB” “Update DB” “Log result”
  129. 129. Pure core Interpreter Actual DB Web Framework Server App runtimeWafer thin E2E tests Input Output Assert
  130. 130. object SearchController { def getList(uid: UserId): Script[Response] = { for { searches <- getSearches(uid) cfg <- getConfig results <- addLinks(searches, cfg) } yield Ok ~> Response(result.toJson) } }
  131. 131. Learning curve:
  132. 132. Learning curve: Smooooth
  133. 133. Mocks
  134. 134. Stubs
  135. 135. Flaky pretend servers just to check a response code
  136. 136. Flaky tests that take 30 min to run
  137. 137. Side effects everywhere
  138. 138. Exceptions
  139. 139. Intrusive DI frameworks
  140. 140. You shouldn’t be dealing with all that complexity and crap! Most business software isn’t rocket science.
  141. 141. There is a better way FP is a tall tree But there is so much low hanging fruit!
  142. 142. Thank you!

×