simplifying development at guardian.co.uk       Graham Tackley - @tackers        architect, guardian.co.uk
Background Decision  ImpactOutcomes
Background  Decision   Impact Outcomes
Background  Decision   Impact Outcomes
Background  Decision   Impact Outcomes
Background  Decision   Impact Outcomes
Background  Decision   Impact Outcomes
Background  Decision   Impact Outcomes
Background2006-2008: “R2”                    Decision                     Impact                   Outcomes
Background                                                       Decision                                                 ...
Background                                                       Decision                                                 ...
Background                                                       Decision                                                 ...
Background                                                       Decision                                                 ...
Background                    Decision                     Impact                   Outcomesit’s ok, but...
Background2009         Decision          Impact        Outcomes
Background  Decision   Impact Outcomes
Background  Decision   Impact Outcomes
Background                                       Decision                                        Impact                   ...
Background  Decision   Impact Outcomes
Background  Decision   Impact Outcomes
Background                                      Decision                                       Impact                     ...
Background                    Decision                     Impact                   Outcomesit’s ok, but...
Background                                        Decision                                         Impact                 ...
BackgroundDecision  ImpactOutcomes
BackgroundDecision  ImpactOutcomes
BackgroundDecision  ImpactOutcomes
BackgroundDecision  ImpactOutcomes
Background                                                         Decisionsame infrastructure: toolset,                  ...
Background                         Decision                           Impact                         Outcomeshuge drop in ...
Background                      Decision                        Impact                      Outcomes     lueWe va         ...
Background                                          Decision                                            Impact            ...
BackgroundDecision  ImpactOutcomes
Background                                                     Decision                                                   ...
Background                                                     Decision                                                   ...
Background                           Decision                             Impact                           Outcomes2009   ...
Background                           Decision                             Impact                           Outcomes2009   ...
Background                                                  Decision                                                    Im...
Background                                                  Decision                                                    Im...
Background                                                  Decision                                                    Im...
Background                                           Decision                                             Impact          ...
Background                                           Decision                                             Impact          ...
Background                                       Decision                                         Impact                  ...
Background Decision ImpactOutcomes
Background Decision ImpactOutcomes
Background                                    Decision                                    ImpactThe roll-it-out-to-everyon...
Background                                    Decision                                    ImpactThe roll-it-out-to-everyon...
Background                     Decision                     Impact                    OutcomesHow did we learn?
Background                     Decision                     Impact                    OutcomesHow did we learn?
Background                     Decision                     Impact                    OutcomesHow did we learn?
Background                     Decision                     Impact                    OutcomesHow did we learn?
Background                     Decision                     Impact                    OutcomesHow did we learn?
Background                     Decision                     Impact                    OutcomesHow did we learn?
Background Decision ImpactOutcomes
Background                                    Decision                                    Impact                          ...
Background                            Decision                            Impact                           OutcomesComplex...
Background Decision ImpactOutcomes
Background                         Decision                         Impact                        Outcomes     lueWe va   ...
Background Decision  ImpactOutcomes
Background Decision  ImpactOutcomes
Background Decision  ImpactOutcomes
Background                                                             Decision                                           ...
Background Decision  ImpactOutcomes
Background                                                   Decision                                                    I...
Background Decision  ImpactOutcomes
Background Decision  ImpactOutcomes
Background Decision  ImpactOutcomes
Background Decision  ImpactOutcomes
Background                                                Decision                                                 Impact ...
Background                                                                             Decision                           ...
Background                                                                             Decision                           ...
graham.tackley@guardian.co.uk                 @tackersWe’re hiring: http://bit.ly/guardian-dev-jobs
Upcoming SlideShare
Loading in …5
×

Scala: simplifying development at guardian.co.uk

5,239 views

Published on

Published in: Technology, News & Politics
1 Comment
8 Likes
Statistics
Notes
No Downloads
Views
Total views
5,239
On SlideShare
0
From Embeds
0
Number of Embeds
584
Actions
Shares
0
Downloads
66
Comments
1
Likes
8
Embeds 0
No embeds

No notes for slide
  • intro me\nqns: how many use scala in prod? looking at scala? consider yourself primarily java dev?\n
  • talk overview - story of our background and key principles we learnt pay a key part in the success of using scala; how we started using scala; impact on day to day; some things we’ve been able to do more easily as a result\n
  • what led up to our decision to use scala\n\n
  • what led up to our decision to use scala\n\n
  • Produce a newspaper since 1821 (191 years ago) - today publish the guardian mon-sat and observer sun.\nRegularly break new ground with quality of our investigative journalism\n
  • Produce a newspaper since 1821 (191 years ago) - today publish the guardian mon-sat and observer sun.\nRegularly break new ground with quality of our investigative journalism\n
  • Produce a newspaper since 1821 (191 years ago) - today publish the guardian mon-sat and observer sun.\nRegularly break new ground with quality of our investigative journalism\n
  • Produce a newspaper since 1821 (191 years ago) - today publish the guardian mon-sat and observer sun.\nRegularly break new ground with quality of our investigative journalism\n
  • Web site since 1995 as guardian unlimited\nnow get 3.5m unique browsers a day (2nd most popular uk newspaper... after the mail)\nin addition a whole set of other platforms - iphone, driod, kindle, ipad ...\n
  • Web site since 1995 as guardian unlimited\nnow get 3.5m unique browsers a day (2nd most popular uk newspaper... after the mail)\nin addition a whole set of other platforms - iphone, driod, kindle, ipad ...\n
  • Web site since 1995 as guardian unlimited\nnow get 3.5m unique browsers a day (2nd most popular uk newspaper... after the mail)\nin addition a whole set of other platforms - iphone, driod, kindle, ipad ...\n
  • Web site since 1995 as guardian unlimited\nnow get 3.5m unique browsers a day (2nd most popular uk newspaper... after the mail)\nin addition a whole set of other platforms - iphone, driod, kindle, ipad ...\n
  • rebuild in java - spring hibernate velocity + oracle\nreligious test driven development and pairing for all production code\n
  • R2 code base as of today - some values redacted :)\nthese are plain wc -l line counts\n(aside: moral of this story: never separately estimate & measure backend and frontend devs)\n
  • R2 code base as of today - some values redacted :)\nthese are plain wc -l line counts\n(aside: moral of this story: never separately estimate & measure backend and frontend devs)\n
  • R2 code base as of today - some values redacted :)\nthese are plain wc -l line counts\n(aside: moral of this story: never separately estimate & measure backend and frontend devs)\n
  • - still running website, still making changes, \n- 2 week release cycle, risky release, poor scalability due to ORM, very slow to work with\n
  • MBS talk yesterday touched on this\nmicroapp framework - kind of works like ssi on steroids\n\n\n
  • [keep going through examples...]\n
  • (commenting) first microapps developed in spring-velocity\nsoon gave that up - tried other approaches\nhuge relief at escaping the 2 week release cycle! \n\n\n
  • (its very easy to come up with reasons to excuse the monolith - i did!) e.g. avoiding issues with multiple versions, easier refactoring\n\n
  • commenting - not throwing content at readers, engaging them in conversation\ndesparately looking for faster ways to develop\n
  • and python django\nrevolution for us - not coding with factory beans, property converters - not even a single decorator - we just wrote the code!\n
  • abstractions are good - but too many of those abstractions were nothing to do with our problem, forced on us by frameworks and/or development style and/or over use of patterns (repositories) - if you have small apps just write the code!\n
  • still live (although porting to scala)\nbut different ecosystem; operational cost outweighted dev benefit\n?? and need to write tests to work around lack of compiler\n
  • revolutions seem great, but its very easy to just switch one set of problems with another\n
  • \n
  • \n
  • \n
  • \n \n
  • \n \n
  • \n \n
  • \n \n
  • broken windows of java boilerplate \n-> you just get used to writing bucket loads of code\n(ide generates get/settings, constructor, delegate.....)\n
  • patterns are good; but even better if your code clearly solves the business problem.\n too often the lack of succitness in java drives over reliance of patterns to try to organise the bucketloads of code\nconciseness of language - far more important than saving keystrokes - mindset change\n
  • excited - other tradional media restricting access to content (paywalls etc) - open access.\n\n
  • \n
  • java + guice + guice servlets + apache solr, 3-4 java devs\nplayed with scala, nothing in prod.\n
  • test in scala using scalatest - just pop in the scala plugin to maven and go\n
  • test in scala using scalatest\n
  • a fair bit of our code was java without semicolons,\nbut despite that we loved it so much...\n
  • after a month - decided to do everything in scala\nno big bang - incremental migration relying on mixed java-scala compilation\nkept using guice\n
  • Live - no issues; it was just a war\n
  • switched from maven to simple build tool, mainly for incremental compilatio\n
  • 5k loc => 3.5 loc (mainly due to writing better scala)\n
  • \n
  • m., most devices, and nearly everything new we’re doing powered by content api \n(leave what about scala till next section)\n\n
  • \n
  • \n
  • didn’t really have a plan to roll out - everyone saw our enthusiasm & excitement and picked up.\nsurvey - out of 21 “back end” devs - 15 replied.\n90 % of devs pleased or v pleased using scala - 2 say ok - none dislike\n
  • how - mostly driven by enthusiasm, (odersky book, twitter info, look at open source projects, lsug)\n1 - 3 months of using scala to be as productive as in java\n\n
  • how - mostly driven by enthusiasm, (odersky book, twitter info, look at open source projects, lsug)\n1 - 3 months of using scala to be as productive as in java\n\n
  • how - mostly driven by enthusiasm, (odersky book, twitter info, look at open source projects, lsug)\n1 - 3 months of using scala to be as productive as in java\n\n
  • how - mostly driven by enthusiasm, (odersky book, twitter info, look at open source projects, lsug)\n1 - 3 months of using scala to be as productive as in java\n\n
  • how - mostly driven by enthusiasm, (odersky book, twitter info, look at open source projects, lsug)\n1 - 3 months of using scala to be as productive as in java\n\n
  • don’t insist on scala skills - weeds out the “Enterprise Java” developers - attracts polyglots\nover half the devs who joined us were influenced in their choice by using scala\nquite a few chose to stay with us because we’re using scala\n\n
  • \n
  • all seen the complexity / performance complaints - google\nnot been our experience at all\n
  • i can read each line of code but cannot see the wood for the trees\ntrade off scala complexity arguments against code like this - false simplicity\n\n
  • the power of scala allows funky stuff - readable code is much more important. \n
  • \n
  • \n
  • val vs var; collections or immutable by default - helps dev at all levels to think sensibly about their code\n\n \n
  • val vs var; collections or immutable by default - helps dev at all levels to think sensibly about their code\n\n \n
  • val vs var; collections or immutable by default - helps dev at all levels to think sensibly about their code\n\n \n
  • val vs var; collections or immutable by default - helps dev at all levels to think sensibly about their code\n\n \n
  • downsides: sbt: especially keeping up with plugins\n\n \n
  • downsides: sbt: especially keeping up with plugins\n\n \n
  • downsides: sbt: especially keeping up with plugins\n\n \n
  • downsides: sbt: especially keeping up with plugins\n\n \n
  • facebook app - a radical change to the way we normally deliver news\ndelivered in two months (with some third party involvement)\nscala + scalatra (web routing like sinatra from ruby) + scalate (scala templates)\n
  • looked at this code for the first time last week\n
  • the new article page (to be)\n\n
  • ... and is open on github\n
  • \n
  • \n
  • Scala: simplifying development at guardian.co.uk

    1. 1. simplifying development at guardian.co.uk Graham Tackley - @tackers architect, guardian.co.uk
    2. 2. Background Decision ImpactOutcomes
    3. 3. Background Decision Impact Outcomes
    4. 4. Background Decision Impact Outcomes
    5. 5. Background Decision Impact Outcomes
    6. 6. Background Decision Impact Outcomes
    7. 7. Background Decision Impact Outcomes
    8. 8. Background Decision Impact Outcomes
    9. 9. Background2006-2008: “R2” Decision Impact Outcomes
    10. 10. Background Decision Impact Outcomesmain java (185k) main xml (35k) velocity (72k) test java (248k) 0 75,000 150,000 225,000 300,000
    11. 11. Background Decision Impact Outcomesmain java (185k) main xml (35k) velocity (72k) test java (248k) 0 75,000 150,000 225,000 300,000
    12. 12. Background Decision Impact Outcomesmain java (185k) main xml (35k) velocity (72k) test java (248k) 0 75,000 150,000 225,000 300,000
    13. 13. Background Decision Impact Outcomesmain java (185k) main xml (35k) velocity (72k) test java (248k) 0 75,000 150,000 225,000 300,000
    14. 14. Background Decision Impact Outcomesit’s ok, but...
    15. 15. Background2009 Decision Impact Outcomes
    16. 16. Background Decision Impact Outcomes
    17. 17. Background Decision Impact Outcomes
    18. 18. Background Decision Impact Outcomes lueWe va Small Independent Components over The Monolith
    19. 19. Background Decision Impact Outcomes
    20. 20. Background Decision Impact Outcomes
    21. 21. Background Decision Impact Outcomes lueWe va Clarity of Intent over Ceremonial Abstraction
    22. 22. Background Decision Impact Outcomesit’s ok, but...
    23. 23. Background Decision Impact Outcomes lueWe va Architectural Evolution over Architectural Revolution
    24. 24. BackgroundDecision ImpactOutcomes
    25. 25. BackgroundDecision ImpactOutcomes
    26. 26. BackgroundDecision ImpactOutcomes
    27. 27. BackgroundDecision ImpactOutcomes
    28. 28. Background Decisionsame infrastructure: toolset, Impact Outcomes libraries, build, runtime it’s just bytecode; a war is a war mixed scala & java projects work huge drop in verbosity and boilerplate
    29. 29. Background Decision Impact Outcomeshuge drop in verbosity and boilerplate
    30. 30. Background Decision Impact Outcomes lueWe va Craft over Patterns
    31. 31. Background Decision Impact Outcomeshttp://www.guardian.co.uk/open-platform http://content.guardianapis.com
    32. 32. BackgroundDecision ImpactOutcomes
    33. 33. Background Decision Impact Outcomes2009 2010 2011 Nov java + guice + guice servlets + apache solr
    34. 34. Background Decision Impact Outcomes2009 2010 2011 Jan java + guice + guice servlets + apache solr
    35. 35. Background Decision Impact Outcomes2009 2010 2011 Jan
    36. 36. Background Decision Impact Outcomes2009 2010 2011 Jan
    37. 37. Background Decision Impact Outcomes2009 2010 2011 Feb scala + guice + guice servlets + apache solr
    38. 38. Background Decision Impact Outcomes Live!2009 2010 2011 May scala + guice + guice servlets + apache solr
    39. 39. Background Decision Impact Outcomes2009 2010 2011 July scala + guice + guice servlets + apache solr
    40. 40. Background Decision Impact Outcomes2009 2010 2011 July scala + lift + apache solr
    41. 41. Background Decision Impact Outcomes2009 2010 2011 July scala + lift + apache solr
    42. 42. Background Decision Impact Outcomes2009 2010 2011 Now scala + lift + apache solr
    43. 43. Background Decision ImpactOutcomes
    44. 44. Background Decision ImpactOutcomes
    45. 45. Background Decision ImpactThe roll-it-out-to-everyone plan Outcomes
    46. 46. Background Decision ImpactThe roll-it-out-to-everyone plan Outcomes
    47. 47. Background Decision Impact OutcomesHow did we learn?
    48. 48. Background Decision Impact OutcomesHow did we learn?
    49. 49. Background Decision Impact OutcomesHow did we learn?
    50. 50. Background Decision Impact OutcomesHow did we learn?
    51. 51. Background Decision Impact OutcomesHow did we learn?
    52. 52. Background Decision Impact OutcomesHow did we learn?
    53. 53. Background Decision ImpactOutcomes
    54. 54. Background Decision Impact Outcomes lueWe va Passionate Developers over Turning the Crank
    55. 55. Background Decision Impact OutcomesComplexity vs Simplicity
    56. 56. Background Decision ImpactOutcomes
    57. 57. Background Decision Impact Outcomes lueWe va Clarity over Cleverness
    58. 58. Background Decision ImpactOutcomes
    59. 59. Background Decision ImpactOutcomes
    60. 60. Background Decision ImpactOutcomes
    61. 61. Background Decision Impact Outcomes preference for immutability scala console encourages viewing code as a living entityawesome test frameworks reduced need for complex frameworks
    62. 62. Background Decision ImpactOutcomes
    63. 63. Background Decision Impact Outcomes love-hate relationship with sbt tool support immature compared with javacompiler is slow rate of change
    64. 64. Background Decision ImpactOutcomes
    65. 65. Background Decision ImpactOutcomes
    66. 66. Background Decision ImpactOutcomes
    67. 67. Background Decision ImpactOutcomes
    68. 68. Background Decision Impact Outcomeshttps://github.com/guardian/frontend-article
    69. 69. Background Decision Impact Outcomes ClariSmall Independent Components ty of Int ent over Cerem over onial The Monolith Abstr actio n Craft Clar ity over Passionate Developers over Clev over Patterns erne ss Turning the Crank Architectu ral Evoluti on over Architectu ral Revolu tion
    70. 70. Background Decision Impact Outcomes ClariSmall Independent Components ty of Int ent over Cerem over onial The Monolith Abstr actio n Craft Clar ity over Passionate Developers over Clev over Patterns erne ss Turning the Crank Architectu ral Evoluti on over Architectu ral Revolu tion
    71. 71. graham.tackley@guardian.co.uk @tackersWe’re hiring: http://bit.ly/guardian-dev-jobs

    ×