Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

ScalaItaly 2015 - Your Microservice as a Function

3,888 views

Published on

SoundCloud's microservice architecture is built mostly in Scala, using Finagle as its distributed systems workhorse. Finagle is an RPC system for the JVM, and it is based on a pipes-and-filters architecture that maps very nicely to functional programming concepts of higher-order functions and combinators. Over the past few years we have found that it is extremely useful to go even a step further and think of microservices as functions themselves. In this talk let's explore how SoundCloud uses Scala and Finagle, and how we started thinking of a microservices architecture as a special case of a functional system.

Published in: Software

ScalaItaly 2015 - Your Microservice as a Function

  1. 1. Your microservice as a Function Phil Calçado SoundCloud
  2. 2. > 11 hours of audio uploaded every minute ~ 300 million people every month
  3. 3. for this talk: implementation: code design: how you abstract concepts in your system architecture: how systems talk to each other
  4. 4. how I got started with object-oriented programming
  5. 5. Relationships Between BOUNDED CONTEXTS Shared Kernel
  6. 6. Gzs%ef.. ..... ,... .... ...... .... ... .., .... ...... .. .. .... .... ...... Permissiontocopywithoutfeeallorpartofthismaterinlis grantedprowdedthatthetopicsarenotmadaordistributedfor directcommcrcioladvantage,theACMcopyrightnoticeondtho lillcofthepublicationanditsdareappear.andnoticeisgiven thatcopyingisbypermissionoftheAssociationforComputing Machinery.Tocopyothcrwisc,ortorepublish.requiresBfee and/orspecificpornlission. HOPL-11/4/93/MA,USA o1993ACM0-89791.571.2/93/0004/0069...$1.50 :LJ 2 2 .xE .m8 al :5a .6 69ACMSIGPLANNotices,Volume28,No.3March1993 😧😆😄😃😀😧
  7. 7. how I got started with functional programming
  8. 8. Gzs%ef.. ..... ,... .... ...... .... ... .., .... ...... .. .. .... .... ...... Permissiontocopywithoutfeeallorpartofthismaterinlis grantedprowdedthatthetopicsarenotmadaordistributedfor directcommcrcioladvantage,theACMcopyrightnoticeondtho lillcofthepublicationanditsdareappear.andnoticeisgiven thatcopyingisbypermissionoftheAssociationforComputing Machinery.Tocopyothcrwisc,ortorepublish.requiresBfee and/orspecificpornlission. HOPL-11/4/93/MA,USA o1993ACM0-89791.571.2/93/0004/0069...$1.50 :LJ 2 2 .xE .m8 al :5a .6 Programming R. Morris Techniques Editor On the Criteria To Be Used in Decomposing Systems into Modules D.L. Parnas Carnegie-Mellon University This paper discusses modularization as a mechanism for improving the flexibility and comprehensibility of a system while allowing the shortening of its development time. The effectiveness of a "modularization" is dependent upon the criteria used in dividing the system into modules. A system design problem is presented and both a conventional and unconventional decomposition are described. It is shown that the unconventional decompositions have distinct advantages for the goals outlined. The criteria used in arriving at the decom- positions are discussed. The unconventional decomposi- tion, if implemented with the conventional assumption that a module consists of one or more subroutines, will be less efficient in most cases. An alternative approach to implementation which does not have this effect is sketched. Key Words and Phrases: software, modules, modularity, software engineering, KWIC index, software design CR Categories: 4.0 Introduction A lucid statement of the philosophy of modular programming can be found in a 1970 textbook on the design of system programs by Gouthier and Pont [1, ¶I0.23], which we quote below: 1 A well-defined segmentation of the project effort ensures system modularity. Each task forms a separate, distinct program module. At implementation time each module and its inputs and outputs are well-defined, there is no confusion in the intended interface with other system modules. At checkout time the in- tegrity of the module is tested independently; there are few sche- duling problems in synchronizing the completion of several tasks before checkout can begin. Finally, the system is maintained in modular fashion; system errors and deficiencies can be traced to specific system modules, thus limiting the scope of detailed error searching. Usually nothing is said about the criteria to be used in dividing the system into modules. This paper will discuss that issue and, by means of examples, suggest some criteria which can be used in decomposing a system into modules. Copyright @ 1972,Association for Computing Machinery, Inc. General permission to republish, but not for profit, all or part of this material is granted, provided that reference is made to this publication, to its date of issue, and to the fact that reprinting privileges were granted by permission of the Association for Com- puting Machinery. Author's address: Department of Computer Science, Carnegie- Mellon University, Pittsburgh, PA 15213. 1053 A Brief Status Report The major advancement in the area of modular programming has been the development of coding techniques and assemblers which (l) allow one module to be written with little knowledge of the code in another module, and (2) allow modules to be reas- sembled and replaced without reassembly of the whole system. This facility is extremely valuable for the production of large pieces of code, but the systems most often used as examples of problem systems are highly- modularized programs and make use of the techniques mentioned above. 1Reprinted by permission of Prentice-Hall, Englewood Cliffs, N.J. Communications December 1972 of Volume 15 the ACM Number 12 1 Introduction The fundamental purpose of a type system is to prevent the occurrence of execution errors dur- ing the running of a program. This informal statement motivates the study of type systems, but requires clarification. Its accuracy depends, first of all, on the rather subtle issue of what consti- tutes an execution error, which we will discuss in detail. Even when that is settled, the absence of execution errors is a nontrivial property. When such a property holds for all of the program runs that can be expressed within a programming language, we say that the language is type sound. It turns out that a fair amount of careful analysis is required to avoid false and embar- rassing claims of type soundness for programming languages. As a consequence, the classifica- tion, description, and study of type systems has emerged as a formal discipline. The formalization of type systems requires the development of precise notations and defi- nitions, and the detailed proof of formal properties that give confidence in the appropriateness of the definitions. Sometimes the discipline becomes rather abstract. One should always remem- ber, though, that the basic motivation is pragmatic: the abstractions have arisen out of necessity and can usually be related directly to concrete intuitions. Moreover, formal techniques need not be applied in full in order to be useful and influential. A knowledge of the main principles of type systems can help in avoiding obvious and not so obvious pitfalls, and can inspire regularity and orthogonality in language design. When properly developed, type systems provide conceptual tools with which to judge the adequacy of important aspects of language definitions. Informal language descriptions often fail to specify the type structure of a language in sufficient detail to allow unambiguous implemen- tation. It often happens that different compilers for the same language implement slightly dif- ferent type systems. Moreover, many language definitions have been found to be type unsound, allowing a program to crash even though it is judged acceptable by a typechecker. Ideally, for- mal type systems should be part of the definition of all typed programming languages. This way, typechecking algorithms could be measured unambiguously against precise specifications and, if at all possible and feasible, whole languages could be shown to be type sound. In this introductory section we present an informal nomenclature for typing, execution er- rors, and related concepts. We discuss the expected properties and benefits of type systems, and we review how type systems can be formalized. The terminology used in the introduction is not completely standard; this is due to the inherent inconsistency of standard terminology arising from various sources. In general, we avoid the words type and typing when referring to run time concepts; for example we replace dynamic typing with dynamic checking and avoid common but ambiguous terms such as strong typing. The terminology is summarized in the Defining Terms section. Type Systems Luca Cardelli Microsoft Research
  9. 9. 😧😠😐😒😓😧
  10. 10. there isn’t a lot out there on engineering for functional programming
  11. 11. HTTP HTTP MySQL
  12. 12. what if we apply function composition?
  13. 13. turns out functional principles correlate with good implementation
  14. 14. going up one level
  15. 15. life of a request to SoundCloud.com
  16. 16. Actual Feature Request Validator User
  17. 17. what if we apply function composition?
  18. 18. basically
  19. 19. basically
  20. 20. Actual Feature Request Validator User
  21. 21. Authentication Geo IP Rate Limiting Available Features Actual Feature User
  22. 22. Actual Feature User Filters Service }
  23. 23. we have heaps of filters
  24. 24. turns out functional principles correlate with good design
  25. 25. going up one level
  26. 26. metadata track credentials user track pageplayback groups facebook integration database api …
  27. 27. what if we apply function composition?
  28. 28. you want your services to be stateless as possible
  29. 29. you want to be careful where you store stuff
  30. 30. 💽 💽 💽 💽 💽 💽 💽 💽 💽
  31. 31. 💽 💽 💽 💽 💽Filters } Services
  32. 32. track page <track <track metadata>stats> <user metadata> 💽 💽 💽 <content filtering> <monetisation rules>
  33. 33. turns out functional principles correlate with good architecture
  34. 34. we still lack material around engineering in functional languages, but the basic principles already offer a good framework
  35. 35. tl;dr small functions + small combinators all the way up
  36. 36. Q&A

×