This document discusses monad transformers in Scala. It begins by introducing the OptionT monad transformer, which lifts an Option into a monad M. It defines the point and map methods for OptionT to make it an instance of the Monad type class. Later sections discuss using monad transformers to compose monads like IO and Option that normally do not compose, and how this allows embedding domain-specific languages within programs.
Introduces the functional programming ideas of Functor, Apply, Applicative And Monad. Shows how to implement each in Scala with Scalaz and how to validate the implementation using property based test using specs2 and scalacheck.
For the past few years in the functional Scala community, the standard approach for adding features to an effect type (features like logging, stateful updates, or accessing config) has been Monad Transformers (EItherT, OptionT, WriterT, ReaderT, etc.).
While elegant and proven, monad transformers were imported directly from Haskell, and in Scala, they have poor ergonomics and poor performance. Using tagless-final on transformers can eliminate some of the boilerplate, but cannot improve performance, and tagless-final makes it insanely hard to locally introduce and eliminate features.
In this presentation, John will introduce an alternate approach he coined ‘effect rotation’, which shares most of the power of monad transformers, but with better ergonomics and no loss of performance. You will see how to use the ZIO library that John created to composably add different features into the ZIO effect type, to solve the same problems as monad transformers, but in a way that feels natural and idiomatic for Scala.
For decades, the Functor, Monoid, and Foldable type class hierarchies have dominated functional programming. Implemented in libraries like Scalaz and Cats, these type classes have an ancient origin in Haskell, and they have repeatedly proven useful for advanced functional programmers, who use them to maximize code reuse and increase code correctness.
Yet, as these type classes have been copied into Scala and aged, there is a growing awareness of their drawbacks, ranging from being difficult to teach to weird operators that don’t make sense in Scala (ap from Applicative), to overlapping and lawless type classes (Semigroupal), to a complete inability to abstract over data types that possess related structure (such as isomorphic applicatives).
In this presentation, John A. De Goes introduces a new Scala library with a completely different factoring of functional type classes—one which throws literally everything away and starts from a clean slate. In this new factoring, type classes leverage Scala’s strengths, including variance and modularity. Pieces fit together cleanly and uniformly, and in a way that satisfies existing use cases, but enables new ones never before possible. Finally, type classes are named, organized, and described in a way that makes teaching them easier, without compromising on algebraic principles.
If you’ve ever thought functional type classes were too impractical or too confusing or too restrictive, now’s your chance to get a fresh perspective on a library that just might make understanding functional programming easier than ever before!
Final tagless. The topic strikes fear into the hearts of Scala developers everywhere—and not without reason. Final tagless allows developers to build composable Domain Specific Languages (DSLs) that model interaction with the outside world. Programs written using the final tagless style can be tested deterministically and reasoned about at compile-time. Yet the technique requires confusing, compiler-choking higher-kinded types, like `F[_]`, and pervasive, non-inferable context bounds like `F[_]: Concurrent: Console: Logging`. Many have looked at final tagless and wondered if all the layers of complexity and ceremony are really worth the benefits.
In this presentation, John A. De Goes provides a gentle and accessible introduction to final tagless, explaining what it is and the problem it intends to solve. John shows that while final tagless is easier to use than free monads, the technique suffers from a litany of drawbacks that push developers away from functional programming in Scala. John then introduces a novel approach that shares some of the benefits of final tagless, but which is idiomatic Scala, easy to explain, doesn’t need any complex type machinery, provides flawless type inference, and works beautifully across Scala 2.x and Scala 3.
Come join John for an evening of fun as you learn how to write functional code in Scala that's easy to test and easy to reason about—all without the complexity of free monads or final tagless.
Blazing Fast, Pure Effects without Monads — LambdaConf 2018John De Goes
Effect monads like IO are the way functional programmers interact with the real world. Yet, monadic effects in programming languages like Scala often perform poorly compared to their Haskell counterparts—as much as 10x slower in some benchmarks. In this presentation, John A. De Goes, author of the Scalaz 8 effect system, dredges up an old paper to cast new light on the question of how to model effects, and comes to the surprising conclusion that in Scala, monads may not be the fastest way to model purely functional effects. Join John as he shows a new model of effects that offers performance improvements without sacrificing the wonderful purity that functional programmers rely on to reason about their software.
Scalaz 8 is the latest edition of the popular functional programming library for Scala. In this whirlwind tour, maintainer John A. De Goes discusses some of the hottest features of Scalaz 8, including all of the following:
* A fast, concurrent, and leak-free effect system, which has small, composable, and powerful primitives for building practical, real-world software;
* A non-linear type class hierarchy, which permits a more powerful hierarchy that infers well without devastating ambiguous implicit errors;
* A new encoding for abstractions in category theory that providers higher fidelity and enables new categories of useful software to be developed;
* A Scala 2.12 encoding of opaque types that powers improved performance and better developer UX.
In this tour, you’ll see how the design of Scalaz 8 was inspired by a desire to provide Scala developers with a principled, performant, and pragmatic library that never sacrifices the safety and equational reasoning properties of functional programming. You’ll see live code snippets that show you how solving complex real world problems is simpler, faster, safer, and more reasonable than in previous versions of Scalaz. And hopefully you’ll be inspired at just how far functional programming in Scala has come in the past decade.
Some languages, like SML, Haskell, and Scala, have built-in support for pattern matching, which is a generic way of branching based on the structure of data.
While not without its drawbacks, pattern matching can help eliminate a lot of boilerplate, and it's often cited as a reason why functional programming languages are so concise.
In this talk, John A. De Goes talks about the differences between built-in patterns, and so-called first-class patterns (which are "do-it-yourself" patterns implemented using other language features).
Unlike built-in patterns, first-class patterns aren't magical, so you can store them in variables and combine them in lots of interesting ways that aren't always possible with built-in patterns. In addition, almost every programming language can support first-class patterns (albeit with differing levels of effort and type-safety).
During the talk, you'll watch as a mini-pattern matching library is developed, and have the opportunity to follow along and build your own pattern matching library in the language of your choice.
Introduces the functional programming ideas of Functor, Apply, Applicative And Monad. Shows how to implement each in Scala with Scalaz and how to validate the implementation using property based test using specs2 and scalacheck.
For the past few years in the functional Scala community, the standard approach for adding features to an effect type (features like logging, stateful updates, or accessing config) has been Monad Transformers (EItherT, OptionT, WriterT, ReaderT, etc.).
While elegant and proven, monad transformers were imported directly from Haskell, and in Scala, they have poor ergonomics and poor performance. Using tagless-final on transformers can eliminate some of the boilerplate, but cannot improve performance, and tagless-final makes it insanely hard to locally introduce and eliminate features.
In this presentation, John will introduce an alternate approach he coined ‘effect rotation’, which shares most of the power of monad transformers, but with better ergonomics and no loss of performance. You will see how to use the ZIO library that John created to composably add different features into the ZIO effect type, to solve the same problems as monad transformers, but in a way that feels natural and idiomatic for Scala.
For decades, the Functor, Monoid, and Foldable type class hierarchies have dominated functional programming. Implemented in libraries like Scalaz and Cats, these type classes have an ancient origin in Haskell, and they have repeatedly proven useful for advanced functional programmers, who use them to maximize code reuse and increase code correctness.
Yet, as these type classes have been copied into Scala and aged, there is a growing awareness of their drawbacks, ranging from being difficult to teach to weird operators that don’t make sense in Scala (ap from Applicative), to overlapping and lawless type classes (Semigroupal), to a complete inability to abstract over data types that possess related structure (such as isomorphic applicatives).
In this presentation, John A. De Goes introduces a new Scala library with a completely different factoring of functional type classes—one which throws literally everything away and starts from a clean slate. In this new factoring, type classes leverage Scala’s strengths, including variance and modularity. Pieces fit together cleanly and uniformly, and in a way that satisfies existing use cases, but enables new ones never before possible. Finally, type classes are named, organized, and described in a way that makes teaching them easier, without compromising on algebraic principles.
If you’ve ever thought functional type classes were too impractical or too confusing or too restrictive, now’s your chance to get a fresh perspective on a library that just might make understanding functional programming easier than ever before!
Final tagless. The topic strikes fear into the hearts of Scala developers everywhere—and not without reason. Final tagless allows developers to build composable Domain Specific Languages (DSLs) that model interaction with the outside world. Programs written using the final tagless style can be tested deterministically and reasoned about at compile-time. Yet the technique requires confusing, compiler-choking higher-kinded types, like `F[_]`, and pervasive, non-inferable context bounds like `F[_]: Concurrent: Console: Logging`. Many have looked at final tagless and wondered if all the layers of complexity and ceremony are really worth the benefits.
In this presentation, John A. De Goes provides a gentle and accessible introduction to final tagless, explaining what it is and the problem it intends to solve. John shows that while final tagless is easier to use than free monads, the technique suffers from a litany of drawbacks that push developers away from functional programming in Scala. John then introduces a novel approach that shares some of the benefits of final tagless, but which is idiomatic Scala, easy to explain, doesn’t need any complex type machinery, provides flawless type inference, and works beautifully across Scala 2.x and Scala 3.
Come join John for an evening of fun as you learn how to write functional code in Scala that's easy to test and easy to reason about—all without the complexity of free monads or final tagless.
Blazing Fast, Pure Effects without Monads — LambdaConf 2018John De Goes
Effect monads like IO are the way functional programmers interact with the real world. Yet, monadic effects in programming languages like Scala often perform poorly compared to their Haskell counterparts—as much as 10x slower in some benchmarks. In this presentation, John A. De Goes, author of the Scalaz 8 effect system, dredges up an old paper to cast new light on the question of how to model effects, and comes to the surprising conclusion that in Scala, monads may not be the fastest way to model purely functional effects. Join John as he shows a new model of effects that offers performance improvements without sacrificing the wonderful purity that functional programmers rely on to reason about their software.
Scalaz 8 is the latest edition of the popular functional programming library for Scala. In this whirlwind tour, maintainer John A. De Goes discusses some of the hottest features of Scalaz 8, including all of the following:
* A fast, concurrent, and leak-free effect system, which has small, composable, and powerful primitives for building practical, real-world software;
* A non-linear type class hierarchy, which permits a more powerful hierarchy that infers well without devastating ambiguous implicit errors;
* A new encoding for abstractions in category theory that providers higher fidelity and enables new categories of useful software to be developed;
* A Scala 2.12 encoding of opaque types that powers improved performance and better developer UX.
In this tour, you’ll see how the design of Scalaz 8 was inspired by a desire to provide Scala developers with a principled, performant, and pragmatic library that never sacrifices the safety and equational reasoning properties of functional programming. You’ll see live code snippets that show you how solving complex real world problems is simpler, faster, safer, and more reasonable than in previous versions of Scalaz. And hopefully you’ll be inspired at just how far functional programming in Scala has come in the past decade.
Some languages, like SML, Haskell, and Scala, have built-in support for pattern matching, which is a generic way of branching based on the structure of data.
While not without its drawbacks, pattern matching can help eliminate a lot of boilerplate, and it's often cited as a reason why functional programming languages are so concise.
In this talk, John A. De Goes talks about the differences between built-in patterns, and so-called first-class patterns (which are "do-it-yourself" patterns implemented using other language features).
Unlike built-in patterns, first-class patterns aren't magical, so you can store them in variables and combine them in lots of interesting ways that aren't always possible with built-in patterns. In addition, almost every programming language can support first-class patterns (albeit with differing levels of effort and type-safety).
During the talk, you'll watch as a mini-pattern matching library is developed, and have the opportunity to follow along and build your own pattern matching library in the language of your choice.
The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...John De Goes
Maybe you've played around with functional programming before, but don't consider yourself a functional programmer. Or maybe you program functionally, but only in a dynamically-typed programming language. Or MAYBE you just like workshops with really long, ridiculously-sounding titles!
No matter, this workshop that teaches the super hot programming language PureScript is guaranteed to cure what ails you!
Come, learn about functions, learn about types, learn about data, and learn about how to smash them all together to build beautiful programs that are easy to test, easy to combine, and easy to reason about.
Become the functional programmer you were born to be!
"Немного о функциональном программирование в JavaScript" Алексей КоваленкоFwdays
Все началось давно, еще в школе, классе эдак 7. Тогда учитель математики впервые произнесла фразу: "Игрек равно эф от икс". В то время я и не догадывался что это самое "эф от икс", является базовым принципом функционального программирования, да и не только функционального.
Functional Programming, Reactive Programming, Transducers, MapReduce и многое другое, так или иначе корнями уходит в то самое, незамысловатое f(x). Это настолько серьезная часть программирования, что ежеминутно, если не ежесекундно, по всему миру на клавиатуре нажимаются клавиши f, u, n, c, t, i, o, n. И нажимаются они именно в этой последовательности.
Пора принять тот факт, что без функционального программирования, программирования не существует!
Пора разобраться. Пора понять для чего нужны функции в программирование, как они должны работать и чем они могут быть полезны в ежедневной работе.
Quark: A Purely-Functional Scala DSL for Data Processing & AnalyticsJohn De Goes
Quark is a new Scala DSL for data processing and analytics that runs on top of the Quasar Analytics compiler. Quark is adept at processing semi-structured data and compiles query plans to operations that run entirely inside a target data source. In this presentation, John A. De Goes provides an overview of the open source library, showing several use cases in data processing and analytics. John also demonstrates a powerful technique that every developer can use to create their own purely-functional, type-safe DSLs in the Scala programming language.
"The joy of Scala" - Maxim Novak / Wix
Around eight years ago I started my journey as a developer. Since then, I've played around with many languages and thought that C# offers the best developer productivity. After joining Wix two years ago, I was exposed to the amazing world of Scala and Functional Programming and never looked back.
In Scala the code is much more concise, less ceremonious, immutable by default, combines functional with object oriented, seamlessly interoperates with Java, and many software engineering patterns are already baked into the language. Most importantly - Scala is FUN! By the end of the session you too will, hopefully, convert to Scala and never look back.
Recording of the lecture (Hebrew) - https://youtu.be/TcnYTwff2xU
scala.concurrent.Future is familiar to nearly all Scala devs.
This presentation first talks about referential transparency and the IO Monad in general. (Monix Task is an impl of the IO Monad.)
Then it compares Future Monix 3.x Task with their Pros and Cons.
Interop with Future: As Scala's Future is used in many environments and libraries, we look at the conversion from Task to Future and - vice versa - from Future to Task.
I will also take a look at Task evaluation, cancelation and memoization as well as tail recursive loops and asynchronous boundaries.
The presentation will include a comparative discussion on ExecutionContext (required for Future) and Scheduler (required for Task, but only to run it).
Often recurring on the valuable Monix Task doumentation at https://monix.io/docs/3x/eval/task.html the presentation can also be seen as an introduction to Monix Task.
ZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional ProgrammingJohn De Goes
As professional software engineers, sometimes messy details of the real world stand in the way of us delivering principled software. Flaky connections, unreliable services, and bulletproof job scheduling in the presence of non-determinism and failure all tricky problems that discourage us from writing principled software. Yet sometimes the shortcuts we take to solve these problems result in downtime for the business and sleepless nights for us.
In this brand-new presentation, created exclusively for Scala in the City, John A. De Goes will show how functional programming can help bring order to even the most chaotic systems. Using ZIO, a new zero-dependency Scala library for building massively scalable asynchronous and concurrent applications, John will demonstrate how functional programming leverages reified effects and algebras to solve the trickiest of reliability and scheduling problems in a principled, composable, flexible way.
Join John for an evening of fun and functional programming as you explore fresh ways of thinking about reliability and scheduling, and come out of the talk with valuable skills for using ZIO to solve the everyday problems you encounter at work.
Moore's Law may be dead, but CPUs acquire more cores every year. If you want to minimize response latency in your application, you need to use every last core - without wasting resources that would destroy performance and throughput. Traditional locks grind threads to a halt and are prone to deadlocks, and although actors provide a saner alternative, they compose poorly and are at best weakly-typed.
In this presentation, created exclusively for Scalar Conf, John reveals a new alternative to the horrors of traditional concurrency, directly comparing it to other leading approaches in Scala. Witness the beauty, simplicity, and power of declarative concurrency, as you discover how functional programming is the most practical solution to solving the tough challenges of concurrency.
In this presentation, John A. De Goes looks at several concurrency and scalability problems similar to the ones all programmers have to face, and shows how purely functional solutions written using Scalaz 8 are shorter, faster, easier to test, and easier to understand than competing solutions written using Akka actors. Discover how functional programming can be your secret superpower when it comes to quickly building bullet-proof business applications!
Free monads and free applicatives have proven an incredibly useful tool in repertoire of the functional programmer: they separate concerns, encourage denotational semantics for program specification, allow easy and type-safe mocking of purely functional code, and allow dynamic introspection and optimization.
Despite these benefits, free monads are notoriously constrained: by themselves, they cannot handle parallelism (only sequentiality), and because they provide only a monad, richer structures (such as monads that fail, or monads that support alternation) cannot be expressed without crude hacks that limit composability and expressiveness.
In this session, John A. De Goes shows how the free monad can be deconstructed for its individual features, and then rebuilt using a more powerful technique that enables more extensibility. The resulting structure — no longer technically a "free monad" — allows reification of as few or as many aspects of computation as are necessary to model the problem domain.
After the session, attendees will know how to augment their existing free programs to add parallelism, racing, failure, and other aspects of computation as required by their problem. In addition, through this thorough deconstruction and reconstruction of the free monad, attendees will have a very deep understanding of reified computation and why the free monad has the structure and limitations it does.
From Functor Composition to Monad TransformersHermann Hueck
In a List[Option[A]] or Future[Option[A]] we want to access the value of type A conveniently without nested mapping and flatMapping.
We can avoid nested mapping with composed Functors. Functors compose, but Monads do not! What can we do?
Monad transformers to the rescue!
After going through Functor composition I show how 2 Monads are bolted together with a Monad transformer and how to use this construct. I demonstrate this with the Option transformer OptionT and will end up with best practices.
The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...John De Goes
Maybe you've played around with functional programming before, but don't consider yourself a functional programmer. Or maybe you program functionally, but only in a dynamically-typed programming language. Or MAYBE you just like workshops with really long, ridiculously-sounding titles!
No matter, this workshop that teaches the super hot programming language PureScript is guaranteed to cure what ails you!
Come, learn about functions, learn about types, learn about data, and learn about how to smash them all together to build beautiful programs that are easy to test, easy to combine, and easy to reason about.
Become the functional programmer you were born to be!
"Немного о функциональном программирование в JavaScript" Алексей КоваленкоFwdays
Все началось давно, еще в школе, классе эдак 7. Тогда учитель математики впервые произнесла фразу: "Игрек равно эф от икс". В то время я и не догадывался что это самое "эф от икс", является базовым принципом функционального программирования, да и не только функционального.
Functional Programming, Reactive Programming, Transducers, MapReduce и многое другое, так или иначе корнями уходит в то самое, незамысловатое f(x). Это настолько серьезная часть программирования, что ежеминутно, если не ежесекундно, по всему миру на клавиатуре нажимаются клавиши f, u, n, c, t, i, o, n. И нажимаются они именно в этой последовательности.
Пора принять тот факт, что без функционального программирования, программирования не существует!
Пора разобраться. Пора понять для чего нужны функции в программирование, как они должны работать и чем они могут быть полезны в ежедневной работе.
Quark: A Purely-Functional Scala DSL for Data Processing & AnalyticsJohn De Goes
Quark is a new Scala DSL for data processing and analytics that runs on top of the Quasar Analytics compiler. Quark is adept at processing semi-structured data and compiles query plans to operations that run entirely inside a target data source. In this presentation, John A. De Goes provides an overview of the open source library, showing several use cases in data processing and analytics. John also demonstrates a powerful technique that every developer can use to create their own purely-functional, type-safe DSLs in the Scala programming language.
"The joy of Scala" - Maxim Novak / Wix
Around eight years ago I started my journey as a developer. Since then, I've played around with many languages and thought that C# offers the best developer productivity. After joining Wix two years ago, I was exposed to the amazing world of Scala and Functional Programming and never looked back.
In Scala the code is much more concise, less ceremonious, immutable by default, combines functional with object oriented, seamlessly interoperates with Java, and many software engineering patterns are already baked into the language. Most importantly - Scala is FUN! By the end of the session you too will, hopefully, convert to Scala and never look back.
Recording of the lecture (Hebrew) - https://youtu.be/TcnYTwff2xU
scala.concurrent.Future is familiar to nearly all Scala devs.
This presentation first talks about referential transparency and the IO Monad in general. (Monix Task is an impl of the IO Monad.)
Then it compares Future Monix 3.x Task with their Pros and Cons.
Interop with Future: As Scala's Future is used in many environments and libraries, we look at the conversion from Task to Future and - vice versa - from Future to Task.
I will also take a look at Task evaluation, cancelation and memoization as well as tail recursive loops and asynchronous boundaries.
The presentation will include a comparative discussion on ExecutionContext (required for Future) and Scheduler (required for Task, but only to run it).
Often recurring on the valuable Monix Task doumentation at https://monix.io/docs/3x/eval/task.html the presentation can also be seen as an introduction to Monix Task.
ZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional ProgrammingJohn De Goes
As professional software engineers, sometimes messy details of the real world stand in the way of us delivering principled software. Flaky connections, unreliable services, and bulletproof job scheduling in the presence of non-determinism and failure all tricky problems that discourage us from writing principled software. Yet sometimes the shortcuts we take to solve these problems result in downtime for the business and sleepless nights for us.
In this brand-new presentation, created exclusively for Scala in the City, John A. De Goes will show how functional programming can help bring order to even the most chaotic systems. Using ZIO, a new zero-dependency Scala library for building massively scalable asynchronous and concurrent applications, John will demonstrate how functional programming leverages reified effects and algebras to solve the trickiest of reliability and scheduling problems in a principled, composable, flexible way.
Join John for an evening of fun and functional programming as you explore fresh ways of thinking about reliability and scheduling, and come out of the talk with valuable skills for using ZIO to solve the everyday problems you encounter at work.
Moore's Law may be dead, but CPUs acquire more cores every year. If you want to minimize response latency in your application, you need to use every last core - without wasting resources that would destroy performance and throughput. Traditional locks grind threads to a halt and are prone to deadlocks, and although actors provide a saner alternative, they compose poorly and are at best weakly-typed.
In this presentation, created exclusively for Scalar Conf, John reveals a new alternative to the horrors of traditional concurrency, directly comparing it to other leading approaches in Scala. Witness the beauty, simplicity, and power of declarative concurrency, as you discover how functional programming is the most practical solution to solving the tough challenges of concurrency.
In this presentation, John A. De Goes looks at several concurrency and scalability problems similar to the ones all programmers have to face, and shows how purely functional solutions written using Scalaz 8 are shorter, faster, easier to test, and easier to understand than competing solutions written using Akka actors. Discover how functional programming can be your secret superpower when it comes to quickly building bullet-proof business applications!
Free monads and free applicatives have proven an incredibly useful tool in repertoire of the functional programmer: they separate concerns, encourage denotational semantics for program specification, allow easy and type-safe mocking of purely functional code, and allow dynamic introspection and optimization.
Despite these benefits, free monads are notoriously constrained: by themselves, they cannot handle parallelism (only sequentiality), and because they provide only a monad, richer structures (such as monads that fail, or monads that support alternation) cannot be expressed without crude hacks that limit composability and expressiveness.
In this session, John A. De Goes shows how the free monad can be deconstructed for its individual features, and then rebuilt using a more powerful technique that enables more extensibility. The resulting structure — no longer technically a "free monad" — allows reification of as few or as many aspects of computation as are necessary to model the problem domain.
After the session, attendees will know how to augment their existing free programs to add parallelism, racing, failure, and other aspects of computation as required by their problem. In addition, through this thorough deconstruction and reconstruction of the free monad, attendees will have a very deep understanding of reified computation and why the free monad has the structure and limitations it does.
From Functor Composition to Monad TransformersHermann Hueck
In a List[Option[A]] or Future[Option[A]] we want to access the value of type A conveniently without nested mapping and flatMapping.
We can avoid nested mapping with composed Functors. Functors compose, but Monads do not! What can we do?
Monad transformers to the rescue!
After going through Functor composition I show how 2 Monads are bolted together with a Monad transformer and how to use this construct. I demonstrate this with the Option transformer OptionT and will end up with best practices.
In this presentation, Akka Team Lead and author Roland Kuhn presents the freshly released final specification for Reactive Streams on the JVM. This work was done in collaboration with engineers representing Netflix, Red Hat, Pivotal, Oracle, Typesafe and others to define a standard for passing streams of data between threads in an asynchronous and non-blocking fashion. This is a common need in Reactive systems, where handling streams of "live" data whose volume is not predetermined.
The most prominent issue facing the industry today is that resource consumption needs to be controlled such that a fast data source does not overwhelm the stream destination. Asynchrony is needed in order to enable the parallel use of computing resources, on collaborating network hosts or multiple CPU cores within a single machine.
Here we'll review the mechanisms employed by Reactive Streams, discuss the applicability of this technology to a variety of problems encountered in day to day work on the JVM, and give an overview of the tooling ecosystem that is emerging around this young standard.
Дизайн REST API для высокопроизводительных систем / Александр Лебедев (Новые ...Ontico
Доклад осветит вопросы устройства REST API для веб-приложений и мобильных клиентов, от которых требуется высокая производительность.
Проектирование высокопроизводительных REST API.
- Кто должен участвовать в проектировании.
- Как узнать, что оптимизировать.
- Как измерять производительность REST API.
Паттерны и антипаттерны.
- Почему pagination - это плохо, и на что лучше заменить.
- Проблема N+1 и как с ней бороться.
- Бесполезные данные - как обнаружить и уничтожить.
- Как не ломать кэширование на клиенте.
- Эффективная работа с интерфейсами "мастер-детали".
Кэширование.
- Три слоя кэширования.
- Самый быстрый запрос - тот, которого не было. Как увеличить их количество.
- Экономия трафика.
- Исключение ненужных вычислений.
- Подходы к инвалидации кэша.
Приемы оптимизации работы с API на клиенте.
- Параллельные запросы.
- Эффективный разбор данных.
- In-memory DB на клиенте.
- Стратегии кэширования на клиенте.
Functional programming in Scala. Looking at various examples of defining a program first and executing it at some later stage, separating pure functions from side effects.
This is going to be a discussion about design patterns. But I promise it’s going to be very different from the Gang of Four patterns that we all have used and loved in Java.
It doesn’t have any mathematics or category theory - it’s about developing an insight that lets u identify code structures that u think may be improved with a beautiful transformation of an algebraic pattern.
In earlier days of Java coding we used to feel proud when we could locate a piece of code that could be transformed into an abstract factory and the factory bean could be injected using Spring DI. The result was we ended up maintaining not only Java code, but quite a bit of XML too, untyped and unsafe. This was the DI pattern in full glory. In this session we will discuss patterns that don’t look like external artifacts, they are part of the language, they have some mathematical foundations in the sense that they have an algebra that actually compose and compose organically to evolve larger abstractions.
Introduction to ad-3.4, an automatic differentiation library in Haskellnebuta
Haskellの自動微分ライブラリ Ad-3.4 の紹介(の試み) If you don't see 21 slides in this presentation, try this one (re-uploaded): http://www.slideshare.net/nebuta/130329-ad-by-ekmett
This is the slide for what I shared in JS Group meetup, 2014, Taiwan. It covers what JavaScript could do for making the program more "functional", the benefits, price and the limitation.
Download for better quality.
Monads do not Compose. Not in a generic way - There is no general way of composing monads.
A comment from Rúnar Bjarnason, coauthor of FP in Scala: "They do compose in a different generic way. For any two monads F and G we can take the coproduct which is roughly Free of Either F or G (up to normalization)".
Another comment from Sergei Winitzki (which caused me to upload https://www.slideshare.net/pjschwarz/addendum-to-monads-do-not-compose): "It is a mistake to think that a traversable monad can be composed with another monad. It is true that, given `Traversable`, you can implement the monad's methods (pure and flatMap) for the composition with another monad (as in your slides 21 to 26), but this is a deceptive appearance. The laws of the `Traversable` typeclass are far insufficient to guarantee the laws of the resulting composed monad. The only traversable monads that work correctly are Option, Either, and Writer. It is true that you can implement the type signature of the `swap` function for any `Traversable` monad. However, the `swap` function for monads needs to satisfy very different and stronger laws than the `sequence` function from the `Traversable` type class. I'll have to look at the "Book of Monads"; but, if my memory serves, the FPiS book does not derive any of these laws." See https://www.linkedin.com/feed/update/urn:li:groupPost:41001-6523141414614814720?commentUrn=urn%3Ali%3Acomment%3A%28groupPost%3A41001-6523141414614814720%2C6532108273053761536%29
Function Programming in Scala.
A lot of my examples here comes from the book
Functional programming in Scala By Paul Chiusano and Rúnar Bjarnason, It is a good book, buy it.
In functional programming, words from Category Theory are thrown around, but how useful are they really?
This session looks at applications of monoids specifically and how using their algebraic properties offers a solid foundation of reasoning in many types of business domains and reduces developer error as computational context complexity increases.
This will provide a tiny peak at Category Theory's practical uses in software development and modeling. Code examples will be in Haskell and Scala, but monoids could be constructed in almost any language by software craftsmen and women utilizing higher orders of reasoning to their code.
Different Ways of Function Composition in Scala:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Fine-grained composability of functions is one of the core advantages of FP.
Treating "Functions as Data" means that we can store, manipulate, pass functions around and compose them in much the same way we do with data.
This talk demonstrates different ways of function composition in Scala.
The focus lies on scala.Function1, because due to tupling and currying we can regard any FunctionN (except Function0) as a Function1. Curried functions are easier to compose.
Starting with the composition methods of scala.Function1: apply, compose and andThen, we will investigate folding a Seq of functions.
We can also define a pipe operator |> as in F# in order to 'pipe' values through a pipeline of functions.
Defining a Monoid for Function1 allows us to combine two or more functions into a new one.
A function can also be seen as a Functor and a Monad. That means: Functions can be mapped and flatMapped over. And we can write for-comprehensions in a Function1 context just as we do with List, Option, Future, Either etc.
Being Monads, we can use functions in any monadic context. We will see that Function1 is the Reader Monad.
The most powerful way of function composition is Kleisli (also known as ReaderT). We will see that Kleisli (defined with the Id context) is the Reader Monad again.
4. SF SCALA
May 2012
* http://marakana.com/s/scala_typeclassopedia_with_john_kodumal_of_atlassian_video,1198/index.html
5. trait Monad[F[_]] extends Applicative[F] {
def flatMap[A, B](fa: F[A])(f :A=>F[B]):F[B]
}
* monad type class
* flatMap also called bind, >>=
6. def point[A](a: => A): M[A]
def map[A,B](ma: M[A])(f: A => B): M[B]
def flatMap[A,B](ma: M[A])(f: A => M[B]): M[B]
* the functions we care about
* lift pure value, lift pure function, chain “operations”
7. scala> import scalaz.Monad
scala> import scalaz.std.option._
scala> val a = Monad[Option].point(1)
a: Option[Int] = Some(1)
scala> Monad[Option].map(a)(_.toString + "hi")
res2: Option[java.lang.String] = Some(1hi)
scala> Monad[Option].bind(a)(i => if (i < 0) None else Some(i + 1))
res4: Option[Int] = Some(2)
* explicit type class usage in scalaz seven
8. scala> import scalaz.syntax.monad._
import scalaz.syntax.monad._
scala> Option(1).flatMap(i => if (i < 0) None else Some(i+1))
res6: Option[Int] = Some(2)
scala> 1.point[Option].flatMap(...)
res7: Option[Int] = Some(2)
* implicit type class usage in scalaz7 using syntax extensions
9. “A MONADIC FOR
COMPREHENSION IS AN
EMBEDDED PROGRAMMING
LANGUAGE WITH SEMANTICS
DEFINED BY THE MONAD”
* “one intuition of monads” - john
12. SIDE NOTE:
SEMANTICS
* to an extent, you can “choose” the meaning of a monad
* Option -- anon. exceptions -- more narrowly, the exception that something is not there. Validation - monad/not monad - can
mean different things in different contexts
21. def composeFunctor[M[_],N[_]](implicit m: Functor[M], n: Functor[N]) =
new Functor[({type MN[A]=[M[N[A]]]})#MN] {
def map[A,B](mna: M[N[A]])(f: A => B): M[N[B]] = ...
}
* generic function that composes any two functors M[_] and N[_]
23. scala> Option("abc").map(f)
res1: Option[Int] = Some(3)
scala> List(Option("abc"), Option("d"), Option("ef")).map2(f)
res2: List[Option[Int]] = List(Some(3), Some(1), Some(2))
* can compose functors infinitely deep but...
* scalaz provides method to compose 2, with nice syntatic sugar, easily (map2)
24. def notPossible[M[_],N[_]](implicit m: Monad[M], n: Monad[N]) =
new Monad[({type MN[A]=[M[N[A]]]})#MN] {
def flatMap[A,B](mna: M[N[A]])(f: A => M[N[B]]): M[N[B]] = ...
}
* cannot write the same function for any two monads M[_], N[_]
25. IT !
def notPossible[M[_],N[_]](implicit m: Monad[M], n: Monad[N]) =
Y
new Monad[({type MN[A]=[M[N[A]]]})#MN] {
R
def flatMap[A,B](mna: M[N[A]])(f: A => M[N[B]]): M[N[B]] = ...
}
T
* best way to understand this is attempt to write it yourself
* it won’t compile
27. STAIR
STEPPING
* the problem in practice
*http://www.flickr.com/photos/caliperstudio/2667302181/
28. val a: IO[Option[MyData]] = ...
val b: IO[Option[MyData]] = ...
* have two values that require we communicate w/ outside world to fetch
* those values may not exist (alternative meaning, fetching may result in exceptions that are anonymous)
29. for {
data1 <- a
data2 <- b
} yield {
data1 merge data2 // fail
}
* want to merge the two pieces of data if they both exist
30. for {
// we've escaped IO, fail
d1 <- a.unsafePerformIO
d2 <- b.unsafePerformIO
} yield d1 merge d2
* don’t want to perform the actions until later (don’t escape the IO monad)
31. for {
od1 <- a for {
od2 <- b
od1 <- a
} yield (od1,od2) match {
od2 <- b
case (Some(d1),Some(d2) =>
} yield for {
Option(d1 merge d2)
d1 <- od1
case (a@Some(d1),_)) => a
d2 <- od2
case (_,a@Some(d2)) => a
case _ => None } yield d1 merge d2
}
* may notice the semi-group here
* can also write it w/ an applicative
* this is a contrived example
32. BUT WHAT IF...
def b(data: MyData): IO[Option[MyData]
* even w/ simple example, this minor change throws a monkey wrench in things
33. for {
):
readRes <- readIO(domain)
res <- readRes.fold(
success = _.cata(
some = meta =>
if (meta.enabledStatus /== status) {
writeIO(meta.copy(enabledStatus = status))
} else meta.successNel[BarneyException].pure[IO],
none = new ReadFailure(domain).failNel[AppMetadata].pure[IO]
),
failure = errors => errors.fail[AppMetadata].pure[IO]
)
} yield res
* example of what not to do from something I wrote a while back
35. case class IOOption[A](run: IO[Option[A]])
define type that boxes box the value, doesn’t need to be a case class, similar to haskell newtype.
36. new Monad[IOOption] {
def point[A](a: => A): IOOption[A] = IOOption(a.point[Option].point[IO])
def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] =
IOOption(fa.run.map(opt => opt.map(f)))
def flatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]):IOOption[B] =
IOOption(fa.run.flatMap((o: Option[A]) => o match {
case Some(a) => f(a).run
case None => (None : Option[B]).point[IO]
}))
}
* can define a Monad instance for new type
37. val a: IOOption[MyData] = ...
val b: IOOption[MyData] = ...
val c: IOOption[MyData] = for {
data1 <- a
data2 <- b
} yield {
data1 merge data2
}
val d: IO[Option[MyData]] = c.run
can use new type to improve previous contrived example
38. type MyState[A] = State[StateData,A]
case class MyStateOption[A](run: MyState[Option[A]])
* what if we don’t need effects, but state we can read and write to produce a final optional value and some new state
* State[S,A] where S is fixed is a monad
* can define a new type for that as well
39. new Monad[MyStateOption] { new Monad[IOOption] {
def map[A,B](fa: MyStateOption[A])(f: A => B): MyStateOption[B] = def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] =
MyStateOption(Functor[MyState].map(fa)(opt => opt.map(f))) IOOption(Functor[IO].map(fa)(opt => opt.map(f)))
def flatMap[A, B](fa: MyStateOption[A])(f :A=>IOOption[B]) = def flatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]) =
MyStateOption(Monad[MyState]].bind(fa)((o: Option[A]) => o match { IOOption(Monad[IO]].bind(fa)((o: Option[A]) => o match {
case Some(a) => f(a).run case Some(a) => f(a).run
case None => (None : Option[B]).point[MyState] case None => (None : Option[B]).point[IO]
})) }))
} }
* opportunity for more abstraction
* if you were going to do this, not exactly the way you would define these in real code, cheated a bit using {Functor,Monad}.apply
41. case class OptionT[M[_], A](run: M[Option[A]]) {
def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B]
def flatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]): OptionT[M,B]
}
* define map/flatMap a little differently, can be done like previous as typeclass instance but convention is to define the interface
on the transformer and later define typeclass instance using the interface
42. case class OptionT[M[_], A](run: M[Option[A]]) {
def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] =
OptionT[M,B](F.map(run)((o: Option[A]) => o map f))
def flatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]): OptionT[M,B] =
OptionT[M,B](M.bind(run)((o: Option[A]) => o match {
case Some(a) => f(a).run
case None => M.point((None: Option[B]))
}))
}
* implementations resemble what has already been shown
43. new Monad[IOOption] {
case class OptionT[M[_], A](run: M[Option[A]]) {
def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] =
def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] =
OptionT[M,B](F.map(run)((o: Option[A]) => o map f)) IOOption(Functor[IO].map(fa)(opt => opt.map(f)))
def flatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]) = def flatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]) =
OptionT[M,B](M.bind(run)((o: Option[A]) => o match {
IOOption(Monad[IO]].bind(fa)((o: Option[A]) => o match {
case Some(a) => f(a).run
case Some(a) => f(a).run
case None => M.point((None: Option[B]))
})) case None => (None : Option[B]).point[IO]
} }))
}
* it the generalization of what was written before
44. type FlowState[A] = State[ReqRespData, A]
val f: Option[String] => FlowState[Boolean] = (etag: Option[String]) => {
val a: OptionT[FlowState, Boolean] = for {
// string <- OptionT[FlowState,String]
e <- optionT[FlowState](etag.point[FlowState])
// wrap FlowState[Option[String]] in OptionT
matches <- optionT[FlowState]((requestHeadersL member IfMatch))
} yield matches.split(",").map(_.trim).toList.contains(e)
a getOrElse false // FlowState[Boolean]
}
* check existence of etag in an http request, data lives in state
* has minor bug, doesn’t deal w/ double quotes as written
* https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/
WebmachineDecisions.scala#L282-285
45. val reqCType: OptionT[FlowState,ContentType] = for {
contentType <- optionT[FlowState](
(requestHeadersL member ContentTypeHeader)
)
mediaInfo <- optionT[FlowState](
parseMediaTypes(contentType).headOption.point[FlowState]
)
} yield mediaInfo.mediaRange
* determine content type of the request, data lives in state, may not be specified
* https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/
WebmachineDecisions.scala#L772-775
46. scala> type EitherTString[M[_],A] = EitherT[M,String,A]
defined type alias EitherTString
scala> val items = eitherT[List,String,Int](List(1,2,3,4,5,6).map(Right(_)))
items: scalaz.EitherT[List,String,Int] = ...
* adding features to a “embedded language”
47. for { i <- items } yield print(i)
// 123456
for {
i <- items
_ <- if (i > 4) leftT[List,String,Unit]("fail")
else rightT[List,String,Unit](())
} yield print(i)
// 1234
* adding error handling, and early termination to non-deterministic computation
51. BOXES A VALUE
run: M[MyMonad[A]
* value is typically called “run” in scalaz7
* often called “value” in scalaz6 (because of NewType)
52. A MONAD
TRANSFORMER
IS A
MONAD TOO
* i mean, its thats kinda the point of this whole exercise isn’t it :)
53. def optTMonad[M[_] : Monad] = new Monad[({type O[X]=OptionT[M,X]]})#O) {
def point[A](a: => A): OptionT[M,A] = OptionT(a.point[Option].point[M])
def map[A,B](fa: OptionT[M,A])(f: A => B): OptionT[M,B] = fa map f
def flatMap[A, B](fa: OptionT[M,A])(f :A=> OptionT[M,B]): OptionT[M, B] =
fa flatMap f
}
* monad instance definition for OptionT
54. HAS INTERFACE
RESEMBLING UNDERLYING
MONAD’S INTERFACE
* can interact with the monad transformer in a manner similar to working with the actual monad
* same methods, slightly different type signatures
* different from haskell, “feature” of scala, since we can define methods on a type
57. TRANSFORMER IS A MONAD
TRANSFORMER CAN WRAP
ANOTHER TRANSFORMER
* at the start, the goal was to stack effects (not just stack 2 effects)
* this makes it possible
58. type VIO[A] = ValidationT[IO,Throwable,A]
def doWork(): VIO[Option[Int]] = ...
val r: OptionT[VIO,Int] = optionT[VIO](doWork())
* wrap the ValidationT with success type Option[A] in an OptionT
* define type alias for connivence -- avoids nasty type lambda syntax inline
59. val action: OptionT[VIO, Boolean] = for {
devDomain <- optionT[VIO] {
validationT(
bucket.fetch[CName]("%s.%s".format(devPrefix,hostname))
).mapFailure(CNameServiceException(_))
}
_ <- optionT[VIO] {
validationT(deleteDomains(devDomain)).map(_.point[Option])
}
} yield true
* code (slightly modified) from one of stackmob’s internal services
* uses Scaliak to fetch hostname data from riak and then remove them
* possible to clean this code up a bit, will discuss shortly (monadtrans)
60. KEEP ON
STACKIN’
ON
* don’t have to stop at 2 levels deep, our new stack is monad too
* each monad/transformer we add to the stack compose more types of effects
61. “ORDER”
MATTERS
* how stack is built, which transformers wrap which monads, determines the overall semantics of the entire stack
* changing that order can, and usually does, change semantics
62. OptionT[FlowState, A]
vs.
StateT[Option,ReqRespData,A]
* what is the difference in semantics between the two?
* type FlowState[A] = State[ReqRespData,A]
63. FlowState[Option[A]]
vs.
Option[State[ReqRespData,A]
* unboxing makes things easier to see
* a state action that returns an optional value vs a state action that may not exist
* the latter probably doesn’t make as much sense in the majority of cases
64. MONADTRANS
The Type Class
* type classes beget more type classes
65. REMOVING REPETITION
===
MORE ABSTRACTION
* previous examples have had a repetitive, annoying, & verbose task
* can be abstracted away...by a type class of course
66. optionT[VIO](validationT(deleteDomains(devDomain)).map(_.point[Option]))
eitherT[List,String,Int](List(1,2,3,4,5,6).map(Right(_)))
resT[FlowState](encodeBodyIfSet(resource).map(_.point[Res]))
* some cases require lifting the value into the monad and then wrap it in the transformer
* from previous examples
67. M[A] -> M[N[A]] -> NT[M[N[_]], A]
* this is basically what we are doing every time
* taking some monad M[A], lifting A into N, a monad we have a transformer for, and then wrapping all of that in N’s monad
transformer
68. trait MonadTrans[F[_[_], _]] {
def liftM[G[_] : Monad, A](a: G[A]): F[G, A]
}
* liftM will do this for any transformer F[_[_],_] and any monad G[_] provided an instance of it is defined for F[_[_],_]
69. def liftM[G[_], A](a: G[A])(implicit G: Monad[G]): OptionT[G, A] =
OptionT[G, A](G.map[A, Option[A]](a)((a: A) => a.point[Option]))
* full definition requires some type ceremony
* https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/scalaz/OptionT.scala#L155-156
70. def liftM[G[_], A](ga: G[A])(implicit G: Monad[G]): ResT[G,A] =
ResT[G,A](G.map(ga)(_.point[Res]))
* implementation for scalamachine’s Res monad
* https://github.com/stackmob/scalamachine/blob/master/scalaz7/src/main/scala/scalamachine/scalaz/res/
ResT.scala#L75-76
71. encodeBodyIfSet(resource).liftM[OptionT]
List(1,2,3).liftM[EitherTString]
validationT(deleteDomains(devDomain)).liftM[OptionT]
* cleanup of previous examples
* method-like syntax requires a bit more work: https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/
scalaz/syntax/MonadSyntax.scala#L9
74. STACKING
MONADS
COMPOSES
EFFECTS
* when monads are stacked an embedded language is being built with multiple effects
* this is not the only intuition of monads/transformers
75. CAN NOT
COMPOSE MONADS
GENERICALLY
* cannot write generic function to compose any two monads M[_], N[_] like we can for any two functors
76. MONAD TRANSFORMERS
COMPOSE M[_] : MONAD WITH
ANY N[_] : MONAD
* can’t compose any two, but can compose a given one with any other
77. MONAD TRANSFORMERS
WRAP OTHER
MONAD TRANSFORMERS
* monad transformers are monads
* so they can be the N[_] : Monad that the transformer composes with its underlying monad
78. MONADTRANS
REDUCES
REPETITION
* often need to take a value that is not entirely lifted into a monad transformer stack and do just that
79. STACK MONADS
DON’T
STAIR-STEP
* monad transformers reduce ugly, stair-stepping or nested code and focuses on core task
* focuses on intuition of mutiple effects instead of handling things haphazardly
80. THANK
YOU
* stackmob, markana, john & atlassian, other sponsors, cosmin