Scalaz-stream introduced the Scala ecosystem to purely functional streams. Although widely deployed in production at Verizon, SlamData, and elsewhere, the last release of scalaz-stream was October 23rd, 2016, and newer projects such as FS2, Monix Iterant, and others have risen to fill the gap, each addressing different use cases in the functional programming community. In this talk, John A. De Goes, architect of the Scalaz 8 effect system, unveils a new design for a next-generation version of scalaz-stream. Designed to embrace the powerful features of the Scalaz 8 effect system—features not available in any other effect type—this design features higher-performance, stronger guarantees, and far less code than previous incarnations. You are invited to discover the practicality of functional programming, as well as its elegance, beauty, and composability, in this unique presentation available exclusively at Scale By the Bay.
At the heart of data processing, event-sourcing, actors, and much more is the queue—a data structure that allows producers to pass work to consumers in a flexible way. On the JVM, most libraries, including Akka, are powered by Java concurrent queues, which are reliable but were designed in a different era, for synchronous (blocking) procedural code. In this presentation, John A. De Goes—architect of the Scalaz 8 effect system—introduces IOQueue, a new type of queue powered by the Scalaz 8 IO monad. IOQueue never blocks and provides seamless, composable back-pressure across an entire application, without users having to think about the problem or write any special code. John will discuss how IOQueue achieves these remarkable properties, and show how the structure can be used to solve hard problems with just a few lines of type-safe, leak-free, composable Scala code. Come learn about the power of Scalaz to solve the hard problems of software development, in a principled way, without compromises.
The Design of the Scalaz 8 Effect SystemJohn De Goes
Purely functional Scala code needs something like Haskell's IO monad—a construct that allows functional programs to interact with external, effectful systems in a referentially transparent way. To date, most effect systems for Scala have fallen into one of two categories: pure, but slow or inexpressive; or fast and expressive, but impure and unprincipled. In this talk, John A. De Goes, the architect of Scalaz 8’s new effect system, introduces a novel solution that’s up to 100x faster than Future and Cats Effect, in a principled, modular design that ships with all the powerful primitives necessary for building complex, real-world, high-performance, concurrent functional programs.
Thanks to built-in concurrency, high performance, lawful semantics, and rich expressivity, Scalaz 8's effect system may just be the effect system to attract the mainstream Scala developers who aren't familiar with functional programming.
Halogen is a popular choice for building front-end user-interfaces with PureScript. Often described as a purely functional version of React, Halogen allows building user-interfaces by composing declarative, self-contained components, including effectful components those built from third-party Javascript libraries.
In this presentation, John presents a high-level summary of where Halogen has come from, how it works right now, and what are the main drawbacks to both FRP and React. John then suggests that incremental computation should be the foundation for the next major version of Halogen, and sketches out a possible way of achieving that in a declarative fashion.
ZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional ProgrammingJohn De Goes
As professional software engineers, sometimes messy details of the real world stand in the way of us delivering principled software. Flaky connections, unreliable services, and bulletproof job scheduling in the presence of non-determinism and failure all tricky problems that discourage us from writing principled software. Yet sometimes the shortcuts we take to solve these problems result in downtime for the business and sleepless nights for us.
In this brand-new presentation, created exclusively for Scala in the City, John A. De Goes will show how functional programming can help bring order to even the most chaotic systems. Using ZIO, a new zero-dependency Scala library for building massively scalable asynchronous and concurrent applications, John will demonstrate how functional programming leverages reified effects and algebras to solve the trickiest of reliability and scheduling problems in a principled, composable, flexible way.
Join John for an evening of fun and functional programming as you explore fresh ways of thinking about reliability and scheduling, and come out of the talk with valuable skills for using ZIO to solve the everyday problems you encounter at work.
In this presentation, John A. De Goes looks at several concurrency and scalability problems similar to the ones all programmers have to face, and shows how purely functional solutions written using Scalaz 8 are shorter, faster, easier to test, and easier to understand than competing solutions written using Akka actors. Discover how functional programming can be your secret superpower when it comes to quickly building bullet-proof business applications!
Free monads and free applicatives have proven an incredibly useful tool in repertoire of the functional programmer: they separate concerns, encourage denotational semantics for program specification, allow easy and type-safe mocking of purely functional code, and allow dynamic introspection and optimization.
Despite these benefits, free monads are notoriously constrained: by themselves, they cannot handle parallelism (only sequentiality), and because they provide only a monad, richer structures (such as monads that fail, or monads that support alternation) cannot be expressed without crude hacks that limit composability and expressiveness.
In this session, John A. De Goes shows how the free monad can be deconstructed for its individual features, and then rebuilt using a more powerful technique that enables more extensibility. The resulting structure — no longer technically a "free monad" — allows reification of as few or as many aspects of computation as are necessary to model the problem domain.
After the session, attendees will know how to augment their existing free programs to add parallelism, racing, failure, and other aspects of computation as required by their problem. In addition, through this thorough deconstruction and reconstruction of the free monad, attendees will have a very deep understanding of reified computation and why the free monad has the structure and limitations it does.
At the heart of data processing, event-sourcing, actors, and much more is the queue—a data structure that allows producers to pass work to consumers in a flexible way. On the JVM, most libraries, including Akka, are powered by Java concurrent queues, which are reliable but were designed in a different era, for synchronous (blocking) procedural code. In this presentation, John A. De Goes—architect of the Scalaz 8 effect system—introduces IOQueue, a new type of queue powered by the Scalaz 8 IO monad. IOQueue never blocks and provides seamless, composable back-pressure across an entire application, without users having to think about the problem or write any special code. John will discuss how IOQueue achieves these remarkable properties, and show how the structure can be used to solve hard problems with just a few lines of type-safe, leak-free, composable Scala code. Come learn about the power of Scalaz to solve the hard problems of software development, in a principled way, without compromises.
The Design of the Scalaz 8 Effect SystemJohn De Goes
Purely functional Scala code needs something like Haskell's IO monad—a construct that allows functional programs to interact with external, effectful systems in a referentially transparent way. To date, most effect systems for Scala have fallen into one of two categories: pure, but slow or inexpressive; or fast and expressive, but impure and unprincipled. In this talk, John A. De Goes, the architect of Scalaz 8’s new effect system, introduces a novel solution that’s up to 100x faster than Future and Cats Effect, in a principled, modular design that ships with all the powerful primitives necessary for building complex, real-world, high-performance, concurrent functional programs.
Thanks to built-in concurrency, high performance, lawful semantics, and rich expressivity, Scalaz 8's effect system may just be the effect system to attract the mainstream Scala developers who aren't familiar with functional programming.
Halogen is a popular choice for building front-end user-interfaces with PureScript. Often described as a purely functional version of React, Halogen allows building user-interfaces by composing declarative, self-contained components, including effectful components those built from third-party Javascript libraries.
In this presentation, John presents a high-level summary of where Halogen has come from, how it works right now, and what are the main drawbacks to both FRP and React. John then suggests that incremental computation should be the foundation for the next major version of Halogen, and sketches out a possible way of achieving that in a declarative fashion.
ZIO Schedule: Conquering Flakiness & Recurrence with Pure Functional ProgrammingJohn De Goes
As professional software engineers, sometimes messy details of the real world stand in the way of us delivering principled software. Flaky connections, unreliable services, and bulletproof job scheduling in the presence of non-determinism and failure all tricky problems that discourage us from writing principled software. Yet sometimes the shortcuts we take to solve these problems result in downtime for the business and sleepless nights for us.
In this brand-new presentation, created exclusively for Scala in the City, John A. De Goes will show how functional programming can help bring order to even the most chaotic systems. Using ZIO, a new zero-dependency Scala library for building massively scalable asynchronous and concurrent applications, John will demonstrate how functional programming leverages reified effects and algebras to solve the trickiest of reliability and scheduling problems in a principled, composable, flexible way.
Join John for an evening of fun and functional programming as you explore fresh ways of thinking about reliability and scheduling, and come out of the talk with valuable skills for using ZIO to solve the everyday problems you encounter at work.
In this presentation, John A. De Goes looks at several concurrency and scalability problems similar to the ones all programmers have to face, and shows how purely functional solutions written using Scalaz 8 are shorter, faster, easier to test, and easier to understand than competing solutions written using Akka actors. Discover how functional programming can be your secret superpower when it comes to quickly building bullet-proof business applications!
Free monads and free applicatives have proven an incredibly useful tool in repertoire of the functional programmer: they separate concerns, encourage denotational semantics for program specification, allow easy and type-safe mocking of purely functional code, and allow dynamic introspection and optimization.
Despite these benefits, free monads are notoriously constrained: by themselves, they cannot handle parallelism (only sequentiality), and because they provide only a monad, richer structures (such as monads that fail, or monads that support alternation) cannot be expressed without crude hacks that limit composability and expressiveness.
In this session, John A. De Goes shows how the free monad can be deconstructed for its individual features, and then rebuilt using a more powerful technique that enables more extensibility. The resulting structure — no longer technically a "free monad" — allows reification of as few or as many aspects of computation as are necessary to model the problem domain.
After the session, attendees will know how to augment their existing free programs to add parallelism, racing, failure, and other aspects of computation as required by their problem. In addition, through this thorough deconstruction and reconstruction of the free monad, attendees will have a very deep understanding of reified computation and why the free monad has the structure and limitations it does.
All Aboard The Scala-to-PureScript Express!John De Goes
Many Scala programmers have embraced functional programming, but the syntax and semantics of programming languages in the Haskell family remains a mystery. In this talk, Scala developers (and to some extent, Java developers) will see how the types, data structures, traits / interfaces, packages, and so forth translate into their PureScript counterparts.
Quark: A Purely-Functional Scala DSL for Data Processing & AnalyticsJohn De Goes
Quark is a new Scala DSL for data processing and analytics that runs on top of the Quasar Analytics compiler. Quark is adept at processing semi-structured data and compiles query plans to operations that run entirely inside a target data source. In this presentation, John A. De Goes provides an overview of the open source library, showing several use cases in data processing and analytics. John also demonstrates a powerful technique that every developer can use to create their own purely-functional, type-safe DSLs in the Scala programming language.
Functional Programming Patterns for the Pragmatic ProgrammerRaúl Raja Martínez
In this talk we will see a pragmatic approach to building a purely functional architecture that delivers cohesive functional components.
We will cover functional patterns such as Free Monads, Transformers, Kleisli arrows, dependently typed checked exceptions
and types as well as how they can be glued together to achieve pure functions that are composable, context free, dependently injectable and testable.
Dome project and code with instructions to run it can be found at:
https://github.com/47deg/func-architecture
How Pony ORM translates Python generators to SQL queriesponyorm
Pony ORM is an Object-Relational Mapper implemented in Python. It uses an unusual approach for writing database queries using Python generators. Pony analyzes the abstract syntax tree of a generator and translates it to its SQL equivalent. The translation process consists of several non-trivial stages.
This talk was given at EuroPython 2014 and reveals the internal details of the translation process.
Final tagless. The topic strikes fear into the hearts of Scala developers everywhere—and not without reason. Final tagless allows developers to build composable Domain Specific Languages (DSLs) that model interaction with the outside world. Programs written using the final tagless style can be tested deterministically and reasoned about at compile-time. Yet the technique requires confusing, compiler-choking higher-kinded types, like `F[_]`, and pervasive, non-inferable context bounds like `F[_]: Concurrent: Console: Logging`. Many have looked at final tagless and wondered if all the layers of complexity and ceremony are really worth the benefits.
In this presentation, John A. De Goes provides a gentle and accessible introduction to final tagless, explaining what it is and the problem it intends to solve. John shows that while final tagless is easier to use than free monads, the technique suffers from a litany of drawbacks that push developers away from functional programming in Scala. John then introduces a novel approach that shares some of the benefits of final tagless, but which is idiomatic Scala, easy to explain, doesn’t need any complex type machinery, provides flawless type inference, and works beautifully across Scala 2.x and Scala 3.
Come join John for an evening of fun as you learn how to write functional code in Scala that's easy to test and easy to reason about—all without the complexity of free monads or final tagless.
Introduction to Swift programming language.Icalia Labs
Take a look to Swift, if you've been developing for iOS in Objective-C many things may look familiar, maybe just "upgraded". If you're a first timer diving into iOS development we strongly recommend you to understand first the basics of Cocoa.
With the announcement on officially supporting Kotlin for Android, this presentation tries to compare Java and Kotlin to encourage people to switch to Kotlin
Kotlin Bytecode Generation and Runtime Performanceintelliyole
In this talk, we'll dive into the details of how various language features supported by Kotlin are translated to Java bytecode. We'll use the JMH microbenchmarking tool to study the relative performance of various constructs and to understand how we can ensure top performance of the Kotlin code that we write.
Twins: Object Oriented Programming and Functional ProgrammingRichardWarburton
Object-Oriented Programming has well established design principles, such as SOLID. For many developers architecture and functional programming are at odds with each other: they don’t know how their existing tricks of the trade convert into functional design. This problem becomes worse as hybrid languages such as Java 8 or Scala become common. We’ll talk about how functional programming helps you implement the SOLID principles, and how a functional mindset can actually help you achieve cleaner and simpler OO design.
Introduction to how to test our programs using imperative programming and functional programming with Tagless Final technique and how to make it simpler using ZIO. ZIO provides many features for concurrency and asynchronous programs. This presentation is about how to use ZIO environment to test your application.
ZIO provides a variety of features for building synchronous, asynchronous and concurrent applications and enables us to use functional effects in Scala.
- Why functional effects?
- How to use functional effects and data types built on ZIO?
- How to manage errors and recover from them?
- How to manage resources and how to make concurrent tasks?
The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...John De Goes
Maybe you've played around with functional programming before, but don't consider yourself a functional programmer. Or maybe you program functionally, but only in a dynamically-typed programming language. Or MAYBE you just like workshops with really long, ridiculously-sounding titles!
No matter, this workshop that teaches the super hot programming language PureScript is guaranteed to cure what ails you!
Come, learn about functions, learn about types, learn about data, and learn about how to smash them all together to build beautiful programs that are easy to test, easy to combine, and easy to reason about.
Become the functional programmer you were born to be!
This presentation explains the difference between concurrency and parallelism, and how could we make parallel computations and how could we design an API for parallel computation following the structure presented in the Functional Programming in Scala book
All Aboard The Scala-to-PureScript Express!John De Goes
Many Scala programmers have embraced functional programming, but the syntax and semantics of programming languages in the Haskell family remains a mystery. In this talk, Scala developers (and to some extent, Java developers) will see how the types, data structures, traits / interfaces, packages, and so forth translate into their PureScript counterparts.
Quark: A Purely-Functional Scala DSL for Data Processing & AnalyticsJohn De Goes
Quark is a new Scala DSL for data processing and analytics that runs on top of the Quasar Analytics compiler. Quark is adept at processing semi-structured data and compiles query plans to operations that run entirely inside a target data source. In this presentation, John A. De Goes provides an overview of the open source library, showing several use cases in data processing and analytics. John also demonstrates a powerful technique that every developer can use to create their own purely-functional, type-safe DSLs in the Scala programming language.
Functional Programming Patterns for the Pragmatic ProgrammerRaúl Raja Martínez
In this talk we will see a pragmatic approach to building a purely functional architecture that delivers cohesive functional components.
We will cover functional patterns such as Free Monads, Transformers, Kleisli arrows, dependently typed checked exceptions
and types as well as how they can be glued together to achieve pure functions that are composable, context free, dependently injectable and testable.
Dome project and code with instructions to run it can be found at:
https://github.com/47deg/func-architecture
How Pony ORM translates Python generators to SQL queriesponyorm
Pony ORM is an Object-Relational Mapper implemented in Python. It uses an unusual approach for writing database queries using Python generators. Pony analyzes the abstract syntax tree of a generator and translates it to its SQL equivalent. The translation process consists of several non-trivial stages.
This talk was given at EuroPython 2014 and reveals the internal details of the translation process.
Final tagless. The topic strikes fear into the hearts of Scala developers everywhere—and not without reason. Final tagless allows developers to build composable Domain Specific Languages (DSLs) that model interaction with the outside world. Programs written using the final tagless style can be tested deterministically and reasoned about at compile-time. Yet the technique requires confusing, compiler-choking higher-kinded types, like `F[_]`, and pervasive, non-inferable context bounds like `F[_]: Concurrent: Console: Logging`. Many have looked at final tagless and wondered if all the layers of complexity and ceremony are really worth the benefits.
In this presentation, John A. De Goes provides a gentle and accessible introduction to final tagless, explaining what it is and the problem it intends to solve. John shows that while final tagless is easier to use than free monads, the technique suffers from a litany of drawbacks that push developers away from functional programming in Scala. John then introduces a novel approach that shares some of the benefits of final tagless, but which is idiomatic Scala, easy to explain, doesn’t need any complex type machinery, provides flawless type inference, and works beautifully across Scala 2.x and Scala 3.
Come join John for an evening of fun as you learn how to write functional code in Scala that's easy to test and easy to reason about—all without the complexity of free monads or final tagless.
Introduction to Swift programming language.Icalia Labs
Take a look to Swift, if you've been developing for iOS in Objective-C many things may look familiar, maybe just "upgraded". If you're a first timer diving into iOS development we strongly recommend you to understand first the basics of Cocoa.
With the announcement on officially supporting Kotlin for Android, this presentation tries to compare Java and Kotlin to encourage people to switch to Kotlin
Kotlin Bytecode Generation and Runtime Performanceintelliyole
In this talk, we'll dive into the details of how various language features supported by Kotlin are translated to Java bytecode. We'll use the JMH microbenchmarking tool to study the relative performance of various constructs and to understand how we can ensure top performance of the Kotlin code that we write.
Twins: Object Oriented Programming and Functional ProgrammingRichardWarburton
Object-Oriented Programming has well established design principles, such as SOLID. For many developers architecture and functional programming are at odds with each other: they don’t know how their existing tricks of the trade convert into functional design. This problem becomes worse as hybrid languages such as Java 8 or Scala become common. We’ll talk about how functional programming helps you implement the SOLID principles, and how a functional mindset can actually help you achieve cleaner and simpler OO design.
Introduction to how to test our programs using imperative programming and functional programming with Tagless Final technique and how to make it simpler using ZIO. ZIO provides many features for concurrency and asynchronous programs. This presentation is about how to use ZIO environment to test your application.
ZIO provides a variety of features for building synchronous, asynchronous and concurrent applications and enables us to use functional effects in Scala.
- Why functional effects?
- How to use functional effects and data types built on ZIO?
- How to manage errors and recover from them?
- How to manage resources and how to make concurrent tasks?
The Easy-Peasy-Lemon-Squeezy, Statically-Typed, Purely Functional Programming...John De Goes
Maybe you've played around with functional programming before, but don't consider yourself a functional programmer. Or maybe you program functionally, but only in a dynamically-typed programming language. Or MAYBE you just like workshops with really long, ridiculously-sounding titles!
No matter, this workshop that teaches the super hot programming language PureScript is guaranteed to cure what ails you!
Come, learn about functions, learn about types, learn about data, and learn about how to smash them all together to build beautiful programs that are easy to test, easy to combine, and easy to reason about.
Become the functional programmer you were born to be!
This presentation explains the difference between concurrency and parallelism, and how could we make parallel computations and how could we design an API for parallel computation following the structure presented in the Functional Programming in Scala book
Building Kafka Connectors with Kotlin: A Step-by-Step Guide to Creation and D...HostedbyConfluent
"Kafka Connect, the framework for building scalable and reliable data pipelines, has gained immense popularity in the data engineering landscape. This session will provide a comprehensive guide to creating Kafka connectors using Kotlin, a language known for its conciseness and expressiveness.
In this session, we will explore a step-by-step approach to crafting Kafka connectors with Kotlin, from inception to deployment using an simple use case. The process includes the following key aspects:
Understanding Kafka Connect: We'll start with an overview of Kafka Connect and its architecture, emphasizing its importance in real-time data integration and streaming.
Connector Design: Delve into the design principles that govern connector creation. Learn how to choose between source and sink connectors and identify the data format that suits your use case.
Building a Source Connector: We'll start with building a Kafka source connector, exploring key considerations, such as data transformations, serialization, deserialization, error handling and delivery guarantees. You will see how Kotlin's concise syntax and type safety can simplify the implementation.
Testing: Learn how to rigorously test your connector to ensure its reliability and robustness, utilizing best practices for testing in Kotlin.
Connector Deployment: go through the process of deploying your connector in a Kafka Connect cluster, and discuss strategies for monitoring and scaling.
Real-World Use Cases: Explore real-world examples of Kafka connectors built with Kotlin.
By the end of this session, you will have a solid foundation for creating and deploying Kafka connectors using Kotlin, equipped with practical knowledge and insights to make your data integration processes more efficient and reliable. Whether you are a seasoned developer or new to Kafka Connect, this guide will help you harness the power of Kafka and Kotlin for seamless data flow in your applications."
Real-time Stream Processing with Apache Flink @ Hadoop SummitGyula Fóra
Apache Flink is an open source project that offers both batch and stream processing on top of a common runtime and exposing a common API. This talk focuses on the stream processing capabilities of Flink.
Apache flink: data streaming as a basis for all analytics by Kostas Tzoumas a...Big Data Spain
Flink is one of the largest and most active Apache big data projects with well over 120 contributors.
Session presented at Big Data Spain 2015 Conference
16th Oct 2015
Kinépolis Madrid
http://www.bigdataspain.org
Event promoted by: http://www.paradigmatecnologico.com
Abstract:http://www.bigdataspain.org/program/fri/slot-31.html
Creating the PromQL Transpiler for Flux by Julius Volz, Co-Founder | PrometheusInfluxData
Flux is not only a new data scripting and query language — it is also a powerful data processing engine. This talk by Julius Volz will focus on how he worked with the InfluxData team to build PromQL support for the Flux engine. Hear about lessons learned from building the transpiler and recommendations on why and how to use PromQL and Flux. This talk will include a demo and will share the current project progress.
Ruby on Rails com certeza é a estrela que elevou Ruby ao patamar de linguagem praticamente obrigatória a todo programador moderno. Porém muitos esquecem que Rails não é a única forma de desenvolver aplicações Web e muitas vezes nem é a melhor opção. O Ecossistema Ruby evolui a passos largos, todos os novos frameworks Web em Ruby adotaram o padrão Rack, que facilita a interoperabilidade entre frameworks Ruby. Além disso muitos estão olhando para o mundo da alta concorrência com novas tecnologias como Node.JS, mas o mundo Ruby tem opções robustas e testadas em produção como EventMachine. Enfim, vamos tentar aumentar o leque de soluções web com Ruby, além do Rails, traçando um paralelo com a plataforma .NET.
Wikipedia’s Event Data Platform, Or: JSON Is Okay Too With Andrew Otto | Curr...HostedbyConfluent
Wikipedia’s Event Data Platform, Or: JSON Is Okay Too With Andrew Otto | Current 2022
The Wikimedia Foundation (which operates Wikipedia) has a different engineering environment than most organizations. We build systems using only Free and Open Source Software. We have a diverse and active developer community that contributes to our software. For privacy reasons, we own and run bare metal hardware. We care about open data, and strive to make our data publicly available.
Because of this, the way we build event driven architectures is different too. The data we produce should be easily consumable for both internal engineers as well as the public
developer community. Avro and other binary formats can make using data difficult, so we intentionally chose to avoid them.
This session will describe how and why we built Wikimedia's Event Data Platform using Kafka, JSON and JSONSchemas, and how we make our event data available to the world.
This presentation held in at Inovex GmbH in Munich in November 2015 was about a general introduction of the streaming space, an overview of Flink and use cases of production users as presented at Flink Forward.
ISTIO is one of big trends these days. Red Hat will support Service Mesh from OpenShift 4.0. This presentation contains interactive demo using chat application.
For decades, the Functor, Monoid, and Foldable type class hierarchies have dominated functional programming. Implemented in libraries like Scalaz and Cats, these type classes have an ancient origin in Haskell, and they have repeatedly proven useful for advanced functional programmers, who use them to maximize code reuse and increase code correctness.
Yet, as these type classes have been copied into Scala and aged, there is a growing awareness of their drawbacks, ranging from being difficult to teach to weird operators that don’t make sense in Scala (ap from Applicative), to overlapping and lawless type classes (Semigroupal), to a complete inability to abstract over data types that possess related structure (such as isomorphic applicatives).
In this presentation, John A. De Goes introduces a new Scala library with a completely different factoring of functional type classes—one which throws literally everything away and starts from a clean slate. In this new factoring, type classes leverage Scala’s strengths, including variance and modularity. Pieces fit together cleanly and uniformly, and in a way that satisfies existing use cases, but enables new ones never before possible. Finally, type classes are named, organized, and described in a way that makes teaching them easier, without compromising on algebraic principles.
If you’ve ever thought functional type classes were too impractical or too confusing or too restrictive, now’s your chance to get a fresh perspective on a library that just might make understanding functional programming easier than ever before!
For the past few years in the functional Scala community, the standard approach for adding features to an effect type (features like logging, stateful updates, or accessing config) has been Monad Transformers (EItherT, OptionT, WriterT, ReaderT, etc.).
While elegant and proven, monad transformers were imported directly from Haskell, and in Scala, they have poor ergonomics and poor performance. Using tagless-final on transformers can eliminate some of the boilerplate, but cannot improve performance, and tagless-final makes it insanely hard to locally introduce and eliminate features.
In this presentation, John will introduce an alternate approach he coined ‘effect rotation’, which shares most of the power of monad transformers, but with better ergonomics and no loss of performance. You will see how to use the ZIO library that John created to composably add different features into the ZIO effect type, to solve the same problems as monad transformers, but in a way that feels natural and idiomatic for Scala.
Modern applications are concurrent, parallel, asynchronous, and synchronous; they utilize many different subsystems, including network systems, actor systems, distributed systems, and more. Across all these modes of computation and different subsystems, the one constant is failure. Errors happen everywhere, and taming their monstrous complexity in a way that helps developers write correct code and troubleshoot failures is one of the hardest challenges of modern application development.
In this presentation, created just for the Dublin Scala Meetup, John A. De Goes and Kai from 7mind.io will take attendees on a tour of error management in Scala, comparing and contrasting Scala's own Future type, and the ZIO effect type. You'll see how functional effects provide features that go way beyond Future: including unified errors across all modes of computation, powerful error operators, lossless error propagation, compiler-assisted error handling, and a stunning new feature for debugging, sponsored by Irish consultancy 7mind.io, will be unveiled exclusively at this presentation.
Come learn about how modern functional effect systems like ZIO provide compelling new solutions to the problems of everyday error management.
Moore's Law may be dead, but CPUs acquire more cores every year. If you want to minimize response latency in your application, you need to use every last core - without wasting resources that would destroy performance and throughput. Traditional locks grind threads to a halt and are prone to deadlocks, and although actors provide a saner alternative, they compose poorly and are at best weakly-typed.
In this presentation, created exclusively for Scalar Conf, John reveals a new alternative to the horrors of traditional concurrency, directly comparing it to other leading approaches in Scala. Witness the beauty, simplicity, and power of declarative concurrency, as you discover how functional programming is the most practical solution to solving the tough challenges of concurrency.
Scalaz-stream introduced the Scala ecosystem to purely functional streams. Although widely deployed in production at Verizon, SlamData, and elsewhere, the last release of scalaz-stream was October 23rd, 2016, and newer projects such as FS2, Monix Iterant, and others have risen to fill the gap, each addressing different use cases in the functional programming community. In this talk, John A. De Goes, architect of the Scalaz 8 effect system, unveils a new design for a next-generation version of scalaz-stream. Designed to embrace the powerful features of the Scalaz 8 effect system—features not available in any other effect type—this design features higher-performance, stronger guarantees, and far less code than previous incarnations. You are invited to discover the practicality of functional programming, as well as its elegance, beauty, and composability, in this unique presentation available exclusively at Scale By the Bay.
Blazing Fast, Pure Effects without Monads — LambdaConf 2018John De Goes
Effect monads like IO are the way functional programmers interact with the real world. Yet, monadic effects in programming languages like Scala often perform poorly compared to their Haskell counterparts—as much as 10x slower in some benchmarks. In this presentation, John A. De Goes, author of the Scalaz 8 effect system, dredges up an old paper to cast new light on the question of how to model effects, and comes to the surprising conclusion that in Scala, monads may not be the fastest way to model purely functional effects. Join John as he shows a new model of effects that offers performance improvements without sacrificing the wonderful purity that functional programmers rely on to reason about their software.
Scalaz 8 is the latest edition of the popular functional programming library for Scala. In this whirlwind tour, maintainer John A. De Goes discusses some of the hottest features of Scalaz 8, including all of the following:
* A fast, concurrent, and leak-free effect system, which has small, composable, and powerful primitives for building practical, real-world software;
* A non-linear type class hierarchy, which permits a more powerful hierarchy that infers well without devastating ambiguous implicit errors;
* A new encoding for abstractions in category theory that providers higher fidelity and enables new categories of useful software to be developed;
* A Scala 2.12 encoding of opaque types that powers improved performance and better developer UX.
In this tour, you’ll see how the design of Scalaz 8 was inspired by a desire to provide Scala developers with a principled, performant, and pragmatic library that never sacrifices the safety and equational reasoning properties of functional programming. You’ll see live code snippets that show you how solving complex real world problems is simpler, faster, safer, and more reasonable than in previous versions of Scalaz. And hopefully you’ll be inspired at just how far functional programming in Scala has come in the past decade.
Well-architected libraries for functional programming are at once immensely beautiful and practical. They are simple but extraordinarily powerful, helping users solve their problems by snapping together Lego-like building blocks, each of which has just one purpose. Yet, there is a surprising dearth of material on how developers can construct their own well-architected functional code. Many functional programming tutorials talk discuss type safety and making illegal states unrepresentable, but few speak on the subject of good functional interface design.
In this presentation, John A. De Goes takes to the stage to discuss a nebulous and underrated tool in the arsenal of every functional programmer. Called *orthogonality*, this tool allows programmers to craft the building blocks of their functional code at "right angles", so so they can be reasoned about simply and composed predictably to solve complex problems. John introduces the concept of orthogonality, looking at its geometric and algebraic origins, presents a way to measure orthogonality, and then walks through a number of interface examples, comparing non-orthogonal designs with orthogonal ones.
By the end of the session, attendees should have a newfound appreciation for how important orthogonality is to constructing good functional interfaces, and they should develop the early stages of an intuition about how to slice up a complex problem into core, single-purpose, composable building blocks.
There are many popular stream libraries for Scala developers, including Akka Streams, scalaz-stream, fs2, plus others in the Java ecosystem. While all excellent choices for building reactive Scala applications, their reliance on effects makes them particularly difficult to test and reason about. In this talk, long-time Scala functional programmer John A. De Goes takes to the stage to demonstrate a new approach to modeling streams that requires less machinery and has more reasoning power, composability, flexibility, and testability than many other approaches. By attending the talk, you'll learn how the best stream library may be the one you get for (co)free!
What's the best way to model modular, composable effects in your purely functional program? In this presentation, I take a look at monad transformers and free monads, discuss their history, and compare how effectively they solve the problem.
SlamData - How MongoDB Is Powering a Revolution in Visual AnalyticsJohn De Goes
Slides from my presentation at MongoDB Days Silicon Valley. I discuss what SlamData is, the challenges it had to solve to build an analytics solution in the NoSQL space, and how specific features of MongoDB help power its advanced analytics functionality.
The Next Great Functional Programming LanguageJohn De Goes
A LambdaConf 2015 talk.
John has no clue what the next great functional programming language will be like, but he's more than happy to hop up on stage and rant against type classes, nominative typing, data, modules, pattern matching, recursion, and, well, basically everything else you associate with functional programming! John will argue that to make radical progress in cheaply building correct software, we need to take one step backward and two steps forward, embracing dependent-typing, total functions, Turing-complete compilers, host languages, and automated proof search. Attend this talk to question your assumptions about functional programming... or just to heckle! Either way, all are welcome.
NoSQL databases like MongoDB are making it easier than ever for developers to quickly build complex, agile applications that scale well. Unfortunately, the rise of NoSQL has dark side: data stored in NoSQL databases is invisible to traditional business analytics and intelligence apps. As a result, operations often spends a massive amount of time and effort developing complex pipelines to transform NoSQL data into a format compatible with traditional RDBMS. In this talk, John A. De Goes introduces SlamData, a new open source project designed to build a NoSQL BI application by extending the operations and data model of relational algebra. The result is backward compatible with SQL, but allows freeform analytics on semi-structured, heterogeneous, deeply nested data. John discusses the design of SlamData and highlights some of the issues involved in bringing the project to life.
Some languages, like SML, Haskell, and Scala, have built-in support for pattern matching, which is a generic way of branching based on the structure of data.
While not without its drawbacks, pattern matching can help eliminate a lot of boilerplate, and it's often cited as a reason why functional programming languages are so concise.
In this talk, John A. De Goes talks about the differences between built-in patterns, and so-called first-class patterns (which are "do-it-yourself" patterns implemented using other language features).
Unlike built-in patterns, first-class patterns aren't magical, so you can store them in variables and combine them in lots of interesting ways that aren't always possible with built-in patterns. In addition, almost every programming language can support first-class patterns (albeit with differing levels of effort and type-safety).
During the talk, you'll watch as a mini-pattern matching library is developed, and have the opportunity to follow along and build your own pattern matching library in the language of your choice.
Quirrel is a statistically-oriented language designed principally for data analysis. It combines a purely-declarative, implicitly parallel design with features needed by data scientists. In this presentation, John A. De Goes (chairman of the Quirrel language committee) introduces Quirrel and shows how it can be used to solve problems across large data sets. Over the past 5 years, R has enjoyed tremendous success in the data science community, and for good reason—it comes with batteries loaded, and sports one of the best communities in the data science world. Although R is not an easy programming language to learn, the basics can be picked up rather quickly. In this talk, John A. De Goes walks through the core syntax and features of R, providing enough training to give anyone the ability to do simple analysis.
Predictive analytics have long lived in the domain of statistical tools like R. Increasingly, however, as companies struggle to deal with exploding volumes of data not easily analyzed by small data tools, they are looking at ways of doing predictive analytics directly inside the primary data store.
This approach, called in-database predictive analytics, eliminates the need to sample data and perform a separate ETL process into a statistical tool, which can decrease total cost, improve the quality of predictive models, and dramatically shorten development time. In this class, you will learn the pros and cons of doing in-database predictive analytics, highlights of its limitations, and survey the tools and technologies necessary to head down the path.
Every company is at a different stage in leveraging analytics to improve their operational efficiency and product offerings. In this presentation, you will learn an analytics maturity model that companies can use to determine how far they are from the most successful analytical companies.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
38. Stream
PERFORMANCEMOTIVATION API DESIGN SUMMARY
type StreamIO[A] = Stream[IOException, A]
val bytes: StreamIO[Byte] = …
val headersAndContent:
Managed[IOException,
(ContentType, StreamIO[Byte])] =
bytes.peel(parseContentType)
headersAndContent.use {
case (ContentType(Json), content) =>
content.transduce(toJson)...
case (ContentType(Xml), content) =>
content.transduce(toXml)...
}
stream.map(f)
stream.flatMap(f)
stream.filter(f)
stream1 ++ stream2
stream.transduce(what)
stream.scan(z)(f)
stream.take(n)
stream.takeWhile(f)
stream.drop(n)
stream.dropWhile(f)
stream.foreach(f)
stream1.zip(stream2)
stream1.merge(stream2)
stream1.joinWith(stream2)(f)
stream.peel(what)
39. Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Sink[E, A0, A, B]
May fail with a checked error of type E
May effectfully produce a result of type B
Consumes values of type A
May possibly output a remainder of type A0
41. Sink
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Monoid
append
zero
Applicative
map
point
ap
Monad
bind
Bifunctor
leftMap
bimap
Alternative
alt
empty
Category
id
compose
Profunctor
lmap
rmap
dimap
Strong
first
second
Choice
left
right
60. Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Start with stream of integers (chunked)
Filter for only even integers
Convert to longs
Sum all integers
61. Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
def akkaChunkFilterMapSum = {
val chunks = (1 to chunkCount).flatMap(i =>
Array.fill(chunkSize)(i))
val program = AkkaSource
.fromIterator(() => chunks.iterator)
.filter(_ % 2 == 0)
.map(_.toLong)
.toMat(AkkaSink.fold(0L)(_ + _))(Keep.right)
Await.result(program.run, Duration.Inf)
}
62. Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
63. Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
def fs2ChunkFilterMapSum = {
val chunks = (1 to chunkCount).map(i =>
FS2Chunk.array(Array.fill(chunkSize)(i)))
val stream = FS2Stream(chunks: _*)
.flatMap(FS2Stream.chunk(_))
.filter(_ % 2 == 0)
.map(_.toLong)
.covary[CatsIO]
.compile
.fold(0L)(_ + _)
stream.unsafeRunSync
}
64. Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
FS2
65. Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
def scalazChunkFilterMapSum = {
val chunks = (1 to chunkCount).map(i =>
Chunk.fromArray(Array.fill(chunkSize)(i)))
val stream = StreamChunk
.fromChunks(chunks: _*)
.filter(_ % 2 == 0)
.map(_.toLong)
val sink = Sink.foldLeft(0L)((s, c) => c.foldl(s)(_ + _))
unsafeRun(stream.run(sink))
}
66. Filter · Map · Sum
PERFORMANCEMOTIVATION API DESIGN SUMMARY
FS2
67. CSV Tokenize
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Start with stream of characters (chunked)
Identify CSV separators and columns
Convert character stream into token stream
76. Initial Encoding
sealed trait Stream[+A] Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
PERFORMANCEMOTIVATION API DESIGN SUMMARY
77. Initial Encoding
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
78. Initial Encoding
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
case class Map[A0, A](s: Stream[A0],
f: A0 => A) extends Stream[A]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
79. Initial Encoding
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
case class Map[A0, A](s: Stream[A0],
f: A0 => A) extends Stream[A]
case class Join[A](s: Stream[Stream[A]])
extends Stream[A]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
80. Initial Encoding
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
case class Map[A0, A](s: Stream[A0],
f: A0 => A) extends Stream[A]
case class Join[A](s: Stream[Stream[A]])
extends Stream[A]
case class Merge[A](l: Stream[A],
r: Stream[A]) extends Stream[A]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
81. Initial Encoding
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
case class Map[A0, A](s: Stream[A0],
f: A0 => A) extends Stream[A]
case class Join[A](s: Stream[Stream[A]])
extends Stream[A]
case class Merge[A](l: Stream[A],
r: Stream[A]) extends Stream[A]
case class Fold[A0, A](a: A,
s: Stream[A0],
f: (A, A0) => A) extends Stream[A]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
82. Initial Encoding
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
sealed trait Stream[+A]
case class Emit[A](a: A) extends Stream[A]
case class Map[A0, A](s: Stream[A0],
f: A0 => A) extends Stream[A]
case class Join[A](s: Stream[Stream[A]])
extends Stream[A]
case class Merge[A](l: Stream[A],
r: Stream[A]) extends Stream[A]
case class Fold[A0, A](a: A,
s: Stream[A0],
f: (A, A0) => A) extends Stream[A]
case class Effect[A](io: IO[A]) extends Stream[A]
83. Initial Encoding
def drain[A](s: Stream[A]): IO[Unit] =
s match {
case Emit(a) => …
...
}
Interpreter
PERFORMANCEMOTIVATION API DESIGN SUMMARY
85. Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit])
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
86. Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit])
object Stream {
def emit[A](a: => A): Stream[A] =
Stream[A](read => read(a))
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
87. Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit]) {
def map[B](f: A => B): Stream[B] =
Stream(r => run(r.compose(f)))
}
object Stream {
def emit[A](a: => A): Stream[A] =
Stream[A](read => read(a))
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
88. Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit]) {
def map[B](f: A => B): Stream[B] =
Stream(r => run(r.compose(f)))
}
object Stream {
def emit[A](a: => A): Stream[A] =
Stream[A](read => read(a))
def join[A](s: Stream[Stream[A]]) =
Stream(r => s.run(sa => sa.run(r)))
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
89. Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit]) {
def map[B](f: A => B): Stream[B] =
Stream(r => run(r.compose(f)))
def merge[A1 >: A](that: Stream[A1]) =
Stream[A1](r =>
run(r).par(that.run(r)).void)
}
object Stream {
def emit[A](a: => A): Stream[A] =
Stream[A](read => read(a))
def join[A](s: Stream[Stream[A]]) =
Stream(r => s.run(sa => sa.run(r)))
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
90. Final Encoding
case class Stream[+A](
run: (A => IO[Unit]) => IO[Unit]) {
def map[B](f: A => B): Stream[B] =
Stream(r => run(r.compose(f)))
def merge[A1 >: A](that: Stream[A1]) =
Stream[A1](r =>
run(r).par(that.run(r)).void)
def fold[S](s: S)(f: (S, A) => S) = ...
}
object Stream {
def emit[A](a: => A): Stream[A] =
Stream[A](read => read(a))
def join[A](s: Stream[Stream[A]]) =
Stream(r => s.run(sa => sa.run(r)))
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Type
Emission
Mapping
Joining
Merging
Folding
Effect
...
93. Constant space processing
Discovered in 2003
Aggressively lazy
ITERATEES
PERFORMANCEMOTIVATION API DESIGN SUMMARY
Final encoding-friendly
Automatically leak-free
“Left folds”
94. Iteratees 101
sealed trait Iteratee[-A, +B]
case class More[A, B](f: A => Iteratee[A, B]) extends Iteratee[A, B]
case class Done[B](value: B) extends Iteratee[Any, B]
sealed trait Enumerator[+A] { self =>
def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B]
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
95. Iteratees 101
sealed trait Enumerator[+A] { self =>
final def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B] =
fold(...)(...)
def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B]
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
96. Iteratees 101
sealed trait Enumerator[+A] { self =>
final def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B] =
fold(...)(...)
def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B]
final def map[B](f0: A => B): Enumerator[B] =
new Enumerator[B] {
def fold[C, S](s: S)(f: (S, B) => Either[S, C]): IO[C] =
self.fold(s)((s, a) => f(s, f0(a)))
}
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
97. Iteratees 101
sealed trait Enumerator[+A] { self =>
final def run[A1 >: A, B](iteratee: Iteratee[A1, B]): IO[B] =
fold(...)(...)
def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B]
final def map[B](f0: A => B): Enumerator[B] =
new Enumerator[B] {
def fold[C, S](s: S)(f: (S, B) => Either[S, C]): IO[C] =
self.fold(s)((s, a) => f(s, f0(a)))
}
final def filter(f0: A => Boolean): Enumerator[A] =
new Enumerator[A] {
def fold[B, S](s: S)(f: (S, A) => Either[S, B]): IO[B] =
self.fold(s)((s, a) => if (f0(a)) f(s, a) else Left(s))
}
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
103. Iteratees 101
sealed trait IterateeT[F[_], -A, +B]
case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B]
case class Done[F[_], B](value: B) extends IterateeT[F, Any, B]
case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B]
PERFORMANCEMOTIVATION API DESIGN SUMMARY
104. Iteratees 101
sealed trait IterateeT[F[_], -A, +B]
case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B]
case class Done[F[_], B](value: B) extends IterateeT[F, Any, B]
case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B]
type Parallel[F[_], A1, A2, B] =
IterateeT[IterateeT[F, A1, ?], A2, B]
def lift[F[_], A1, A2, B](it: IterateeT[F, A1, B]): Parallel[F, A1, A2, B] =
Effect[IterateeT[F, A1, ?], B](it)
PERFORMANCEMOTIVATION API DESIGN SUMMARY
105. Iteratees 101
sealed trait IterateeT[F[_], -A, +B]
case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B]
case class Done[F[_], B](value: B) extends IterateeT[F, Any, B]
case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B]
type Parallel[F[_], A1, A2, B] =
IterateeT[IterateeT[F, A1, ?], A2, B]
def lift[F[_], A1, A2, B](it: IterateeT[F, A1, B]): Parallel[F, A1, A2, B] =
Effect[IterateeT[F, A1, ?], B](it)
def readLeft[F[_], A1, A2]: Parallel[F, A1, A2, A1] =
lift(More(a => Done(a)))
def readRight[F[_], A1, A2]: Parallel[F, A1, A2, A2] =
More(a => Done(a))
PERFORMANCEMOTIVATION API DESIGN SUMMARY
106. Iteratees 101
sealed trait IterateeT[F[_], -A, +B]
case class More[F[_], A, B](f: A => IterateeT[F, A, B]) extends IterateeT[F, A, B]
case class Done[F[_], B](value: B) extends IterateeT[F, Any, B]
case class Effect[F[_], B](fx: F[B]) extends IterateeT[F, Any, B]
type Parallel[F[_], A1, A2, B] =
IterateeT[IterateeT[F, A1, ?], A2, B]
def lift[F[_], A1, A2, B](it: IterateeT[F, A1, B]): Parallel[F, A1, A2, B] =
Effect[IterateeT[F, A1, ?], B](it)
def readLeft[F[_], A1, A2]: Parallel[F, A1, A2, A1] =
lift(More(a => Done(a)))
def readRight[F[_], A1, A2]: Parallel[F, A1, A2, A2] =
More(a => Done(a))
PERFORMANCEMOTIVATION API DESIGN SUMMARY
107. ZIO Stream
trait Stream[E, A] {
def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]]
}
object Stream {
sealed trait Step[S]
object Step {
case class Stop[S](value: S) extends Step[S]
case class Cont[S](value: S) extends Step[S]
}
}
PERFORMANCEMOTIVATION API DESIGN SUMMARY
108. ZIO Stream
trait Stream[E, A] {
def fold[S](s: S)(f: (S, A) => IO[E, Step[S]]): IO[E, Step[S]]
}
object Stream {
sealed trait Step[S]
object Step {
case class Stop[S](value: S) extends Step[S]
case class Cont[S](value: S) extends Step[S]
}
}
Continuation Monad!
PERFORMANCEMOTIVATION API DESIGN SUMMARY
117. Fusion, Queue &
complex combinators
Performance
Outstanding Work
Comprehensive test
suites and great docs
Tests & Docs
Combinators for
every reason
Combinators
PERFORMANCEMOTIVATION API DESIGN SUMMARY
118. Eric Torreborre — Beautiful Folds
The FS2 team — Michael Pilquist, Fabio Labella, Pavel Chlupacek, et al
Special thanks to all people who made ZIO Stream possible.
Credits
Oleg Kiselyov — Iteratees 2003
PERFORMANCEMOTIVATION API DESIGN SUMMARY
119. Any questions?
Thanks!
John A. De Goes
@jdegoes - degoes.net
Itamar Ravid
@itrvd - iravid.com
PERFORMANCEMOTIVATION API DESIGN SUMMARY