Command/Query responsibility separation and event sourcing are two great patterns for structuring microservices. They are a natural fit for architectures based around Kafka and streams, and help us achieve immutability and replaybility for our data. However, it's not always clear *how* you should create a CQRS-based application, and where do all the moving parts fit. In this talk, I will show how we can implement CQRS and ES in a purely functional manner. We'll see how doing this is nothing more than activating the data types and type classes already written for us: we'll use traversable functors and the State and Writer monads to elegantly construct the components that we need. We'll also touch the pitfalls here and how to integrate these components with a streaming library such as FS2.
The document discusses using functional programming techniques to solve common programming problems in a type-safe way. It presents examples using data types like WriterT, IndexedStateT, and Codensity to handle time measurement, state machines, and resource management. WriterT allows logging computations to a shared log. IndexedStateT encodes state machine transitions in the type system. Codensity models resource acquisition and release in a way that ensures cleanup. The examples demonstrate how functional patterns can address common tasks in a pure and safe manner.
Monitoring Complex Systems: Keeping Your Head on Straight in a Hard WorldBrian Troutwine
This talk will provide motivation for the extensive instrumentation of complex computer systems and make the argument that such systems. This talk will provide practical starting points in Erlang projects and maintain a perspective on the human organization around the computer system. Brian will focus on getting started with instrumentation in a systematic way and follow up with the challenge of interpreting and acting on metrics emitted from a production system in a way which does not overwhelm operators’ ability to effectively control or prioritize faults in the system. He’ll use historical examples and case studies from my work to keep the talk anchored in the practical.
Talk objectives:
Brian hopes to convince the audience of two things:
* that monitoring and instrumentation is an essential component of any long-lived system and
* that it's not so hard to get started, after all.
He’ll keep a clear-eyed view of what works and is difficult in practice so that the audience can make a reasoned decision after the talk.
Target audience:
This talk would appeal to engineers with long-running production employments, operations folks and Erlangers in general.
Bei Jimdo sammeln wir jede Menge Metriken über alle Teile unseres Systems. Dabei fallen Daten auf allen Ebenen des Systems an: Infrastruktur, System und Applikation. Wichtig ist, dass alle Entwickler zu jedem Zeitpunkt Einblick in die Echtzeit-Metriken ihrer Services nehmen können. Um das zu garantieren, haben wir uns einige Zeit mit der Integration von Prometheus in unsere Systeme beschäftigt.
In unserem Talk werden wir sowohl über den Betrieb von Prometheus als auch über die Integrationen mit dem Rest der Jimdo-Plattform sprechen. Wir werden von Stolpersteinen und Tricks berichten, die wir gelernt haben, sowie einen Einblick in unserer Tool-Landschaft geben.
Why do we build applications based on microservices? To gain flexibility and resilience - that’s what everybody tells us. But building and application with a lot of microservices leads to a highly distributed system - and distribution hurts! How can we keep dependencies between microservices at the lowest possible level and how can we reduce communication between them? We will have a look at „Event-Sourcing“ as a possible (and very cool) technique for asynchronous communication and data exchange/replication between microservices - in theory and also in code.
This document provides an overview of Azure Stream Analytics. It discusses how Stream Analytics is an easy to use real-time event processing engine that can be used for scenarios like personalized stock trading analysis, fraud detection, and IoT applications. It describes key capabilities like scalability and ease of use. It also outlines the steps to create a Stream Analytics job, including adding inputs, outputs, and queries. Finally it provides examples of using Stream Analytics for Twitter sentiment analysis and real-time fraud detection.
Functors show up everywhere in our day-to-day programming. They're so common, we take them for granted - especially in typed functional programming. Beside being common, they're incredibly useful for code reuse. However, functors have several relatively unknown variants: profunctors, bifunctors, contravariant functors, and so on. And guess what - they're amazingly useful, especially combined with other abstractions in the functional programming toolkit! In this talk, we'll cover the many species of functors and see how they can help us with tasks such as serialization, stream processing, and more.
The document discusses common mistakes and pitfalls when developing with Ruby on Rails, including:
1) Not understanding the difference between 'and' and '&&' operators and operator precedence issues this can cause.
2) Not sanitizing user input passed to functions like strip_tags, which can lead to security issues.
3) Caching controller actions without checking for initialized instance variables, which can cause crashes.
4) Issues with validating database uniqueness across multiple processes that can lead to duplicate entry errors.
5) Problems composing conditions for queries when categories or subcategories are empty.
6) Breaking the callback chain filter by returning from callbacks.
7) Perform
The document discusses using functional programming techniques to solve common programming problems in a type-safe way. It presents examples using data types like WriterT, IndexedStateT, and Codensity to handle time measurement, state machines, and resource management. WriterT allows logging computations to a shared log. IndexedStateT encodes state machine transitions in the type system. Codensity models resource acquisition and release in a way that ensures cleanup. The examples demonstrate how functional patterns can address common tasks in a pure and safe manner.
Monitoring Complex Systems: Keeping Your Head on Straight in a Hard WorldBrian Troutwine
This talk will provide motivation for the extensive instrumentation of complex computer systems and make the argument that such systems. This talk will provide practical starting points in Erlang projects and maintain a perspective on the human organization around the computer system. Brian will focus on getting started with instrumentation in a systematic way and follow up with the challenge of interpreting and acting on metrics emitted from a production system in a way which does not overwhelm operators’ ability to effectively control or prioritize faults in the system. He’ll use historical examples and case studies from my work to keep the talk anchored in the practical.
Talk objectives:
Brian hopes to convince the audience of two things:
* that monitoring and instrumentation is an essential component of any long-lived system and
* that it's not so hard to get started, after all.
He’ll keep a clear-eyed view of what works and is difficult in practice so that the audience can make a reasoned decision after the talk.
Target audience:
This talk would appeal to engineers with long-running production employments, operations folks and Erlangers in general.
Bei Jimdo sammeln wir jede Menge Metriken über alle Teile unseres Systems. Dabei fallen Daten auf allen Ebenen des Systems an: Infrastruktur, System und Applikation. Wichtig ist, dass alle Entwickler zu jedem Zeitpunkt Einblick in die Echtzeit-Metriken ihrer Services nehmen können. Um das zu garantieren, haben wir uns einige Zeit mit der Integration von Prometheus in unsere Systeme beschäftigt.
In unserem Talk werden wir sowohl über den Betrieb von Prometheus als auch über die Integrationen mit dem Rest der Jimdo-Plattform sprechen. Wir werden von Stolpersteinen und Tricks berichten, die wir gelernt haben, sowie einen Einblick in unserer Tool-Landschaft geben.
Why do we build applications based on microservices? To gain flexibility and resilience - that’s what everybody tells us. But building and application with a lot of microservices leads to a highly distributed system - and distribution hurts! How can we keep dependencies between microservices at the lowest possible level and how can we reduce communication between them? We will have a look at „Event-Sourcing“ as a possible (and very cool) technique for asynchronous communication and data exchange/replication between microservices - in theory and also in code.
This document provides an overview of Azure Stream Analytics. It discusses how Stream Analytics is an easy to use real-time event processing engine that can be used for scenarios like personalized stock trading analysis, fraud detection, and IoT applications. It describes key capabilities like scalability and ease of use. It also outlines the steps to create a Stream Analytics job, including adding inputs, outputs, and queries. Finally it provides examples of using Stream Analytics for Twitter sentiment analysis and real-time fraud detection.
Functors show up everywhere in our day-to-day programming. They're so common, we take them for granted - especially in typed functional programming. Beside being common, they're incredibly useful for code reuse. However, functors have several relatively unknown variants: profunctors, bifunctors, contravariant functors, and so on. And guess what - they're amazingly useful, especially combined with other abstractions in the functional programming toolkit! In this talk, we'll cover the many species of functors and see how they can help us with tasks such as serialization, stream processing, and more.
The document discusses common mistakes and pitfalls when developing with Ruby on Rails, including:
1) Not understanding the difference between 'and' and '&&' operators and operator precedence issues this can cause.
2) Not sanitizing user input passed to functions like strip_tags, which can lead to security issues.
3) Caching controller actions without checking for initialized instance variables, which can cause crashes.
4) Issues with validating database uniqueness across multiple processes that can lead to duplicate entry errors.
5) Problems composing conditions for queries when categories or subcategories are empty.
6) Breaking the callback chain filter by returning from callbacks.
7) Perform
This document discusses the importance of software testing and outlines some major software failures caused by defects. It provides examples of defects in software requirements and code. The document also summarizes reasons to pursue a career in quality assurance testing, noting that it is a stable, recession-proof career path with competitive salaries starting at $45,000-$130,000 annually. It outlines the training program, which includes theoretical concepts, practical testing exercises, homework assignments, and sessions on resume preparation and domain knowledge.
You can watch the replay for this Geek Sync webcast in the IDERA Resource Center: http://ow.ly/tt9w50A5g7u
Jeffrey will cover ways to avoid people thinking your code is bad, some common coding fallacies, and presents two case studies on rewriting bad SQL. The first case study contains three iterations of code written by the instructor, the second case study comes from an eCommerce site that had a great idea but horrible execution.
Explores common patterns in microservice architectures and how these are addressed in the gilmour library.
Discusses async signal-slot as well as synchronous request-response architectures.
Introduces unix inspired composition of microservices for more modular and flexible design.
The Challenge of Bringing FEZ to PlayStation PlatformsMiguel Angel Horna
This presentation explores the process of bringing FEZ to the Sony PlayStation platforms. Beginning with the conversion from the existing C# codebase to C++ native code, the talk will then cover memory management without a garbage collector, tuned optimization techniques for each platform, and will finish with special features added to PlayStation builds
Mondrian MDX requests can be slow in production environments for several reasons: large schemas with many dimensions and measures can cause performance issues; querying large datasets from the database can also impact performance. To address these issues, the document discusses profiling Mondrian requests, optimizing the JVM and database, using caching, and tuning the Mondrian schema. A Mondrian schema pool is also described that reuses schema objects and periodically flushes unused schemas to free memory.
The System of Automatic Searching for Vulnerabilities or how to use Taint Ana...Positive Hack Days
This document discusses using taint analysis to automatically find vulnerabilities. It describes Alex Bazhanyuk and Nikita Tarakanov, who work on security research projects. Their presentation covers the System of Automatic Searching for Vulnerabilities (SASV), which uses taint analysis, BitBlaze tools like TEMU and VINE, and the STP constraint solver to automatically find vulnerabilities. SASV traces target programs, converts the traces to intermediate language code, and uses path exploration and symbolic execution to generate new test inputs that maximize code coverage and have the potential to trigger vulnerabilities.
Reduce SRE Stress: Minimizing Service Downtime with Grafana, InfluxDB and Tel...InfluxData
NetApp is a global cloud-led, data-centric software company. They are an industry leader in hybrid cloud data services and data management solutions. Their platform enables their customers to store and share large quantities of digital data across physical and hybrid cloud environments. NetApp Engineering’s Site Reliability Engineering team is tasked with supporting their internal build environment, test, and automation infrastructure. After collecting their time-stamped data in InfluxDB, they are using Kapacitor to push alerts directly to Slack via webhooks. Their globally distributed SRE team are able to seamlessly collaborate and troubleshoot. Discover how NetApp uses a time series platform to detect trends in real time that can result in failures within their environments, and to provide key metrics used in SRE postmortems.
Join this webinar as Dustin Sorge will dive into:
NetApp's approach to monitoring their SRE team's metrics — including SLO's and SLI's
Their best practices and techniques for monitoring memory usage and CPU usage
How they use InfluxDB and Telegraf to detect trends and coordinate fixes faster.
This document discusses randomization using SystemVerilog. It begins by introducing constraint-driven test generation and random testing. It explains that SystemVerilog allows specifying constraints in a compact way to generate random values that meet the constraints. The document then discusses using objects to model complex data types for randomization. It provides examples of using SystemVerilog functions like $random, $urandom, and $urandom_range to generate random numbers. It also discusses constraining randomization using inline constraints and randomizing objects with the randomize method.
This document discusses various strategies for optimizing MySQL performance, including:
1. Using partitioning, indexes, and caching to improve query performance. Functional and multi-level partitioning techniques are described.
2. Denormalizing data, adding columns, and flagging rows to reduce the number of joins and optimize for read-heavy workloads.
3. Monitoring tools like the slow query log, server status, and InnoTOP are recommended for identifying bottlenecks and tracking improvements over time. Testing and an iterative process of addressing the biggest problems is advised.
Jay Clifford [InfluxData] | Tips & Tricks for Analyzing IIoT in Real-Time | I...InfluxData
Transforming raw machine data into real business outcomes such as OE and OEE is a journey. In this talk we will learn some tips and tricks for analyzing and transforming your machine data using InfluxDB and other third-party platforms.
Dino2 - the Amazing Evolution of the VA Smalltalk Virtual MachineESUG
Dino2 - the Amazing Evolution of the VA Smalltalk Virtual Machine
First Name: John
Last Name: O'Keefe
Type: Talk
Video1: https://www.youtube.com/watch?v=Ii8Dwq1b6YI
Video2: https://www.youtube.com/watch?v=30L7fWvtddU
Over the last 18 months we have evolved the VA Smalltalk VM from a Smalltalk model-based 32-bit VM to a C-based 32/64-bit VM. During this talk I will tell the story of our journey along this evolutionary path, describe some of the innovative techniques and approaches we took to reach our goal, and demonstrate the running 64-bit VM.
Bio:
I have been developing software for over 45 years. I joined the original IBM Smalltalk prototype team in 1990 and was a founding member of the IBM VisualAge Smalltalk development team. I was Team Lead and Chief Architect of IBM VisualAge Smalltalk from 1997 to 2007. In February 2007, I joined Instantiations to lead the VA Smalltalk development team. I am currently the CTO and Principal Smalltalk Architect focusing on future product architecture and development. I live in Durham, NC and work in Raleigh, NC.
Trends and development practices in Serverless architecturesDiUS
AWS ISV Event - Unlocking Business Agility with the AWS Serverless Application Model
Matt Fellows, Principal Consultant from DiUS will talk about the evolution to serverless architectures, and discuss key development and testing practices for these modern distributed systems
Vertical Recommendation Using Collaborative Filteringgorass
This document discusses using collaborative filtering to generate recommendations for large-scale vertical recommendation. It describes collaborative filtering techniques like matrix factorization that can be used to learn relationships between users and merchants from transaction data. It then discusses how the collaborative filtering model can be adapted to generate recommendations for the next best vertical by grouping related merchants and calculating vertical scores. The recommendation workflow is broken down into steps like matrix generation, model training, and vertical grouping that can be implemented using technologies like Hadoop, Pig, and Mahout.
This document discusses optimizing JavaScript performance in Node.js. It covers benchmarking Node applications, tips for writing efficient JavaScript that avoids hidden classes and dictionary mode in the V8 engine, profiling Node to find hot spots, and how the V8 optimizing compiler works. The presenter emphasizes the importance of speed and provides resources for further optimizing Node applications.
This document discusses records in Pascal programming. It defines records as a complex data type that allows combining simple variables. Examples of records include a person record with fields for name, age, etc. and an employee record with fields for address, salary, and a person subrecord. Procedures are provided to input, output, combine, and calculate the absolute value of complex number records. Homework involves writing a program to input, output, sort, and update employee records for a company with n employees.
This document discusses records in Pascal programming. It defines records as a complex data type that allows combining simple variables. Examples of records include a person record with fields for name, age, etc. and an employee record with fields for address, salary, and a person subrecord. Procedures are provided to input, output, combine, and calculate the absolute value of complex number records. Homework involves writing a program to input, output, sort, and update employee records for a company with n employees.
This document discusses using Puppet roles and profiles to organize Puppet code into logical layers (roles, profiles, resources) and tie together different modules. Roles apply directly to nodes and may only include profiles. Profiles contain resources and parameters and are applied via roles. Resources are reusable components declared with defines. Data is configured separately in Hiera.
¿Pensando en montar una arquitectura de microservicios?¿O quizás ya estás en mitad de una y sientes que estás perdiendo el control o te produce ansiedad cualquier subida a producción? En esta charla hablaré de cómo podemos volver a tomar el control de la situación combinando tres cosas: GraphQL, service directory and contract testing.
Find out more at https://madrid2018.codemotionworld.com/speakers/
No software engineer is exempt from writing batch jobs, its just part of every software app stack. Lack of a standards and a reusable batch architecture in the Java platform has resulted in the proliferation of many one-off in-house batch solutions.
In this talk I presented about Spring Batch Framework, how it could help an organization to standardize their batching needs. I also talked about a Real World Use Case in Dealer dot com. How we used Spring Batch along with Spring Integration to solve our job concurrency, data flow control, job resiliency & additional requirements - thus enabling us to build beautiful batch jobs.
GraphSummit Paris - The art of the possible with Graph TechnologyNeo4j
Sudhir Hasbe, Chief Product Officer, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
This document discusses the importance of software testing and outlines some major software failures caused by defects. It provides examples of defects in software requirements and code. The document also summarizes reasons to pursue a career in quality assurance testing, noting that it is a stable, recession-proof career path with competitive salaries starting at $45,000-$130,000 annually. It outlines the training program, which includes theoretical concepts, practical testing exercises, homework assignments, and sessions on resume preparation and domain knowledge.
You can watch the replay for this Geek Sync webcast in the IDERA Resource Center: http://ow.ly/tt9w50A5g7u
Jeffrey will cover ways to avoid people thinking your code is bad, some common coding fallacies, and presents two case studies on rewriting bad SQL. The first case study contains three iterations of code written by the instructor, the second case study comes from an eCommerce site that had a great idea but horrible execution.
Explores common patterns in microservice architectures and how these are addressed in the gilmour library.
Discusses async signal-slot as well as synchronous request-response architectures.
Introduces unix inspired composition of microservices for more modular and flexible design.
The Challenge of Bringing FEZ to PlayStation PlatformsMiguel Angel Horna
This presentation explores the process of bringing FEZ to the Sony PlayStation platforms. Beginning with the conversion from the existing C# codebase to C++ native code, the talk will then cover memory management without a garbage collector, tuned optimization techniques for each platform, and will finish with special features added to PlayStation builds
Mondrian MDX requests can be slow in production environments for several reasons: large schemas with many dimensions and measures can cause performance issues; querying large datasets from the database can also impact performance. To address these issues, the document discusses profiling Mondrian requests, optimizing the JVM and database, using caching, and tuning the Mondrian schema. A Mondrian schema pool is also described that reuses schema objects and periodically flushes unused schemas to free memory.
The System of Automatic Searching for Vulnerabilities or how to use Taint Ana...Positive Hack Days
This document discusses using taint analysis to automatically find vulnerabilities. It describes Alex Bazhanyuk and Nikita Tarakanov, who work on security research projects. Their presentation covers the System of Automatic Searching for Vulnerabilities (SASV), which uses taint analysis, BitBlaze tools like TEMU and VINE, and the STP constraint solver to automatically find vulnerabilities. SASV traces target programs, converts the traces to intermediate language code, and uses path exploration and symbolic execution to generate new test inputs that maximize code coverage and have the potential to trigger vulnerabilities.
Reduce SRE Stress: Minimizing Service Downtime with Grafana, InfluxDB and Tel...InfluxData
NetApp is a global cloud-led, data-centric software company. They are an industry leader in hybrid cloud data services and data management solutions. Their platform enables their customers to store and share large quantities of digital data across physical and hybrid cloud environments. NetApp Engineering’s Site Reliability Engineering team is tasked with supporting their internal build environment, test, and automation infrastructure. After collecting their time-stamped data in InfluxDB, they are using Kapacitor to push alerts directly to Slack via webhooks. Their globally distributed SRE team are able to seamlessly collaborate and troubleshoot. Discover how NetApp uses a time series platform to detect trends in real time that can result in failures within their environments, and to provide key metrics used in SRE postmortems.
Join this webinar as Dustin Sorge will dive into:
NetApp's approach to monitoring their SRE team's metrics — including SLO's and SLI's
Their best practices and techniques for monitoring memory usage and CPU usage
How they use InfluxDB and Telegraf to detect trends and coordinate fixes faster.
This document discusses randomization using SystemVerilog. It begins by introducing constraint-driven test generation and random testing. It explains that SystemVerilog allows specifying constraints in a compact way to generate random values that meet the constraints. The document then discusses using objects to model complex data types for randomization. It provides examples of using SystemVerilog functions like $random, $urandom, and $urandom_range to generate random numbers. It also discusses constraining randomization using inline constraints and randomizing objects with the randomize method.
This document discusses various strategies for optimizing MySQL performance, including:
1. Using partitioning, indexes, and caching to improve query performance. Functional and multi-level partitioning techniques are described.
2. Denormalizing data, adding columns, and flagging rows to reduce the number of joins and optimize for read-heavy workloads.
3. Monitoring tools like the slow query log, server status, and InnoTOP are recommended for identifying bottlenecks and tracking improvements over time. Testing and an iterative process of addressing the biggest problems is advised.
Jay Clifford [InfluxData] | Tips & Tricks for Analyzing IIoT in Real-Time | I...InfluxData
Transforming raw machine data into real business outcomes such as OE and OEE is a journey. In this talk we will learn some tips and tricks for analyzing and transforming your machine data using InfluxDB and other third-party platforms.
Dino2 - the Amazing Evolution of the VA Smalltalk Virtual MachineESUG
Dino2 - the Amazing Evolution of the VA Smalltalk Virtual Machine
First Name: John
Last Name: O'Keefe
Type: Talk
Video1: https://www.youtube.com/watch?v=Ii8Dwq1b6YI
Video2: https://www.youtube.com/watch?v=30L7fWvtddU
Over the last 18 months we have evolved the VA Smalltalk VM from a Smalltalk model-based 32-bit VM to a C-based 32/64-bit VM. During this talk I will tell the story of our journey along this evolutionary path, describe some of the innovative techniques and approaches we took to reach our goal, and demonstrate the running 64-bit VM.
Bio:
I have been developing software for over 45 years. I joined the original IBM Smalltalk prototype team in 1990 and was a founding member of the IBM VisualAge Smalltalk development team. I was Team Lead and Chief Architect of IBM VisualAge Smalltalk from 1997 to 2007. In February 2007, I joined Instantiations to lead the VA Smalltalk development team. I am currently the CTO and Principal Smalltalk Architect focusing on future product architecture and development. I live in Durham, NC and work in Raleigh, NC.
Trends and development practices in Serverless architecturesDiUS
AWS ISV Event - Unlocking Business Agility with the AWS Serverless Application Model
Matt Fellows, Principal Consultant from DiUS will talk about the evolution to serverless architectures, and discuss key development and testing practices for these modern distributed systems
Vertical Recommendation Using Collaborative Filteringgorass
This document discusses using collaborative filtering to generate recommendations for large-scale vertical recommendation. It describes collaborative filtering techniques like matrix factorization that can be used to learn relationships between users and merchants from transaction data. It then discusses how the collaborative filtering model can be adapted to generate recommendations for the next best vertical by grouping related merchants and calculating vertical scores. The recommendation workflow is broken down into steps like matrix generation, model training, and vertical grouping that can be implemented using technologies like Hadoop, Pig, and Mahout.
This document discusses optimizing JavaScript performance in Node.js. It covers benchmarking Node applications, tips for writing efficient JavaScript that avoids hidden classes and dictionary mode in the V8 engine, profiling Node to find hot spots, and how the V8 optimizing compiler works. The presenter emphasizes the importance of speed and provides resources for further optimizing Node applications.
This document discusses records in Pascal programming. It defines records as a complex data type that allows combining simple variables. Examples of records include a person record with fields for name, age, etc. and an employee record with fields for address, salary, and a person subrecord. Procedures are provided to input, output, combine, and calculate the absolute value of complex number records. Homework involves writing a program to input, output, sort, and update employee records for a company with n employees.
This document discusses records in Pascal programming. It defines records as a complex data type that allows combining simple variables. Examples of records include a person record with fields for name, age, etc. and an employee record with fields for address, salary, and a person subrecord. Procedures are provided to input, output, combine, and calculate the absolute value of complex number records. Homework involves writing a program to input, output, sort, and update employee records for a company with n employees.
This document discusses using Puppet roles and profiles to organize Puppet code into logical layers (roles, profiles, resources) and tie together different modules. Roles apply directly to nodes and may only include profiles. Profiles contain resources and parameters and are applied via roles. Resources are reusable components declared with defines. Data is configured separately in Hiera.
¿Pensando en montar una arquitectura de microservicios?¿O quizás ya estás en mitad de una y sientes que estás perdiendo el control o te produce ansiedad cualquier subida a producción? En esta charla hablaré de cómo podemos volver a tomar el control de la situación combinando tres cosas: GraphQL, service directory and contract testing.
Find out more at https://madrid2018.codemotionworld.com/speakers/
No software engineer is exempt from writing batch jobs, its just part of every software app stack. Lack of a standards and a reusable batch architecture in the Java platform has resulted in the proliferation of many one-off in-house batch solutions.
In this talk I presented about Spring Batch Framework, how it could help an organization to standardize their batching needs. I also talked about a Real World Use Case in Dealer dot com. How we used Spring Batch along with Spring Integration to solve our job concurrency, data flow control, job resiliency & additional requirements - thus enabling us to build beautiful batch jobs.
Similar to Traversals and Scans and CQRS, oh my! (20)
GraphSummit Paris - The art of the possible with Graph TechnologyNeo4j
Sudhir Hasbe, Chief Product Officer, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Flutter is a popular open source, cross-platform framework developed by Google. In this webinar we'll explore Flutter and its architecture, delve into the Flutter Embedder and Flutter’s Dart language, discover how to leverage Flutter for embedded device development, learn about Automotive Grade Linux (AGL) and its consortium and understand the rationale behind AGL's choice of Flutter for next-gen IVI systems. Don’t miss this opportunity to discover whether Flutter is right for your project.
Graspan: A Big Data System for Big Code AnalysisAftab Hussain
We built a disk-based parallel graph system, Graspan, that uses a novel edge-pair centric computation model to compute dynamic transitive closures on very large program graphs.
We implement context-sensitive pointer/alias and dataflow analyses on Graspan. An evaluation of these analyses on large codebases such as Linux shows that their Graspan implementations scale to millions of lines of code and are much simpler than their original implementations.
These analyses were used to augment the existing checkers; these augmented checkers found 132 new NULL pointer bugs and 1308 unnecessary NULL tests in Linux 4.4.0-rc5, PostgreSQL 8.3.9, and Apache httpd 2.2.18.
- Accepted in ASPLOS ‘17, Xi’an, China.
- Featured in the tutorial, Systemized Program Analyses: A Big Data Perspective on Static Analysis Scalability, ASPLOS ‘17.
- Invited for presentation at SoCal PLS ‘16.
- Invited for poster presentation at PLDI SRC ‘16.
Mobile App Development Company In Noida | Drona InfotechDrona Infotech
Looking for a reliable mobile app development company in Noida? Look no further than Drona Infotech. We specialize in creating customized apps for your business needs.
Visit Us For : https://www.dronainfotech.com/mobile-application-development/
Odoo ERP software
Odoo ERP software, a leading open-source software for Enterprise Resource Planning (ERP) and business management, has recently launched its latest version, Odoo 17 Community Edition. This update introduces a range of new features and enhancements designed to streamline business operations and support growth.
The Odoo Community serves as a cost-free edition within the Odoo suite of ERP systems. Tailored to accommodate the standard needs of business operations, it provides a robust platform suitable for organisations of different sizes and business sectors. Within the Odoo Community Edition, users can access a variety of essential features and services essential for managing day-to-day tasks efficiently.
This blog presents a detailed overview of the features available within the Odoo 17 Community edition, and the differences between Odoo 17 community and enterprise editions, aiming to equip you with the necessary information to make an informed decision about its suitability for your business.
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Crescat
Crescat is industry-trusted event management software, built by event professionals for event professionals. Founded in 2017, we have three key products tailored for the live event industry.
Crescat Event for concert promoters and event agencies. Crescat Venue for music venues, conference centers, wedding venues, concert halls and more. And Crescat Festival for festivals, conferences and complex events.
With a wide range of popular features such as event scheduling, shift management, volunteer and crew coordination, artist booking and much more, Crescat is designed for customisation and ease-of-use.
Over 125,000 events have been planned in Crescat and with hundreds of customers of all shapes and sizes, from boutique event agencies through to international concert promoters, Crescat is rigged for success. What's more, we highly value feedback from our users and we are constantly improving our software with updates, new features and improvements.
If you plan events, run a venue or produce festivals and you're looking for ways to make your life easier, then we have a solution for you. Try our software for free or schedule a no-obligation demo with one of our product specialists today at crescat.io
Artificia Intellicence and XPath Extension FunctionsOctavian Nadolu
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
Atelier - Innover avec l’IA Générative et les graphes de connaissancesNeo4j
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Allez au-delà du battage médiatique autour de l’IA et découvrez des techniques pratiques pour utiliser l’IA de manière responsable à travers les données de votre organisation. Explorez comment utiliser les graphes de connaissances pour augmenter la précision, la transparence et la capacité d’explication dans les systèmes d’IA générative. Vous partirez avec une expérience pratique combinant les relations entre les données et les LLM pour apporter du contexte spécifique à votre domaine et améliorer votre raisonnement.
Amenez votre ordinateur portable et nous vous guiderons sur la mise en place de votre propre pile d’IA générative, en vous fournissant des exemples pratiques et codés pour démarrer en quelques minutes.
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsPeter Muessig
The UI5 tooling is the development and build tooling of UI5. It is built in a modular and extensible way so that it can be easily extended by your needs. This session will showcase various tooling extensions which can boost your development experience by far so that you can really work offline, transpile your code in your project to use even newer versions of EcmaScript (than 2022 which is supported right now by the UI5 tooling), consume any npm package of your choice in your project, using different kind of proxies, and even stitching UI5 projects during development together to mimic your target environment.
DDS Security Version 1.2 was adopted in 2024. This revision strengthens support for long runnings systems adding new cryptographic algorithms, certificate revocation, and hardness against DoS attacks.
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Łukasz Chruściel
No one wants their application to drag like a car stuck in the slow lane! Yet it’s all too common to encounter bumpy, pothole-filled solutions that slow the speed of any application. Symfony apps are not an exception.
In this talk, I will take you for a spin around the performance racetrack. We’ll explore common pitfalls - those hidden potholes on your application that can cause unexpected slowdowns. Learn how to spot these performance bumps early, and more importantly, how to navigate around them to keep your application running at top speed.
We will focus in particular on tuning your engine at the application level, making the right adjustments to ensure that your system responds like a well-oiled, high-performance race car.
Do you want Software for your Business? Visit Deuglo
Deuglo has top Software Developers in India. They are experts in software development and help design and create custom Software solutions.
Deuglo follows seven steps methods for delivering their services to their customers. They called it the Software development life cycle process (SDLC).
Requirement — Collecting the Requirements is the first Phase in the SSLC process.
Feasibility Study — after completing the requirement process they move to the design phase.
Design — in this phase, they start designing the software.
Coding — when designing is completed, the developers start coding for the software.
Testing — in this phase when the coding of the software is done the testing team will start testing.
Installation — after completion of testing, the application opens to the live server and launches!
Maintenance — after completing the software development, customers start using the software.
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI AppGoogle
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-fusion-buddy-review
AI Fusion Buddy Review: Key Features
✅Create Stunning AI App Suite Fully Powered By Google's Latest AI technology, Gemini
✅Use Gemini to Build high-converting Converting Sales Video Scripts, ad copies, Trending Articles, blogs, etc.100% unique!
✅Create Ultra-HD graphics with a single keyword or phrase that commands 10x eyeballs!
✅Fully automated AI articles bulk generation!
✅Auto-post or schedule stunning AI content across all your accounts at once—WordPress, Facebook, LinkedIn, Blogger, and more.
✅With one keyword or URL, generate complete websites, landing pages, and more…
✅Automatically create & sell AI content, graphics, websites, landing pages, & all that gets you paid non-stop 24*7.
✅Pre-built High-Converting 100+ website Templates and 2000+ graphic templates logos, banners, and thumbnail images in Trending Niches.
✅Say goodbye to wasting time logging into multiple Chat GPT & AI Apps once & for all!
✅Save over $5000 per year and kick out dependency on third parties completely!
✅Brand New App: Not available anywhere else!
✅ Beginner-friendly!
✅ZERO upfront cost or any extra expenses
✅Risk-Free: 30-Day Money-Back Guarantee!
✅Commercial License included!
See My Other Reviews Article:
(1) AI Genie Review: https://sumonreview.com/ai-genie-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
#AIFusionBuddyReview,
#AIFusionBuddyFeatures,
#AIFusionBuddyPricing,
#AIFusionBuddyProsandCons,
#AIFusionBuddyTutorial,
#AIFusionBuddyUserExperience
#AIFusionBuddyforBeginners,
#AIFusionBuddyBenefits,
#AIFusionBuddyComparison,
#AIFusionBuddyInstallation,
#AIFusionBuddyRefundPolicy,
#AIFusionBuddyDemo,
#AIFusionBuddyMaintenanceFees,
#AIFusionBuddyNewbieFriendly,
#WhatIsAIFusionBuddy?,
#HowDoesAIFusionBuddyWorks
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j
Dr. Jesús Barrasa, Head of Solutions Architecture for EMEA, Neo4j
Découvrez les dernières innovations de Neo4j, et notamment les dernières intégrations cloud et les améliorations produits qui font de Neo4j un choix essentiel pour les développeurs qui créent des applications avec des données interconnectées et de l’IA générative.
2. About me
• Freelance software developer
• Doing functional programming, microservices, DevOps
• Co-organize the Underscore Scala meetup
• Hit me up if you want to work together!
Itamar Ravid - @itrvd 2
3. Agenda
• Why Event Sourcing and CQRS are worth your time
• How we can use functions to implement them
• How abstractions from FP help us
• (shortly) Hooking this up to a stream
Itamar Ravid - @itrvd 3
28. CQRS: Command/Query Responsibility
Separation
We store the events in an append only stream.
We handle reads using a specialized store, derived from the
events.
Now you know CQRS!
Itamar Ravid - @itrvd 28
29. What do we gain?
• Faster response time
• Availability
• Determinism, take your debugging home
•
!
Event-sourced services are like pure functions!
Itamar Ravid - @itrvd 29
30. And speaking of pure functions
Let's see some code!
The examples ahead assume:
import cats._, cats.data._, cats.implicits._
Itamar Ravid - @itrvd 30
31. Zoom in
We'll now focus on the reservation service.
Let's event-storm this!
What commands/events are we going to use?
Itamar Ravid - @itrvd 31
32. Commands
We might be asked to:
• Create a reservation
• Add a person to a reservation
• Cancel a reservation
Itamar Ravid - @itrvd 32
33. Events
And these will cause the:
• Reservation to be created
• Reservation to be updated
• Reservation to be canceled
Itamar Ravid - @itrvd 33
34. Command Definitions
I promised some code, right?
sealed trait ReservationCommand
case class Create(roomId: String, guests: Int)
extends ReservationCommand
case class ModifyGuests(reservationId: String, amount: Int)
extends ReservationCommand
case class Cancel(reservationId: String)
extends ReservationCommand
Itamar Ravid - @itrvd 34
35. Event Definitions
And we have the corresponding events:
sealed trait ReservationEvent
case class Created(id: String, roomId: String, guests: Int)
extends ReservationEvent
case class Updated(id: String, roomId: String, guests: Int)
extends ReservationEvent
case class Canceled(id: String, roomId: String, guests: Int)
extends ReservationEvent
Itamar Ravid - @itrvd 35
42. Breakdown
So technically, we need:
1. Validation of commands
2. State representation, state mutation
3. Event generation
4. Persistence
Itamar Ravid - @itrvd 42
45. Validation
So we want a function, right?
def processCommand(command: ReservationCommand): ???
Itamar Ravid - @itrvd 45
46. Validation
So we want a function, right?
def processCommand(command: ReservationCommand): Either[Error, Unit]
Itamar Ravid - @itrvd 46
47. Validation
So we want a function, right?
type Error = String
type Result[A] = Either[Error, A]
def processCommand(command: ReservationCommand): Result[Unit]
Itamar Ravid - @itrvd 47
48. Validation
Let's start with simple validation: the number of guests should be
positive.
def validateGuests(command: Create): Result[Unit] =
if (command.guests <= 0) Left("Non-positive guests")
else Right(())
Itamar Ravid - @itrvd 48
50. Stateful validation
Next, we want a stateful validation - no duplicate reservations.
To do that, we need some sort of state:
case class Reservation(id: String)
trait Reservations {
def byRoomId(id: String): Option[Reservation]
}
Itamar Ravid - @itrvd 50
51. Validation
And here's our validation:
def validateDup(reservations: Reservations,
command: Create): Result[Unit] =
reservations.byRoomId(command.roomId) match {
case Some(_) => Left("Duplicate reservation")
case None => Right(())
}
Itamar Ravid - @itrvd 51
52. Validation - continued
Our command processing now looks like this:
def processCommand(reservations: Reservations,
command: ReservationCommand): Result[Unit] =
command match {
case create: Create =>
(validateGuests(create), validateDup(create, reservations))
.tupled.void
// case modifyGuests, case cancelReservation, etc.
}
Itamar Ravid - @itrvd 52
53. Validation - continued
What's that (a, b).tupled function?
This will combine the result of both validations into a single
value:
val guestValidation: Result[Unit] = validateGuests(create)
val dupValidation: Result[Unit] = validateDup(create, reservations)
val tupled: Result[(Unit, Unit)] = (guestValidation, dupValidation).tupled
Itamar Ravid - @itrvd 53
54. Validation - continued
What's that (a, b).tupled function?
This will combine the result of both validations into a single
value:
val guestValidation: Result[Unit] = validateGuests(create)
val dupValidation: Result[Unit] = validateDup(create, reservations)
val tupled: Result[(Unit, Unit)] = (guestValidation, dupValidation).tupled
If one of them fails - both fails too.
Itamar Ravid - @itrvd 54
56. Validation - continued
What else can we say about Result?
It's an Applicative - so we can use tupled.
Itamar Ravid - @itrvd 56
57. Validation - continued
What else can we say about Result?
It's also a Monad, so we can use for comprehensions:
for {
a <- validate1(cmd)
b <- validate2(cmd, a)
} yield f(a, b)
Itamar Ravid - @itrvd 57
58. Validation - continued
A few side notes:
• Our validation unfortunately short circuits. We don't see all
errors.
Check out cats.data.Validated for a solution.
• Applicative s are essential for joining independent
computations. Definitely read up on them!
Check out the mapN, *>, <* combinators.
Itamar Ravid - @itrvd 58
59. Validation - summary
This is all I'm going to say about validation!
• Write an Either returning function
• Give it what it needs to compute
• Combine with the Applicative combinators
• Profit Validated!
Itamar Ravid - @itrvd 59
61. Generating events
The path of least resistance is another function for generating:
def events(reservations: Reservations,
command: ReservationCommand): List[ReservationEvent]
Itamar Ravid - @itrvd 61
62. Generating events
We return the events inside Result:
def processCommand(reservations: Reservations,
command: ReservationCommand)
: Result[List[ReservationEvent]] =
command match {
case create: Create =>
(validateGuests(create), validateDup(create, reservations))
.tupled
.map(_ => events(reservations, commands))
}
Since we're using List, we can output 0, 1 or many events.
Itamar Ravid - @itrvd 62
63. Generating events
This is great if we can separate validation and event generation.
Itamar Ravid - @itrvd 63
64. Generating events
This is great if we can separate validation and event generation.
This isn't always possible!
Itamar Ravid - @itrvd 64
65. Generating events
This is great if we can separate validation and event generation.
This isn't always possible!
We'll upgrade the two functions with event generation.
Itamar Ravid - @itrvd 65
67. Generating events
There's a cool data type called WriterT we can use here:
case class WriterT[F[_], Log, Value](run: F[(Log, Value)])
Itamar Ravid - @itrvd 67
68. Generating events
We'll modify the validation return type to be:
WriterT[Result, List[ReservationEvent], A]
Which wraps values of type
Result[(List[ReservationEvent], A)]
Itamar Ravid - @itrvd 68
69. Generating events
Now, when we use the .tupled syntax, we get back:
val writer: WriterT[Result, List[ReservationEvent], Unit] =
(validateDup(reservations, command),
validateGuests(command)).tupled.void
And it actually concatenated the lists for us!
Itamar Ravid - @itrvd 69
70. Generating events
We can "peel" this using .run:
val writer: WriterT[Result, List[ReservationEvent], Unit] =
(validateDup(reservations, command),
validateGuests(command)).tupled.void
val result: Result[(List[ReservationEvent], Unit)] =
writer.run
Itamar Ravid - @itrvd 70
71. Generating events
Or, if we only want the log:
val result: Result[List[ReservationEvent]] =
writer.written
Itamar Ravid - @itrvd 71
72. Generating events
So let's give this new type a name:
type EventsAnd[A] = WriterT[Result, List[ReservationEvent], A]
Itamar Ravid - @itrvd 72
73. Actually writing those functions
Here's an example of how one of those functions would look like:
def validateGuests(command: Create): EventsAnd[Unit] =
if (command.guests <= 0)
WriterT.liftF("Non-positive guests".asLeft[Unit])
else
WriterT.putT(().asRight[Error])(List(ReservationCreated(command.id)))
Itamar Ravid - @itrvd 73
74. Generating events - summary
We're done! Our functions now validate and generate events:
def validateGuests(command: Create): EventsAnd[Unit]
def validateDup(reservations: Reservations,
command: Create): EventsAnd[Unit]
val result = (validateGuests(c), validateDup(s, c))
.tupled.void
Itamar Ravid - @itrvd 74
75. Generating events - summary
We're done! Our functions now validate and generate events:
def validateGuests(command: Create): EventsAnd[Unit]
def validateDup(reservations: Reservations,
command: Create): EventsAnd[Unit]
val result = (validateGuests(c), validateDup(s, c))
.tupled.void
Of course, if validation fails - everything still fails.
Itamar Ravid - @itrvd 75
76. A few side notes on WriterT
• WriterT is also a Monad and an Applicative.
• Lots of combinators - check out the docs!
• WriterT can be quite allocation-heavy.
We'll touch a possible solution towards the end.
Itamar Ravid - @itrvd 76
78. State mutation
Our function doesn't do anything currently:
def processCommand(reservations: Reservations,
command: ReservationCommand): EventsAnd[Unit]
Itamar Ravid - @itrvd 78
79. State mutation
It's pretty awesome that we can reason about it just by looking at
the type:
(Reservations, Command) => WriterT[Result, List[Event], Unit]
Nothing else happens. WYSIWYG.
Itamar Ravid - @itrvd 79
80. State mutation
The path of least resistance is to return a new state:
def processCommand(reservations: Reservations,
command: ReservationCommand): (Reservations, EventsAnd[Unit])
Itamar Ravid - @itrvd 80
81. State mutation
The path of least resistance is to return a new state:
def processCommand(reservations: Reservations,
command: ReservationCommand): (Reservations, EventsAnd[Unit])
We're not saying anything about whether it was modified.
Itamar Ravid - @itrvd 81
82. State mutation
However, we want to guarantee that nothing happens if
validation fails.
def processCommand(reservations: Reservations,
command: ReservationCommand): EventsAnd[Reservations]
If validation is fine, we get a new state.
If it fails, we keep the previous state.
Itamar Ravid - @itrvd 82
83. State mutation
But - how do we reconcile the two resulting states?
def validateGuests(s: Reservations, c: Create): EventsAnd[Reservations]
def validateDup(s: Reservations, c: Create): EventsAnd[Reservations]
val result: EventsAnd[(Reservations, Reservations)] =
(validateGuests(s, c),
validateDup(s, c)).tupled
Itamar Ravid - @itrvd 83
87. State mutation
Functional programming saves the day again- this is StateT:
case class StateT[F[_], State, Value](run: State => F[(State, Value)])
Itamar Ravid - @itrvd 87
88. State mutation
We add yet another type alias on top:
type CommandProcessor[A] = StateT[EventsAnd, Reservations, A]
def validateGuests(c: Create): CommandProcessor[Unit]
def validateDup(c: Create): CommandProcessor[Unit]
Itamar Ravid - @itrvd 88
89. State mutation
We can now compose our command processors:
def validateGuests(c: Create): CommandProcessor[Unit]
def validateDup(c: Create): CommandProcessor[Unit]
val resultProcessor: CommandProcessor[Unit] =
(validateGuests(c),
validateDup(c)).tupled.void
But where did our initial state disappear to?
Itamar Ravid - @itrvd 89
90. State mutation
We haven't run anything yet! Let's run the processor:
val result: EventsAnd[(Reservations, Unit)] = resultProcessor.run(state)
This does a lot of stuff!
Itamar Ravid - @itrvd 90
91. State mutation
Here's how we actually write one of these functions:
def validateGuests(command: Create): CommandProcessor[Unit] =
if (command.guests <= 0)
StateT.liftF(WriterT.liftF("Non-positive guests".asLeft[Unit]))
else
for {
state <- StateT.get[EventsAnd, Reservations]
events = generateEventsSomehow(state)
_ <- StateT.liftF(WriterT.tell[Result, List[ReservationEvent]](events))
newState = generateNewStateSomehow(reservations, event)
_ <- StateT.set[EventsAnd, Reservations](newState)
} yield ()
Itamar Ravid - @itrvd 91
92. State mutation
Here's how we actually write one of
these functions:
def validateGuests(command: Create): CommandProcessor[Unit] =
if (command.guests <= 0)
StateT.liftF(WriterT.liftF("Non-positive guests".asLeft[Unit]))
else
for {
state <- StateT.get[EventsAnd, Reservations]
events = generateEventsSomehow(state)
_ <- StateT.liftF(WriterT.tell[Result, List[ReservationEvent]](events))
newState = generateNewStateSomehow(reservations, event)
_ <- StateT.set[EventsAnd, Reservations](newState)
} yield ()
Itamar Ravid - @itrvd 92
93. Summary
• I bet you're loving the boilerplate by now ;-)
It gets worse as you add more layers!
• Don't worry though! This is solvable.
• As before, StateT is also a Monad and an Applicative.
Itamar Ravid - @itrvd 93
94. Recap
We now have a pretty powerful type as a building block:
type Result[A] = Either[Error, A]
type EventsAnd[A] = WriterT[Result, List[ReservationEvent], A]
type CommandProcessor[A] = StateT[EventsAnd, Reservations, A]
Itamar Ravid - @itrvd 94
95. Recap
If we expand everything, we get:
type CommandProcessor[A] =
Reservations => Either[Error, (List[Event], (Reservations, A))]
Itamar Ravid - @itrvd 95
100. Is that it?
type Result[A] = Either[Error, A]
type EventsAnd[A] = WriterT[Result, List[ReservationEvent], A]
type CommandProcessor[A] = StateT[EventsAnd, Reservations, A]
The command processor can return values, signal failures, log
events, and accumulate state.
Itamar Ravid - @itrvd 100
101. Is that it?
type Result[A] = Either[Error, A]
type EventsAnd[A] = WriterT[Result, List[ReservationEvent], A]
type CommandProcessor[A] = StateT[EventsAnd, Reservations, A]
The command processor can return values, signal failures, log
events, and accumulate state.
No magic! These are just functions!
Itamar Ravid - @itrvd 101
102. Fitting this into an application
We still have some work to do:
• we need to hook this up to a source of commands,
• and we actually need to persist the state and events.
Itamar Ravid - @itrvd 102
103. Fitting this into an application
Where can commands come from?
Treating these as infinite streams makes our life easier.
Itamar Ravid - @itrvd 103
104. Streams
We'll use fs2 for our examples.
Akka Streams can definitely work too! Check out the slides
when I publish them.
Itamar Ravid - @itrvd 104
105. fs2
fs2 is based on one type:
Stream[Effect[_], Element]
A possibly infinite stream of Element, with effect Effect.
Effect can be cats IO, scalaz IO, Monix Task, etc.
Itamar Ravid - @itrvd 105
106. fs2
fs2 is based on one type:
Stream[Effect[_], Element]
A possibly infinite stream of Element, with effect Effect.
Itamar Ravid - @itrvd 106
107. fs2
To run our command processor, we can use mapAccumulate:
class Stream[F[_], Element] {
def mapAccumulate[State, Out](init: State)
(f: (State, Element) => (State, Out)): Stream[F, (State, Out)]
}
Itamar Ravid - @itrvd 107
108. fs2
To run our command processor, we can use mapAccumulate:
class Stream[F[_], Element] {
def mapAccumulate[State, Out](init: State)
(f: (State, Element) => (State, Out)): Stream[F, (State, Out)]
}
We'll set:
• Element = Command, State = Reservations
• Out = (Reservations, List[ReservationEvent])
Itamar Ravid - @itrvd 108
112. Persisting events and state
def persist(reservations: Reservations): IO[Unit]
def produce(events: List[ReservationEvent]): IO[Unit]
stream.evalMap {
case (state, events) =>
(persist(state), produce(events))
.tupled
.void
}
Itamar Ravid - @itrvd 112
113. Where to go from here
• The solution to monad transformer overhead: finally tagless
• Keep the effect abstract, interpret into Task+IORef.
• Check out the links on next slide.
• You can also hire me and I'll help you make it work ;-)
Itamar Ravid - @itrvd 113
114. Where to go from here
Resources:
• Ben Stopford, Designing Event-Driven Systems:
https://www.confluent.io/designing-event-driven-systems
• Martin Kleppman's blog:
https://martin.kleppmann.com
• The FS2 guide:
https://functional-streams-for-scala.github.io/fs2/
• Finally tagless and Free:
https://softwaremill.com/free-tagless-compared-how-not-to-commit-to-monad-too-early/
Itamar Ravid - @itrvd 114
118. Digression: Folds
If you squint, our function looks like a foldLeft:
def foldLeft[S, A](init: S)
(f: (S, A ) => S ): S
def processCommand: (Reservations, ReservationCommand) => Reservations
Itamar Ravid - @itrvd 118
119. Digression: Folds
There's also an interesting variant of foldLeft when the result
has an effect:
def foldLeftM[S, A](init: S)(f: (S, A) => IO[S]): IO[S]
Itamar Ravid - @itrvd 119
120. Digression: Folds
Since EventsAnd is a Monad, we can use foldLeftM:
val commands: List[ReservationCommand]
val resultState: EventsAnd[Reservations] =
commands.foldLeftM(Reservations.empty)(processCommand)
Any failed validation will halt the processing,
and all the events would be accumulated.
Itamar Ravid - @itrvd 120
122. Digression: Folds
Another interesting type of fold is a scan:
def scanLeft[S, A](init: S)(f: (S, A) => S): List[S]
Looks exactly like a fold, but you get a list of intermediate states.
Itamar Ravid - @itrvd 122
123. Digression: Folds
Scanning commands would get us a list of intermediate
Reservations:
val commands: List[ReservationCommand]
val states = commands.scanLeft(Reservations.empty)(processCommand)
// List(reservations1, reservations2, ...)
Itamar Ravid - @itrvd 123
124. Traversals
What's the shortest way to make processCommand work
on a List[ReservationCommand]?
def processCommand(command: ReservationCommand): CommandProcessor[Unit]
Itamar Ravid - @itrvd 124
125. Traversals
Let's try mapping.
val cmds: List[ReservationCommand]
val mapped: List[CommandProcessor[Unit]] = cmds.map(processCommand)
We get back a list of functions; not very helpful.
Itamar Ravid - @itrvd 125
127. Traversals
Lucky for us, we have the sequence function:
val cmds: List[ReservationCommand]
val mapped: List[CommandProcessor[Unit]] = cmds.map(processCommand)
val sequenced: CommandProcessor[List[Unit]] = mapped.sequence
So now we get one giant processing function that runs
everything!
Itamar Ravid - @itrvd 127
128. Traversals
As the title gave away, this can be expressed as:
val traversed: CommandProcessor[List[Unit]] =
cmds.traverse(processCommand)
And we can actually run it:
traversed.run(initState) match {
case Left(_) =>
// log the error
case Right(resultState, (events, _)) =>
// persist the events and state
}
Itamar Ravid - @itrvd 128
129. Traversals
This is our Traversable typeclass:
trait Traversable[F[_]] extends Functor[F]
with Foldable[F] {
def traverse[G[_]: Applicative, A, B](fa: F[A])
(f: A => G[B]): G[F[B]]
}
Itamar Ravid - @itrvd 129
130. Traversals
If you line traverse up with map, the effectful map analogy is
clearer:
def map [A, B](fa: F[A])(f: A => B ): F[B]
def traverse[G[_], A, B](fa: F[A])(f: A => G[B]): G[F[B]]
Itamar Ravid - @itrvd 130
131. Akka Streams
With Akka Streams, we can use the scan operator to process
commands:
class Source[Element] {
def scan[State](zero: State)(f: (State, Element) => State): Source[State]
}
If we set:
• Element = ReservationCommand
• State = (Reservations, List[ReservationEvent])
Itamar Ravid - @itrvd 131