Malli is a new Clojure/Script library for elegant data-driven data modelling. It is born from the real world needs of dynamic, distributed and multi-tenant systems: Schemas to describe the data should be first-class data. They should drive the runtime value transformations, web forms, rules and processes. Configuration should be explicit – no magic, macros or mutability. We should be able to infer, create and modify the models at runtime, persist and load them back from database and share over the wire, for both Clojure and ClojureScript. Like JSON Schema, but for EDN & Clojure/Script.
In this talk, I’ll walk through the core concepts, features and the lessons learned on creating Malli. I’ll also demonstrate some cool new things we can do with literal data schemas and compare it to prior art, including clojure.spec, plumatic schema and JSON Schema.
The talk will compare Cascalog, fully-featured data processing and querying library on top of Hadoop, and Sparkling – A Clojure API for Apache Spark. How both of these compare in terms of performance and code complexity for Big Data processing and why you shouldn’t be writing MapReduce jobs in plain Hadoop API.
Domain Modeling Made Functional (KanDDDinsky 2019)Scott Wlaschin
(video at https://fsharpforfunandprofit.com/ddd/)
Statically typed functional programming languages encourage a very different way of thinking about types. The type system is your friend, not an annoyance, and can be used in many ways that might not be familiar to OO programmers. Types can be used to represent the domain in a fine-grained, self documenting way. And in many cases, types can even be used to encode business rules so that you literally cannot create incorrect code. You can then use the static type checking almost as an instant unit test — making sure that your code is correct at compile time. In this talk, we'll look at some of the ways you can use types as part of a domain driven design process, with some simple real world examples in F#. No jargon, no maths, and no prior F# experience necessary.
The Functional Programmer's Toolkit (NDC London 2019)Scott Wlaschin
(slides and video at https://fsharpforfunandprofit.com/fptoolkit)
The functional programming community has a number of patterns with strange names such as monads, monoids, functors, and catamorphisms.
In this beginner-friendly talk, we'll demystify these techniques and see how they all fit together into a small but versatile "tool kit".
We'll then see how the tools in this tool kit can be applied to a wide variety of programming problems, such as handling missing data, working with lists, and implementing the functional equivalent of dependency injection.
This is presentation, that covers all the important topics related to strings in python. It covers storing, slicing, format, concatenation, modification, escape characters and string methods.
The file attatched also includes examples related to the slides shown.
Tommi Reiman (https://twitter.com/ikitommi) will be presenting Malli
(https://github.com/metosin/malli) is a fresh new data-driven data
validation and specification library for Clojure/Script. In this talk,
Tommi will give a quick introduction to Malli, compare it to prior art
including Plumatic Schema and clojure.spec and demonstrate how to
elegantly solve real-world problems with it. Also, peek beyond the
runtime validation.
This document provides an overview of JavaScript arrays, including:
- Declaring and initializing different types of arrays such as associative arrays and indexed arrays
- Common array methods like push(), pop(), splice(), and slice()
- Array attributes including length, indexOf, and typeOf
- Techniques for adding, removing, and modifying array elements
The document discusses Python lists, which are ordered collections of items of any data type. It describes how to create, access, and modify lists using various functions and operations like append, insert, remove, sort, and slice. These include adding/removing elements, sorting/reversing lists, using lists as stacks with push/pop, and extracting sublist sections. The document provides examples of creating lists with different syntax, accessing elements by index, and using common list methods for operations.
Malli is a new Clojure/Script library for elegant data-driven data modelling. It is born from the real world needs of dynamic, distributed and multi-tenant systems: Schemas to describe the data should be first-class data. They should drive the runtime value transformations, web forms, rules and processes. Configuration should be explicit – no magic, macros or mutability. We should be able to infer, create and modify the models at runtime, persist and load them back from database and share over the wire, for both Clojure and ClojureScript. Like JSON Schema, but for EDN & Clojure/Script.
In this talk, I’ll walk through the core concepts, features and the lessons learned on creating Malli. I’ll also demonstrate some cool new things we can do with literal data schemas and compare it to prior art, including clojure.spec, plumatic schema and JSON Schema.
The talk will compare Cascalog, fully-featured data processing and querying library on top of Hadoop, and Sparkling – A Clojure API for Apache Spark. How both of these compare in terms of performance and code complexity for Big Data processing and why you shouldn’t be writing MapReduce jobs in plain Hadoop API.
Domain Modeling Made Functional (KanDDDinsky 2019)Scott Wlaschin
(video at https://fsharpforfunandprofit.com/ddd/)
Statically typed functional programming languages encourage a very different way of thinking about types. The type system is your friend, not an annoyance, and can be used in many ways that might not be familiar to OO programmers. Types can be used to represent the domain in a fine-grained, self documenting way. And in many cases, types can even be used to encode business rules so that you literally cannot create incorrect code. You can then use the static type checking almost as an instant unit test — making sure that your code is correct at compile time. In this talk, we'll look at some of the ways you can use types as part of a domain driven design process, with some simple real world examples in F#. No jargon, no maths, and no prior F# experience necessary.
The Functional Programmer's Toolkit (NDC London 2019)Scott Wlaschin
(slides and video at https://fsharpforfunandprofit.com/fptoolkit)
The functional programming community has a number of patterns with strange names such as monads, monoids, functors, and catamorphisms.
In this beginner-friendly talk, we'll demystify these techniques and see how they all fit together into a small but versatile "tool kit".
We'll then see how the tools in this tool kit can be applied to a wide variety of programming problems, such as handling missing data, working with lists, and implementing the functional equivalent of dependency injection.
This is presentation, that covers all the important topics related to strings in python. It covers storing, slicing, format, concatenation, modification, escape characters and string methods.
The file attatched also includes examples related to the slides shown.
Tommi Reiman (https://twitter.com/ikitommi) will be presenting Malli
(https://github.com/metosin/malli) is a fresh new data-driven data
validation and specification library for Clojure/Script. In this talk,
Tommi will give a quick introduction to Malli, compare it to prior art
including Plumatic Schema and clojure.spec and demonstrate how to
elegantly solve real-world problems with it. Also, peek beyond the
runtime validation.
This document provides an overview of JavaScript arrays, including:
- Declaring and initializing different types of arrays such as associative arrays and indexed arrays
- Common array methods like push(), pop(), splice(), and slice()
- Array attributes including length, indexOf, and typeOf
- Techniques for adding, removing, and modifying array elements
The document discusses Python lists, which are ordered collections of items of any data type. It describes how to create, access, and modify lists using various functions and operations like append, insert, remove, sort, and slice. These include adding/removing elements, sorting/reversing lists, using lists as stacks with push/pop, and extracting sublist sections. The document provides examples of creating lists with different syntax, accessing elements by index, and using common list methods for operations.
Functional programming can be an excellent approach to designing decoupled, reusable systems with a rich domain model. In fact, the lessons from applying DDD in a functional language translate well to object-oriented programming.
Learn about how to define and invoke methods in Java, how to use parameters and return results. Watch the video lesson here:
https://softuni.org/code-lessons/java-foundations-certification-methods
Templates allow functions and classes to operate on generic types in C++. There are two types of templates: class templates and function templates. Function templates are functions that can operate on generic types, allowing code to be reused for multiple types without rewriting. Template parameters allow types to be passed to templates, similar to how regular parameters pass values. When a class, function or static member is generated from a template, it is called template instantiation.
Domain Driven Design with the F# type System -- NDC London 2013Scott Wlaschin
(Video of these slides here http://fsharpforfunandprofit.com/ddd)
Statically typed functional programming languages like F# encourage a very different way of thinking about types. The type system is your friend, not an annoyance, and can be used in many ways that might not be familiar to OO programmers.
Types can be used to represent the domain in a fine-grained, self documenting way. And in many cases, types can even be used to encode business rules so that you literally cannot create incorrect code. You can then use the static type checking almost as an instant unit test — making sure that your code is correct at compile time.
In this talk, we'll look at some of the ways you can use types as part of a domain driven design process, with some simple real world examples in F#. No jargon, no maths, and no prior F# experience necessary.
Code, links to video, etc., at http://fsharpforfunandprofit.com/ddd
For more on DDD and F# see:
http://fsharpforfunandprofit.com/ddd/
http://tomasp.net/blog/type-first-development.aspx/
http://gorodinski.com/blog/2013/02/17/domain-driven-design-with-fsharp-and-eventstore/
Operator overloading allows operators like + and - to have different implementations depending on the types of their operands. It allows user-defined types to be manipulated using the same syntax as built-in types. The document discusses various aspects of operator overloading like overloading unary, binary, and stream insertion/extraction operators. It also covers type conversions between basic and user-defined types using constructors and conversion functions. Restrictions on which operators can be overloaded and how different conversion scenarios are handled are explained.
Operators and Control Statements in PythonRajeswariA8
This document discusses operators and conditional statements in Python programming. It defines operators as symbols that instruct the computer to perform tasks on operands or values. The main types of operators covered are arithmetic, comparison, assignment, logical, bitwise, membership and identity operators. Conditional statements like if, else if and else are described as ways to control program flow based on conditions. Examples of using different operators and conditional statements are provided.
Common Strategies for Improving Performance on Your Delta LakehouseDatabricks
The Delta Architecture pattern has made the lives of data engineers much simpler, but what about improving query performance for data analysts? What are some common places to look at for tuning query performance? In this session we will cover some common techniques to apply to our delta tables to make them perform better for data analysts queries. We will look at a few examples of how you can analyze a query, and determine what to focus on to deliver better performance results.
Sum and Product Types -The Fruit Salad & Fruit Snack Example - From F# to Ha...Philip Schwarz
- The document describes how product types (built using AND) and sum types (built using OR) are used to define types for representing fruit salads and fruit snacks in F#, Haskell, Scala, and Java.
- Product types combine elements and are used to define the FruitSalad type, while sum types allow alternatives and are used to define the FruitSnack type consisting of different fruits.
- Pattern matching is demonstrated to write functions that analyze and return different strings based on the values of FruitSalad and FruitSnack types.
This session is about Django, which is a web framework build in python. It has several features like admin interface and ORM. The architecture of Django has Model, View, and template and it's ORM saves the pain of writing database queries.
The document discusses various Java operators including:
- Assignment operators like = that store values
- Arithmetic operators like +, -, *, /, % for math operations
- Relational operators like >, <, ==, != for comparisons
- Logical operators like &&, ||, ! for boolean logic
- Bitwise operators like &, |, ^, ~ for bit manipulation
- Shift operators like <<, >>, >>> for shifting bits left or right
Examples are provided to demonstrate the usage of each operator type.
How to Actually Tune Your Spark Jobs So They WorkIlya Ganelin
This document summarizes a USF Spark workshop that covers Spark internals and how to optimize Spark jobs. It discusses how Spark works with partitions, caching, serialization and shuffling data. It provides lessons on using less memory by partitioning wisely, avoiding shuffles, using the driver carefully, and caching strategically to speed up jobs. The workshop emphasizes understanding Spark and tuning configurations to improve performance and stability.
METHODS DESCRIPTION
copy() They copy() method returns a shallow copy of the dictionary.
clear() The clear() method removes all items from the dictionary.
pop() Removes and returns an element from a dictionary having the given key.
popitem() Removes the arbitrary key-value pair from the dictionary and returns it as tuple.
get() It is a conventional method to access a value for a key.
dictionary_name.values() returns a list of all the values available in a given dictionary.
str() Produces a printable string representation of a dictionary.
update() Adds dictionary dict2’s key-values pairs to dict
setdefault() Set dict[key]=default if key is not already in dict
keys() Returns list of dictionary dict’s keys
items() Returns a list of dict’s (key, value) tuple pairs
has_key() Returns true if key in dictionary dict, false otherwise
fromkeys() Create a new dictionary with keys from seq and values set to value.
type() Returns the type of the passed variable.
cmp() Compares elements of both dict.
Regular expressions (regex) use metacharacters to form patterns for string matching. The basic regex structure includes character classes, quantifiers, and alternation/grouping. In Java, the Pattern class compiles a regex string into a pattern object, and Matcher finds matches in input. Matcher methods like start(), end(), and group() retrieve match details. Regex strings in Java require escaping metacharacters with a backslash to avoid compiler errors.
JavaScript Arithmetic Operators
Arithmetic operators are used to perform arithmetic between variables and/or values.
JavaScript Assignment Operators
Assignment operators are used to assign values to JavaScript variables.
JavaScript String Operators
The + operator, and the += operator can also be used to concatenate (add) strings.
Comparison Operators
Comparison operators are used in logical statements to determine equality or difference between variables or values.
Conditional (Ternary) Operator
The conditional operator assigns a value to a variable based on a condition.
Logical Operators
Logical operators are used to determine the logic between variables or values.
JavaScript Bitwise Operators
Bit operators work on 32 bits numbers. Any numeric operand in the operation is converted into a 32 bit number. The result is converted back to a JavaScript number.
The delete Operator
The delete operator deletes a property from an object:
Introduction to Apache Airflow - Data Day Seattle 2016Sid Anand
Apache Airflow is a platform for authoring, scheduling, and monitoring workflows or directed acyclic graphs (DAGs) of tasks. It includes a DAG scheduler, web UI, and CLI. Airflow allows users to author DAGs in Python without needing to bundle many XML files. The UI provides tree and Gantt chart views to monitor DAG runs over time. Airflow was accepted into the Apache Incubator in 2016 and has over 300 users from 40+ companies. Agari uses Airflow to orchestrate message scoring pipelines across AWS services like S3, Spark, SQS, and databases to enforce SLAs on correctness and timeliness. Areas for further improvement include security, APIs, execution scaling, and on
This document contains an agenda and slides for a presentation on Spring Boot. The presentation introduces Spring Boot, which allows developers to rapidly build production-grade Spring applications with minimal configuration. It demonstrates how to quickly create a "Hello World" application using Spring Boot and discusses some of the features it provides out-of-the-box like embedded servers and externalized configuration. The presentation also shows how to add additional functionality like Thymeleaf templates and actuator endpoints to monitor and manage applications.
The document discusses Scala, a programming language designed to be scalable. It can be used for both small and large programs. Scala combines object-oriented and functional programming. It interoperates seamlessly with Java but allows developers to add new constructs like actors through libraries. The Scala community is growing, with industrial adoption starting at companies like Twitter.
(video and more at http://fsharpforfunandprofit.com/fppatterns)
In object-oriented development, we are all familiar with design patterns such as the Strategy pattern and Decorator pattern, and design principles such as SOLID. The functional programming community has design patterns and principles as well. This talk will provide an overview of some of these patterns (such as currying, monads), and present some demonstrations of FP design in practice. We'll also look at some of the ways you can use these patterns as part of a domain driven design process, with some simple real world examples in F#. No jargon, no maths, and no prior F# experience necessary.
Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes. It is written in Java and uses a pluggable backend. Presto is fast due to code generation and runtime compilation techniques. It provides a library and framework for building distributed services and fast Java collections. Plugins allow Presto to connect to different data sources like Hive, Cassandra, MongoDB and more.
Functional programming can be an excellent approach to designing decoupled, reusable systems with a rich domain model. In fact, the lessons from applying DDD in a functional language translate well to object-oriented programming.
Learn about how to define and invoke methods in Java, how to use parameters and return results. Watch the video lesson here:
https://softuni.org/code-lessons/java-foundations-certification-methods
Templates allow functions and classes to operate on generic types in C++. There are two types of templates: class templates and function templates. Function templates are functions that can operate on generic types, allowing code to be reused for multiple types without rewriting. Template parameters allow types to be passed to templates, similar to how regular parameters pass values. When a class, function or static member is generated from a template, it is called template instantiation.
Domain Driven Design with the F# type System -- NDC London 2013Scott Wlaschin
(Video of these slides here http://fsharpforfunandprofit.com/ddd)
Statically typed functional programming languages like F# encourage a very different way of thinking about types. The type system is your friend, not an annoyance, and can be used in many ways that might not be familiar to OO programmers.
Types can be used to represent the domain in a fine-grained, self documenting way. And in many cases, types can even be used to encode business rules so that you literally cannot create incorrect code. You can then use the static type checking almost as an instant unit test — making sure that your code is correct at compile time.
In this talk, we'll look at some of the ways you can use types as part of a domain driven design process, with some simple real world examples in F#. No jargon, no maths, and no prior F# experience necessary.
Code, links to video, etc., at http://fsharpforfunandprofit.com/ddd
For more on DDD and F# see:
http://fsharpforfunandprofit.com/ddd/
http://tomasp.net/blog/type-first-development.aspx/
http://gorodinski.com/blog/2013/02/17/domain-driven-design-with-fsharp-and-eventstore/
Operator overloading allows operators like + and - to have different implementations depending on the types of their operands. It allows user-defined types to be manipulated using the same syntax as built-in types. The document discusses various aspects of operator overloading like overloading unary, binary, and stream insertion/extraction operators. It also covers type conversions between basic and user-defined types using constructors and conversion functions. Restrictions on which operators can be overloaded and how different conversion scenarios are handled are explained.
Operators and Control Statements in PythonRajeswariA8
This document discusses operators and conditional statements in Python programming. It defines operators as symbols that instruct the computer to perform tasks on operands or values. The main types of operators covered are arithmetic, comparison, assignment, logical, bitwise, membership and identity operators. Conditional statements like if, else if and else are described as ways to control program flow based on conditions. Examples of using different operators and conditional statements are provided.
Common Strategies for Improving Performance on Your Delta LakehouseDatabricks
The Delta Architecture pattern has made the lives of data engineers much simpler, but what about improving query performance for data analysts? What are some common places to look at for tuning query performance? In this session we will cover some common techniques to apply to our delta tables to make them perform better for data analysts queries. We will look at a few examples of how you can analyze a query, and determine what to focus on to deliver better performance results.
Sum and Product Types -The Fruit Salad & Fruit Snack Example - From F# to Ha...Philip Schwarz
- The document describes how product types (built using AND) and sum types (built using OR) are used to define types for representing fruit salads and fruit snacks in F#, Haskell, Scala, and Java.
- Product types combine elements and are used to define the FruitSalad type, while sum types allow alternatives and are used to define the FruitSnack type consisting of different fruits.
- Pattern matching is demonstrated to write functions that analyze and return different strings based on the values of FruitSalad and FruitSnack types.
This session is about Django, which is a web framework build in python. It has several features like admin interface and ORM. The architecture of Django has Model, View, and template and it's ORM saves the pain of writing database queries.
The document discusses various Java operators including:
- Assignment operators like = that store values
- Arithmetic operators like +, -, *, /, % for math operations
- Relational operators like >, <, ==, != for comparisons
- Logical operators like &&, ||, ! for boolean logic
- Bitwise operators like &, |, ^, ~ for bit manipulation
- Shift operators like <<, >>, >>> for shifting bits left or right
Examples are provided to demonstrate the usage of each operator type.
How to Actually Tune Your Spark Jobs So They WorkIlya Ganelin
This document summarizes a USF Spark workshop that covers Spark internals and how to optimize Spark jobs. It discusses how Spark works with partitions, caching, serialization and shuffling data. It provides lessons on using less memory by partitioning wisely, avoiding shuffles, using the driver carefully, and caching strategically to speed up jobs. The workshop emphasizes understanding Spark and tuning configurations to improve performance and stability.
METHODS DESCRIPTION
copy() They copy() method returns a shallow copy of the dictionary.
clear() The clear() method removes all items from the dictionary.
pop() Removes and returns an element from a dictionary having the given key.
popitem() Removes the arbitrary key-value pair from the dictionary and returns it as tuple.
get() It is a conventional method to access a value for a key.
dictionary_name.values() returns a list of all the values available in a given dictionary.
str() Produces a printable string representation of a dictionary.
update() Adds dictionary dict2’s key-values pairs to dict
setdefault() Set dict[key]=default if key is not already in dict
keys() Returns list of dictionary dict’s keys
items() Returns a list of dict’s (key, value) tuple pairs
has_key() Returns true if key in dictionary dict, false otherwise
fromkeys() Create a new dictionary with keys from seq and values set to value.
type() Returns the type of the passed variable.
cmp() Compares elements of both dict.
Regular expressions (regex) use metacharacters to form patterns for string matching. The basic regex structure includes character classes, quantifiers, and alternation/grouping. In Java, the Pattern class compiles a regex string into a pattern object, and Matcher finds matches in input. Matcher methods like start(), end(), and group() retrieve match details. Regex strings in Java require escaping metacharacters with a backslash to avoid compiler errors.
JavaScript Arithmetic Operators
Arithmetic operators are used to perform arithmetic between variables and/or values.
JavaScript Assignment Operators
Assignment operators are used to assign values to JavaScript variables.
JavaScript String Operators
The + operator, and the += operator can also be used to concatenate (add) strings.
Comparison Operators
Comparison operators are used in logical statements to determine equality or difference between variables or values.
Conditional (Ternary) Operator
The conditional operator assigns a value to a variable based on a condition.
Logical Operators
Logical operators are used to determine the logic between variables or values.
JavaScript Bitwise Operators
Bit operators work on 32 bits numbers. Any numeric operand in the operation is converted into a 32 bit number. The result is converted back to a JavaScript number.
The delete Operator
The delete operator deletes a property from an object:
Introduction to Apache Airflow - Data Day Seattle 2016Sid Anand
Apache Airflow is a platform for authoring, scheduling, and monitoring workflows or directed acyclic graphs (DAGs) of tasks. It includes a DAG scheduler, web UI, and CLI. Airflow allows users to author DAGs in Python without needing to bundle many XML files. The UI provides tree and Gantt chart views to monitor DAG runs over time. Airflow was accepted into the Apache Incubator in 2016 and has over 300 users from 40+ companies. Agari uses Airflow to orchestrate message scoring pipelines across AWS services like S3, Spark, SQS, and databases to enforce SLAs on correctness and timeliness. Areas for further improvement include security, APIs, execution scaling, and on
This document contains an agenda and slides for a presentation on Spring Boot. The presentation introduces Spring Boot, which allows developers to rapidly build production-grade Spring applications with minimal configuration. It demonstrates how to quickly create a "Hello World" application using Spring Boot and discusses some of the features it provides out-of-the-box like embedded servers and externalized configuration. The presentation also shows how to add additional functionality like Thymeleaf templates and actuator endpoints to monitor and manage applications.
The document discusses Scala, a programming language designed to be scalable. It can be used for both small and large programs. Scala combines object-oriented and functional programming. It interoperates seamlessly with Java but allows developers to add new constructs like actors through libraries. The Scala community is growing, with industrial adoption starting at companies like Twitter.
(video and more at http://fsharpforfunandprofit.com/fppatterns)
In object-oriented development, we are all familiar with design patterns such as the Strategy pattern and Decorator pattern, and design principles such as SOLID. The functional programming community has design patterns and principles as well. This talk will provide an overview of some of these patterns (such as currying, monads), and present some demonstrations of FP design in practice. We'll also look at some of the ways you can use these patterns as part of a domain driven design process, with some simple real world examples in F#. No jargon, no maths, and no prior F# experience necessary.
Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes. It is written in Java and uses a pluggable backend. Presto is fast due to code generation and runtime compilation techniques. It provides a library and framework for building distributed services and fast Java collections. Plugins allow Presto to connect to different data sources like Hive, Cassandra, MongoDB and more.
The Road To Reactive with RxJava JEEConf 2016Frank Lyaruu
This document introduces Reactive Programming with RxJava and how it can be used to create non-blocking applications. It discusses the limitations of blocking code and how RxJava uses Observables and Subscribers to implement reactive and asynchronous operations. It provides examples of converting blocking servlets and HTTP calls to non-blocking using RxJava. While non-blocking code is not always faster, it allows asynchronous operations to utilize threads more efficiently.
This document provides an overview of using Prometheus for monitoring and alerting. It discusses using Node Exporters and other exporters to collect metrics, storing metrics in Prometheus, querying metrics using PromQL, and configuring alert rules and the Alertmanager for notifications. Key aspects covered include scraping configs, common exporters, data types and selectors in PromQL, operations and functions, and setting up alerts and the Alertmanager for routing alerts.
The document discusses Apache Camel, an open source framework for integrating applications and building messaging endpoints. It provides an overview of Camel's capabilities including its lightweight nature, large number of components, support for various data formats, and use of Domain Specific Languages. Basic examples are shown using XML and Java DSLs. Key components, data formats, and Enterprise Integration Patterns supported by Camel are listed. Pros include its flexibility and large community, while documentation is listed as a con. Overall the document serves as an introduction to Apache Camel, outlining its main features and providing simple code examples.
Presto generates Java bytecode at runtime to optimize query execution. Key query operations like filtering, projections, joins and aggregations are compiled into efficient Java methods using libraries like ASM and Fastutil. This bytecode generation improves performance by 30% through techniques like compiling row hashing for join lookups directly into machine instructions.
Router is one of the most important feature or component in Web application framework,
ant it is also one of the performance bottlenecks of framework.
In this session, I'll show you how to make router much faster than ever.
How and why i roll my own node.js frameworkBen Lin
1) The document discusses the author's experience building their own node.js web framework, including their background with other technologies like Ruby on Rails.
2) It describes the key features of their framework, such as MVC structure, middleware support, asset packaging, and command line tools.
3) The author explains that they rolled their own framework to learn more about how frameworks work, have more control over the technology stack, and because node.js performance is better than Ruby on Rails. They emphasize that building the framework was a fun learning experience.
PuppetDB: Sneaking Clojure into Operationsgrim_radical
The document provides an overview of PuppetDB, which is a system for storing and querying data about infrastructure as code and system configurations. Some key points:
- PuppetDB stores immutable data about systems and allows querying of this data to enable higher-level infrastructure operations.
- It uses techniques like command query responsibility separation (CQRS) to separate write and read pipelines for better performance and reliability.
- The data is stored in a relational database for efficient querying, and queries are expressed in an abstract syntax tree (AST)-based language.
- The system is designed for speed, reliability, and ease of deployment in operations. It leverages techniques from Clojure and the JVM.
This document describes an integration framework and its components. It includes:
- FUSE ESB as the integration bus based on JBI and OSGi standards.
- ActiveMQ as the message broker based on JMS.
- CXF for creating or consuming web services.
- Camel as the mediation router for creating integration patterns with a simple Java or XML DSL.
- Details on configuring ActiveMQ and Camel within a OSGi container.
- Code examples of using Camel routes and processors to integrate and transform messages between endpoints.
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak PROIDEA
This document summarizes a presentation about using DTrace on OS X. It introduces DTrace as a dynamic tracing tool for user and kernel space. It discusses the D programming language used for writing DTrace scripts, including data types, variables, operators, and actions. Example one-liners and scripts are provided to demonstrate syscall tracking, memory allocation snooping, and hit tracing. The presentation outlines some past security work using DTrace and similar dynamic tracing tools. It concludes with proposing future work like more kernel and USDT tracing as well as Python bindings for DTrace.
This document provides an introduction and overview of a Node.js tutorial presented by Tom Hughes-Croucher. The tutorial covers topics such as building scalable server-side code with JavaScript using Node.js, debugging Node.js applications, using frameworks like Express.js, and best practices for deploying Node.js applications in production environments. The tutorial includes exercises for hands-on learning and demonstrates tools and techniques like Socket.io, clustering, error handling and using Redis with Node.js applications.
Node has captured the attention of early adopters by clearly differentiating itself as being asynchronous from the ground up while remaining accessible. Now that server side JavaScript is at the cutting edge of the asynchronous, real time web, it is in a much better position to establish itself as the go to language for also making synchronous, CRUD webapps and gain a stronger foothold on the server.
This talk covers the current state of server side JavaScript beyond Node. It introduces Common Node, a synchronous CommonJS compatibility layer using node-fibers which bridges the gap between the different platforms. We look into Common Node's internals, compare its performance to that of other implementations such as RingoJS and go through some ideal use cases.
How Many Ohs? (An Integration Guide to Apex & Triple-o)OPNFV
Dan Radez, Red Hat, Tim Rozet, Red Hat
The OPNFV ecosystem is made up of projects that need to integrate with each other. Project Apex uses Triple-o under the covers which most people usually need some assistance to integrate with.
Come and spend a session with the Apex development team learning the ins and outs of Triple-o.
In this session participants will learn about the deployment process that is run when an Apex/Triple-o deployment is executed and how to assign services to nodes and generate networking configurations withing Triple-o to successfully integrate and deploy a new component in OpenStack.
Come learn how to untangle the learning curve presented when integrating and using Triple-o and simplify your future development and deployment endeavors with a new found intimate knowledge of the Apex & Triple-o platform.
Exploring Async PHP (SF Live Berlin 2019)dantleech
(note slides are missing animated gifs and video)
As PHP programmers we are used to waiting for network I/O, in general we may not even consider any other option. But why wait? Why not jump on board the Async bullet-train and experience life in the fast lane and give Go and NodeJS a run for the money. This talk will aim to make the audience aware of the benefits, opportunities, and pitfalls of asynchronous programming in PHP, and guide them through the native functionality, frameworks and PHP extensions though which it can be facilitated.
JRuby allows developers to write plugins for data processing systems like Norikra and Embulk in Ruby while taking advantage of Java libraries and the JVM. Norikra is a stream processing system that allows SQL queries over data streams. It is written in JRuby and uses the Java Esper library. Embulk is an open-source ETL tool that loads data between databases and file formats using plugins. Both systems use a plugin architecture where plugins can be written in JRuby or Java and are distributed as Ruby gems. This allows for a pluggable ecosystem that benefits from Ruby's productivity while utilizing Java libraries and the JVM's performance.
These are the outline slides that I used for the Pune Clojure Course.
The slides may not be much useful standalone, but I have uploaded them for reference.
Exposing the hidden caveats of technical debt with machine learning and A.I
Leevi Rantala's slides from his talk introducing his research work on technical debt at University of Oulu.
Creating an experimental GraphQL formatter using Clojure, Instaparse, and Gra...Metosin Oy
The document discusses the creation of an experimental GraphQL formatter using Clojure, Instaparse, and GraalVM. It provides background on the creator and their motivation to build a GraphQL formatter due to frustrations with inconsistencies across existing tools. It then details the initial scope, progress over time implementing a parser and formatter, lessons learned, performance testing, and opportunities for future improvement.
Serverless Clojure and ML prototyping: an experience reportMetosin Oy
This document summarizes a serverless Clojure and machine learning application for prototyping. It describes the team building the application, the serverless AWS infrastructure using Terraform, CI/CD pipelines with GitHub Actions, the Clojure applications including a dashboard SPA and event processor, and natural language processing using Haystack and SageMaker batch transforms. The application allows users to upload documents and questionnaires, select questions, and trigger inferences to find answers from the documents using ML models.
Clojure is a modern dynamically typed lisp. Dynamical typing is ofter associated with poor performance and runtime failures. In this talk, I'll present some of the lessons learned on building Clojure/Script systems that are both ridiculously fast and will fail fast on errors. Will compare the performance of mutable, persistent & zero-copy data structures and show how we can use interpreters and compilers to build beautiful and performant abstractions. A quick demo on how to build a simple non-blocking web server that runs idiomatic Clojure to serve millions of requests per sec.
Clojure is awesome, and it can be fast too.
Video: https://www.youtube.com/watch?v=3SSHjKT3ZmA
Clojutre Real Life (2012 ClojuTRE Retro Edition)Metosin Oy
The document discusses using Clojure for a real-life public sector project. It finds that Clojure enables building a simple RESTful web application using CQRS and optionally event sourcing for managing domain models over time. The project evolved from patterns to tools effectively. Clojure allows rapid development through its functional programming approach and tooling.
The Ancient Art of Data-Driven - reitit, the library - Metosin Oy
The document summarizes a presentation about the Ancient Art of Data-Driven development using Clojure. It introduces the speaker and his company Metosin, which builds software using Clojure. It then discusses the benefits of taking a simple, data-driven approach to building systems by making everything explicit and understandable. Several Clojure libraries that take this approach are highlighted, including reitit for routing and DataScript for domain modeling. The presentation encourages taking a data-first approach and composing data, specifications, and routers to build optimized systems.
The document summarizes Clojure, a functional programming language that runs on the JVM and JavaScript virtual machines. It notes that Clojure is strongly typed, functional, immutable, and allows macros and runtime polymorphism. It then discusses Clojure adoption in Finland, providing examples of startups and consultancies using it. Finally, it recommends some resources for learning Clojure, including books and online courses.
Tommi Reiman discusses optimizing Clojure performance and abstractions. He shares lessons learned from optimizing middleware performance and JSON serialization. Data-driven approaches can enable high performance while maintaining abstraction. Reitit is a new routing library that aims to have the fastest performance through techniques like compiled routing data. Middleware can also benefit from data-driven approaches without runtime penalties. Overall performance should be considered but not obsessively, as many apps do not require extreme optimization.
ClojuTRE2016 was held in Tampere, Finland at Pakkahuone. The conference focused on Clojure and included pre-conference surveys, speaking areas, sponsor areas, toilets, a terrace, and a club. Anna-Kaisa Ikonen, the Mayor of Tampere, gave the opening words for the conference.
Schema tools-and-trics-and-quick-intro-to-clojure-spec-22.6.2016Metosin Oy
The document discusses Schema and Clojure.spec, two libraries for data validation and specification in Clojure. It provides an overview of Schema's capabilities for runtime validation, documentation, and data transformations. It also summarizes Clojure.spec's focus on specification through generative testing. The document compares the two approaches and argues that Schema is currently better for runtime validation in web applications, while Clojure.spec will be important for system-level specification.
Wieldy remote apis with Kekkonen - ClojureD 2016Metosin Oy
Today, most of the apps we build involve the browser and thus need an remoting layer between the ui and the backend. New Technology like Falcor, Relay and Om Next are leading the innovation in this space. Is REST dead? CQRS to the rescue? How should I build my next api?
Our 2 cents is Kekkonen, a fresh new multi-paradigm api library for Clojure.
ClojuTRE2015: Kekkonen - making your Clojure web APIs more awesomeMetosin Oy
Thanks to REST and Swagger, we can build beautiful apis to feed both our browser front-ends and external applications. But, wrapping your Clojure code into resources mostly for your ClojureScript front-end doesn't feel right? Just use RPC? Meet in the middle?
Kekkonen is a small library for managing your (web) apis as commands and queries. No magic, data-driven, un-restful and non-rpc. It's goals are to be small, explicit, extendable and to help enforce your business rules both on the server side and on the ClojureScript frontend. Besides Swagger, it provides run-time context-aware apidocs for Clojure(Script).
This document discusses Clojure, an immutable, functional programming language that runs on the JVM. It provides an overview of Clojure basics like its syntax, data structures, concurrency features and macros. It then discusses how Clojure is used in practice, including common tools, frameworks and patterns for web development, testing, persistence and more. Real-world examples are given throughout.
Euroclojure2014: Schema & Swagger - making your Clojure web APIs more awesomeMetosin Oy
- The document discusses using Prismatic Schema and Swagger to describe, validate, and document Clojure web APIs. It covers topics like Schema validation and coercion, Ring-Swagger, Compojure-API, and Fnhouse-Swagger integration.
- Ring-Swagger implements the Swagger specification for Ring and uses Schema for data models and coercion. Compojure-API is an extendable web API library that uses Schema and generates Swagger documentation. Fnhouse-Swagger provides Swagger support for the Fnhouse library.
- The document encourages adopting Schema and Swagger tools to build robust and well-documented Clojure web APIs in a consistent manner across different frameworks.
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on automated letter generation for Bonterra Impact Management using Google Workspace or Microsoft 365.
Interested in deploying letter generation automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxSitimaJohn
Ocean Lotus cyber threat actors represent a sophisticated, persistent, and politically motivated group that poses a significant risk to organizations and individuals in the Southeast Asian region. Their continuous evolution and adaptability underscore the need for robust cybersecurity measures and international cooperation to identify and mitigate the threats posed by such advanced persistent threat groups.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Tatiana Kojar
Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI.
With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.
Introduction of Cybersecurity with OSS at Code Europe 2024Hiroshi SHIBATA
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
This presentation provides valuable insights into effective cost-saving techniques on AWS. Learn how to optimize your AWS resources by rightsizing, increasing elasticity, picking the right storage class, and choosing the best pricing model. Additionally, discover essential governance mechanisms to ensure continuous cost efficiency. Whether you are new to AWS or an experienced user, this presentation provides clear and practical tips to help you reduce your cloud costs and get the most out of your budget.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
2. Me(tosin)
Sinclair Spectrum in 1983
...
Co-founded Metosin 2012
20 developers, Tampere & Helsinki / Finland
Projects, Startups and Design
FP, mostly Clojure/Script
https://github.com/metosin
3. ClojuTRE 2019 (8th)
September 26-27th 2019, Helsinki
FP & Clojure, 300++ on last two years
Basic flights & hotels for speakers
Free Student & Diversity tickets
Call For Speakers Open
https://clojutre.org
4. This talk
Routing and dispatching
Reitit, the library
Reitit, the framework
Performance
Takeaways
5. Routing in Clojure/Script
Lot's of options: Ataraxy, Bide, Bidi, Compojure,
Keechma, Pedestal, Silk, Secretary, ...
Something we need in mostly all the (web)apps
Mapping from path -> match
Mapping from name -> match (reverse routing)
HTTP (both server & browser)
Messaging
Others
6. Dispatching in Clojure/Script
Here: composing end executing functionality in
the request-response pipeline
Common patterns
Middleware (Ring, nREPL, ..)
Interceptors (Pedestal, Re-Frame, ..)
Controllers (Keechma)
Promises
Sync & Async
7. New Routing Library?
More data-driven
Leveraging clojure.spec
Built for performance
Friendly & explicit API
Reach: browser, JVM, Node, Others
Both routing and dispatching
The Yellow (Hiccup) Castle of Routing?
10. Basics
reitit.core/router to create a router
Matching by path or by name (reverse routing)
Functions to browse the route trees
(defprotocol Router
(router-name [this])
(routes [this])
(compiled-routes [this])
(options [this])
(route-names [this])
(match-by-path [this path])
(match-by-name [this name] [this name path-params]))
18. What is in the route data?
Can be anything, the Router doesn't care.
Returned on successful Match
Can be queried from a Router
Build your own interpreters for the data
A Route First Architecture: Match -> Data -> React
(r/match-by-path router "/api/admin/db")
;#Match{:template "/api/admin/db",
; :data {:interceptors [::api ::db]
; :roles #{:db-admin}},
; :result nil,
; :path-params {},
; :path "/api/admin/db"}
19. Example use cases
Authorization via :roles
Frontend components via :view
Dispatching via :middleware and :interceptors
Stateful dispatching with :controllers
Coercion via :parameters and :coercion
Run requests in a server io-pool with :server/io
Deploy endpoints separately to :server/target
21. Route Data Validation
Route Data can be anything -> data spaghetti?
Leverage clojure.spec at router creation
Define and enforce route data specs
Fail-fast, missing, extra or misspelled keys
With help of spec-tools and spell-spec
Closed specs coming to Spec2 (TBD)
25. Failing Fast
During static analysis (e.g. linter)
At compile-time (macros, defs)
At development-time (schema/spec annos)
At creation-time
At runtime
At runtime special condition
(c) https://www.artstation.com/kinixuki
26. Coercion (Déjà-vu)
Coercion is a process of transforming values between
domain (e.g. JSON->EDN, String->EDN)
Route data keys :parameters & :coercion
Utilities to apply coercion in reitit.coercion
Implementation for Schema & clojure.spec <-- !!!
(defprotocol Coercion
"Pluggable coercion protocol"
(-get-name [this])
(-get-options [this])
(-get-apidocs [this specification data])
(-compile-model [this model name])
(-open-model [this model])
(-encode-error [this error])
(-request-coercer [this type model])
(-response-coercer [this model]))
28. Framework
You call the library, but the framework calls you
Reitit ships with multiple Routing Frameworks
reitit-ring Middleware-dispatch for Ring
reitit-http Async Interceptor-dispatch for http
reitit-pedestal Reitit Pedestal
reitit-frontend (Keechma-style) Controllers,
History & Fragment Router, helpers
30. Ring Routing
A separate lightweight module
Routing based on path and :request-method
Adds support for :middleware dispatch
chain is executed after a match
ring-handler to create a ring-compatible handler
Supports Both sync & async
Supports Both JVM & Node
No magic, no default middleware
32. Accessing route data
Match and Router are injected into the request
Components can read these at request time and
do what ever they want, "Ad-hoc extensions"
Pattern used in Kekkonen and in Yada
(defn wrap-roles [handler]
;; roles injected via session-middleware
(fn [{:keys [roles] :as request}]
;; read the route-data at request-time
(let [required (-> request (ring/get-match) :data :roles)]
(if (and (seq required)
(not (set/subset? required roles)))
{:status 403, :body "forbidden"}
(handler request)))))
33. Middleware as data
Ring middleware are opaque functions
Reitit adds a first class values, Middleware records
Recursive IntoMiddleware protocol to expand to
Attach documentation, specs, requirements, ...
Can be used in place of middleware functions
Zero runtime penalty
(defn roles-middleware []
{:name ::roles-middleware
:description "Middleware to enforce roles"
:requires #{::session-middleware}
:spec (s/keys :opt-un [::roles])
:wrap wrap-roles})
34. Compiling middleware
Each middleware knows the endpoint it's mounted to
We can pass the route data in at router creation time
Big win for optimizing chains
(def roles-middleware
{:name ::roles-middleware
:description "Middleware to enforce roles"
:requires #{::session-middleware}
:spec (s/keys :opt-un [::roles])
:compile (fn [{required :roles} _]
;; unmount if there are no roles required
(if (seq required)
(fn [handler]
(fn [{:keys [roles] :as request}]
(if (not (set/subset? required roles))
{:status 403, :body "forbidden"}
(handler request))))))})
35. Partial Specs Example
"All routes under /account should require a role"
;; look ma, not part of request processing!
(def roles-defined
{:name ::roles-defined
:description "requires a ::role for the routes"
:spec (s/keys :req-un [::roles])})
["/api" {:middleware [roles-middleware]} ;; behavior
["/ping"] ;; unmounted
["/account" {:middleware [roles-defined]} ;; :roles mandatory
["/admin" {:roles #{:admin}}] ;; ok
["/user" {:roles #{:user}}] ;; ok
["/manager"]]] ;; fail!
36. Middleware chain as data
Each endpoint has it's own vector of middleware
Documents of what is in the chain
Chain can be manipulated at router creation time
Reordering
Completing
Interleaving
Interleaving a request diff console printer:
reitit.ring.middleware.dev/print-request-diffs
37.
38. Data definitions (as data)
The core coercion for all (OpenAPI) paramerer &
response types ( :query , :body , header , :path etc)
Separerate Middleware to apply request & response
coercion and format coercion errors
Separate modules, spec coercion via spec-tools
["/plus/:y"
{:get {:parameters {:query {:x int?},
:path {:y int?}}
:responses {200 {:body {:total pos-int?}}}
:handler (fn [{:keys [parameters]}]
;; parameters are coerced
(let [x (-> parameters :query :x)
y (-> parameters :path :y)]
{:status 200
:body {:total (+ x y)}}))}}]
41. Going Async
Interceptors are much better fit for async
reitit-http uses the Interceptor model
Requires an Interceptor Executor
reitit-pedestal or reitit-sieppari
Sieppari supports core.async , Manifold and Promesa
Pedestal is more proven, supports core.async
Target Both JVM & Node (via Sieppari)
45. Route-driven frameworks
Routing and dispatching is separated, middleware (or
interceptors) are applied only after a match
Each endpoint has a unique dispatch chain
Each component can be compiled and optimized against
the endpoint at creation time
Components can define partial route data specs that
only effect the routes they are mounted to.
We get both Performance & Correctness
... this is Kinda Awesome.
(c) https://www.artstation.com/kinixuki
47. Performance
how can we make compojure-api faster?
we moved from Clojure to GO because of perf
How fast are the current Clojure libraries?
How fast can we go with Java/Clojure?
48. Measuring Performance
Always measure
Both micro & macro benchmarks
In the end, the order of magnitude matters
Lot's of good tools, some favourites:
clojure.core/time
criterium
com.clojure-goes-fast/clj-async-profiler
com.clojure-goes-fast/clj-java-decompiler
https://github.com/wg/wrk
49. (def defaults {:keywords? true})
(time
(dotimes [_ 10000]
(merge defaults {})))
; "Elapsed time: 4.413803 msecs"
(require '[criterium.core :as cc])
(cc/quick-bench
(merge defaults {}))
; Evaluation count : 2691372 in 6 samples of 448562 calls.
; Execution time mean : 230.346208 ns
; Execution time std-deviation : 10.355077 ns
; Execution time lower quantile : 221.101397 ns ( 2.5%)
; Execution time upper quantile : 245.331388 ns (97.5%)
; Overhead used : 1.881561 ns
50. (require '[clj-async-profiler.core :as prof])
(prof/serve-files 8080) ;; serve the svgs here
(prof/profile
(dotimes [_ 40000000] ;; ~10sec period
(merge defaults {})))
51. Reitit performance
Designed group up to be performant
Perf suite to see how performance evolves
Measured against Clojure/JavaScript/GO Routers
Performance toolbox:
Optimized routing algorithms
Separation of creation & request time
The Usual Suspects
52. Routing algorithms
When a router is created, route tree is inspected
and a best possible routing algorith is chosen
No regexps, use linear-router as a last effort
lookup-router , single-static-path-router
trie-router , linear-router
mixed-router , quarantine-router
53. trie-router
For non-conflicting trees with wildcards
First insert data into Trie AST, then compile it into
fast lookup functions using a TrieCompiler
On JVM, backed by a fast Java-based Radix-trie
60. Java Trie
Set of matchers defined by the TrieCompiler
Order of magnitude faster than the original impl
@Override
public Match match(int i, int max, char[] path) {
boolean hasPercent = false;
boolean hasPlus = false;
if (i < max && path[i] != end) {
int stop = max;
for (int j = i; j < max; j++) {
final char c = path[j];
hasPercent = hasPercent || c == '%';
hasPlus = hasPlus || c == '+';
if (c == end) {
stop = j;
break;
}
}
final Match m = child.match(stop, max, path);
if (m != null) {
m.params = m.params.assoc(key, decode(new String(path, i, stop - i), hasPercent, hasPlus));
}
return m;
}
return null;
}
61. The Usual Suspects
Persistent Data Structures -> Records, Reify
Multimethods -> Protocols
Map Destructuring -> Manually
Unroll recursive functions ( assoc-in , ...)
Too generic functions ( walk , zip , ...)
Dynamic Binding
Manual inlining
Regexps
63. RESTful api test
50+ routes, mostly wildcards
Reitit is orders of magnitude faster
220ns vs 22000ns, actually matters for busy sites
64.
65. Looking out of the box
https://github.com/julienschmidt/httprouter
One of the fastest router in GO
(and source of many of the optimizations in reitit)
In Github api test, Reitit is ~40% slower
That's good! Still work to do.
67. Current Status
Most Clojure libraries don't even try to be fast
And that's totally ok for most apps
Compojure+ring-defaults vs reitit, with same response
headers, simple json echo
;; 10198tps
;; http :3000/api/ping
;; wrk -d ${DURATION:="30s"} http://127.0.0.1:3000/api/ping
(http/start-server defaults-app {:port 3000})
;; 48084tps
;; http :3002/api/ping
;; wrk -d ${DURATION:="30s"} http://127.0.0.1:3002/api/ping
(http/start-server reitit-app {:port 3002})
=> that's 5x more requests per sec.
71. Web Stacks
We know Clojure is a great tool for building web stuff
We can build a REALLY fast server stack for Clojure
aleph or immutant-nio as web server (nio, zero-copy)
reitit -based routing
Fast formatters like jsonista for JSON
next.jdbc (or porsas ) for database access
Good tools for async values & streams
lot's of other important components
Simple tools on top, the (coastal) castles?
72. Clojure Web & Data Next?
We have Ring, Pedestal, Yada & friends
We have nice templates & examples
Ring2 Spec? Spec for interceptors?
New performant reference architecture?
Bigger building blocks for rapid prototypes?
Making noice that Clojure is kinda awesome?
Community built Error Formatter?
Data-driven tools & inventories?
clojure.spec as data?
74. Reitit bubbin' under
Support for OpenAPI3
JSON Schema validation
Spec2 support (when it's out)
Developer UI with remote debugger
More Batteries & Guides (frontend)
(Help make next.jdbc java-fast)
Sieppari.next
(c) https://www.artstation.com/kinixuki
75. Wrap-up
Reitit is a new routing library & framework
Embrace data-driven design, all the way down
Clojure is a Dynamic Language -> fail fast
clojure.spec to validate & transform data
Understand performance, test on your libraries
We can build beautiful & fast things with Clojure
Try out https://github.com/metosin/reitit
Chatting on #reitit in Slack
(c) https://www.artstation.com/kinixuki