Building Scalable Systems: What you can learn from Erlang - DotScale 2016
•
0 likes•975 views
Here are the slides from my talk at DotScale 2016 conference. Erlang is a programming language dedicated to building robust and scalable services. It has matured for many years and finally reached a point where it is one of the secret weapon used by many prominent companies. To help you understand Erlang, I am going to walk you through old programming concepts that are at the root of Erlang using … A time machine !
Report
Share
Report
Share
Download to read offline
Recommended
Introduction to Elixir
A brief overview of the Elixir programming language, mainly focusing on its speed, fault-tolerance, and power as a functional language.
Akka dotnet presentation ndc 2017
The slides for my presentation at NDC Oslo 2017 about building a near real time system with Akka.NET, React and Redux.
Jenkins and the Future of Software Delivery
This document discusses trends in software delivery past, present and future. It notes how software delivery has shifted from manual and localized to automated and distributed. It argues continuous delivery is about dealing with increasing chaos and diversity. The document suggests several trends for the future including machine learning, augmented reality, IoT and self-driving vehicles. It envisions a future where computers write more of the code, delivery is further automated through self-healing systems, and improved visualization and interfaces change how software is developed and delivered.
Phoenix for Rubyists
This document introduces Phoenix, a web framework for building scalable and fault-tolerant distributed systems with Elixir and Erlang. It discusses how Moore's Law has led to more multi-core machines requiring better support for concurrency. Phoenix provides productivity benefits like Rails while enabling applications to handle massive concurrency through Elixir and Erlang's actor model and lightweight processes. The document demonstrates building basic and real-time web apps with Phoenix as well as using it as the web layer for distributed systems.
Demystifying Distributed Graph Processing
dotScale 2016 presentation
Writing distributed graph applications is inherently hard. In this talk, Vasia gives an overview of high-level programming models and platforms for distributed graph processing. She exposes and discusses common misconceptions, shares lessons learnt, and suggests best practices.
The Expanding Boundaries of CSS
Is CSS a simple technology for non programmers? It certainly seems simple at first. Let's explore the boundaries of CSS and cover the ways in which those boundaries are changing and what that means for how we think about ourselves as CSS authors.
OpenZFS dotScale
The document discusses the architecture of ZFS and how it has evolved. It describes ZFS as a combined file system and volume manager that provides data integrity and manages storage pools. It supports various access interfaces like NFS, SMB, iSCSI and can be used with virtualization technologies like containers and virtual machines. The document also shows how networking speeds and storage densities have increased dramatically over the last 30 years enabling technologies like ZFS.
The low level awesomeness of Go
This document discusses the appeal of Go for C++ developers and how some key Go features are implemented under the hood. It notes that concurrent I/O programming in C++ can be complicated, verbose, and fragile, whereas Go's goroutines, netpoller, and channels make concurrency easier. Goroutines use fast context switching via thread local storage and assembly code. The netpoller leverages OS syscalls like epoll to unpark goroutines only when I/O is available. Channels use lock-free operations and fast mutexes to efficiently pass data between goroutines.
Recommended
Introduction to Elixir
A brief overview of the Elixir programming language, mainly focusing on its speed, fault-tolerance, and power as a functional language.
Akka dotnet presentation ndc 2017
The slides for my presentation at NDC Oslo 2017 about building a near real time system with Akka.NET, React and Redux.
Jenkins and the Future of Software Delivery
This document discusses trends in software delivery past, present and future. It notes how software delivery has shifted from manual and localized to automated and distributed. It argues continuous delivery is about dealing with increasing chaos and diversity. The document suggests several trends for the future including machine learning, augmented reality, IoT and self-driving vehicles. It envisions a future where computers write more of the code, delivery is further automated through self-healing systems, and improved visualization and interfaces change how software is developed and delivered.
Phoenix for Rubyists
This document introduces Phoenix, a web framework for building scalable and fault-tolerant distributed systems with Elixir and Erlang. It discusses how Moore's Law has led to more multi-core machines requiring better support for concurrency. Phoenix provides productivity benefits like Rails while enabling applications to handle massive concurrency through Elixir and Erlang's actor model and lightweight processes. The document demonstrates building basic and real-time web apps with Phoenix as well as using it as the web layer for distributed systems.
Demystifying Distributed Graph Processing
dotScale 2016 presentation
Writing distributed graph applications is inherently hard. In this talk, Vasia gives an overview of high-level programming models and platforms for distributed graph processing. She exposes and discusses common misconceptions, shares lessons learnt, and suggests best practices.
The Expanding Boundaries of CSS
Is CSS a simple technology for non programmers? It certainly seems simple at first. Let's explore the boundaries of CSS and cover the ways in which those boundaries are changing and what that means for how we think about ourselves as CSS authors.
OpenZFS dotScale
The document discusses the architecture of ZFS and how it has evolved. It describes ZFS as a combined file system and volume manager that provides data integrity and manages storage pools. It supports various access interfaces like NFS, SMB, iSCSI and can be used with virtualization technologies like containers and virtual machines. The document also shows how networking speeds and storage densities have increased dramatically over the last 30 years enabling technologies like ZFS.
The low level awesomeness of Go
This document discusses the appeal of Go for C++ developers and how some key Go features are implemented under the hood. It notes that concurrent I/O programming in C++ can be complicated, verbose, and fragile, whereas Go's goroutines, netpoller, and channels make concurrency easier. Goroutines use fast context switching via thread local storage and assembly code. The netpoller leverages OS syscalls like epoll to unpark goroutines only when I/O is available. Channels use lock-free operations and fast mutexes to efficiently pass data between goroutines.
Unbundling the Future Web Runtime
The document discusses the integration of ES modules into JavaScript build tooling and how tools like SystemJS can support static and dynamic imports of ES modules and WebAssembly. It notes how the native module loader enables code sharing across pages and lazy loading, and how build tools can optimize modules through techniques like tree shaking, module splitting, and output generation while fully exposing the module graph to enable cross-module optimizations for better performance.
Swift Generics in Theory and Practice
Generics are one of the features of Swift that set it apart from many other languages. Since Swift debuted, developers have been wondering when to use them and what to use them for. By looking at the standard library, which is the first piece of software to use generics, we can better understand their purpose and utility.
The Value in Trees
Swift introduces new ways to model data through value types like structs and enums. I’ll discuss my experiences rewriting the data model of the vector graphics app Sketch to use value trees. I’ll finish off pondering whether future data modelling frameworks could be based on value trees, rather than entities and relationships, and introduce the experimental open source project Impeller (https://github.com/mentalfaculty/impeller), which is a step in that direction.
This talk was given at dotSwift 2017.
NoSQL Injections in Node.js - The case of MongoDB
This document discusses NoSQL injections in Node.js applications using MongoDB. It provides examples of how request body parameters can be used to alter MongoDB queries and presents best practices for validating user input to prevent injection attacks. These include using middleware to validate request data matches expected types and structures, or using libraries like Joi and Celebrate for schema-based validation. The document emphasizes that input validation is crucial to secure MongoDB queries from manipulation through user-supplied values.
The Ins and Outs of Easing - dotCSS 2016
The document discusses different techniques for easing or smoothing animation transitions, including linear easing with a constant rate of change, easing curves like ease in/out quad that accelerate or decelerate, Penner easing equations that precisely define different curve functions, cubic beziers that allow customizing the curve, lerping that calculates incremental changes each frame, and spring physics with damping that simulates a spring. It recommends different techniques based on the type of animation and provides resources for further information.
dotScale 2014
CTF links:
NotSoSecure CTF: http://ctf.notsosecure.com
Security Shepherd: https://www.owasp.org/index.php/OWASP_Security_Shepherd
http://hax.tor.hu/
https://pwn0.com/
http://www.smashthestack.org/
http://www.hellboundhackers.org/
http://www.overthewire.org/wargames/
http://counterhack.net/Counter_Hack/Challenges.html
http://www.hackthissite.org/
http://exploit-exercises.com/
http://vulnhub.com/
Content Security Policy - The application security Swiss Army Knife
This document discusses Content Security Policy (CSP), which defines a set of directives to allow websites to restrict resources the browser can load for increased security. It covers the basic CSP directives like default-src, script-src, and style-src for controlling what content is allowed to execute or render on a page. It also discusses additional CSP directives for features like forms, frames, and mixed content blocking. The document recommends starting with a basic CSP policy and then fine-tuning it by specifying allowed sources for each directive. It also provides examples of how CSP can help mitigate cross-site scripting attacks and reporting errors to help migrate sites from HTTP to HTTPS.
The jsdom
Two years ago I inherited jsdom, a project of breathtaking scope. In essence, it aims to simulate a browser in JavaScript, by implementing JavaScript versions of a wide variety of web standards (much more than just the DOM). While maintaining jsdom for the last two years, and eventually bringing it to a 1.0 release, I learned an incredible amount about the web. I want to share with you what I’ve learned: the history of the standards and implementations of them that make up the web; the interaction between seemingly-disparate parts of the platform; and all about the strange APIs we’ve come to know and love. You should walk away from this talk with a new appreciation for how browsers work—and how, through the ongoing effort of a community of contributors and package maintainers, we’ve been able to recreate one in pure JavaScript.
Actors evolved- Rotem Hermon
The actor model is a novel approach to writing concurrent software. It is based on the concept of small computational units communicating through asynchronous message passing, thus allowing concurrency and scalability while negating a lot of the problems of concurrent programming. Though the actor model got some adoption with the Erlang language and the Akka framework, it remained a rather niche approach and has not become a commonly used practice. But this may be changing now with the introduction of “Virtual Actors” - a new abstraction for writing distributed applications. This abstraction was introduced with the Orleans framework by Microsoft and adopted to Java by EA with their Orbit framework. This talk will include a short introduction to the actor model. We will then explore the Virtual Actors model, how it’s different from the classic model, and why it makes distributed application programming a lot simpler.
Intro to elixir and phoenix
This document provides an overview of Elixir and the Phoenix framework. It discusses how Elixir runs on the Erlang VM and inherits properties like high availability and distribution. Phoenix is introduced as a web framework for Elixir that focuses on productivity, reliability, and speed. Key Phoenix concepts like the request pipeline, router, controllers, models and views are briefly outlined.
Akka Fundamentals
This document provides an overview of Akka fundamentals including:
- The actor model which uses message passing between encapsulated state and behavior units to achieve concurrency
- The Akka actor API for building fault tolerant distributed applications in Scala and Java
- Fault tolerance techniques in Akka like supervision which allows actors to monitor and respond to failures of child actors
- Routing which provides abstractions for distributing messages to multiple receiver actors
Why Play Framework is fast
Typesafe trainer and consultant Will Sargent describes just how Play Framework is so "fast" for Java and Scala production apps.
More Play, Akka, Scala and Apache Spark webinars, presentations, and videos:
http://typesafe.com/resources/videos
VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang: two very promising beasts, made for the new parallel world, but still lingering in the wings. Not only are they addressing todays challenges but they are using parallel architectures as corner stone of their new and surprising approach to be faster and more productive. What are they good for? Why are we working to team them up?
Erlang promises faster implementation, way better maintenance and 4 times shorter code. VoltDB claims to be two orders of magnitude faster than its competitors. The two share many similarities: both are the result of scientific research and designed from scratch to address the new reality of parallel architectures with full force.
This talk presents the case for Erlang as server language, where it shines, how it looks, and how to get started. It details Erlang's secret sauce: microprocesses, actors, atoms, immutable variables, message passing and pattern matching. (Note: for a longer version of this treatment of Erlang only see: Why Erlang? http://www.slideshare.net/eonblast/why-erlang-gdc-online-2012)
VoltDB's inner workings are explained to understand why it can be so incredibly fast and still better than its NoSQL competitors. The well publicized Node.js benchmark clocking in at 695,000 transactions per second is described and the simple steps to get VoltDB up and running to see the prodigy from up close.
Source examples are presented that show Erlang and VoltDB in action.
The speaker is creator and maintainer of the Erlang VoltDB driver Erlvolt.
Introduction to Actor Model and Akka
The slides of the speech I given during coscup 2011. The topic is "Programming for the Future, Introduction to Actor Model and Akka"
applicationlayer.pptx
This document summarizes key points from Chapter 7 of Andrew Tanenbaum's book "Computer Networks" which covers the application layer of the OSI model. It discusses the Domain Name System (DNS) which maps domain names to IP addresses, electronic mail (email) protocols and formats, and the basic architecture of email systems including user agents and message transfer. Real applications like the World Wide Web and multimedia are also mentioned.
Evented Ruby VS Node.js
This document discusses evented programming in Node.js and Ruby. It explains that evented programming uses callbacks and asynchronous non-blocking I/O. Node.js uses this approach to improve concurrency over blocking I/O models. Ruby can also implement evented programming using libraries like EventMachine that provide asynchronous abstractions while keeping app code procedural. The document provides examples of building evented applications in both languages.
Scalable game-servers-tgc
Scalable Game Servers talk I gave at TGC 2017
The slides describe the case for building distributed systems using a statteful architecture to achieve high throughput and low latency using Microsoft Orleans as the best provider and inventor of the virtual actor model.
http://github.com/dotnet/orleans
Fastest Servlets in the West
The venerable Servlet Container still has some performance tricks up its sleeve - this talk will demonstrate Apache Tomcat's stability under high load, describe some do's (and some don'ts!), explain how to performance test a Servlet-based application, troubleshoot and tune the container and your application and compare the performance characteristics of the different Tomcat connectors. The presenters will share their combined experience supporting real Tomcat applications for over 20 years and show how a few small changes can make a big, big difference.
VIRTUAL MACHINE VERSATILE PLATFORM01~chapter 1 (1).ppt
This document provides an introduction to virtual machines. It defines virtual machines as software implementations that emulate an entire physical computer system. It describes two main types of virtual machines - system virtual machines that virtualize an entire operating system and process virtual machines that virtualize individual applications. The document outlines the advantages of virtual machines in enabling hardware and software innovation as well as the ability to run different operating systems on the same physical machine. It also provides examples of various virtual machine technologies throughout history.
Sharing-akka-pub
This document provides an introduction and overview of Akka and the actor model. It begins by discussing reactive programming principles and how applications can react to events, load, failures, and users. It then defines the actor model as treating actors as the universal primitives of concurrent computation that process messages asynchronously. The document outlines the history and origins of the actor model. It defines Akka as a toolkit for building highly concurrent, distributed, and resilient message-driven applications on the JVM. It also distinguishes between parallelism, which modifies algorithms to run parts simultaneously, and concurrency, which refers to applications running through multiple threads of execution simultaneously in an event-driven way. Finally, it provides examples of shared-state concurrency issues
Orchestrating Linux Containers while tolerating failures
lthough containers are bringing a refreshing flexibility when deploying services in production, the management of those containers in such an environment still requires special care in order to keep the application up and running. In this regard, orchestration platforms like Docker, Kubernetes and Nomad have been trying to alleviate this responsibility, facilitating the task of deploying and maintaining the entire application stack in its desired state. This ensures that a service will be always running, tolerating machine failures, network erratic behavior or software updates and downtime. The purpose of this talk is to explain the mechanisms and architecture of the Docker Engine orchestration platform (using a framework called swarmkit) to tolerate failures of services and machines, from cluster state replication and leader-election to container re-scheduling logic when a host goes down.
More Related Content
Viewers also liked
Unbundling the Future Web Runtime
The document discusses the integration of ES modules into JavaScript build tooling and how tools like SystemJS can support static and dynamic imports of ES modules and WebAssembly. It notes how the native module loader enables code sharing across pages and lazy loading, and how build tools can optimize modules through techniques like tree shaking, module splitting, and output generation while fully exposing the module graph to enable cross-module optimizations for better performance.
Swift Generics in Theory and Practice
Generics are one of the features of Swift that set it apart from many other languages. Since Swift debuted, developers have been wondering when to use them and what to use them for. By looking at the standard library, which is the first piece of software to use generics, we can better understand their purpose and utility.
The Value in Trees
Swift introduces new ways to model data through value types like structs and enums. I’ll discuss my experiences rewriting the data model of the vector graphics app Sketch to use value trees. I’ll finish off pondering whether future data modelling frameworks could be based on value trees, rather than entities and relationships, and introduce the experimental open source project Impeller (https://github.com/mentalfaculty/impeller), which is a step in that direction.
This talk was given at dotSwift 2017.
NoSQL Injections in Node.js - The case of MongoDB
This document discusses NoSQL injections in Node.js applications using MongoDB. It provides examples of how request body parameters can be used to alter MongoDB queries and presents best practices for validating user input to prevent injection attacks. These include using middleware to validate request data matches expected types and structures, or using libraries like Joi and Celebrate for schema-based validation. The document emphasizes that input validation is crucial to secure MongoDB queries from manipulation through user-supplied values.
The Ins and Outs of Easing - dotCSS 2016
The document discusses different techniques for easing or smoothing animation transitions, including linear easing with a constant rate of change, easing curves like ease in/out quad that accelerate or decelerate, Penner easing equations that precisely define different curve functions, cubic beziers that allow customizing the curve, lerping that calculates incremental changes each frame, and spring physics with damping that simulates a spring. It recommends different techniques based on the type of animation and provides resources for further information.
dotScale 2014
CTF links:
NotSoSecure CTF: http://ctf.notsosecure.com
Security Shepherd: https://www.owasp.org/index.php/OWASP_Security_Shepherd
http://hax.tor.hu/
https://pwn0.com/
http://www.smashthestack.org/
http://www.hellboundhackers.org/
http://www.overthewire.org/wargames/
http://counterhack.net/Counter_Hack/Challenges.html
http://www.hackthissite.org/
http://exploit-exercises.com/
http://vulnhub.com/
Content Security Policy - The application security Swiss Army Knife
This document discusses Content Security Policy (CSP), which defines a set of directives to allow websites to restrict resources the browser can load for increased security. It covers the basic CSP directives like default-src, script-src, and style-src for controlling what content is allowed to execute or render on a page. It also discusses additional CSP directives for features like forms, frames, and mixed content blocking. The document recommends starting with a basic CSP policy and then fine-tuning it by specifying allowed sources for each directive. It also provides examples of how CSP can help mitigate cross-site scripting attacks and reporting errors to help migrate sites from HTTP to HTTPS.
The jsdom
Two years ago I inherited jsdom, a project of breathtaking scope. In essence, it aims to simulate a browser in JavaScript, by implementing JavaScript versions of a wide variety of web standards (much more than just the DOM). While maintaining jsdom for the last two years, and eventually bringing it to a 1.0 release, I learned an incredible amount about the web. I want to share with you what I’ve learned: the history of the standards and implementations of them that make up the web; the interaction between seemingly-disparate parts of the platform; and all about the strange APIs we’ve come to know and love. You should walk away from this talk with a new appreciation for how browsers work—and how, through the ongoing effort of a community of contributors and package maintainers, we’ve been able to recreate one in pure JavaScript.
Viewers also liked (9)
Content Security Policy - The application security Swiss Army Knife
Content Security Policy - The application security Swiss Army Knife
Similar to Building Scalable Systems: What you can learn from Erlang - DotScale 2016
Actors evolved- Rotem Hermon
The actor model is a novel approach to writing concurrent software. It is based on the concept of small computational units communicating through asynchronous message passing, thus allowing concurrency and scalability while negating a lot of the problems of concurrent programming. Though the actor model got some adoption with the Erlang language and the Akka framework, it remained a rather niche approach and has not become a commonly used practice. But this may be changing now with the introduction of “Virtual Actors” - a new abstraction for writing distributed applications. This abstraction was introduced with the Orleans framework by Microsoft and adopted to Java by EA with their Orbit framework. This talk will include a short introduction to the actor model. We will then explore the Virtual Actors model, how it’s different from the classic model, and why it makes distributed application programming a lot simpler.
Intro to elixir and phoenix
This document provides an overview of Elixir and the Phoenix framework. It discusses how Elixir runs on the Erlang VM and inherits properties like high availability and distribution. Phoenix is introduced as a web framework for Elixir that focuses on productivity, reliability, and speed. Key Phoenix concepts like the request pipeline, router, controllers, models and views are briefly outlined.
Akka Fundamentals
This document provides an overview of Akka fundamentals including:
- The actor model which uses message passing between encapsulated state and behavior units to achieve concurrency
- The Akka actor API for building fault tolerant distributed applications in Scala and Java
- Fault tolerance techniques in Akka like supervision which allows actors to monitor and respond to failures of child actors
- Routing which provides abstractions for distributing messages to multiple receiver actors
Why Play Framework is fast
Typesafe trainer and consultant Will Sargent describes just how Play Framework is so "fast" for Java and Scala production apps.
More Play, Akka, Scala and Apache Spark webinars, presentations, and videos:
http://typesafe.com/resources/videos
VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang: two very promising beasts, made for the new parallel world, but still lingering in the wings. Not only are they addressing todays challenges but they are using parallel architectures as corner stone of their new and surprising approach to be faster and more productive. What are they good for? Why are we working to team them up?
Erlang promises faster implementation, way better maintenance and 4 times shorter code. VoltDB claims to be two orders of magnitude faster than its competitors. The two share many similarities: both are the result of scientific research and designed from scratch to address the new reality of parallel architectures with full force.
This talk presents the case for Erlang as server language, where it shines, how it looks, and how to get started. It details Erlang's secret sauce: microprocesses, actors, atoms, immutable variables, message passing and pattern matching. (Note: for a longer version of this treatment of Erlang only see: Why Erlang? http://www.slideshare.net/eonblast/why-erlang-gdc-online-2012)
VoltDB's inner workings are explained to understand why it can be so incredibly fast and still better than its NoSQL competitors. The well publicized Node.js benchmark clocking in at 695,000 transactions per second is described and the simple steps to get VoltDB up and running to see the prodigy from up close.
Source examples are presented that show Erlang and VoltDB in action.
The speaker is creator and maintainer of the Erlang VoltDB driver Erlvolt.
Introduction to Actor Model and Akka
The slides of the speech I given during coscup 2011. The topic is "Programming for the Future, Introduction to Actor Model and Akka"
applicationlayer.pptx
This document summarizes key points from Chapter 7 of Andrew Tanenbaum's book "Computer Networks" which covers the application layer of the OSI model. It discusses the Domain Name System (DNS) which maps domain names to IP addresses, electronic mail (email) protocols and formats, and the basic architecture of email systems including user agents and message transfer. Real applications like the World Wide Web and multimedia are also mentioned.
Evented Ruby VS Node.js
This document discusses evented programming in Node.js and Ruby. It explains that evented programming uses callbacks and asynchronous non-blocking I/O. Node.js uses this approach to improve concurrency over blocking I/O models. Ruby can also implement evented programming using libraries like EventMachine that provide asynchronous abstractions while keeping app code procedural. The document provides examples of building evented applications in both languages.
Scalable game-servers-tgc
Scalable Game Servers talk I gave at TGC 2017
The slides describe the case for building distributed systems using a statteful architecture to achieve high throughput and low latency using Microsoft Orleans as the best provider and inventor of the virtual actor model.
http://github.com/dotnet/orleans
Fastest Servlets in the West
The venerable Servlet Container still has some performance tricks up its sleeve - this talk will demonstrate Apache Tomcat's stability under high load, describe some do's (and some don'ts!), explain how to performance test a Servlet-based application, troubleshoot and tune the container and your application and compare the performance characteristics of the different Tomcat connectors. The presenters will share their combined experience supporting real Tomcat applications for over 20 years and show how a few small changes can make a big, big difference.
VIRTUAL MACHINE VERSATILE PLATFORM01~chapter 1 (1).ppt
This document provides an introduction to virtual machines. It defines virtual machines as software implementations that emulate an entire physical computer system. It describes two main types of virtual machines - system virtual machines that virtualize an entire operating system and process virtual machines that virtualize individual applications. The document outlines the advantages of virtual machines in enabling hardware and software innovation as well as the ability to run different operating systems on the same physical machine. It also provides examples of various virtual machine technologies throughout history.
Sharing-akka-pub
This document provides an introduction and overview of Akka and the actor model. It begins by discussing reactive programming principles and how applications can react to events, load, failures, and users. It then defines the actor model as treating actors as the universal primitives of concurrent computation that process messages asynchronously. The document outlines the history and origins of the actor model. It defines Akka as a toolkit for building highly concurrent, distributed, and resilient message-driven applications on the JVM. It also distinguishes between parallelism, which modifies algorithms to run parts simultaneously, and concurrency, which refers to applications running through multiple threads of execution simultaneously in an event-driven way. Finally, it provides examples of shared-state concurrency issues
Orchestrating Linux Containers while tolerating failures
lthough containers are bringing a refreshing flexibility when deploying services in production, the management of those containers in such an environment still requires special care in order to keep the application up and running. In this regard, orchestration platforms like Docker, Kubernetes and Nomad have been trying to alleviate this responsibility, facilitating the task of deploying and maintaining the entire application stack in its desired state. This ensures that a service will be always running, tolerating machine failures, network erratic behavior or software updates and downtime. The purpose of this talk is to explain the mechanisms and architecture of the Docker Engine orchestration platform (using a framework called swarmkit) to tolerate failures of services and machines, from cluster state replication and leader-election to container re-scheduling logic when a host goes down.
You Can Run but You Can’t Read: Preventing Disclosure Exploits in Executable ...
An unofficial slides for paper "You Can Run but You Can’t Read: Preventing Disclosure Exploits in Executable Code".
All contents come from my understanding on this paper. So it may contain errors. If you see it, kindly point out, and let me know, thanks.
Akka.Net Overview
An overview of Akka.Net
- Intro to Actor Model
- Asynchronous Design
Actor Lifecycle
- Actor Hierarchies
- Location transparencies
- Configuration
- Clustering
- Routing
- Persistence
P05-slides
This document provides an overview of using jail(8) on FreeBSD to host virtual servers for an Internet Service Provider (ISP). It discusses how jail(8) isolates processes and users, similar to chroot but with additional network bindings. It also describes some of the technical and management challenges an ISP faces in scaling jail-based virtual hosting, such as monitoring disk/memory usage, restricting processes, and handling security issues like fork bombs. The document uses the example of iMeme, one of the first ISPs to extensively use jail(8) hosting, and how it addressed these challenges.
Wars I’ve Seen From Java EE to Spring and more, Azure has you covered
Ed Burns discusses his career journey in software development and how the cloud has disrupted the industry. He began working on early web browsers and frameworks and witnessed "wars" between competing technologies. The rise of cloud computing inserted major vendors into the ecosystem and changed how software is developed and deployed. Azure aims to support all types of Java workloads throughout the development lifecycle and provides fully managed services for Spring Boot applications.
Alternate concurrency models
This document discusses alternate concurrency models including the actor model and software transactional memory (STM). It provides overviews of common problems with multi-threading like atomicity, visibility and race conditions. The actor model is described as involving autonomous units that communicate asynchronously via message passing without shared state. STM is analogous to database transactions for controlling concurrency in shared memory through atomic reads and writes. Examples of actor model applications and STM implementations in Java are also mentioned.
From 100s to 100s of Millions
Urban Airship is a mobile platform that provides services to over 160 million active application installs across 80 million devices. They initially used PostgreSQL but needed a system that could scale writes more easily. They tried several NoSQL databases including MongoDB, but ran into issues with MongoDB's locking, long queries blocking writes, and updates causing heavy disk I/O. They are now converging on Cassandra and PostgreSQL for transactions and HBase for analytics workloads.
Performance testing meets the cloud - Artem Shendrikov
Yet another way to run distributed performance/load tests on AWS: Jmeter + Docker + Terraform
Who is Artem?
A Test Automation Engineer at SoftServe with 5 years of IT experience overall and about 3 years dealing with test automation (Web UI, Mobile, API and performance tests).
Similar to Building Scalable Systems: What you can learn from Erlang - DotScale 2016 (20)
VIRTUAL MACHINE VERSATILE PLATFORM01~chapter 1 (1).ppt
VIRTUAL MACHINE VERSATILE PLATFORM01~chapter 1 (1).ppt
Orchestrating Linux Containers while tolerating failures
Orchestrating Linux Containers while tolerating failures
You Can Run but You Can’t Read: Preventing Disclosure Exploits in Executable ...
You Can Run but You Can’t Read: Preventing Disclosure Exploits in Executable ...
Wars I’ve Seen From Java EE to Spring and more, Azure has you covered
Wars I’ve Seen From Java EE to Spring and more, Azure has you covered
Performance testing meets the cloud - Artem Shendrikov
Performance testing meets the cloud - Artem Shendrikov
More from Mickaël Rémond
Go for Real Time Streaming Architectures - DotGo 2017
Streaming applications are at the core of realtime information system. With microservices streaming components are becoming increasingly important.
With the rise of cloud architecture, it is now easier and more elegant to use Go to write realtime high-performance components for streaming architecture than to use traditional tools.
Mickaël explains what are streaming architectures and demonstrates the benefits of using Go to build such platform.
Phoenix Presence: Le service temps réel de Phoenix - Paris.ex #8
Mickaël Rémond présente l'implementation d'une application de chat simple sur le web à l'aide de Phoenix Presence.
Pour réaliser cette application d'exemple, une migration de Phoenix vers Phoenix 1.2 a été effectuée. Toutes les étapes de la migration sont présentés et peuvent aider les développeurs a effectuer les bonnes étapes pour mettre à jour leur application Phoenix.
Fighting XMPP abuse and spam with ejabberd - ejabberd Workshop #1
Mickaël Rémond explains the features, options and API that can be used to protect ejabberd servers from spam and various other types of attacks.
Managing ejabberd Platforms with Docker - ejabberd Workshop #1
Christophe Romain (ProcessOne) introduces Docker and ProcessOne work in progress to support Docker-based ejabberd deployments.
Messaging temps réel avec Go
Présentation pour le Meetup Go Paris (Juin 2016): Créer des objets connectés en Go avec XMPP et MQTT
Property-based testing of XMPP: generate your tests automatically - ejabberd ...
Property-based testing uses properties of software to automatically generate test cases rather than writing tests manually. It can test combinations of features that would be impractical to test manually. QuickCheck is used to generate random inputs based on properties and shrink failing tests to minimal cases. The document provides an example of property-based testing of an XML parser by generating random XML and checking it roundtrips through parsing and pretty printing. It discusses challenges like invalid characters and improving the generator to handle attributes and namespaces properly. Larger examples like testing Ejabberd hooks against specifications are also mentioned.
IoT Studio #1: Protocols introduction and connected jukebox
This is the slides ProcessOne IoT Studio session #1.
At the core of the discussion is an introduction to Internet of Things protocols.
Here are the topics / questions that I have been addressing:
- What are the main classes of protocols for the Internet of Things ?
- What are the strengths of XMPP for the Internet of things ?
- What are the main XMPP Extensions for building the Internet of Things ?
- How can I leverage XMPP to build a real device / appliance ?
As such, the session ends with a hands-on demonstration of a Soundcloud connected Jukebox built with:
- Raspberry Pi 2
- Connected to ejabberd
- Client is developed in Go using Go XMPP library Gox
Video of the session is available on Youtube: https://www.youtube.com/watch?v=YtommTbCxCc
Deep Dive Into ejabberd Pubsub Implementation
Christophe Romain goes into the details of ejabberd Pubsub implementation. He explains the Pubsub plugin systems and how to leverage it to optimize ejabberd Pubsub for your own use cases.
This talk will teach you how to get more performance and scalability from your Pubsub implementation.
XMPP Academy #3
During this XMPP academy session I talk about Google and Apple Push Notifications, XEP-0357, Websockets, XMPP playground and Message Archiving.
Full session can be seen on Youtube: https://www.youtube.com/watch?v=LToKLTf-N_E
XMPP Academy #2
- Backends in ejabberd are pluggable modules that store data to provide to feature modules. This allows ejabberd to be more stateless. Common backends include Mnesia, MySQL, PostgreSQL, Riak, and custom backends.
- Message carbons allow messages to be copied to all connected clients so the full conversation history is visible, but they do not work in MUC rooms due to the anonymous nature of MUC chats.
- To learn XMPP, it is best to start at the protocol level by using a client like PSI that allows inspecting and crafting XMPP packets directly rather than starting from libraries that abstract the protocol.
2015: L'année d'Elixir, Code, écosystème et communauté
ElixirParis Meetup: Cette présentation fait le point sur l'écosystème et la communauté Elixir et explique les raisons du succès.
XMPP Academy #1
This is the slide deck for ProcessOne first live XMPP Academy.
Here are the questions covered:
1. ejabberd SaaS architecture questions
- What is the best way to archive user messages if we do not want to sync data from user device?
- Why does ejabberd SaaS not use async mechanisms for archiving messages to customer back-end server?
- Mobile XMPP support: Explain standby, push and detached modes.
2. XMPP / ejabberd questions
- How does ejabberd internally store messages which are not yet delivered?
- How are privacy lists managed in ejabberd?
- What is on the ejabberd roadmap ? OAuth !
Archipel Introduction - ejabberd SF Meetup
This is slide deck for talk given by Archipel Project Team, with Nicolas Ochem as speaker. It introduces the concept behind Archipel, a solution to manage and supervise virtual machines based on XMPP.
A vision for ejabberd - ejabberd SF Meetup
This is Mickaël Rémond presentation from ejabberd San Francisco Bay Area Meetup in march 2015. He presents latest achievements and progress made on ejabberd during the last few months. He then share his vision about the next steps to bring ejabberd XMPP platform to the next level.
Multitasking in iOS 7
This document discusses background capabilities in iOS 7. It summarizes the three new background modes available: background fetching, remote notifications, and background transfers. Background tasks now run for 3 minutes instead of 10 minutes. Apps can now download content in the background in response to push notifications or on a scheduled basis using background fetch APIs. Developers have close control while Apple prioritizes battery life and network usage.
ProcessOne Push Platform: XMPP-based Push Solutions
XMPP can provide a flexible and scalable solution for real-time push notifications across devices and platforms. ProcessOne offers an XMPP-based push platform as a service to enable reliable delivery of notifications to users. Case studies demonstrate how the platform supports use cases like radio program updates, social media feeds, and mobile applications. ProcessOne's expertise in XMPP pubsub helps make these services highly scalable and able to support new features over time.
WaveOne server and client by ProcessOne
WaveOne is ProcessOne client and server implementation of Google Wave protocol. It allows real time collaboration on the web.
Real time Web Application with XMPP and Wave
This is the slide of my talk given at Erlang Factory 2010 in San Francisco. It introduces XMPP, Wave and ProcessOne Wave server implementation, to build realtime web collaborative platform.
Erlang White Label
Geoff Cant Whitelabel Erlang covers the trials, tribulations and technologies used in building a whitelabel ejabberd hosting service.
The talk explains the rationale behind the choices of web server, templating system, email module, page optimizers and other web framework components. The talk also covers the documented and undocumented DNS query and caching modules in OTP and introduces 'dns' - a convenient new API for using them. Finally, the talk describes some tips and tricks useful for whitelabel hosting solutions that require customer DNS updates.
OneTeam Media Server
This presentation describes progress on ProcessOne new product, OneTeam Media Server. This is a Flash Media Server used to write interactive, collaborative and real time Flash applications.
More from Mickaël Rémond (20)
Go for Real Time Streaming Architectures - DotGo 2017
Go for Real Time Streaming Architectures - DotGo 2017
Phoenix Presence: Le service temps réel de Phoenix - Paris.ex #8
Phoenix Presence: Le service temps réel de Phoenix - Paris.ex #8
Fighting XMPP abuse and spam with ejabberd - ejabberd Workshop #1
Fighting XMPP abuse and spam with ejabberd - ejabberd Workshop #1
Managing ejabberd Platforms with Docker - ejabberd Workshop #1
Managing ejabberd Platforms with Docker - ejabberd Workshop #1
Property-based testing of XMPP: generate your tests automatically - ejabberd ...
Property-based testing of XMPP: generate your tests automatically - ejabberd ...
IoT Studio #1: Protocols introduction and connected jukebox
IoT Studio #1: Protocols introduction and connected jukebox
2015: L'année d'Elixir, Code, écosystème et communauté
2015: L'année d'Elixir, Code, écosystème et communauté
ProcessOne Push Platform: XMPP-based Push Solutions
ProcessOne Push Platform: XMPP-based Push Solutions
Recently uploaded
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
How to Get CNIC Information System with Paksim Ga.pptx
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
HCL Notes and Domino License Cost Reduction in the World of DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
UiPath Test Automation using UiPath Test Suite series, part 5
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.Microsoft - Power Platform_G.Aspiotis.pdf
Revolutionizing Application Development
with AI-powered low-code, presentation by George Aspiotis, Sr. Partner Development Manager, Microsoft
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Removing Uninteresting Bytes in Software Fuzzing
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Full-RAG: A modern architecture for hyper-personalization
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Recently uploaded (20)
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Building Scalable Systems: What you can learn from Erlang - DotScale 2016
- 1. Building Scalable Systems What you can Learn from Erlang
- 4. Actor Model • Carl Hewitt
- 5. Actor Model Principle • An actor can: • Send Messages to other Actors • Create other Actors • Process Next Message
- 7. “An Island of Sanity in a Sea of Concurrency" Actor State Actor State Actor State Actor State Actor State
- 10. Birth of Erlang Joe Armstrong, Robert Virding, Mike Williams, Bjarne Däcker
- 11. Share Nothing with Lightweight Processes
- 12. Let it Crash
- 13. Handle Failures with Process Links and Supervisors
- 15. Next stop: 1998
- 16. Erlang VM released as Open Source
- 17. Erlang strength lies in the VM
- 18. Vertical scaling: Multicore and Scheduler
- 20. Erlang strengths match Server applications properties • Server apps are most of the time IO bound • Server apps are often memory bound.
- 21. Network Stack Accepted connections Requests processing Server threads Pool Thread Thread Thread Thread Thread Thread Thread Thread Thread … Client Requests X X X Connections Pool i.e. Apache
- 22. Network Stack Accepted connections Requests processing Lightweight processes Client Requests … One Connection per Thread Erlang webservers i.e. Yaws, Cowboy, etc.
- 23. Design comparison • Number of requests handled in peak time: 1Ks to 10ks • Throughput: reduce quickly when max concurrency reached • More complex code • Number of concurrent requests handle in peak time: 100Ks to 1Ms • Throughput: Higher for large number of concurrent requests. • Simple code, no pool to manage Design with worker thread pools Design with Erlang processes
- 24. Back to 2016
- 25. Erlang is still the Reference for Scalable Systems
- 26. Makes it easier to build Advanced Concurrency Patterns
- 27. New languages using the Erlang VM: Elixir, LFE, etc
- 28. Inspiration for new languages / Frameworks: Go, Scala, Akka, Clojure, …
- 29. Next Stop ?