This whitepaper features the transition from traditional networking to software-defined networking or SDN. Find outlines of next-generation architectures.
Data center interconnect seamlessly through SDNFelecia Fierro
Data Center Interconnect Seamlessly through SDN
Data Center Interconnect, or DCI, has become a hot topic as IT infrastructures transform from islands of connectivity to pools of resources for efficiency purposes. Properly deployed, DCI enables all computing and storage resources to be pooled, regardless of where they physically reside, and it is the quality of this abstraction and the associated visibility that counts.
Enter Pluribus Networks running on Broadcom chipsets
Join us for this On-Demand Webinar where we will discuss why Data Center Interconnect is a key opportunity to simplify any network and how Broadcom chipsets enable this with their industry-leading VXLAN capabilities.
Pluribus Netvisor®-powered switches running on Broadcom include the industry’s most powerful and open DCI technology, VXLAN, which enables all resources across the entire planet to be shared. Along with VXLAN itself, we will explain the role of visibility in a widely distributed VXLAN based environment.
In this On-Demand Webinar, we'll discuss how DCI running on Broadcom VXLAN can:
Share IT resources and increase utilization of those resources
Provide enterprise scale, simplicity and agility – reducing the cost and complexity of IT
Support modern applications including Converged Infrastructure and VDI
This whitepaper features the transition from traditional networking to software-defined networking or SDN. Find outlines of next-generation architectures.
Data center interconnect seamlessly through SDNFelecia Fierro
Data Center Interconnect Seamlessly through SDN
Data Center Interconnect, or DCI, has become a hot topic as IT infrastructures transform from islands of connectivity to pools of resources for efficiency purposes. Properly deployed, DCI enables all computing and storage resources to be pooled, regardless of where they physically reside, and it is the quality of this abstraction and the associated visibility that counts.
Enter Pluribus Networks running on Broadcom chipsets
Join us for this On-Demand Webinar where we will discuss why Data Center Interconnect is a key opportunity to simplify any network and how Broadcom chipsets enable this with their industry-leading VXLAN capabilities.
Pluribus Netvisor®-powered switches running on Broadcom include the industry’s most powerful and open DCI technology, VXLAN, which enables all resources across the entire planet to be shared. Along with VXLAN itself, we will explain the role of visibility in a widely distributed VXLAN based environment.
In this On-Demand Webinar, we'll discuss how DCI running on Broadcom VXLAN can:
Share IT resources and increase utilization of those resources
Provide enterprise scale, simplicity and agility – reducing the cost and complexity of IT
Support modern applications including Converged Infrastructure and VDI
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
#VirtualDesignMaster 3 Challenge 3 – James Brownvdmchallenge
While things on Mars have been going well, since we now have multiple options for our infrastructure, the fact remains that we are working on the colonization of a foreign planet.
Talon systems - Distributed multi master replication strategySaptarshi Chatterjee
Data Replication is the process of storing data in more than one site or node.It is useful in improving the availability of data. The result is a distributed database in which users can access data relevant to their tasks without interfering with the work of others.Data Replication is generally performed to provide a consistent copy of data across all the database nodes.
Traditionally it’s done by copying data from one database server to another, so that all the servers can have the same data. Our implementation, proposes a completely different approach. Instead of copying data from one node to another, in our design , master replicas do not directly communicate between each other and work virtually independently for write queries. For read queries, an independent process consults all the replicas to constitute a quorum. and returns the result if a majority of the machines in the system response with the same result
In this lecture we analyze key-values databases. At first we introduce key-value characteristics, advantages and disadvantages.
Then we analyze the major Key-Value data stores and finally we discuss about Dynamo DB.
In particular we consider how Dynamo DB: How is implemented
1. Motivation Background
2. Partitioning: Consistent Hashing
3. High Availability for writes: Vector Clocks
4. Handling temporary failures: Sloppy Quorum
5. Recovering from failures: Merkle Trees
6. Membership and failure detection: Gossip Protocol
Micro Processors Present Technology and Up gradations Requiredijtsrd
In this paper we will deal with the current technology being used in microprocessor. We will analyse both the hardware and the software and their interfacing with each other and How to better them to increase speed and reduce size. Assembly language with its constituent syntax in NASM on Linux Operating System will be discussed and we will show how it can be used to be executed with minimum time gap. Assembler will be discussed and user friendly high level language to be used as fast as machine language will be discussed with recommendations. We will be also comparing and contrasting various micro processor, their architecture and speed with each other so that to highlight advantages of each over other and also to suggest issues and their respective solutions. Vividh Bansal "Micro-Processors: Present Technology and Up-gradations Required" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-6 | Issue-6 , October 2022, URL: https://www.ijtsrd.com/papers/ijtsrd52123.pdf Paper URL: https://www.ijtsrd.com/engineering/electronics-and-communication-engineering/52123/microprocessors-present-technology-and-upgradations-required/vividh-bansal
Concurrency and Parallelism, Asynchronous Programming, Network ProgrammingPrabu U
The presentation starts with concurrency and parallelism. Then the concepts of reactive programming is covered. Finally network programming is detailed
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
#VirtualDesignMaster 3 Challenge 3 – James Brownvdmchallenge
While things on Mars have been going well, since we now have multiple options for our infrastructure, the fact remains that we are working on the colonization of a foreign planet.
Talon systems - Distributed multi master replication strategySaptarshi Chatterjee
Data Replication is the process of storing data in more than one site or node.It is useful in improving the availability of data. The result is a distributed database in which users can access data relevant to their tasks without interfering with the work of others.Data Replication is generally performed to provide a consistent copy of data across all the database nodes.
Traditionally it’s done by copying data from one database server to another, so that all the servers can have the same data. Our implementation, proposes a completely different approach. Instead of copying data from one node to another, in our design , master replicas do not directly communicate between each other and work virtually independently for write queries. For read queries, an independent process consults all the replicas to constitute a quorum. and returns the result if a majority of the machines in the system response with the same result
In this lecture we analyze key-values databases. At first we introduce key-value characteristics, advantages and disadvantages.
Then we analyze the major Key-Value data stores and finally we discuss about Dynamo DB.
In particular we consider how Dynamo DB: How is implemented
1. Motivation Background
2. Partitioning: Consistent Hashing
3. High Availability for writes: Vector Clocks
4. Handling temporary failures: Sloppy Quorum
5. Recovering from failures: Merkle Trees
6. Membership and failure detection: Gossip Protocol
Micro Processors Present Technology and Up gradations Requiredijtsrd
In this paper we will deal with the current technology being used in microprocessor. We will analyse both the hardware and the software and their interfacing with each other and How to better them to increase speed and reduce size. Assembly language with its constituent syntax in NASM on Linux Operating System will be discussed and we will show how it can be used to be executed with minimum time gap. Assembler will be discussed and user friendly high level language to be used as fast as machine language will be discussed with recommendations. We will be also comparing and contrasting various micro processor, their architecture and speed with each other so that to highlight advantages of each over other and also to suggest issues and their respective solutions. Vividh Bansal "Micro-Processors: Present Technology and Up-gradations Required" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-6 | Issue-6 , October 2022, URL: https://www.ijtsrd.com/papers/ijtsrd52123.pdf Paper URL: https://www.ijtsrd.com/engineering/electronics-and-communication-engineering/52123/microprocessors-present-technology-and-upgradations-required/vividh-bansal
Concurrency and Parallelism, Asynchronous Programming, Network ProgrammingPrabu U
The presentation starts with concurrency and parallelism. Then the concepts of reactive programming is covered. Finally network programming is detailed
Dynamic Load Calculation in A Distributed System using centralized approachIJARIIT
The building of networks and the establishment of communication protocols have led to distributed systems, in which computers that are linked in a network cooperate on a task. The task is divided by the master node into small parts (sub problems) and is given to the nodes of the distributed system to solve, which gives better performance in time complexity to solve the problem compared to the time required to solve the problem in a single machine. Load balancing is the process of redistributing the work load among nodes of the distributed system to improve both resource utilization and job response time while also avoiding a situation where some nodes are heavily loaded while others are idle or doing little work. So before sending these parts of problem by the master to the nodes, master node should know the actual work load of all the nodes. We try a dynamic approach to find out the work load of each participating nodes in the distributed system by the master before sending the parts of the problem to the nodes.
This paper describes an algorithm which runs in the master machine and collects information from the nodes of the distributed system (client server application) and calculates the current work load of the nodes of the distributed system. The algorithm is developed in such a way that it can calculate the loads of the nodes dynamically. This means the loads can be evaluated if new nodes are added or deleted or during current performance of the nodes. The whole system is implemented on linux machine and local area network.
Optimization of Remote Core Locking Synchronization in Multithreaded Programs...ITIIIndustries
This paper proposes the algorithms for optimization of Remote Core Locking (RCL) synchronization method in multithreaded programs. The algorithm of initialization of RCL-locks and the algorithms for threads affinity optimization are developed. The algorithms consider the structures of hierarchical computer systems and non-uniform memory access (NUMA) to minimize execution time of RCLprograms. The experimental results on multi-core computer systems represented in the paper shows the reduction of RCLprograms execution time.
1. Software-Defined Networks (SDN) is a new paradigm in network ma.docxjackiewalcutt
1. Software-Defined Networks (SDN) is a new paradigm in network management that adds another layer (i.e., Network Operating System) to the architecture. Answer the following questions in the context of SDN with your reasoning.
(a) Is it scalable? Why?
(b) Is it less responsive? Why?
(c) Does it create a single point of failure? Why?
(d) Is it inherently less secure? Why?
(e) Is it incrementally deployable? Why?
2.RED randomly drops packets when it experience congestion. The probability of drop increases as the average queue size increases.
(a) Does it do a better job for uniform or bursty traffic? and why?
(b) Does it drop packets from the head of the queue or from the tail of the queue? and why?
(c) Does it make any difference; head/tail drop? and why?
3. Carefully read the short article OpenFlow: A Radical New Idea in Networking (http://queue.acm.org/detail.cfm?id=2305856), and answer the following questions.
The author argues that the deployment of SDN in general and OpenFlow in specific towards network democratization is a crazy idea. Do you agree? If yes, how come SDN has been supported and being deployed by many networking vendors. If not, give one scenario that SDN could cause disruptions.
NET WORKS
1
OpenFlow:
A Radical New Idea in Networking
An open standard that enables software-defined networking
Thomas A. Limoncelli
Computer networks have historically evolved box by box, with individual network elements
occupying specific ecological niches as routers, switches, load balancers, NATs (network address
translations), or firewalls. Software-defined networking proposes to overturn that ecology, turning
the network as a whole into a platform and the individual network elements into programmable
entities. The apps running on the network platform can optimize traffic flows to take the shortest
path, just as the current distributed protocols do, but they can also optimize the network to
maximize link utilization, create different reachability domains for different users, or make device
mobility seamless.
OpenFlow, an open standard that enables software-defined networking in IP networks, is a new
network technology that will enable many new applications and new ways of managing networks.
Here are three real, though somewhat fictionalized, applications:
EXAMPLE 1: BANDWIDTH MANAGEMENT. A typical wide area network has 30 percent utilization;
it must “reserve” bandwidth for “burst” times. Using OpenFlow, however, a system was developed
in which internal application systems (consumers) that need bulk data transfer could use the
spare bandwidth. Typical uses include daily replication of datasets, database backups, and the
bulk transmission of logs. Consumers register the source, destination, and quantity of data to
be transferred with a central service. The service does various calculations and sends the results
to the routers so they know how to forward this bulk data when links are otherwise unused.
Communication ...
Just like you can't defeat the laws of physics there are natural laws that ultimately decide software performance. Even the latest technology beta is still bound by Newton's laws, and you can't change the speed of light, even in the cloud!
Hardback solution to accelerate multimedia computation through mgp in cmpeSAT Publishing House
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Yet another talk :)
This time in a lecture called “Markup Languages und Anwendungen”
Went pretty ok, but as usual I talked a little bit too fast ^_^
http://www.marc-seeger.de
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Eventdriven I/O - A hands on introduction
1. Event-Driven I/O
A hands-on introduction
Marc Seeger
HdM Stuttgart
July 13, 2010
Abstract
This paper gives a short introduction to the topic of asynchronous,
event-driven I/O, explains the underlying "reactor" pattern and introduces
the usage of lightweight threads to help with readability of event-driven
code.
1
3. 1 Introduction
When it comes to application development, there are a few things that separate
regular single-user desktop applications from server applications. The one
main thing however is, that server applications tend to have many concurrent
users while desktop applications can usually be designed for a single user at
any given time.
One of the most important differences is the way that the applications handles
data input and output. Desktop Applications usually receive data over the
network or load files from disk using "synchronous" or "blocking" I/O. This
is usually the easy way and is fine for a single user. As soon as an application
has to interact with several users or large amounts of input/output, using an
non-blocking I/O becomes the only way of keeping the application responsive
and allowing it to scale.
2 General scalability problems
When trying to create a single service that will scale for thousands of incoming
requests, there are two main things that influence the performance of your
application: Computation and I/O.
2.1 Computations
Computations are the part of your application that actually use most of the
available CPU. They are the functional components that transform data from
one form into another. This can be compressing images, translating text into
other languages, sorting numbers according to size.
The main problem is that your server’s CPU has only a limited amount of
operations per second. Your algorithm needs a certain number of cycles for
every bit of input. Once you max out this ratio, your server is at peak capacity
and every added request will have to wait for the other ones to finish.
The simplest solution to this problem is adding more and faster hardware to
the system in question. You can buy faster CPUs to be able to process more
cycles per second. The problem is that the cost and the amount of additional
scalability you can gather is very limited by the manufacturing process and
clockspeeds of current CPUs. There is usually only little difference between
the clock speed of a regular CPU and a high-end CPU. There is however
a difference in the amount of available cores between the regular (2) and
the high end (8) Processors. This leads us to the next way to solve the
computational problem. A solution that is a little bit more sophisticated
2
4. is the partitioning of data by the algorithm. This allows each part of the
split-up computation to be processed by a different core of the CPU. A simple
assumption would be, that an 8 core CPU will now solve a given problem
4 times as fast as a dual-core CPU. This is however a bit oversimplified. If
you’re interested in this topic, I can recommend Kai Jäger’s Bachelor Thesis
"Finding parallelism - How to survive in a multi-core world" (2008) [2].
Taking the parallelization to the next step means that we distribute the
problem not only between several cores of a CPU, but between several
networked computers and thus have a simple way of scaling our system
by adding additional machines to the network. A generalized approach to
this problem has been introduced by Jeffrey Dean and Sanjay Ghemawat
(both employed by Google) in a paper called "MapReduce: Simplified Data
Processing on Large Clusters". If the scalability of the application is CPU
bound, transforming the applications different tasks into MapReduce elements
might be the best way to go. An open-source set of frameworks that (among
other things) implements MapReduce and helps with coordination and network
interaction between the processing nodes is called Apache Hadoop. It is widely
used and worth a look if your problem is the processing of vast amounts of
data on a large amount of machines/cores.
2.2 I/O
The other problem that keeps applications from scaling is Input and Output
of data to attached devices. These devices can be network cards, harddiscs
or anything else that isn’t directly connected to the reasonably fast RAM or
the CPU.
Using MapReduce helps to process a large amounts of data in a short amount
of time and thus solves the computation problem, but it doesn’t help with
large numbers of small operations.
The single pieces of data are usually pretty easy to process and distributing
the single piece of data among more than one node, sometimes even among
more than one CPU core, isn’t usually worth the overhead that comes with it.
Most of the time between the start of the request and the result is actually
being spent waiting for data to be copied in and out of buffers and checking
if the response has arrived back yet. A simple, single-threaded application
would completely lock down while it waits for the harddisc to read a file, the
network card to collect the whole request packet, the database to answer a
query or a user to push a button. Even multi-threaded applications suffer
from problems when it comes to I/O. Every time an application is waiting for
an I/O request using a thread, it doesn’t block the rest of the application. It
does however add another piece of memory to your applications footprint and
3
5. depending on the implementation, might cause a lot of unnecessary switches
between threads that add a lot of overhead to the whole process.
This is why it is important to look at the way a single server process can
handle incoming data without locking down completely.
3 Threads: Concurrency the old fashioned way
To understand what concurrency using threads does, one has to understand
the process an application has to go through, to offer its services over the
network.
An application has to attach itself to a certain TCP/UDP port. This way,
the operating system will pass all incoming data that is addressed to that
port to the application listening on it. To create a simple Service, one doesn’t
have to do much.
As a little example, here is a little server that does echo back everything that
is being sent to it:
require ’socket’
server = TCPServer.new(2803)
while client = server.accept
input = client.readline
client.write "You said: #{input}"
client.close
end
This application will run to the server.accept call which is "blocking". This
means that the application will stop until somebody actually opens up a
connection to the specified port (2803 in this case). This application does
a pretty good job as long as there is only one user at a time. As soon as 2
users want to connect to the server at the same time, one of them will have
to wait for the server to message back to the first user.
The problem in the simple solution is, that everything that happens between
"server.accept" and "end" will keep our server from servicing a second
connection. This is where Threads come in.
Since a thread usually returns immediately and does its work somewhere in
the background, one can simply but everything between "server.accept" and
"end" into a thread, spawn it off and immediately start working on the next
connection that came in.
The modifications to our application are minimal:
4
6. require ’socket’
server = TCPServer.new(2803)
loop do
Thread.new(server.accept){ |client|
input = client.readline
client.write "You said: #{input}"
client.close
}
end
The only thing one has to pay attention to is, that the application has to be
"thread-safe" now. This means that whatever data your application changes
has to be secured by some kind of locking. Otherwise, two threads that
run in parallel would be able to produce inconsistent data in the backend
or end up in a race-condition. In our example, we don’t change any data
besides the output that gets echoed back to the user. All of the "state" of the
thread is embedded in the thread itself. This is comparable to the main ideas
behind functional programming and allows us to run these threads without
any locking backend data.
This approach isn’t without problems. For each request, a new thread has
to be spawned up, kept in memory and managed by the operating system.
This leads to a situation in which you have a 1:1 ratio between threads and
users and are limited by the ability of your operating system to balance these
threads and the RAM it takes to keep them in memory. While this allows a
server that only does lightweight operations in the threads (e.g. an HTTP
server) to scale for a while, at a few thousand concurrent connections, the
sheer management of the threads becomes a problem and the performance
drops.
4 Reactor and Proactor
There are two design patterns that try to solve the problem of having a
1:1 ratio between threads and users. Reactor and proactor are both I/O
multiplexers. They form a central instance and allow a programmer to register
event-handlers for specific I/O events. They employ an event demultiplexer
that calls the registered handlers whenever the event occurs. This means
that the programmer doesn’t have to keep all of his threads in memory at
the same time. The application logic for dealing with the I/O responses is
executed when the response is actually there and not beforehand. This keeps
the amount of switching between threads and blocking for results close to
5
7. zero.
The difference between reactor and proactor are the events that are being
monitored.
In the (synchronous) reactor pattern, the programmer registers callbacks
on "ready to read/write" events. Whenever one of these events happens,
the callback is being invoked and the application logic reads from or writes
to the available file descriptor. It has to be noted that file descriptors in
Unix can either be an actual file on the file system or a representation of a
network-socket.
In the (asynchronous) proactor pattern, the programmer starts an asynchronous
write/read operation by executing the operation-system’s asynchronous read/write
call and passing it a user-defined buffer. Additionally, a callback on a
"write/read complete" event is set. While the event demultiplexer waits
for changes, the operating system launches a parallel kernel thread that does
the actual write/read operation and copies the data from/to the user-defined
buffer. Once the operation is complete, the "complete"-event gets fired
and the invoked callback can continue writing/reading additional data with
another systemcall. In conclusion: The main difference between reactor and
proactor is the way they do the actual write/read. While the reactor is
only being signalled when the socket is ready to write/read and then does
the reading/writing itself in a synchronous manor, the proactor will use an
asynchronous I/O call to the operating system and only supply the buffers to
read from/write to.
A more detailed comparison between the two can be found in "Comparing Two
High-Performance I/O Design Patterns" by Alexander Libman and Vladimir
Gilbourd [1]
6
8. Figure 1: event-loop
4.1 Working with the reactor
When working with the reactor pattern, there are a few things that one has
to pay attention to.
The most important one is: Don’t stop the reactor! Most frameworks that
provide a generic implementation of the reactor pattern are single-threaded.
This makes perfect sense as the thread overhead is exactly what we wanted
to stay away from. This also means that the reactor loop will be executing
your callbacks and resume once they finished. Making your callback code as
lightweight as possible is a best practice when working with this pattern.
This means mainly that these things should be avoided:
• loops that iterate over large collections of objects
• computation intensive things
• calling synchronous I/O methods (printing to stdout might also be part
of this depending on its implementation)
• sleep/pause statements
If there really is a need for these operations, they should be done within an
extra thread.
7
9. 5 Fighting complexity - Lightweight threads as
syntactic sugar
While the reactor pattern offers a great solution to I/O problems when trying
to scale, it is definitely not a "drop-in" solution. Using the pattern requires
evented code to be registered as callbacks.
This code is in almost all cases more complex (and bigger) than its ’regular’
counterpart. Especially things like nested callbacks require a lot of concentration
and are lacking readability.
While this certainly is a problem, some programming languages are able to
help by adding a lightweight alternative to threads to the mix which are
usually known as "Coroutines".
These lightweight threads are e.g. "Coroutines" in Lua, "Fibers" in Ruby,
"tasklets" in stackless Python or in general: lightweight constructs that are
able to "yield" and "resume" at a given time.
An overview of programming languages supporting this construct can be
found on the "Coroutine" page on Wikipedia1 .
A good example of this can be found in Ilya Grigorik’s wonderful post blogpost
on "Untangling evented code with ruby fibers"[3]. He shows how wrapping
our callback functions in Ruby Fibers can help bring some readability back
to our code and still keep it asynchonous at heart. By wrapping them, we
can define them as a "regular" function outside of our reactor-loop and still
call them from within the loop without the risk of accidentally blocking the
reactor. Since Fibers, Ruby’s lightweight concurrency-elements, are basically
user-level Threads without preemption. They have to be yielded and resumed
by hand. This allows us to do the following workflow:
1. set up asynchronous call
2. add a callback that will resume the Fiber once the call it is done
3. immediately yield the Fiber
To give a better example, here is a generic method that would allow us
to call it from within the reactor with minimal overhead, thus keeping the
reactor loop small. The difference to a "regular" function call is, that this
piece of code will almost immediately return and NOT block the reactor.
1
http://en.wikipedia.org/wiki/Coroutine)
8
10. def do_something(parameter)
# get the ID of our current Fiber
f = Fiber.current
# set up an asyncronous call
my_async_call = Library.do_something()
# register callback to resume fiber
# once the async call is done
result = my_async_call.callback { f.resume() }
# give control back to the main Process
Fiber.yield
# This is where we’ll return to after after the fiber
# has been resumed by the callback. Our "result"
# variable should be set and we can return it
return result
end
To process the return value of this function, we can’t just access it in the
reactor loop, because we never know when the Fiber got called and actually
returns something.
This can be solved by just having a simple check like "if my_result != nil" in
the reactor loop. This way, we can basically wait for the method to return
something without blocking the reactor and therefore still allowing other
operations to happen while we wait.
Ilya Grigorik has automated this behaviour within his "em-synchrony" library
for the Eventmachine framework.
6 Asynchronous I/O Frameworks
There are numerous frameworks for programming languages that help programmers
to abstract over the complexity of underlying OS functionality and keep the
code focused on business logic.
This list is by no means complete, but it mentions some well known Frameworks
for mainstream programming languages. I would strongly advice to look into
these Frameworks before trying to implement an independent solution. Almost
all of these Frameworks offer generic support for TCP/UDP connections that
allow to implement any given protocol on top of an asynchronous base:
• Eventmachine: A Ruby Framework for Ruby (and JRuby). It provides
9
11. clients for HTTP, Memcached, SMTP, Socks and numerous other
protocols. It supports epoll (Linux), kqueue (BSD/OS X) and /dev/poll
(Solaris)
• Twisted: A Python Framework. Basis for lots of Projects implementing
HTTP, NNTP, IRC, FTP and others.
• Node.js: An evented I/O Framework for serverside Javascript. Node.js
is using Google’s V8 Javascript Engine.
• Apache Java, MINA: Allows programmers to create their own asynchronous
servers. Shipping with SSL support.
• Netty: Java, Supposedly faster than MINA. It is the underlying basis
of JXTA.
7 When to use it
While asynchronous I/O is a great tool for deployments that are I/O intensive,
it sometimes isn’t worth the additional complexity in code. Just doing a simple
test with HTTP libraries that offer easy access to evented I/O operations
shows that the total time actually waiting for a thread to start, execute and
terminate itself again is really close to what the asynchronous libraries do
with, in this case, epoll. Just as a small proof of concept, I used a benchmark
script to run requests to a mix of large sites (google, yahoo, ...) and a site that
responded slowly (simple web-application that just uses a sleep() command
before answering).
The Ruby script used 3 different approaches:
• em-http: A HTTP library for Eventmachine. Eventmachine is an
asynchronous I/O framework using the reactor pattern. The reactor is
written in C to get arround the threading constraints imposed by the
Ruby VM
• typhoeus: A library that has bindings to libcurl and is able to use
libcurl’s "multi’ interface which internally relies on epoll for evented
I/O
• threaded net::http: Ruby’s default HTTP library. Since There is no
way to request multiple URLs at once, we just launch a new thread for
each request.
10
12. After running each script 20 times, these are the averaged results:
em-http 2.66 s
typhoeus 2.92 s
net::http 2.65 s
We can see that, in this case, the three approaches don’t differ from each
other by more than the margin of error in this quick experiment. For most
"simple" operations with only 50-100 connections, the overhead created by
the reactor is comparable to the overhead that regular threads create in this
situation.
8 Conclusion
Asynchronous I/O is a powerful tool to help applications that have to handle
large amounts of parallel I/O operations scale. While it is also possible to
use it for generic applications with lesser requirements, it usually results in
overly complex code without a lot of gained performance for the specific task.
While there are frameworks and techniques that make the necessary code
look cleaner, it might be best to have at least some experience with either
functional programming or event-driven systems to be able to maintain a
codebase that uses evented I/O.
References
[1] "Comparing Two High-Performance I/O Design Patterns"
Alexander Libman and Vladimir Gilbourd
2005
http://www.artima.com/articles/io_design_patterns.html
[2] Finding parallelism - How to survive in a multi-core world
Kai Jäger
2008
http://kaijaeger.com/publications/finding-parallelism-how-to-survive-in-a-multi-core-world.
pdf
[3] Untangling Evented Code with Ruby Fibers
Ilya Grigorik
2010
http://www.igvita.com/2010/03/22/untangling-evented-code-with-ruby-fibers/
11