Eliminating every last bit of downtime caused by deployment and application errors takes some work. Learn how a combination of domains, sensible handling of uncaught exceptions, graceful connection termination, and process management with the cluster module and its friends can give you confidence that your application is always available.
Counting with Prometheus (CloudNativeCon+Kubecon Europe 2017)Brian Brazil
Counters are one of the two core metric types in Prometheus, allowing for tracking of request rates, error ratios and other key measurements. Learn why are they designed the way they are, how client libraries implement them and how rate() works.
If you'd like more information about Prometheus, contact us at prometheus@robustperception.io
Counting with Prometheus (CloudNativeCon+Kubecon Europe 2017)Brian Brazil
Counters are one of the two core metric types in Prometheus, allowing for tracking of request rates, error ratios and other key measurements. Learn why are they designed the way they are, how client libraries implement them and how rate() works.
If you'd like more information about Prometheus, contact us at prometheus@robustperception.io
How to improve the availability of your Plone site using RelStorage and PostgreSQL, with the help of repmgr.
A brief introduction to HA is given, before dwelling deep into the setup of RelStorage and PostgreSQL, the use of repmgr, and how to avoid common pitfalls and unexpected traps.
We rubyists historically haven’t been in the habit of thinking about concurrency but the reality is that our thread-unsafe code often works by sheer luck. There are different implementations of Ruby with their own semantics that can unearth challenging and unexpected concurrency bugs in our code. We have to become more accustomed to writing threadsafe code in order to anticipate these potential surprises, especially in light of the rise in popularity of JRuby.
I will discuss approaches to writing threadsafe code in this talk, with a specific focus on performance considerations and testing. I'll start by explaining some basic concurrency concepts, describe methods for handling shared mutable data, and touch on the subtleties of concurrency primitives (Mutex, ConditionVariable). Hair-raising, real-world bugs will be used throughout the presentation to illustrate specific concurrency issues and techniques for solving them.
Advanced Percona XtraDB Cluster in a nutshell... la suiteKenny Gryp
Percona XtraDB Cluster is a high availability and high scalability solution for MySQL clustering. Percona XtraDB Cluster integrates Percona Server with the Galera synchronous replication library in a single product package which enables you to create a cost-effective MySQL cluster.
Since three years during Percona Live we initiate people to this technology... but what's next ? This tutorial is the continuation. It targets users that already have experience with PXC and want to go further.
This tutorial will cover the following topics:
- monitoring and trending
- problem solving
- limitations, when not to choose for PXC
- how to test ? (benchmark)
- schema changes
- backups
- multi datacenter
- advanced load balancing with HA Pproxy and Maxscale
- fine tune some important variables like galera cache, flow control limit, ...
Securing Containers, One Patch at a Time - Michael Crosby, DockerDocker, Inc.
Responsible disclosure is a key ingredient of any solid security strategy. In this session, Docker maintainer Michael Crosby will explain the ins and outs of CVE-2016-9962: how it was discovered, how it could even happen in the first place, and how it was addressed. A vertiginous abseil at the boundaries of the kernel, in the fascinating land of system calls and randomized address space. You will think twice before leaking a file descriptor again.
Describe synchronization techniques used by programmers who develop .pdfexcellentmobiles
Describe synchronization techniques used by programmers who develop applications for
LUBUNTU (LINUX)
Solution
Concurrency and locking: Synchronization methods are necessary when the property of
concurrency exists. Concurrency exists when two or more processes execute over the same time
period and potentially interact with one another. The Linux kernel supports concurrency in both
modes. The kernel itself is dynamic, and race conditions can be created in a number of ways. The
Linux kernel also supports multiprocessing known as symmetric multiprocessing (SMP).
Concurrency can occur on uniprocessor (UP) hosts where multiple threads share the same CPU
and preemption creates race conditions. Preemption is sharing the CPU transparently by
temporarily pausing one thread to allow another to execute. A race condition occurs when two or
more threads manipulate a shared data item and the result depends upon timing of the execution.
Concurrency also exists in multiprocessor (MP) machines, where threads executing
simultaneously in each processor share the same data. Note that in the MP case there is true
parallelism because the threads execute simultaneously. In the UP case, parallelism is created by
preemption. The difficulties of concurrency exist in both modes.
To combat the issue of race conditions, the concept of a critical section was created. A critical
section is a portion of code that is protected against multiple access. This portion of code can
manipulate shared data or a shared service (such as a hardware peripheral). Critical sections
operate on the principle of mutual exclusion.
Race condition Situation where simultaneous manipulation of a resource by two or more threads
causes inconsistent results.
Critical section Segment of code that coordinates access to a shared resource.
Mutual exclusion Property of software that ensures exclusive access to a shared resource.
Deadlock Special condition created by two or more processes and two or more resource locks
that keep processes from doing productive work.
Linux synchronization methods
Now that you have a little theory under your belt and an understanding of the problem to be
solved, let\'s look at the various ways that Linux supports concurrency and mutual exclusion. In
the early days, mutual exclusion was provided by disabling interrupts, but this form of locking is
inefficient (even though you can still find traces of it in the kernel). This method also doesn\'t
scale well and doesn\'t guarantee mutual exclusion on other processors.
The atomic operators are ideal for situations where the data you need to protect is simple,
such as a counter. While simple, the atomic API provides a number of operators for a variety of
situations. Here\'s a sample use of the API.
To declare an atomic variable, you simply declare a variable of type atomic_t. This structure
contains a single int element. Next, you ensure that your atomic variable is initialized using the
ATOMIC_INIT symbolic constant. In the case .
For More information, refer to Java EE 7 performance tuning and optimization book:
The book is published by Packt Publishing:
http://www.packtpub.com/java-ee-7-performance-tuning-and-optimization/book
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
How to improve the availability of your Plone site using RelStorage and PostgreSQL, with the help of repmgr.
A brief introduction to HA is given, before dwelling deep into the setup of RelStorage and PostgreSQL, the use of repmgr, and how to avoid common pitfalls and unexpected traps.
We rubyists historically haven’t been in the habit of thinking about concurrency but the reality is that our thread-unsafe code often works by sheer luck. There are different implementations of Ruby with their own semantics that can unearth challenging and unexpected concurrency bugs in our code. We have to become more accustomed to writing threadsafe code in order to anticipate these potential surprises, especially in light of the rise in popularity of JRuby.
I will discuss approaches to writing threadsafe code in this talk, with a specific focus on performance considerations and testing. I'll start by explaining some basic concurrency concepts, describe methods for handling shared mutable data, and touch on the subtleties of concurrency primitives (Mutex, ConditionVariable). Hair-raising, real-world bugs will be used throughout the presentation to illustrate specific concurrency issues and techniques for solving them.
Advanced Percona XtraDB Cluster in a nutshell... la suiteKenny Gryp
Percona XtraDB Cluster is a high availability and high scalability solution for MySQL clustering. Percona XtraDB Cluster integrates Percona Server with the Galera synchronous replication library in a single product package which enables you to create a cost-effective MySQL cluster.
Since three years during Percona Live we initiate people to this technology... but what's next ? This tutorial is the continuation. It targets users that already have experience with PXC and want to go further.
This tutorial will cover the following topics:
- monitoring and trending
- problem solving
- limitations, when not to choose for PXC
- how to test ? (benchmark)
- schema changes
- backups
- multi datacenter
- advanced load balancing with HA Pproxy and Maxscale
- fine tune some important variables like galera cache, flow control limit, ...
Securing Containers, One Patch at a Time - Michael Crosby, DockerDocker, Inc.
Responsible disclosure is a key ingredient of any solid security strategy. In this session, Docker maintainer Michael Crosby will explain the ins and outs of CVE-2016-9962: how it was discovered, how it could even happen in the first place, and how it was addressed. A vertiginous abseil at the boundaries of the kernel, in the fascinating land of system calls and randomized address space. You will think twice before leaking a file descriptor again.
Describe synchronization techniques used by programmers who develop .pdfexcellentmobiles
Describe synchronization techniques used by programmers who develop applications for
LUBUNTU (LINUX)
Solution
Concurrency and locking: Synchronization methods are necessary when the property of
concurrency exists. Concurrency exists when two or more processes execute over the same time
period and potentially interact with one another. The Linux kernel supports concurrency in both
modes. The kernel itself is dynamic, and race conditions can be created in a number of ways. The
Linux kernel also supports multiprocessing known as symmetric multiprocessing (SMP).
Concurrency can occur on uniprocessor (UP) hosts where multiple threads share the same CPU
and preemption creates race conditions. Preemption is sharing the CPU transparently by
temporarily pausing one thread to allow another to execute. A race condition occurs when two or
more threads manipulate a shared data item and the result depends upon timing of the execution.
Concurrency also exists in multiprocessor (MP) machines, where threads executing
simultaneously in each processor share the same data. Note that in the MP case there is true
parallelism because the threads execute simultaneously. In the UP case, parallelism is created by
preemption. The difficulties of concurrency exist in both modes.
To combat the issue of race conditions, the concept of a critical section was created. A critical
section is a portion of code that is protected against multiple access. This portion of code can
manipulate shared data or a shared service (such as a hardware peripheral). Critical sections
operate on the principle of mutual exclusion.
Race condition Situation where simultaneous manipulation of a resource by two or more threads
causes inconsistent results.
Critical section Segment of code that coordinates access to a shared resource.
Mutual exclusion Property of software that ensures exclusive access to a shared resource.
Deadlock Special condition created by two or more processes and two or more resource locks
that keep processes from doing productive work.
Linux synchronization methods
Now that you have a little theory under your belt and an understanding of the problem to be
solved, let\'s look at the various ways that Linux supports concurrency and mutual exclusion. In
the early days, mutual exclusion was provided by disabling interrupts, but this form of locking is
inefficient (even though you can still find traces of it in the kernel). This method also doesn\'t
scale well and doesn\'t guarantee mutual exclusion on other processors.
The atomic operators are ideal for situations where the data you need to protect is simple,
such as a counter. While simple, the atomic API provides a number of operators for a variety of
situations. Here\'s a sample use of the API.
To declare an atomic variable, you simply declare a variable of type atomic_t. This structure
contains a single int element. Next, you ensure that your atomic variable is initialized using the
ATOMIC_INIT symbolic constant. In the case .
For More information, refer to Java EE 7 performance tuning and optimization book:
The book is published by Packt Publishing:
http://www.packtpub.com/java-ee-7-performance-tuning-and-optimization/book
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
18. tycth
r / a c doesn't do
async.
ty{
r
vrf=fnto( {
a
ucin)
trwnwErr"ho";
ho e ro(u-h)
}
;
stieu(,10;
eTmotf 0)
}cth(x {
ac e)
cnoelg"r /cthwntcth,e)
osl.o(ty
ac o' ac" x;
}
19. Domains are a bit like
tycth
r / a c for async.
vrd=rqie'oan)cet(;
a
eur(dmi'.rae)
do(err,fnto (r){
.n'ro' ucin er
cnoelg"oancuh" er;
osl.o(dmi agt, r)
};
)
vrf=dbn(ucin){
a
.idfnto(
trwnwErr"ho";
ho e ro(u-h)
};
)
stieu(,10;
eTmotf 0)
26. In Express, this might look like:
vrdmiWapr=fnto(e,rs nx){
a oanrpe
ucinrq e, et
vrrqoan=dmi.rae)
a eDmi
oancet(;
rqoanadrq;
eDmi.d(e)
rqoanadrs;
eDmi.d(e)
rqoanoc(err,fnto(r){
eDmi.ne'ro' uciner
rssn(0) / o nx(r)
e.ed50; / r eter;
};
)
rqoanrnnx)
eDmi.u(et;
}
;
Based on
https://github.com/brianc/node-domain-middleware
https://github.com/mathrawka/express-domain-errors
27. Domain methods.
a dbind an EE to the domain.
d:
r nrun a function in context of domain.
u:
b n : bind one function.
id
i t r e tlike bind but handles 1st arg e r
necp:
r.
d s o ecancels IO and timers.
ips:
34. Cluster module.
Node = one thread per process.
Most machines have multiple CPUs.
One process per CPU = cluster.
35. master / workers
1 master process forks n
workers.
Master and workers communicate state via IPC.
When workers want to listen to a socket, master registers them
for it.
Each new connection to socket is handed off to a worker.
No shared application state between workers.
38. Another use case for cluster:
Deployment.
Want to replace all existing servers.
Something must manage that = cluster master process.
39. Zero downtime deployment.
When master starts, give it a symlink to worker code.
After deploy new code, update symlink.
Send signal to master: fork new workers!
Master tells old workers to shut down, forks new workers from
new code.
Master process never stops running.
40. Signals.
A way to communicate with running processes.
S G U : reload workers (some like S G S 2
IHP
I U R ).
$kl - HP<i>
il s U pd
$srie<oesrienm>rla
evc nd-evc-ae eod
51. 1. Call s r e . l s .
evrcoe
vratrroHo =fnto(r){
a feErrok
uciner
sre.ls(;/ <-esr n nwcnetos
evrcoe) / - nue o e oncin
}
52. 2. Shut down keep-alive
connections.
vratrroHo =fnto(r){
a feErrok
uciner
apst"shtigon,tu) / <-stsae
p.e(iSutnDw" re; / - e tt
sre.ls(;
evrcoe)
}
vrsudwMdl =fnto(e,rs nx){
a htonide
ucinrq e, et
i(p.e(iSutnDw" { / <-ceksae
fapgt"shtigon)
/ - hc tt
rqcneto.eTmot1; / <-kl ke-lv
e.oncinstieu()
/ - il epaie
}
nx(;
et)
}
Idea from https://github.com/mathrawka/express-graceful-exit
53. 3. Then call p o e s e i
rcs.xt
in s r e . l s callback.
evrcoe
vratrroHo =fnto(r){
a feErrok
uciner
apst"shtigon,tu)
p.e(iSutnDw" re;
sre.ls(ucin){
evrcoefnto(
poesei() / <-alcert ei
rcs.xt1; / - l la o xt
};
)
}
56. On startup:
Cluster master comes up (for example, via Upstart).
Cluster master forks workers from symlink.
Each worker's server starts accepting connections.
57. On deploy:
Point symlink to new version.
Send signal to cluster master.
Master tells existing workers to stop accepting new connections.
Master forks new workers from new code.
Existing workers shut down gracefully.
60. Back to where we started:
1. Sensibly handle uncaught
exceptions.
We have minimized these by using domains.
But they can still happen.
61. Node docs say not to keep running.
An unhandled exception means your
application — and by extension node.js
itself — is in an undefined state. Blindly
resuming means anything could happen.
You have been warned.
http://nodejs.org/api/process.html#process_event_uncaughtexception
65. On uncaught exception:
Log error.
Server stops accepting new connections.
Worker tells cluster master it's done.
Master forks a replacement worker.
Worker exits gracefully when all connections are closed, or after
timeout.
67. People are also under the illusion that it is
possible to trace back [an uncaught]
exception to the http request that caused
it...
-felixge, https://github.com/joyent/node/issues/2582
78. Good reading:
Node.js Best Practice Exception Handling (some answers more
helpful than others)
Remove uncaught exception handler?
Isaacs stands by killing on uncaught
Domains don't incur performance hits compared to try catch
Rejected PR to add domains to Mongoose, with discussion
Don't call enter / exit across async
Comparison of naught and forever
What's changing in cluster
79. If you thought this was interesting,
We're hiring.
careers.fluencia.com