TellApart uses the gevent library to build highly concurrent and asynchronous Python applications. Gevent allows synchronous Python code to run asynchronously by using greenlets. It monkey-patches blocking libraries so that when blocking calls occur, other greenlets can run instead of the entire process blocking. This allows TellApart to build applications like their front-end server and database proxy that can handle millions of requests per day with low latency using a single process per core.
Presentation on how GRNET uses Ceph as a storage backend on its Cloud Computing services. Technical specs, lessons learned, future plans.
Presentation held at the 1st GEANT SIG-CISS Meeting in Amsterdam, 2017-09-25.
GRNET - Greek Research and Technology network is the state-owned Greek NREN.
Want to learn how Facebook scales their load balancing infrastructure to support more than 1.3 billion users? We will be revealing the technologies and methods we use to global route and balance Facebook's traffic. The Traffic team at Facebook has built several systems for managing and balancing our site traffic, including both a DNS load balancer and a software load balancer capable of handling several protocols. This talk will focus on these technologies and how they have helped improve user performance, manage capacity, and increase reliability.
This is the story of how we managed to scale and improve Tappsi’s RoR RESTful API to handle our ever-growing load - told from different perspectives: infrastructure, data storage tuning, web server tuning, RoR optimization, monitoring and architecture design.
Talk presented by Aarón Fas & Andrés Viedma at the JBcnConf 2015.
'Microservices' is one of the most popular buzzwords in the industry now, but are they really a step forward? Or they might be more a problem than a solution? When are they really helpful? How should they be addressed? What challenges will we face if we decide to implement a microservices based architecture?
One year ago, Tuenti moved from a monolithic PHP backend to a Java + PHP microservices architecture. In this talk, we'll share our experiences so far: how we addressed the change, how we implemented it, why we think it's been valuable for us (and how is that related to the company culture), why it might not be a good idea for your company / application and, mostly, what lessons we have learned from this experience.
Presentation on how GRNET uses Ceph as a storage backend on its Cloud Computing services. Technical specs, lessons learned, future plans.
Presentation held at the 1st GEANT SIG-CISS Meeting in Amsterdam, 2017-09-25.
GRNET - Greek Research and Technology network is the state-owned Greek NREN.
Want to learn how Facebook scales their load balancing infrastructure to support more than 1.3 billion users? We will be revealing the technologies and methods we use to global route and balance Facebook's traffic. The Traffic team at Facebook has built several systems for managing and balancing our site traffic, including both a DNS load balancer and a software load balancer capable of handling several protocols. This talk will focus on these technologies and how they have helped improve user performance, manage capacity, and increase reliability.
This is the story of how we managed to scale and improve Tappsi’s RoR RESTful API to handle our ever-growing load - told from different perspectives: infrastructure, data storage tuning, web server tuning, RoR optimization, monitoring and architecture design.
Talk presented by Aarón Fas & Andrés Viedma at the JBcnConf 2015.
'Microservices' is one of the most popular buzzwords in the industry now, but are they really a step forward? Or they might be more a problem than a solution? When are they really helpful? How should they be addressed? What challenges will we face if we decide to implement a microservices based architecture?
One year ago, Tuenti moved from a monolithic PHP backend to a Java + PHP microservices architecture. In this talk, we'll share our experiences so far: how we addressed the change, how we implemented it, why we think it's been valuable for us (and how is that related to the company culture), why it might not be a good idea for your company / application and, mostly, what lessons we have learned from this experience.
Shared Memory Performance: Beyond TCP/IP with Ben Cotton, JPMorganHazelcast
In this webinar you’ll learn the importance of Advanced Data Locality and Data IPC Transports with respect to Java distributed cache data grids. This is information that is super-crucial to the HPC Linux supercomputing community. The presenter will show how by using native /dev/shm as an IPC transport we can achieve latencies 1,000x faster than TCP/IP.
We’ll cover the following topics:
-Why going off-heap is fundamental to meeting real-time SLAs
-Why traditional grid transports (TCP/IP) are not good enough for the HPC Linux supercomputing community
-Why we need something more than TCP/IP
-Live Q&A Session
Presenter:
Ben Cotton, Consultant at JPMorgan
Ben is has been an IT Consultant to Financial Services Industry for nearly 20 years. His specializations lie in open source, transactions, caching, datagrids, fixed income & derivatives trading systems.
Ben is active in the following communities:
Java Community Process Member
JSR-156 expert group: Java XML Transactions API
JSR-107 expert group: Java Caching API
JSR-347 expert group: Java Data Grids API
RedHat Community Member (Fedora, Infinispan, JBoss XTS) Code Contributor
Kubernetes at NU.nl (Kubernetes meetup 2019-09-05)Tibo Beijen
Slides of the presentation about Kubernetes practices and learnings at NU.nl.
This presentation was the first of two at the Dutch Kubernetes meetup at the Sanoma Netherlands offices, that took place on Sept. 5th 2019
Keeping Latency Low and Throughput High with Application-level Priority Manag...ScyllaDB
Throughput and latency are at a constant tension. ScyllaDB CTO and co-founder Avi Kivity will show how high throughput and low latency can both be achieved in a single application by using application-level priority scheduling.
WebLogic Stability; Detect and Analyse Stuck ThreadsMaarten Smeets
Stuck threads are a major cause for stability issues of WebLogic Server environments. Often people in operations and development who are confronted with stuck threads, are at a loss what to do. In this presentation we will talk about what stuck threads actually are and how you can detect them. We will elaborate on how you can get to the root cause of a stuck thread and which tools can help you with that. In order to reduce the impact of having stuck threads in an application, we will talk about using workmanagers. In order to prevent stuck threads we will illustrate several patterns which can be implemented in infrastructure and applications. Next time you see a stuck thread, you will know what to do!
Unless you have a problem which scales to many independent tasks easily e.g. web services, you may find that the best way to improve throughput is by reducing latency. This talk starts with Little's Law and it's consequences for high performance computing.
Shared Memory Performance: Beyond TCP/IP with Ben Cotton, JPMorganHazelcast
In this webinar you’ll learn the importance of Advanced Data Locality and Data IPC Transports with respect to Java distributed cache data grids. This is information that is super-crucial to the HPC Linux supercomputing community. The presenter will show how by using native /dev/shm as an IPC transport we can achieve latencies 1,000x faster than TCP/IP.
We’ll cover the following topics:
-Why going off-heap is fundamental to meeting real-time SLAs
-Why traditional grid transports (TCP/IP) are not good enough for the HPC Linux supercomputing community
-Why we need something more than TCP/IP
-Live Q&A Session
Presenter:
Ben Cotton, Consultant at JPMorgan
Ben is has been an IT Consultant to Financial Services Industry for nearly 20 years. His specializations lie in open source, transactions, caching, datagrids, fixed income & derivatives trading systems.
Ben is active in the following communities:
Java Community Process Member
JSR-156 expert group: Java XML Transactions API
JSR-107 expert group: Java Caching API
JSR-347 expert group: Java Data Grids API
RedHat Community Member (Fedora, Infinispan, JBoss XTS) Code Contributor
Kubernetes at NU.nl (Kubernetes meetup 2019-09-05)Tibo Beijen
Slides of the presentation about Kubernetes practices and learnings at NU.nl.
This presentation was the first of two at the Dutch Kubernetes meetup at the Sanoma Netherlands offices, that took place on Sept. 5th 2019
Keeping Latency Low and Throughput High with Application-level Priority Manag...ScyllaDB
Throughput and latency are at a constant tension. ScyllaDB CTO and co-founder Avi Kivity will show how high throughput and low latency can both be achieved in a single application by using application-level priority scheduling.
WebLogic Stability; Detect and Analyse Stuck ThreadsMaarten Smeets
Stuck threads are a major cause for stability issues of WebLogic Server environments. Often people in operations and development who are confronted with stuck threads, are at a loss what to do. In this presentation we will talk about what stuck threads actually are and how you can detect them. We will elaborate on how you can get to the root cause of a stuck thread and which tools can help you with that. In order to reduce the impact of having stuck threads in an application, we will talk about using workmanagers. In order to prevent stuck threads we will illustrate several patterns which can be implemented in infrastructure and applications. Next time you see a stuck thread, you will know what to do!
Unless you have a problem which scales to many independent tasks easily e.g. web services, you may find that the best way to improve throughput is by reducing latency. This talk starts with Little's Law and it's consequences for high performance computing.
Typesafe trainer and consultant Will Sargent describes just how Play Framework is so "fast" for Java and Scala production apps.
More Play, Akka, Scala and Apache Spark webinars, presentations, and videos:
http://typesafe.com/resources/videos
In this session, we'll discuss architectural, design and tuning best practices for building rock solid and scalable Alfresco Solutions. We'll cover the typical use cases for highly scalable Alfresco solutions, like massive injection and high concurrency, also introducing 3.3 and 3.4 Transfer / Replication services for building complex high availability enterprise architectures.
Slides from the second meeting of the Toronto High Scalability Meetup @ http://www.meetup.com/toronto-high-scalability/
-Basics of High Scalability and High Availability
-Using a CDN to Achieve 99% Offload
-Caching at the Code Layer
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInLinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn. This was a presentation made at QCon 2009 and is embedded on LinkedIn's blog - http://blog.linkedin.com/
Hangfire
An easy way to perform background processing in .NET and .NET Core applications. No Windows Service or separate process required.
Why Background Processing?
Lengthy operations like updating lot of records in DB
Checking every 2 hours for new data or files
Invoice generation at the end of every billing period
Monthly Reporting
Rebuild data, indexes or search-optimized index after data change
Automatic subscription renewal
Regular Mailings
Send an email due to an action
Background service provisioning
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
2. TellApart’s
Infrastructure
Overview
• Millions
of
daily
acIve
users
• Page-‐views
across
mulIple
sites
• Real-‐Time
Bidding
integraIon
- Very
high
volume,
low
latency
- Response
Ime:
50
percenIle:
17ms,
95
percenIle:
50
ms
• All
requests
require
user
data
• EnIrely
Amazon
Web
Services
(AWS),
in
2
parallel
regions
2
3. What
is
gevent?
gevent
is
a
corouIne-‐based
Python
networking
library
that
uses
greenlet
to
provide
a
high-‐level
synchronous
API
on
top
of
the
libevent
event
loop.
• EssenIally,
allows
normally
synchronous
code
to
run
asynchronously
3
4. What
is
gevent?
lib·∙e·∙vent
(ˈlib-‐i-‐ˈvent):
efficient
cross-‐pla]orm
library
for
execuIng
callbacks
when
specific
events
occur
or
a
Imeout
has
been
reached.
Includes
several
networking
libraries
(e.g.
DNS,
HTTP)
green·∙let
(ˈgrēn-‐lət):
lightweight
co-‐rouInes
for
in-‐process
concurrent
programming.
Ported
from
Stackless
Python
as
a
library
for
the
CPython
interpreter
4
5. How
does
gevent
work?
• One
gevent
“hub”
per
process
• Monkey-‐patch
blocking
libraries
- socket,
thread,
select,
etc.
• Use
greenlets
like
threads
• Blocking
calls
switch
to
another
(ready)
greenlet
5
7. Example
Server
• Server
implementaIon
is
the
same
• DB
lookup
blocks
on
network
IO
• With
gevent,
greenlet
gets
swapped
out
so
another
request
can
be
served
• When
the
DB
request
finishes,
the
greenlet
will
conInue
where
it
lej
off
7
8. Advantages
• Write
code
as
though
it
were
synchronous
(mostly)
- No
‘callback
spaghen’
like
with
a
callback
framework
- Exact
same
code
can
run
synchronously
(e.g.
unit
tests)
• Greenlets
are
very
lightweight
- 100’s
or
1000’s
can
run
concurrently
- No
context
switch
o Same
order
of
magnitude
as
a
funcIon
call
- No
GIL
related
performance
issues
• Co-‐operaIve
concurrency
makes
synchronizaIon
easy
- Greenlets
cannot
be
preempted
- No
need
for
in-‐process
atomic
locks
- Ojen
eliminates
the
need
for
synchronizaIon
o As
long
as
there
are
no
blocking
calls
in
the
criIcal
secIon
8
9. Advantages
(conInued)
• gevent
is
fast
- Very
thorough
set
of
benchmarks
by
Nicholas
Piël
hrp://nichol.as/benchmark-‐of-‐python-‐web-‐servers
And
then
there
is
Gevent
[...]
[…]
if
you
want
to
dive
into
high
performance
websockets
with
lots
of
concurrent
connecIons
you
really
have
to
go
with
an
asynchronous
framework.
Gevent
seems
like
the
perfect
companion
for
that,
at
least
that
is
what
we
are
going
to
use.
9
10. Problems
• Monkey-‐patching
- Doesn’t
play
well
with
C
extensions
o Blocking
code
in
C
libraries
will
cause
the
process
to
block
- Can
confuse
some
libraries
o e.g.
thread-‐local
storage
• Breaks
analysis
tools
- cProfile
produces
garbage
- AlternaIve
tools
available
o gevent-‐profiler
(Meebo)
o gevent_request_profiler
(TellApart)
• Co-‐operaIve
scheduling
- Rogue
greenlets
can
Ie
up
the
enIre
process
o e.g.
CPU
bound
background
worker
- Long-‐running
tasks
have
to
periodically
yield
10
11. Problems
• Same
server
as
before
• Processing
in
loop
can
take
long
• Can
hurt
latency
of
other
requests
• Add
‘gevent.sleep(0)’
to
loop
• Allows
other
greenlets
to
run
11
12. Uses
• We
use
gevent
everywhere
we
use
Python
• TellApart
Front
End
(TAFE)
- gevent
WSGI
server
with
a
micro-‐framework
- One
process
per
core
- Nginx
reverse-‐proxy
in
front
• Database
Proxy
(moxie)
- Thrij
service
- ConnecIon
pooling
across
clients
- Minimal
addiIonal
latency
(~2ms)
12
13. Case
Study
-‐
Taba
• Taba
is
a
distributed
Event
AggregaIon
Service
• Provides
near
real-‐Ime
metrics
from
across
a
cluster
• At
TellApart:
- 10,000
individual
Tabs
- 100’s
of
event
source
clients
- 20,000,000
events
/
minute
- 25
seconds
latency
from
real-‐Ime
13
14. Case
Study
-‐
Taba
• Implement
Imeouts
very
easily
• FuncIon
doesn’t
need
to
know
it’s
being
Imed
14
15. Case
Study
–
Taba
• Perform
simultaneous
lookups
to
a
sharded
database
• No
thread
pools
• No
need
for
locking
15
16. Case
Study
–
Taba
• Streaming
from
DB
in
batches
• No
thread
pool
• Trivial
synchronizaIon
• Process
data
while
the
next
batch
is
retrieved
16
17. Thank
you!
Kevin
Ballard
kevin(at)tellapart(dot)com
17