Synchronous Reads, Asynchronous Writes refers to an architectural approach where data reads are performed synchronously through services, while data writes are performed asynchronously through a messaging system. This allows for decoupling of services, horizontal scaling of reads and writes, and loose coupling between systems. The key aspects are performing JSON RESTful reads through services like Sinatra, and pushing writes to a messaging system like RabbitMQ with routing keys to trigger downstream processing. This approach can help solve issues with monolithic Rails applications that do not scale effectively.
Sherlock Homepage - A detective story about running large web services - WebN...Maarten Balliauw
The site was slow. CPU and memory usage everywhere! Some dead objects in the corner. Something terrible must have happened! We have some IIS logs. Some traces from a witness. But not enough to find out what was wrong. In this session, we’ll see how effective telemetry, a profiler or two as well as a refresher of how IIS runs our ASP.NET web applications can help solve this server murder mystery.
Slide deck from my presentation on multi-threading with .NET. The presentation covers from beginner onwards and looks at current technologies (i.e. pre .NET 4.0) specifically.
What makes this extra special is the entire process of how I prepared for it, from finding content to slide deck layout to presentation prep is documented at: http://www.sadev.co.za/content/how-i-build-presentations-series-index
Sherlock Homepage - A detective story about running large web services - WebN...Maarten Balliauw
The site was slow. CPU and memory usage everywhere! Some dead objects in the corner. Something terrible must have happened! We have some IIS logs. Some traces from a witness. But not enough to find out what was wrong. In this session, we’ll see how effective telemetry, a profiler or two as well as a refresher of how IIS runs our ASP.NET web applications can help solve this server murder mystery.
Slide deck from my presentation on multi-threading with .NET. The presentation covers from beginner onwards and looks at current technologies (i.e. pre .NET 4.0) specifically.
What makes this extra special is the entire process of how I prepared for it, from finding content to slide deck layout to presentation prep is documented at: http://www.sadev.co.za/content/how-i-build-presentations-series-index
author: Kamil Durski, Senior Ruby Developer at Polcode
Twitter: @kdurski Github: github.com/kdurski
Ruby has built-in support for threads yet it’s barely used, even in situations where it could be very handy, such as crawling the web. While it can be a pretty slow process, the majority of the time is spent on waiting for IO data from the remote server. This is the perfect case to use threads.
You can read more about multi-threaded web crawler in Ruby here: http://www.polcode.com/en/multi-threaded-web-crawler-in-ruby/
Talk given at the Erlang User Conference, june 2013, Stockholm, about the performance of Zotonic, the Erlang Web Framework and CMS.
It highlights Zotonic's architecture, performance charts and provides a glimpse into the future of this web development framework.
It is official: MySQL listens to HTTP and speaks JSON. MySQL got a new plugin that lets HTTP clients and JavaScript users connect to MySQL using HTTP. The development preview brings three APIs: key-document for nested JSON documents, CRUD for JSON mapped SQL tables and plain SQL with JSON replies. More so: MySQL 5.7.4 has SQL functions for modifying JSON, for searching documents and new indexing methods!
author: Kamil Durski, Senior Ruby Developer at Polcode
Twitter: @kdurski Github: github.com/kdurski
Ruby has built-in support for threads yet it’s barely used, even in situations where it could be very handy, such as crawling the web. While it can be a pretty slow process, the majority of the time is spent on waiting for IO data from the remote server. This is the perfect case to use threads.
You can read more about multi-threaded web crawler in Ruby here: http://www.polcode.com/en/multi-threaded-web-crawler-in-ruby/
Talk given at the Erlang User Conference, june 2013, Stockholm, about the performance of Zotonic, the Erlang Web Framework and CMS.
It highlights Zotonic's architecture, performance charts and provides a glimpse into the future of this web development framework.
It is official: MySQL listens to HTTP and speaks JSON. MySQL got a new plugin that lets HTTP clients and JavaScript users connect to MySQL using HTTP. The development preview brings three APIs: key-document for nested JSON documents, CRUD for JSON mapped SQL tables and plain SQL with JSON replies. More so: MySQL 5.7.4 has SQL functions for modifying JSON, for searching documents and new indexing methods!
Similar to Synchronous Reads Asynchronous Writes RubyConf 2009 (20)
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
8. Data Reads Through
Well it means creating systems that
perform data reads through services.
data reads typically have to be
synchronous because a user is waiting on
Services
the operation. So they have to occur
inside the request/response life-cycle.
9. data writes through a
messaging or queuing
system
Often, a user doesn’t have to wait for
data to be written to receive a
response. So writes can be done
asynchronously outside of the request/
response life-cycle which mean you can
put them straight into a queue
14. Monolithic Applications
Also, having your entire application in one code
base and system doesn’t scale. This leads to
test suites that take more than 30 minutes to
Don’t Scale
run, deploys that push your entire application
just for a simple update.
24. Service Oriented Architecture
service oriented
architecture, but that’s
commonly associated with
things like SOAP, WSDL, and
a bunch of other heinous
things.
37. For your message format you should use
JSON. I know you can use XML, but...
JSON
38. XML is too
bloated and
XML Makes Children Cry
complex and
besides, it makes
children cry
39. I previously glossed over this picture. It’s
something called an asynchronous electric
motor, which is the only image I could conjure
Asynchronous
up to go with the term “asynchronous”
Writes
40. requires a
Messaging System
messaging
system to write
data through
41. For that I suggest
RabbitMQ , which is a
RabbitMQ
powerful messaging
system in addition to
having stuff as mundane
as a queue
42. And you’ll of course need a data
Data Store
store. I don’t care which you
use, but it should probably be
designed to solve the problem
for a particular piece of your
application.
51. Joel Spolsky calls people
that exhibit this behavior
“Architecture Astronauts”
“Sometimes smart thinkers just don't know when to stop, and they create
these absurd, all-encompassing, high-level pictures of the universe that
are all good and fine, but don't actually mean anything at all.
These are the people I call Architecture Astronauts. It's very hard to get
them to write code or design programs, because they won't stop thinking
about Architecture.”
58. and then you add
some background
processing...
and then you realize that you can’t do
everything inside the request/response
cycle so you add in a background process.
For now we’ll assume you’re using a
database backed queue like dj, bj, or some
other kind of “j”.
59. and then
you add
memcache...
but wait, then you realize
you need additional
performance so you add
memcache
60. server, duh!
and let’s not
forget that it’s
all fronted by
nginx or apache
61. and then you need more app
processing power so you add
t wo more ser vers and front
all that by ha proxy
62. so once you’ve done
all that, where do
Where to from here?
you go?
63. maybe you add redis because
you heard Ezra or Chris or
somebody say it’s awesome
and scales to infinity
64. and then you add
a read database
to eek out a
little more
performance
65. and the whole time your Rails
application code base is
growing with more logic and
additional background
processing
74. millions of RSS and
Atom feeds
Since we’re pre-launch we
definitely don’t have the too
many users problem. The traffic
and complexity comes from
having to update millions of rss
and atom feeds
75. data from external
sources
Pulling in real time
engagement from
multiple external
sources
76. complex business logic
and complex business logic. every
time something enters our system
we have to perform many different
tasks that are interdependent.
Here’s just a taste of it: our feed
fetcher pulls in a new blog post from
somewhere
90. originally we set up
a ser vices based
design that looked
kind of like this. as
you can see there
are a bunch of
interconnections
and it’s hard to
comprehend.
troubleshooting
failures was hard.
91. Each ser vice had to implement
HTTP + JSON
an http interface with json
formatted messages. This was
the only method for ser vice-
to-ser vice communication.
99. keep the HTTP
http services
for data reads,
which can be
cached and
Services for data reads
optimized
100. push writes through a
messaging system
data writes through a messaging
system with built in routing. It
also helps if it’s optimized for
processing thousands of messages
per second and supports the
pubsub style
103. require 'rubygems'
require 'sinatra'
get '/entries/:id' do
Entry.find(params[:id]).to_json
end
now sinatra is awesome
because it makes creating a
service this easy.
112. first_request.on_complete do |response|
post = Post.new(JSON.parse(response.body))
# get the first url in the post
third_request =
Typhoeus::Request.new(post.links.first)
third_request.on_complete do |response|
# do something with that
end
hydra.queue third_request
post
end
115. 20.times do
r = Typhoeus::Request.new(
"http://localhost:3000/users/1")
hydra.queue r
end
hydra.run
116. hydra.cache_setter do |request|
@cache.set(
request.cache_key,
request.response,
request.cache_timeout) if
request.cache_timeout
end
hydra.cache_getter do |request|
@cache.get(request.cache_key)
end
128. these features enable you to build
an event based system, which is
Event Based System
exactly what we needed. when
certain updates happen, it should
kick off calculations elsewhere in
the system. I’ll get into that in a bit,
but first some rabbit specifics
129. rabbit is an implementation
of an open protocol called
Advanced Message
Queueing Protocol or AMQP
AMQP
131. it has Exchanges and
it has a bunch of
features, but for the
purposes of
Asynchronous Writes,
Routing Keys too
exchanges and routing
keys are what we care
about most
138. So we have a fanout exchange called
entry.write. every queue bound to this
exchange will get messages published to it.
Here we have the three things we want to do.
First, index it for searching. Second, store it in
our key valuer store. Third, index in a
completely separate index used for data
research. So the search is Solr/lucene and the
research is Hadoop. Completely decoupled
systems.
139. That’s how we write entries. Here’s
how we do event based processing on
those writes. so here’s an example
where we have a topic exchange
named ‘entry.notify’. queues can be
bound to exchanges. so we have these
three queues
140. so take the example where
you have a message published
to the exchange with a
routing key of ‘insert’.
141. the message would get
routed to the queue
bound with insert and
to the queue bound with
hash
142. now let’s look at a message
with a routing key of
‘update.clicks.rank’
143. based on the bindings, the
message gets dropped into the
update and hash queue (ones
on the right err left?)
156. http://localhost:3000/locks/names/
pauldix
one way is to have the
ser vice responsible
expose a uniqueness
getter. so once you GET
a lock, you write
through the queue.
159. Eric Brewer’s CAP
theorem
in brewer’s CAP theorem he talked
about the relationship bet ween three
requirements when building
distributed systems. consistency,
availability, and partition tolerance.
162. partition tolerance
when you replicate data across multiple
systems, you create the possibility of forming
a partition. this happens when one or more
systems lose connectivity to other systems.
partition tolerance is defined formally as “no
set of failures less than total net work failure
is allowed to cause the system to respond
incorrectly”
164. Werner Vogels’
eventual consistency
“is a special form of weak
consistency. if no new updates are
made to an object, eventually all
accesses will return the last
updated value.”