CQRS (Command Query Responsibility Segregation) is an architectural pattern that's talked about much and understood less. The gist of CQRS is that commands in your architecture are made differently than queries. The implication of this is that your entire architecture becomes unidirectional. No more request response models from simple APIs! How the heck do you build something like that? Despite being originally popularized in the .NET world over a decade ago, CQRS is making a comeback as unidirectional UI frameworks like Flux and Redux gain in popularity. If you're going to have a reactive, unidirectional user interface, why not have the same patterns reflected in your entire architecture? Matt will go over the tooling, patterns, and practices he's used to build CQRS architectures in marketing, finance, and more since he hopped on the Node bandwagon in 2011!
Find the video at http://www.nycnode.com/videos/matt-walters-how-to-cqrs-in-node-eventually-consistent-unidirectional-systems-with-microservices
Eddystone Beacons - Physical Web - Giving a URL to All ObjectsJeff Prestes
More mobile technologies are empowering people and machines to become more autonomous. In the same way as people, machines need ways to be identified to other sources in a connected environment. This begs the question, why not give a URL to objects? With Eddystone, a new Google specification for Beacon data, this is possible, and it works with both Android and iOS based devices.
With it you can implement what physical-web.org stands
Service Delivery Assembly Line with Vagrant, Packer, and AnsibleIsaac Christoffersen
Leverage Packer, Vagrant, and Ansible as part of a service delivery pipeline. Streamline your continuous delivery process while also targeting multiple cloud providers.
Eddystone Beacons - Physical Web - Giving a URL to All ObjectsJeff Prestes
More mobile technologies are empowering people and machines to become more autonomous. In the same way as people, machines need ways to be identified to other sources in a connected environment. This begs the question, why not give a URL to objects? With Eddystone, a new Google specification for Beacon data, this is possible, and it works with both Android and iOS based devices.
With it you can implement what physical-web.org stands
Service Delivery Assembly Line with Vagrant, Packer, and AnsibleIsaac Christoffersen
Leverage Packer, Vagrant, and Ansible as part of a service delivery pipeline. Streamline your continuous delivery process while also targeting multiple cloud providers.
The Flying Circus is an Operations-as-a-Service platform that supports project development teams to run their custom-develop software for clients. Earlier in 2014 we experienced a major data loss and had to perform massive disaster recovery. Unfortunately our Bacula setup was not up to the task and it took us longer and more effort to restore the data than we and our customers expected.
In this case study I’d like to present our public and very honest root cause analysis on how we managed to lose a lot of VMs’ data, how the restore happened, what we learned and how we’re trying to get better. After investigating our options for the future we decided to move away from Bacula’s file and VTL-oriented model and are currently implementing a solution based on CoW-filesystems (ZFS/btrfs), block-layer snapshots and diffing, and a small utility to glue things together.
Behind the Scenes at LiveJournal: Scaling StorytimeSergeyChernyshev
Brad talks about clustering setups using MySQL and DRDB and their Open Source software most of which he wrote initially and continues to develop.
A lot of these techniques and/or software is used by many other companies as well - among them Flickr/Yahoo! and Facebook.
RSVP Node.js class at www.nycdatascience.com
NYC data science academy's free workshop, given at NYC Open Data Meetup, http://www.meetup.com/NYC-Open-Data/events/163300552/
In this talk, Chris Mills discusses the historic problems with web apps and which technologies are stepping up to fill the holes. This includes device APIs such as Web Activities, Camera, device orientation and nfc, offline apps (which are finally looking realistic thanks to service workers), installable apps, and high quality games and other immersive high performance experiences using such features as Web audio API, Pointer lock, asm.js and Emscripten.
Introduction to Packer and Suitcase: A Packer-based OS Image Build SystemHubSpot Product Team
Introduction to Packer, a tool for building OS images and Suitcase, our framework for building Packer images. Presentation by Tom McLaughlin (@tmclaughbos) from HubSpot engineering.
How to scale up, out or down in Windows Azure - WebinarCommon Sense
Webinar presented on Jan 26th 2011 by Juan De Abreu.
Learn how to achieve:
• Scalability linear scale, scale up vs. scale out, choose VM sizes
• Storage Cache
• Elasticity, scale out, scale back and automation of scaling
Intended for:CIOs, CTOs, IT Managers, IT Developers, Lead Developers
Presentation from DDD Sydney, May 28th, 2016
Buzz word! More buzz words! And another buzz word!! Now that that's out of the way, if you're thinking of heading down the microservices path, then how do you do it? How do you build the services? What do you need to think about if you're starting from scratch? What if you're converting a legacy app? How do we deal with versioning? Do we have to use a NoSQL solution, just because Netflix does? Do we need to use docker/containers? What about the code? Show me the code! Well, that's what this session is all about. Designing and building microservices in .NET and then handling a bunch of other concerns that a microservices approach will force you to think about. Sounds interesting, doesn't it? You betcha.
The Flying Circus is an Operations-as-a-Service platform that supports project development teams to run their custom-develop software for clients. Earlier in 2014 we experienced a major data loss and had to perform massive disaster recovery. Unfortunately our Bacula setup was not up to the task and it took us longer and more effort to restore the data than we and our customers expected.
In this case study I’d like to present our public and very honest root cause analysis on how we managed to lose a lot of VMs’ data, how the restore happened, what we learned and how we’re trying to get better. After investigating our options for the future we decided to move away from Bacula’s file and VTL-oriented model and are currently implementing a solution based on CoW-filesystems (ZFS/btrfs), block-layer snapshots and diffing, and a small utility to glue things together.
Behind the Scenes at LiveJournal: Scaling StorytimeSergeyChernyshev
Brad talks about clustering setups using MySQL and DRDB and their Open Source software most of which he wrote initially and continues to develop.
A lot of these techniques and/or software is used by many other companies as well - among them Flickr/Yahoo! and Facebook.
RSVP Node.js class at www.nycdatascience.com
NYC data science academy's free workshop, given at NYC Open Data Meetup, http://www.meetup.com/NYC-Open-Data/events/163300552/
In this talk, Chris Mills discusses the historic problems with web apps and which technologies are stepping up to fill the holes. This includes device APIs such as Web Activities, Camera, device orientation and nfc, offline apps (which are finally looking realistic thanks to service workers), installable apps, and high quality games and other immersive high performance experiences using such features as Web audio API, Pointer lock, asm.js and Emscripten.
Introduction to Packer and Suitcase: A Packer-based OS Image Build SystemHubSpot Product Team
Introduction to Packer, a tool for building OS images and Suitcase, our framework for building Packer images. Presentation by Tom McLaughlin (@tmclaughbos) from HubSpot engineering.
How to scale up, out or down in Windows Azure - WebinarCommon Sense
Webinar presented on Jan 26th 2011 by Juan De Abreu.
Learn how to achieve:
• Scalability linear scale, scale up vs. scale out, choose VM sizes
• Storage Cache
• Elasticity, scale out, scale back and automation of scaling
Intended for:CIOs, CTOs, IT Managers, IT Developers, Lead Developers
Presentation from DDD Sydney, May 28th, 2016
Buzz word! More buzz words! And another buzz word!! Now that that's out of the way, if you're thinking of heading down the microservices path, then how do you do it? How do you build the services? What do you need to think about if you're starting from scratch? What if you're converting a legacy app? How do we deal with versioning? Do we have to use a NoSQL solution, just because Netflix does? Do we need to use docker/containers? What about the code? Show me the code! Well, that's what this session is all about. Designing and building microservices in .NET and then handling a bunch of other concerns that a microservices approach will force you to think about. Sounds interesting, doesn't it? You betcha.
(ARC402) Deployment Automation: From Developers' Keyboards to End Users' Scre...Amazon Web Services
Some of the best businesses today are deploying their code dozens of times a day. How? By making heavy use of automation, smart tools, and repeatable patterns to get process out of the way and keep the workflow moving. Come to this session to learn how you can do this too, using services such as AWS OpsWorks, AWS CloudFormation, Amazon Simple Workflow Service, and other tools. We'll discuss a number of different deployment patterns, and what aspects you need to focus on when working toward deployment automation yourself.
Vilius Lukošius - Decomposing distributed monolith with Node.js (WIX.com)Agile Lietuva
About the topic:
Wix transitioned from monolith to microservices around 8 years ago. Or did it? We have hundreds of microservices in production with teams working on separate independent services or groups of services with decoupled deployments and SLAs. Sounds like a microservice architecture, but there is a dark truth to it – single platform (jvm), shared libraries (framework), fat clients (binary dependencies). In industry it is called a distributed monolith. There is no easy way or incentive to get rid of it. But there is a way!
Let me tell you a story on how much fun it was, what path we took and where we are now with getting to a proper polyglot microservice architecture.
John About Vilius:
Guild Master @ Wix with experience in different roles and companies (enterprise, start-up, you name it). No matter how much life/circumstances are pushing me to lead/management roles, I always find ways to come back and keep on hacking.
Twitter: https://twitter.com/viliusl
GitHub: https://github.com/viliusl
Containerisation Hack of a Legacy Software Solution - Alex Carter - CodeMill ...CodeMill digital skills
Details
Alexandra Carter - Callcredit, Numero and Microsoft: Containerisation Hack of a Legacy Software Solution
This is the story of how we took a legacy solution and pushed it into containers on windows in just three days. This was also a great chance to work with Microsoft at the cutting edge of their work on containerisation, VSTS and Azure. Moving on from our Hackathon, we have continued adding new components, experimenting with orchestration and showcasing our work. I’ll talk you through the prep work, the 3 day hack and the subsequent work; what it means for the product roadmap, the experimentation we have done and how stakeholders are responding. Finally, we’ll look ahead to next steps.
Case study: https://microsoft.github.io/techcasestudies/devops/2017/06/16/Callcredit_DevOps.html
Alex Carter
"I have worked in IT, Marketing, Software Support and Software Delivery before moving into my current System Build (DevOps) role within Callcredit. I live and breathe DevOps and am currently focussing on anything around containerisation in Windows. A day without Metal and motor racing is a dull one."
@smileandeliver (https://twitter.com/smileandeliver)
From CodeMill digital skills meetup https://www.meetup.com/CodeMill-Digital-Skills/events/243110732/
Modern Cloud-Native Streaming Platforms: Event Streaming Microservices with K...confluent
Microservices, events, containers, and orchestrators are dominating our vernacular today. As operations teams adapt to support these technologies in production, cloud-native platforms like Cloud Foundry and Kubernetes have quickly risen to serve as force multipliers of automation, productivity and value. Kafka is providing developers a critically important component as they build and modernize applications to cloud-native architecture. This talk will explore:
• Why cloud-native platforms and why run Kafka on Kubernetes?
• What kind of workloads are best suited for this combination?
• Tips to determine the path forward for legacy monoliths in your application portfolio
• Running Kafka as a Streaming Platform on Container Orchestration
[Capitole du Libre] #serverless - mettez-le en oeuvre dans votre entreprise...Ludovic Piot
Tout comme le Cloud IaaS avant lui, le serverless promet de faciliter le succès de vos projets en accélérant le Time to Market et en fluidifiant les relations entre Devs et Ops.
Mais sa mise en œuvre au sein d’une entreprise reste complexe et coûteuse.
Après 2 ans à mettre en place des plateformes managées de ce type, nous partagons nos expériences de ce qu’il faut faire pour mettre en œuvre du serverless en entreprise, en évitant les douleurs et en limitant les contraintes au maximum.
Tout d’abord l’architecture technique, avec 2 implémentations très différentes : Kubernetes et Helm d’un côté, Clever Cloud on-premise de l’autre.
Ensuite, la mise en place et l’utilisation d’OpenFaaS. Comment tester et versionner du Function as a Service. Mais aussi les problématiques de blue/green deployment, de rolling update, d’A/B testing. Comment diagnostiquer rapidement les dépendances et les communications entre services.
Enfin, en abordant les sujets chers à la production : * vulnerability management et patch management, * hétérogénéïté du parc, * monitoring et alerting, * gestion des stacks obsolètes, etc.
Integrating Infrastructure as Code into a Continuous Delivery Pipeline | AWS ...Amazon Web Services
Ansible is a simple, but powerful automation tool with an agentless footprint that allows for the definition of architecture, intent, and policy as code that can be deployed across both on-prem and cloud infrastructure. This enables customers to extend their enterprise and applications into AWS in a way that maintains a consistent, secure posture as part of a continuous delivery pipeline. Customers can then natively integrate with AWS to seamlessly configure and deploy a range of AWS services such as Amazon Aurora, Amazon Redshift, Amazon EMR, Amazon Athena, Amazon CloudFront, Amazon Route 53, and Elastic Load Balancing from within Red Hat OpenShift across a secure, consistent hybrid cloud infrastructure. In this session, we will demonstrate how infrastructure can be instantiated with code as part of a continuous delivery pipeline and describe how that integrates with an OpenShift hybrid cloud deployment. Learn More: https://aws.amazon.com/government-education/
"Microservices Lessons Learned" talk at Voxxed Days Microservices, ParisSusanne Kaiser
The journey from monolith to microservices is different for every organization. A variety of challenges come with introducing microservices itself, but also organizational circumstances impacting the transformation that needed to be considered.
In this talk I would like to share some microservices lessons learned from a startup perspective - and in hindsight, what to watch out for if starting the journey again.
In an increasingly competitive marketplace, speed and business agility are paramount. And integration between customer-facing systems and back-end applications is more crucial than ever.
At this event, you'll learn how open source software built by communities, like Apache Camel, Docker, Kubernetes, OpenShift Origin, and Fabric8, can help organizations integrate services and establish effective continuous integration and delivery (CI/CD) pipelines.
Nagios Conference 2014 - David Josephsen - Alert on What You DrawNagios
David Josephsen's presentation on Alert on What You Draw.
The presentation was given during the Nagios World Conference North America held Oct 13th - Oct 16th, 2014 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/conference
Original interactive slides here: http://slides.com/yunzhilin/microservices-and-friends#/
A tongue-in-cheek presentation to TrunkPlatform interns 2015.
WinOps Conf 2016 - Michael Greene - Release PipelinesWinOps Conf
There are benefits to be gained when patterns and practices from developer techniques are applied to operations. Notably, a fully automated solution where infrastructure is managed as code and all changes are automatically validated before reaching production. This is a process shift that is recognized among industry innovators. For organizations already leveraging these processes, it should be clear how to leverage Microsoft platforms. For organizations that are new to the topic, it should be clear how to bring this process to your environment and what it means to your organizational culture. This presentation explains the components of a Release Pipeline for configuration as code, the value to operations, and solutions that are used when designing a new Release Pipeline architecture.
Evolution of a cloud start up: From C# to Node.jsSteve Jamieson
ComputeNext started 3 years ago to develop the first open marketplace for cloud computing services.
We started by using the technologies we were most familiar with - C# and SQL Server, and our initial architecture and implementation was based on these technologies.
Over time, we have progressively introduced more open source elements, including MongoDB, RabbitMQ and Node.js.
Now we are at the point where most of our back-end services rely on Node.js. The talk will talk about why we did this, how we did this, and discuss our experiences - both good and bad.
Similar to How to CQRS in node: Eventually Consistent, Distributed Microservice Systems.. (20)
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
2. It’s real. It works!
Former TechStars Co. CTO. Now consultant.
Built two businesses’ platforms from scratch using
CQRS in Node. Both large, distributed systems.
CQRS made both more maintainable and extendable.
Marketing platform crunched the Twitter firehose in
realtime. 3 Engineers managed around 20 services.
- GoChime.com
Bond Exchange with over $1.75B in trades. 6 engineers
managed around 40 services.
- Electronifie.com
Open Source my tooling and frameworks.
Other companies using them too!
(and I’ll tell you how)
That’s me ^^ !!
3. When to CQRS
• Realtime, reactive systems
• When preferring small, modular services
• When aiming for learning and growth
• When aiming to grow or eventually split teams
• Want ability to scale different parts of your
system separately
When not to CQRS
• Standalone, static sites
• Standalone, simple CRUD applications
5. First, a primer from history.
Bertrand Meyer, regarding object interfaces:
“Every method should either be a command that
performs an action, or a query that returns data to the
caller, but not both. In other words, Asking a question
should not change the answer.”
Long before CQRS was CQS:
Command Query Separation.
this guy
6. Some Type
doSomething() : void
getSomeStuff() : Stuff
Either change stuff<——
Or get stuff<——
CQS
Command-Query
Separation
doAndGetStuff() : Stuff Never both!
——————
7. Command-Query
Responsibility Segregation
A system-wide architecture that states - externally
facing subsystems (apps and apis) send commands to
perform actions which update the system’s state and
request queries to determine system’s state.
*Basically CQS on a system-wide scale. Calls between
services should change stuff, or get stuff. Never both.
CQRS
8. Command-Query
Responsibility Segregation
CQRS also denotes that queries and command
processing are provided by different subsystems.
Queries are made against a data store. Commands are
sent to and processed by services.
CQRS One more thing!
10. How about a larger system?
denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
web
client
mobile
client
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
Queries
Commands
unidirectional flow
eventually consistent
the dance!
Events
11. What’s that dance you’re doing?
denormalizer
dbdenormalizer
dbdenormalizer
db
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
the dance!
chain reaction of events which play out as a result of an
incoming command.
each service subscribes to the events they care about
choreography!
(not orchestration)
Events
12. commands tell services
when an actor wants an action
clients send commands to instruct a service to do work
commands are sent asynchronously; fire and forget
commands are present-tense, directives: order.create
web app order-svc
order.create
commands are sent directly to a single receiving service
13. events tell the world
when you’re done
services publish to inform other services of work / actions performed,
and state updated
services publish (broadcast) events to any services
that wish to subscribe
events past-tense, describe what happened: order.created
order-svc fulfillment-svc
order.createdorder.created
order.created
order.created
14. Two types of services
denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
web
client
mobile
client
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
front end
17. web-uiweb-uiweb-app
web
client
web-uiweb-uisvc
front end (an app’s perspective)
denormalizer
dbdenormalizer
dbdenormalizer
db
What’s different?
Apps (and apis) still query a db to get the state of the system
Never directly modify the db they read from
Let’s focus on:
20. rabbitmq
messaging that just works
• direct send to queue
• fanout / topic routing
• highly available
• highly performant
• used in financial exchanges,
industrial applications and more
• open source
• free
23. servicebus
super simple messaging in node
• direct send
• pub / sub / fanout / topic -routing
• simple to set up
• highly performant
• used in financial exchanges,
online advertising and more
• open source
• free
• perfect for creating microservices!
27. Sending commands
from the front end
// web-app, onButtonClick. instead of updating db.
const bus = require('servicebus').bus();
bus.send(‘order.create', {
order: {
userId: userId,
orderItems: items
}
});
web-uiweb-uiweb-app
web-ui
web-uisvc
// fire and forget.
command name
command itself
order.create command
28. Then what from the front end?
web-uiweb-uiweb-app
denormalizer
dbdenormalizer
dbdenormalizer
db
We wait.
• reactive / realtime:
• mongo oplog tailing (meteor)
• rethinkdb
• redis notifications
• couchdb
• graphQL
• polling
• non-realtime:
• product design: thanks! we’re processing your
order! check back later for updates!
queries!
29. We wait.
For the backend.
Choreography.
It’s eventually consistent!
33. back end (a service’s perspective)
web-uiweb-uiweb-app
svc
Commands
Events• Listen for commands and subscribe to events
• Performs business logic to process commands and events
• Update local state (optionally)
• Publish events to tell external services of updated state
Let’s focus on:
34. Sample service.
web-uiweb-uiweb-app
web-ui
web-uisvc
command name
command object
// order-svc index.js
const bus = require(‘./bus’);
const create = require(‘./lib/create’);
bus.listen(‘order.create', (event) => {
create(event, (err, order) => {
if (err) return event.handle.reject();
bus.publish(‘order.created’, order, () => {
event.handle.ack();
});
});
});
service publishes to the world when it’s done!
order.create command
order.created
event
36. back end (a downstream service’s perspective)
web-uiweb-uiweb-app
svc
Commands
Events• Listen for commands and subscribe to events
• Performs business logic to process commands and events
• Update local state (optionally)
• Publish events to tell external services of updated state
svc-2
Events
Events
Let’s focus on:
Same thing!
37. Sample downstream service.
web-uiweb-uiweb-app
web-ui
web-uisvc
// fulfillment-svc index.js
const bus = require(‘./bus’);
const fulfill = require(‘./lib/fulfill’);
bus.subscribe(‘order.created', (event) => {
fulfill(event, (err, order) => {
if (err) return event.handle.reject();
bus.publish(‘order.fulfilled’, order, () => {
event.handle.ack();
});
});
});
order.created
order.fulfilled
subscribe for events instead of
listening for commands
no different, from any other service!
39. servicebus-register-handlers
convention based event handler definition for
distributed services using servicebus.
automatically registers
event & command
handlers saved as
modules in folder
initialize at startup
40. servicebus-register-handlers
const bus = require(‘./lib/bus'); // instantiate servicebus instance
const config = require('cconfig')();
const log = require('llog');
const registerHandlers = require('servicebus-register-handlers');
registerHandlers({
bus: bus,
handleError: function handleError (msg, err) {
log.error('error handling %s: %s. rejecting message w/ cid %s and correlationId %s.', msg.type,
err, msg.cid, this.correlationId);
log.error(err);
msg.handle.reject(function () {
throw err;
});
},
path: './lib/handlers',
queuePrefix: 'my-svc-name'
});
initialize at startup:
provide initialized bus
define your error handling
path to your handlers
prefix to differentiate similar queues
41. servicebus-register-handlers
const log = require("llog");
module.exports.ack = true;
module.exports.queueName = 'my-service-name-order';
module.exports.routingKey = "order.create";
module.exports.listen = function (event, cb) {
log.info(`handling listened event of type ${event.type} with routingKey $
{this.routingKey}`);
/*
do something with your event
*/
cb();
};
each handler is a file
no params marks success.
pass back error to retry or fail.
callback based transactions!
differentiate queues
for different services
specify which commands or
events to listen or subscribe to
43. What about the ‘work’ part?
const log = require("llog");
module.exports.ack = true;
module.exports.queueName = 'my-service-name-order';
module.exports.routingKey = "order.create";
module.exports.listen = function (event, cb) {
log.info(`handling listened event of type ${event.type}
with routingKey ${this.routingKey}`);
/*
do something with your event
*/
cb();
};
// order-svc index.js
const bus = require(‘./bus’);
const create = require(‘./lib/create’);
bus.listen(‘order.create', (event) => {
create(event, (err, order) => {
if (err) return event.handle.reject();
bus.publish(‘order.created’, order, () => {
event.handle.ack();
});
});
});
these parts
44. What about the ‘work’ part?
That’s up to you!
Need an audit trail? Targeting finance? Consider event sourcing.
*and my framework, ‘sourced’
Depending on your problem, the right choice could be
mongoose and mongodb, a graph database, an in-memory data
structure, or even flat files.
CQRS makes no assertions about what technology you should
use, and in fact frees you to make a different decision for each
particular problem.
and depends on the problem you’re solving
45. But wait! There’s more!
servicebus middleware!
middleware can inspect and modify incoming
and outgoing messages
// ./lib/bus.js required as single bus instance used anywhere in service
const config = require('cconfig')();
const servicebus = require('servicebus');
const retry = require('servicebus-retry');
const bus = servicebus.bus({
url: config.RABBITMQ_URL
});
bus.use(bus.package());
bus.use(bus.correlate());
bus.use(retry({
store: new retry.RedisStore({
host: config.REDIS.HOST,
port: config.REDIS.PORT
})
}));
module.exports = bus;
bus.use() middleware
into bus message
pipeline. middleware can
act on incoming and/or
outgoing messages
46. But wait! There’s more!
servicebus middleware!
middleware can inspect and modify incoming
and outgoing messages
// ./lib/bus.js required as single bus instance used anywhere in service
const config = require('cconfig')();
const servicebus = require('servicebus');
const retry = require('servicebus-retry');
const bus = servicebus.bus({
url: config.RABBITMQ_URL
});
bus.use(bus.package());
bus.use(bus.correlate());
bus.use(retry({
store: new retry.RedisStore({
host: config.REDIS.HOST,
port: config.REDIS.PORT
})
}));
module.exports = bus;
packages outgoing
message data and
adds useful type,
timestamp, and
other properties
47. But wait! There’s more!
servicebus middleware!
middleware can inspect and modify incoming
and outgoing messages
// ./lib/bus.js required as single bus instance used anywhere in service
const config = require('cconfig')();
const servicebus = require('servicebus');
const retry = require('servicebus-retry');
const bus = servicebus.bus({
url: config.RABBITMQ_URL
});
bus.use(bus.package());
bus.use(bus.correlate());
bus.use(retry({
store: new retry.RedisStore({
host: config.REDIS.HOST,
port: config.REDIS.PORT
})
}));
module.exports = bus;
adds a correlationId
for tracing related
commands and events
through your system
48. But wait! There’s more!
servicebus middleware!
middleware can inspect and modify incoming
and outgoing messages
// ./lib/bus.js required as single bus instance used anywhere in service
const config = require('cconfig')();
const servicebus = require('servicebus');
const retry = require('servicebus-retry');
const bus = servicebus.bus({
url: config.RABBITMQ_URL
});
bus.use(bus.package());
bus.use(bus.correlate());
bus.use(retry({
store: new retry.RedisStore({
host: config.REDIS.HOST,
port: config.REDIS.PORT
})
}));
module.exports = bus;
now, every failed
message will retry
3 times if errors occur.
after that, the message
will automatically be
put on an error queue
for human inspection!
49. And more!
distributed tracing middleware!
var trace = require('servicebus-trace');
bus.use(trace({
serviceName: 'my-service-name',
store: new trace.RedisStore({
host: config.REDIS_HOST || 'localhost',
port: config.REDIS_PORT || 6379
})
}));
51. back end services
web-uiweb-uiweb-app
svc
Commands
Events• Listen for commands and subscribe to events
• Performs business logic to process commands and events
• Update local state (optionally)
• Publish events to tell external services of updated state
Recapping:
54. back end
What’s a denormalizer?
front end
• Just another back end service
• Has one job to do
• Subscribe to all events that the UI cares about
• Persist events in a format most efficient for the UI to view
• Completes the eventually consistent, unidirectional flow
denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web
client
denormalizer
Events
svc
56. Recapping the big picture.
denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
web
client
mobile
client
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
Queries
Commands
unidirectional flow
eventually consistent
the dance!
Events